VeeamBackup and the use of changed block tracking

[schedlbauerm]

VeeamBackup as well as for example VMWare DataRecovery uses 'Changed Block Tracking'.

Can this cause a problem if I use Veeam Backup and VMWare VDR with the same VMs ?

If I make a VDR-Backup of a VM before a VeeamBackup, can VDR label some blocks as 'backuped' so that VeeamBackup (with enabled changed block tracking)would not include those blocks in the synthetic fullbackup ?

Like this the Veeam Backup of the VM would be completly useless ?

I already asked the VeeamSupport this question and this was the answer:

Yes, that could be a problem when using VMware and Veeam products one after another.
We advise you to use Veeam product for all your data protection and recovery needs.

I didn't expect that answer. Can someone please tell me, that this is not true !!!

The same problem could occur if I have 2 or more VeeamBackup-Servers or if I do replicas and normal Veeam-Backups (with enabled changed block tracking) of the same VM at the same time.

The worst part is, that I wouldn't even notice that my fullsynthetic backups which use changed block tracking are inconsistent.

I searched the forum for this problem, but nobody ever brought this up.

I need some clear answers on this issue.

Martin

[tsightler]

One of the technical gurus can answer this question better, but I think what you might be missing is this, CBT doesn't work by simply saying "tell me what blocks have changed since the last backup" it works by saying "tell me what blocks have been changed since this specific point it time". So in other words, if your last replication ran at 1AM, and you next one runs at 3AM, then the backup says "tell me all blocks that changed since 1AM". If you have another job that ran a midnight, and runs again at noon, it would say, "tell me all block that changed since midnight" so it would still work fine.

[Gostev]

Martin, this is actually covered in the stickied FAQ thread, you have probably missed this Q and A.

Tom is absolutely correct, perfect summary and really hard to add anything else without going too technical. I believe that support reply was not relevant to changed block tracking, but rather it was about something else. I can investigate this for you, if you PM me your support case number. Thanks!

[schedlbauerm]

Thanks for the quick answer.

This makes sense.

As I understand it now, CBT keeps track of all the changed blocks of a vm. This information is stored in some kind of a database.

If a Backup program for example VeeamBackup or VMWare DataRecovery wants to make a backup, the CBT-Database tells the backup-program what blocks changed since the last backup made by this backup-programm. So it's not the way a normal backup program works which is using the archive bit to keep track of the changed files.

I still have some questions.

1) How long is the CB-Information kept in the CBT-Database ? I could make a backup today and the next backup in a week, month or even a year. For the backups to be consistent the CBT-Database would have to keep the information for a very long time.

2) What happens if the time on the esx or backupserver is changed, because the clock was to slow or fast ? Would the backup then miss some blocks.

3) What exactly is a block ? Is it a block of the partitiontable of the vm ? If yes, what happens If in- or decrease the partiton ?

I would be thankful for more detailed information about CBT and how this affects the use of VeeamBackup.

Thanks, Martin

[Gostev]

1. "Forever". Each VMDK block update also initiates USN (universal sequence number) update for the corresponding block in the CTK file. So if the application properly memorizes current USN during backup, later it will always be able to find out whether the specific block has changed since that USN moment, whether it has been 10 minutes or 10 years - by simply comparing memorized USN to block's USN from the CTK file.

2. Since changed block tracking is not clock based, time does not matter.

3. Definitely does not have to deal with neither guest partitions, nor VMFS block size. I did calculated this once based on VM size and change tracking engine debug output, but I don't recall what it was. I'll try to look up through my records when I get to the office.

[tsightler]

In my simplified explanation I used the concept of "tell me what blocks changed since 1AM" for simplicity, I wasn't trying to be 100% technically accurate. As Anton said, it's actually based on USN, but I thought explaining it as a point in time would be easier. Certainly the tracking information is not based on actual wall clock time, what a disaster that would be if it were designed that poorly. Fortunately, block change tracking is not some type of new-fangled technology, it's just new to VMware. It's been around in other products for quite some time. Oracle has included block change tracking in their RMAN for quite a while and it works pretty much the same way.

I believe that a block is a block based on whatever block size your VMFS is formatted in, but I'm not sure how that works with RDM. So, if you have a VMDK file, and you grow it, well, that's not really any different that what old Veeam would do without block change tracking, it would recognize the the virtual disk is larger, and ask if those blocks were "changed".