Full disk read when a new VMware disk is attached to VM

[TRE3]

Hi Veeam Forum,

one of our customers added a new disk to his file server VM (vSphere) and although CBT is enabled (Veeam shows [CBT] in the job), all the OTHER disks get FULLY read. No other changes have been made (no disk extension). Is this a default behavior? Can we fix this?

Cheers, Thomas

(Case ID: 06129466)

[HiHoItsOffToWorkWeGo]

Although the entire source disk gets read, you will find that only the usual tiny portion of changed data gets sent over the network to the repository thanks to CBT still working and doing its thing.
You can confirm this yourself by looking at the backup. You will find that the incremental is the same size as that of the previous day, becuase, just like the previous day, the CBT technology only backs up the portions of the disk that have actually changed.
On the next backup the situation will go back to normal, with only a tiny bit read and sent.

[TRE3]

Yes, that is what we see. The problem here is: The backup has a speed of 30 MB/s (infrastructure limitation) and it has to read 20 TB of data. So the file server needs several days before the next backup can start. In the near future, there are more disks to be added to this server. So he needs a workaround for the future that really uses CBT and where the backup does not fully read the untouched disks.

[HiHoItsOffToWorkWeGo]

Sounds like you might be doing backups using the Veeam agent installed inside the VM.
Have you considered performing an image backup of the VM instead?
You might see much faster read speeds, e.g. over 200MB/s.
At this speed the entire 20TB can be read in just 1 day.
And image level backups are much less susceptible to the problem you mentioned. Changing one disk means only the changed disk needs to be read. The other disks continue having only their changed data read.

[TRE3]

The customer does VMware based backups (no agent / NBD). SAN-based is not possible.

[HiHoItsOffToWorkWeGo]

NBD is what's slowing this down.
Can you use a Veeam proxy with virtual appliance mode (with NBD disabled) to backup that file server instead?

[TRE3]

Thank you for your suggestions but what the customer wants to clarify is the following:

Is it an expected behaviour that if you add a virtual disk to a VM with CBT enabled, that EVERY UNCHANGED disk gets also fully read instead of using CBT data?

[chris.childerhose]

I believe that is the expected behaviour of Veeam, as you are changing the disk configuration for the VM even though you are just adding a new disk. So Veeam does a full read, and CBT still works but kicks in for the subsequent runs after this one.

[TRE3]

Thanks, I'll submit that info to the customer!

[PetrM]

Hello,

@TRE3 please wait if it's possible

I asked my colleagues from QA to test this scenario in our lab and we clearly see that there is no read of disk (btw we tested NBD only) which had no changes even after a new disk was added to the VM. It looks like that there are some specific conditions, I'll ask our support team to research your case a bit deeper.

May I ask you to clarify which transport mode was used in your test?

Thanks!

[TRE3]

NBD was used

[PetrM]

Let's wait for our support team. I'm not ready to confirm that the behavior described in your first post is by design.

Thanks!

[Rumple]

my thought is..how in the world are they running a vmware server with a virtual 20+TB file server and absolutely no way to install run a helper VM on the same server that could mount the file server disks so it wouldn't use NBD?
All they need is basic vmware licensing (which they need anyway to even do backups) and just a barebones server VM with the backup proxy installed....

[TRE3]

The customer just bought a new NIC for the physical backup server that will be used for iSCSI traffic (SAN based backup) in a few days. Nevertheless, when adding new virtual disks to the file server, he cannot be sure that not all the disks get read fully again.

[HiHoItsOffToWorkWeGo]

My experience with a 40TB VM with 20x VMDK disks is that if one disk is changed, such as expanded in size, then only that one changed disk gets read in full during the next incremental Veeam backup.

[TRE3]

That's what I also thought would be default behaviour but let's wait what the support team says.