Host-based backup of VMware vSphere VMs.
Post Reply
pshute
Veteran
Posts: 254
Liked: 14 times
Joined: Nov 23, 2015 10:56 pm
Full Name: Peter Shute
Contact:

Incrementals of some VMs take as long as full backups

Post by pshute »

Can anyone please tell me why some of my servers take just as long to do an Incremental backup as an Active Full, despite the TransferredSize being much smaller.

Some example durations for one of the servers concerned (you can tell which are incrementals by the smaller Transferred Size):
DurationReadSize in GBTransferredSize in GB
4:29:01853.8485.8
4:54:11853.8490.8
4:57:16833.321.9
4:55:18813.422.3
5:04:04817.828.3
4:28:31761.931.4
4:46:14836.333.4
4:54:49836.935.7
4:23:10763.643.3
4:41:17817.947.4
4:41:31813.8484.1

For that particular server, the history entries show that one particular operation takes most of the time:
Full 11/01/2017 9:52:27 AM :: Hard disk 7 (600.0 GB) 600.0 GB read at 45 MB/s [CBT] - duration 3:51:06
Incr 13/01/2017 12:22:29 AM :: Hard disk 7 (600.0 GB) 600.0 GB read at 45 MB/s [CBT] - duration 3:43:56

Does that mean the problem is to do with how long it's taking to read that disk? I have other servers nearly as big that are much quicker doing incrementals than fulls.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by Mike Resseler »

Peter,

Can you tell us what the bottleneck is? It seems indeed that it is a bit slow. How is the disk read of your other servers?
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by Shestakov »

Do other VMs use same datastore and proxy servers?
Thanks!
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by foggy »

The fact that it reads the entire disk during incremental run tells that CBT isn't working...
pshute
Veteran
Posts: 254
Liked: 14 times
Joined: Nov 23, 2015 10:56 pm
Full Name: Peter Shute
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by pshute »

Yes, thank you, I see this line in the history for that machine:
"16/01/2017 11:35:06 PM :: CBT data is invalid, failing over to legacy incremental backup. No action is required, next job run should start using CBT again. If CBT data remains invalid, follow KB1113 to perform CBT reset. Usual cause is power loss. "

I'll try resetting CBT.

I would never have spotted that myself, as it's a bit buried in the job report, and it would take a bit of work to check if other machines are affected. Is there an easier way of finding them?

And shouldn't CBT failure be mentioned in the backup job report we get emailed?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by foggy » 2 people like this post

You can monitor job duration using Backup Infrastructure Custom Data Veeam ONE report to be aware of such issues.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by Vitaliy S. »

Or if you want to be notified about this behavior in real-time, you can use backup job duration alarm in Veeam ONE, that will notify you when your job has exceeded its usual duration period.
pshute
Veteran
Posts: 254
Liked: 14 times
Joined: Nov 23, 2015 10:56 pm
Full Name: Peter Shute
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by pshute »

foggy wrote:The fact that it reads the entire disk during incremental run tells that CBT isn't working...
Now that I've enabled deleted block exclusion, it's not reading the whole disk. How can I now tell if CBT is working for this disk?

I'd like a way of auditing CBT for all the disks on all our VMs. Monitoring job duration won't help if they've always been failing, as I have no benchmark, nor will it tell me which disk has the problem. Looking for the warning in the job history will only tell me that at least one disk has a CBT problem in that job. It won't tell me how many or which ones.

Would the information be available to a Powershell script, or can I only extract the same information that's in the job histories?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Incrementals of some VMs take as long as full backups

Post by foggy »

pshute wrote:Now that I've enabled deleted block exclusion, it's not reading the whole disk. How can I now tell if CBT is working for this disk?
With CBT in action, Veeam B&R will read just the changes occurred inside the VM, which are typically 3-5% of the total size for regular servers.
Post Reply

Who is online

Users browsing this forum: Semrush [Bot] and 96 guests