Comprehensive data protection for all workloads
Post Reply
montague.thomas
Novice
Posts: 3
Liked: never
Joined: Jan 19, 2012 2:52 pm
Contact:

Feature Request: CBT Estimation

Post by montague.thomas »

During my last set of upgrades on some VMs, I realized a great deal of performance slowdowns with Veeam backups come from change block tracking and the number of changes to a VMDK since the last backup. For example, I upgraded our vCenter from 5.1 to 5.5. During the upgrade the database, as expected, had numerous changes which causes many changed blocks. In this case, the VM had two VMDKs, 80 GBs and 200 GBs. Of these, the 80 GB had 17.3 GBs of changed blocks and the 200 GB had a whopping 177.1 GBs of changed blocks.

So my suggestion is to add an option, per job, to estimate total changed blocks and data that would have to be transferred and if it reaches a percentage threshold setting (e.g. 50% or even 30%), it will change from CBT to a Full backup for the VM / VMDK getting backed up. Does this make sense to implement?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Feature Request: CBT Estimation

Post by Vitaliy S. »

Hello,

Can you please clarify why running active full would be a preferred option in your scenario rather than proceeding with CBT-enabled job run? The time required to do full backup will be longer than backup with CBT or there is another use case I'm not aware of?

Thank you!
HJAdams123
Enthusiast
Posts: 72
Liked: 16 times
Joined: Jul 16, 2012 1:54 pm
Full Name: Harold Adams
Contact:

Re: Feature Request: CBT Estimation

Post by HJAdams123 »

I think I know what the original poster is referring to. I know in my case if you are doing reversed incrementals, and you get a day where the virtual machine has experienced allot of change in the past 24 hours, it can be very disk intensive. (and CPU intensive) to create that next vrb file. With so much change, it is often times faster to just perform a new full backup. (assuming space on the repository allows it). So I am guessing he wants a way to forecast the amount of change that has happened before that next job kicks off, so he can make an informed decision to just perform a new full backup...

I don't know if such a tool exists that tells you this information or not....

-Harold
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Feature Request: CBT Estimation

Post by Vitaliy S. »

Hi Harold,

Well...if you're using forward incremental backup mode then this feature does not really make sense. Yes, you seems to be right about OPs proposal, however sometimes lack of space is more of an issue than longer incremental job run. Do you agree?

Anyway, as far as I remember in order to forecast changes you have to create snapshot and then start transferring data, actual data. CBT does not return the size of changed blocks, what it does return are block IDs that have changed. Not sure it is possible to estimate the size (though I might be wrong with this, need to double-check).

Thanks!
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Feature Request: CBT Estimation

Post by veremin »

On a side note - the OP mentioned that the huge amount of changes was somewhat expected. So, next time once the operation affecting many blocks is finished (like maintenance, database upgrade, etc.) it might be worth running full backup manually, instead of waiting for automatic incremental run. This way, the backup time will be decreased.

Thanks.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Feature Request: CBT Estimation

Post by Gostev »

montague.thomas wrote:So my suggestion is to add an option, per job, to estimate total changed blocks and data that would have to be transferred and if it reaches a percentage threshold setting (e.g. 50% or even 30%), it will change from CBT to a Full backup for the VM / VMDK getting backed up. Does this make sense to implement?
The idea makes sense generally, but may result in multiple unexpected full backups created, and backup repository disk space usage getting out of hand.

The primary reason why people choose to use reversed incremental backup mode is exactly the disk space savings through having a single full backup file, so adding a feature that kills the primary benefit is questionable. I mean, if you are mentally ready to have multiple full backups on disk, why not just switch to forward incremental mode with periodic fulls. Not only this mode will resolve your backup performance issues when there is a large amount of changed blocks, but unlike your suggested approach, it will also give you predictable disk space usage by full backups.
montague.thomas
Novice
Posts: 3
Liked: never
Joined: Jan 19, 2012 2:52 pm
Contact:

Re: Feature Request: CBT Estimation

Post by montague.thomas »

Just to clarify, yes, I'm referring to Reverse incremental jobs taking longer to run incremental then full backups.

Thank you all for your feedback and suggestions. It was my understanding that using Incrementals with synthetic fulls would cause much high disk i/o then reverse incrementals. I'll be looking into using Incrementals and see if that provides any improved performance. Thanks!
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Feature Request: CBT Estimation

Post by Vitaliy S. »

The difference in IO load for these modes is described in this thread, please take a look > Read and Write
montague.thomas
Novice
Posts: 3
Liked: never
Joined: Jan 19, 2012 2:52 pm
Contact:

Re: Feature Request: CBT Estimation

Post by montague.thomas »

Vitaliy S. wrote:Anyway, as far as I remember in order to forecast changes you have to create snapshot and then start transferring data, actual data. CBT does not return the size of changed blocks, what it does return are block IDs that have changed. Not sure it is possible to estimate the size (though I might be wrong with this, need to double-check).
Vitaliy, I understand that CBT only returns the blocks changed. You should be able to tell how many blocked changed out of total possible blocks, thus giving you the ability to calculate a percentage of blocks changed. If that's not the case, oh well. But either way, as discussed, there are normally other considerations with actual backup disk storage that would contradict a setting like this. Granted, when using Reverse incremental, when you have so much change on a disk, you'll be backing that up anyway with a standard incremental run. So it's really you saying "hey, I'm ok with wasting 40% of disk storage to save on disk i/o and backup time.". Also, with Veeam inline deduplication, theoretically, if those unchanged blocks get backed up again, shouldn't they be deduplicated down to only be stored once in the back files?

Thanks!
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Feature Request: CBT Estimation

Post by Vitaliy S. »

montague.thomas wrote:Also, with Veeam inline deduplication, theoretically, if those unchanged blocks get backed up again, shouldn't they be deduplicated down to only be stored once in the back files?
Yes, these blocks will be deduplicated within the VBK file for reversed incremental backup mode. Thank you!
Post Reply

Who is online

Users browsing this forum: Google [Bot], khang1231, Semrush [Bot] and 178 guests