How can I determine size of CBT deltas between backups?

[cparker4486]

I'm asking this question because I'm not sure if the obvious answer is the best answer. The obvious answer (in my mind) is, "Look at the size of each .vrb file. That is how big the deltas are."

The reason I wonder if this is the best (or only) answer is because my backups are currently done daily and as a result the .vrb files are very large. If I want to see how big my .vrb files would be, if I did a backup every 30 minutes, I have to change my job and hope I do it right so as to not push my existing .vrb files off the chain prematurely.

Let me know if I am not being clear and I will clarify.

[Vitaliy S.]

Hi Chris,

Yes, the quickest way to determine the size of all deltas is to look through the VRB/VIB files size on your target repository. To estimate the possible size of the "30 minutes" increments, you can take any existing VRB file and divide its size by 48. This will give you a ballpark number of the VRB/VIB files after you change the backup job schedule.

Thanks!

[Gostev]

This data is provided in the email report. You can also get the same report from the job toolbar.
Per-disk, you can see this data in the job's real-time statistics view.

Please note that VRB size cannot be used for this, as VRB contains replaced data - not changed data.
VIB size can be used for approximation, on the other hand (VIB contains changed data).

[cparker4486]

Hi Gostev,

Which value in the report is the one I should be looking at? Is it Transferred or Read?

Follow up question on this is what's the difference between Transferred and Read?

[Gostev]

You should be looking at Read counter.

The counters are just what they are called:
- Read is amount of data read from disk (all blocks that were reported as changed)
- Transferred is amount of data transferred to target (after source-side dedupe and compression).

[cparker4486]

In that case it sounds like Transferred is what I should really be paying attention to because that is the amount that would be sent to another site for replication. Is that correct?

[Gostev]

If you are interested in WAN bandwidth consumption part, use transferred. I understood you want to determine how much data gets changed in your source disks, for that you need read.

[StefanSpecht]

Hi,

could someone explain why the "transferred" data is not the same as the size on disk?

e.g.:
vbk-file -> 570GB
transferred: 628,4 GB

vib-file -> 42,8 GB
transferred -> 42,7

So in one case the real file size is bigger...in the other case the real file size is smaller then the transferred data...

[veremin]

e.g.:
vbk-file -> 570GB
transferred: 628,4 GB

This is expected.

vib-file -> 42,8 GB
transferred -> 42,7

As to second example, I’m wondering whether you see these stats always during incremental run. If it’s just a temporary problem, it might have to do with residual information related to previous backup run, etc.

Thanks.

[StefanSpecht]

Why is the first example expected?
When happens deduplication/compression? Before or after "Transferring"? Or both?

I just checked the incremental backup from last night: About 2/3 of the jobs has a bigger real file size then the "transferred" data. At about 1/3 of the jobs it is smaller.
For the Full backups the real size on disk seems to be always smaller then the transferred.

[veremin]

When happens deduplication/compression? Before or after "Transferring"? Or both?

Both:

* on the source side: if a block is the same within the same VM, only the hash will be send and not the block data
* on the target side: if a block is the same from another VM in the same backup job, both the hash and the block are send but only the hash is stored

Why is the first example expected?

It’s expected, since the data is transferred between two VB&R agents (source - proxy , target - repository) and it is then deduped by target one. Thus, there is a difference between amount of transferred data and backup size.

About 2/3 of the jobs has a bigger real file size then the "transferred" data.

Is there a fixed difference between sizes or it varies? Additionally, if the amount of transferred data and actual backup size are approximately the same or backup file is a little bit bigger, it might be related to certain amount of metadata data stored in backup file, etc.

Thanks.

[cparker4486]

...VRB contains replaced data - not changed data.
VIB size can be used for approximation, on the other hand (VIB contains changed data).

I've recently got some successful runs of my backup copy job (finally). I noticed that two of the VIB files are several times larger than their VRB counter parts and was wondering if you could clarify the above statement. Are you saying that if a block changed 8 times between backups that only the final change is stored in a VRB whereas all 8 changes are stored in a VIB?

I've included an image to illustrate the issue.

[Gostev]

No, in both cases only the last state of the block is stored in the backup file.

[cparker4486]

In that case, what would cause the VIBs from the backup copy job to be so large?

[Gostev]

Many changes all across the virtual disk blocks (either because of disk fragmentation, or application-specific workload). Note that actual changes do not have to be huge, can be just a few bytes per 1 MB block for example. Any change within the virtual disk block will require that the block is picked up by the incremental backup.

And again, keep in mind that you cannot compare VRB and VIB, because they contain different data, as I explained above. VRB size is useless when looking for amount of changes for the given run. On the other hand, VIB does contains the actual incremental changes between current and previous state, so that is what you should be referring to.

[cparker4486]

I don't have experience with VIB files as I've always performed reversed incremental backups so this may be where I'm getting thrown off... Is the VIB a the delta of the VBK and not the previous VIB? If so, that would make a lot more sense.

[Gostev]

The VIB is a delta from the previous VIB (in other words, classic incremental backup).

[cparker4486]

Then I'm lost. So far your explanations, as appreciated as they are, have not made the distinction for me. Especially your usage of the words changed and replaced. All replacements are changes, no? I don't see how a copy job can "find" more changes than what exist in the backup files if the backup files contain only replaced data.

[tsightler]

All replacements are changes, no?

All replacements are changes, but there might also be new data that didn't actually replace anything, but simply caused the VBK file to grow. This "new" data wouldn't be represented in a VRB since it's not roll back data, but it would show up in a VIB file. Example:

VBK: 500GB

A new backup that has 50GB of changed blocks but 25GB of new data might look like this:

VBK: 525GB
VRB: 50GB

But the backup copy would store all of the changed data in the VIB: so you'd have:

VBK: 500GB
VIB: 75GB

Note that the final results is the same amount of space, but where the blocks are stored is different since, with reverse incremental, the replaced blocks ended up in the VRB file while the new data was in the VBK.

It seems obvious from your screenshot that there's new data being added as your VBK file was only 476GB on 11/8 while the one on 12/3 is 617GB.