Determine size of backed up data

[Tijz]

Hi all,

We need(ed) to know the size of all data being backed up by Veeam, without compression and/or deduplication.

So I opened job statistics of one of our jobs and looked at the "Data: Processed" figure. This showed 4.4TB.
Data read showed 4.5TB
Data Transferred: showed 2.9TB

So I assumed that our total backup data before dedupe en compression was 4.4TB.

Then we disabled compression (as we were implementing a dedupe appliance). Now the "Data Processed" figure shows 6.3TB (!!).
Read: 6.3TB
Transferred: 4.4TB
When I open statistics of an incremental backup of the same job, the data processed figure shows 5.4TB.


So I guess I have two questions:
1. What does the figure "data processed" actually signify?
2. How can I determine the size of a full backup?

regards,
Tijs

[Vitaliy S.]

Hi Mattijs,

1. When you start the backup job our engine reads VM virtual disk blocks, skips zeroed blocks, and this results in processed data number.
2. Since you have disabled compression, then you should be able to calculate full backup size pretty easily via job reports (see our User Guide, page 295, for further details).

Thanks!

[Tijz]

Hi Vitaliy,

thanks for your reply, but I still don't understand.
When I go to the job settings, to the Virtual Machines and hit "recalculate", it shows me a total size of 5.6TB. This is the sum of all VMDK's including empty space.

So my total backup size should be smaller right!? (sum of all vmdk's - empty space).

But in the reports it shows a "total size" of 6.3TB.

How is this possible? What is what?

[Vitaliy S.]

The total size of the VM on "Virtual Machines" tab of the backup job wizard does not include empty space, it just shows you the actual space consumed by VM virtual disks. However, the "total size" in the HTML report shows the provisioned disk space of source VMs, that is why you have this difference.

[Tijz]

The total size of the VM on "Virtual Machines" tab of the backup job wizard does not include empty space, it just shows you the actual space consumed by VM virtual disks. However, the "total size" in the HTML report shows the provisioned disk space of source VMs, that is why you have this difference.

Sorry, but this is not true for all VM's. When I add one VM with a disk of 1TB, it shows me the total size is 1.0TB. But I know for a fact that only 700GB is used within the VMDK.
On the other hand, when I add another VM with a total size of


Sorry, but my question is a fairly simple one:

How can I determine the amount of data that is being backed up, before deduplication and compression, but ofcourse without the empty space.
Where can I find this figure?

[Vitaliy S.]

Ok, now I'm a bit confused with what you want to achieve. Total size of the virtual machines tab shows you the size of source disks (uncompressed and not deduplicated), it doesn't show the size of the backup files. If you want to see how much space was consumed by the backup job on your target repository for the selected job run, then you need to take a look at "Backup Size" in the HTML report table.

[Vitaliy S.]

Sorry, but this is not true. When I add one VM with a disk of 1TB, it shows me the total size is 1.0TB. But I know for a fact that only 700GB is used within the VMDK.

Is this a thin or thick provisioned disk? What does vSphere Client show for this disk?

[Tijz]

Thick and thin

The VM;s that are shown correctly are thin, the other ones are thick.

But still, when using thick provisioned vmdk's, the zero bytes will be left out right?
So I need to find out how much data was backed up.

The size of the backup file will be smaller than that due to compression, but how much data was being backed up?


EDIT: I would assume that the "data processed" would indicate this figure, but this is different for full and incremental backups. (full: 6.3TB, incr: 5.4TB). But backupfile size for incrementals are 300GB. Full are 4.4TB
The data "read" for incrementals is 300GB. so this is correct. But the data "read" for full's is 6.3TB. But when I look at the virtual machine tab is shows me 5.6TB... And this is including empty space of the thick disks

[Vitaliy S.]

But still, when using thick provisioned vmdk's, the zero bytes will be left out right?

Correct.

So I need to find out how much data was backed up.

To find out how much data was backed up (transferred to the target storage), you need to use "Transferred" counter.

2. How can I determine the size of a full backup?

For this you need to use "Backup Size" in the HTML report table.

[Tijz]

I think i've got it. But the GUI is quite unclear, but I think it's something like this:


1. When performing a Full Backup without compression, the figure "Data Processed" is the sum of all VMDK's including empty space.
2. When performing an Incremental Backup without compression , the figure "Data Processed" is a also the sum of all VMDK's, but only the empty space of thick vmdk's is included. So if you have thick and thin vmdk's as we have, data processed seems less when performing an incremental backup. This figure corresponds to the figure shown when editing the job, on the virtual machine tab.
3. When performing backups with compression I don't dare guess what those number mean.. But it seems, when doing a Full at least, that it's close to the amount of the actual data being backed up.

But to find out what the actual data is, the best indicator seems to be "Data Transferred" while not using compression.

The "Backup Size" in the HTML report as Vitaliy suggests, just gives you a sum of all vmdk's including empty space... not what I was asking. I don't care about empty space. I can sum the VMDK's myself...

[foggy]

The "Backup Size" in the HTML report as Vitaliy suggests, just gives you a sum of all vmdk's including empty space... not what I was asking. I don't care about empty space. I can sum the VMDK's myself...

The "Backup Size" indicates the size of the restore point file created during this particular job run. Should equal to the "Transferred" in case where both dedupe and compression are disabled (+ metadata overhead).

[Tijz]

Ok, that might explain the difference, since I did use "optimize for local target" and left "inline deduplication" checked.

But would I uncheck inline dedupe, would Veeam backup a complete thick VMDK including it's empty space? That would be good to know. Makes sense in a way though.

But that would suggest though that there is no good way of determining the total amount of backed up data.
Because using inline dedupe at it's weakest level (biggest block size) still might dedupe more than just empty space.

I think the answer to my question should have been: you can't, exactly.
Whick makes sense now that I have thought about it.. actually quite obvious..

[foggy]

But would I uncheck inline dedupe, would Veeam backup a complete thick VMDK including it's empty space? That would be good to know.

No, zero blocks are removed by the source-side deduplication (though we still need to read them from the VMDK).

[Tijz]

I also opened a support case for this, as I had a 3.7TB of backup data (within the vmdk's), but my backupfile was 4.4TB. The extra space is roughly the same as the free space in two thick 1TB vmdk's.

This might be because the vmdk's have "deleted files" in them, which are not actually zero blocks within the vmdk. This weekend I ran sdelete -z to claim al empty space and set the bytes to zero.

Will let you know what happens after the full next week.