Veeam Backup Statistics

[HJAdams123]

Hello and Good Day Veeam Community...

I am right in the middle of a Veeam B & R evaluation. I noticed that if you go to Backups - Disk and select a Backup Job, right click and select Properties, you can get detailed information about your backup footprint over time. I had a few questions though on what some of the figures mean?

Say for instance the de-dupe ratio. Does that percentage mean the percentage of data that was found to be deduplicated? Or does the dedup ratio mean the reduction of data as a result of deduplication? And then where does the compression ratio come in? Does that percentage only work against data that has already been de-duped? I guess what I am asking is, does de-dupe happen first, and then whats left over from that is compressed? I am just trying to make sense of the numbers....

For example, I am looking at the properties of one of my backup jobs in the Veeam B&R console under Backups - Disks (right click one and select properties) This job is a reverse incremental, and I don't do active fulls or anything like that. On a particular day, I got this figures....

Data Size: 5.3GB
Backup Size: 833.4MB
De-Dupe Ratio: 84%
Compress Ratio: 18%

So, if I understand correctly, de-duplication reduced that backup size by 84%, which is 5.3GB - 4.5= 800MB. So we are left with 800MB after de-dupe. Then compression comes in and reduces that by another 18%. So, 800MB -144MB now equals 656MB. Now 656MB does not equal 833MB. I thought that 833.4MB figure was the result of de-dupe and compression. Or perhaps I am not working with the numbers correctly? Can someone claify how these numbers relate?

[foggy]

Harold, the correct math is as follows: 84% dedupe ratio means that the data was deduped to the 84% size of the original data, the same with compression ratio. So having initially 5.3GB deduped to 84% and then compressed to 18% (5.3 * 0,84 * 0,18) you get somewhere close to the reported backup size.

[HJAdams123]

So this is a case in which lower percentage numbers are better....better in that compared to the original datasize how much data is actually written to the repository...

[foggy]

Correct.

[ccitrano]

All,

Can I get some quick input on interpreting this output from a job that I ran. I was recreating some jobs and switching them to a forward incremental model. This job reflects the first run, which would be a full backup. I agree with the Total size and the Backup Size number which make me pretty happy. I'm trying to understand the Dedupe and Compression numbers so that I can experiment. What does 1.1x mean in terms of Deduping and 4.0x in terms of Compression? I am using the default compression and dedupe options in the job.


Tuesday, February 19, 2013 8:00:13 PM
Success 8 Start time 8:00:13 PM Total size 1.1 TB Backup size 172.0 GB
Warning 0 End time 11:01:36 PM Data read 769.5 GB Dedupe 1.1x
Error 0 Duration 3:01:23 Transferred 195.4 GB Compression 4.0x
Details
Name Status Start time End time Size Read Transferred Duration Details
CLT-SQ826 Success 8:01:37 PM 8:14:39 PM 160.0 GB 43.3 GB 16.7 GB 0:13:01
CLT-SQ828 Success 8:14:45 PM 8:44:53 PM 160.0 GB 134.4 GB 32.1 GB 0:30:07
CLT-SQ822 Success 8:44:56 PM 9:20:57 PM 180.0 GB 164.0 GB 32.3 GB 0:36:01
CLT-SQ827 Success 9:20:59 PM 9:53:34 PM 160.0 GB 139.3 GB 39.1 GB 0:32:34
CLT-SQ825 Success 9:53:36 PM 10:05:26 PM 80.0 GB 40.7 GB 13.2 GB 0:11:49
CLT-SQ821 Success 10:05:28 PM 10:24:25 PM 100.0 GB 84.0 GB 19.6 GB 0:18:57
CLT-SQ823 Success 10:24:28 PM 10:53:37 PM 160.0 GB 144.0 GB 34.4 GB 0:29:09
CLT-SQ824 Success 10:53:39 PM 11:01:30 PM 160.0 GB 19.8 GB 7.9 GB 0:07:50

[veremin]

1.1x means that the data was deduped to (1/1.1)*100%= 91% size of the original data. The same situation with compression: (1/4)*100% = 25%

Having aforesaid points in mind we get these numbers:

769.5*0.91*0.25 = 175,061

175,061 is approximately equal to the reported backup size.

Hope this helps.
Thanks.

[StefanSpecht]

Hi,

I try to understand the backup statistics, and try to understand where exactly compression and deduplication happens.
We using several virtual proxy servers, one physical backup server and a NetApp CIFS repository, and my current understanding for a full backup is as follows:

(1) The virtual proxy reads all blocks from disk, except zero-data blocks, swap files etc.
(2) Proxy compresses all data that has been read
(3) These compressed data is transfered to the target-side transport service. Most of the time this the physical backup server, as it is near to the NetApp
(4) Target-side transport service deduplicates the received data
(5) Target-side transport service deduplicates writes deduplicated data to disk

Is that correct so far?

This is what I get from the Backup Report:

Total Size -> 764 GB
Data Read -> 718,1 GB
Data Transferred -> 451,6 GB
Backup Size -> 420.6 GB
Deduplication -> 1.7 x (1/1,7) *100 % = 58,82% from originzal size ( -> - 41,18)
Compression -> 1.1 x (1/1,1)*100% = 91% from original size (-> - 9%)

My calculation would be as follows; but I get completly diffrent values:


764 GB (Toal Size) - Zero-blocks,swap-files = 718,1 GB (Data Read)
718,1 GB (Data Read) - 9% Compression = 653 GB (Data Transferred)
653 GB (Data Transferred) - 41,18% Deduplication = 384 GB (Backup Size)

What I am also confused about is, that from the real time statistics for this job, I can see:
Data Read -> 718,1 GB
Data Transferred -> 451,6 GB (1,6 x)

I see that 451,6 GB x 1,6 = 718,1 GB. But how does this relate to the 1,1x and 1,7x from the report?

Stefan

[foggy]

Stefan, your understanding is basically correct. The confusion comes from the fact that compression and deduplication values from the Backup Report relate to the backup file size (in relation to the total data size), not the data read and transferred during the job session. So maths should be as follows:

764 GB / 1.7 / 1.1 = 410GB, which is somewhere close to the reported 420GB size of the resulting backup file.

[rpraseet]

Harold, the correct math is as follows: 84% dedupe ratio means that the data was deduped to the 84% size of the original data, the same with compression ratio. So having initially 5.3GB deduped to 84% and then compressed to 18% (5.3 * 0,84 * 0,18) you get somewhere close to the reported backup size.

Thanks for this. It was helpful for me as well. I am also trying to find data volume that changes daily by looking at the incremental backup size.

Success 110 | Start time 16:30:02 | Total size 6.0 TB | Backup size 806.8 GB
Warning 0 | End time 17:56:42 | Data read 106.8 GB | Dedupe 4.5x
Error 0 Duration 1:26:40 | Transferred 58.9 GB | Compression 1.7x

The latest vbk file has a size of 806 GB and the vrb files are all around 50 to 60 GB. How could i calculate an average daily data change with this informa

[foggy]

Data read 106.8 GB

Hi Ravi, this is the daily data change for this job run.

[techuni]

Hi,

I need some explanation what Backup size / Data read / Transferred.

Is backup size restore point size of my vm or something else?
Is the transferred size the compression size of the total data read that goes to my backup disk?

How do i use deupe and compression for calculation.

Total size4.6 TBBackup size16.1 GB
Data read96.5 GBDedupe1.0x
Transferred15.5 GBCompression2.3x

Thanks,
techuni

[Mike Resseler]

Hi Techuni,

First: Welcome to the forums!

The Total Size is the size of your VM. Looks like you have a 4.6 TB VM
The Backup Size is the size of that specific restore point. Since I see it is 16.1 GB I am going to guess that this is an incremental restore point (so you have full already). This is the actual file size stored on the backup repository.
The transferred size is what is being sent from the source-side data mover (probably) over the network to the target data mover. This is AFTER applying compression and deduplication. As you can see this is not the final size of the backup file. B&R will do some additional activities with the transferred data.
The Data Read is the actual data that is being read from the VM prior to dedupe and compression. Since we only read data that has been changed since the last job session.

Makes sense?
Mike

[techuni]

Hi Mike,

Thank you for the reply. I am understanding the logs better now. Correct me if i am wrong.

1. Backup size in my logs shows as being incremental restore points. If i were to keep 40 restore points daily avg. 16.1*40=644gb would I be storing 644gb to my backup repository?

2. My Total size vm is 4.6TB. That would be my total size of the vhdx static disk partitions that does not mean that i am actually using 4.7tb data. If i were to do a full backup what is a typical compress/duplication estimate i would be storing to my backup repository?

3. What is the best practice for a backup would I do a full backup monthly and incremental after?

i am trying to estimate the storage size i need for growth.

Thanks,
techuni

[Mike Resseler]

Hey techuni,

If you try to estimate the storage size, I would advise you to go through this: http://rps.dewin.me/

While obviously it will be an estimate, this tool is created by one of our engineers as a side project and is used by many customers to make estimations

Hope it helps
Mike

PS: Ideally you do a full backup, forever incremental with synthetic fulls in between depending on your need