Understanding replication job progress information

[gparker]

Hi, I'm reletively new to Veeam Backup & Replication. I've got a Prod site and a DR site, each running VMware vSphere. I am setting up Veeam to replicate some VM's from Prod to DR over our WAN link. The WAN bandwith is only 10Mbps, so I'm doing the seeding method and so I've installed Veeam at the prod site, created a full backup of a VM, brought the backup to the DR site, added it as a Backup Repository on the DR Veeam server and restored the VM onto my DR vSphere environment OK.
I then created a replication job with the replica seeding option checked and I checked the Replica Mapping option and mapped the source VM to the restored replica VM. I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% odf the size of the VM, right? The next thing it says is that it's read 61.0GB. What does this mean? The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all). What does the 113.7x in the brackets represent?

Please help me to understand these figures.

George.

[Gostev]

I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% of the size of the VM, right?

Correct. This is overall VM processing pointer.

The next thing it says is that it's read 61.0GB. What does this mean?

The amount of data that has been read from source VM disk. Less than 68.8GB because some blocks are not being read - as because we do not care about their content, or because the content is already known (zeroed block).

The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far

Yes. There are 549.5MB of compressed blocks which are different between your seed, and the current state of source VM disks.

and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all).

Well, your link is only 1 MB/s, so what do you expect when there is 549.4 MB of data to move across. Of course the link will be saturated.

What does the 113.7x in the brackets represent?

Read divided by transferred. If you are doing seeding, ignore this counter - it only makes sense for "normal" runs.

[homerjnick]

Does he have a 10 or 1 Mbs link?

[Gostev]

The WAN bandwith is only 10Mbps

[homerjnick]

Well, your link is only 1 MB/s, so what do you expect when there is 549.4 MB of data to move across. Of course the link will be saturated.

Then this post is not quite right?

[Gostev]

10 Mpbs = 1.25 MB/s

[homerjnick]

Oopps I misread...sorry!

[gparker]

Thanks everyone for your posts, answers and comments

I can confirm that the WAN link is 10Mbps and the Veeam progress box is reporting transfer rate of 1MB/sec. I don't mind that it says 1MB/sec as opposed to 1.25MB/sec, but what does concern me is that it's constantly saying 1MB/sec while the job is active. The job has been active for something like 13 hours and has tranferred only 549.4MB in all that time while the transfer rate hasn't moved from 1MB/sec. So what it going on on the WAN link?

George.

[tsightler]

Do you have proxies on both sides of the link? Have you manually selected them in the job? It sounds like you may have a proxy on the target side that is pulling the replica via network mode across the link, instead of proxy-to-proxy. This is the most common configuration error that I see with V6 replication, either having a proxy only on one side, or having a proxy on both sides, in network mode, but not specifically selecting the proxies on each end.

[gparker]

Hi tsightler,

You are right, I only have a proxie on the target side. In the job progress information box, it actually says "using source proxy VMware Backup Proxy [nbd]" and "using target proxy VMware Backup Proxy [nbd]". I presume the "[nbd]" means it is pulling the replica via network mode across the link, right?

How does this relate to there being a constant 1MB/sec processing rate when it has only transferred 549.4MB?

George.

[tsightler]

So the "VMware Backup Proxy" is the Veeam server itself. Is this located on the source or target side?

The number for "transferred" indicates the amount of data that would have been sent between the two proxies, but since you are using the same proxy for both source and target, this number effectively has no meaning at all. Not having a proxy on both ends will lead to very poor replication performance. I'm suspicious that your VMware server is on the target side, and thus the proxy is having to read the entire data across the 10MB link. That would mean it would pull the entire 61GB "read" over the wire which would be about right for a 10Mb link.

Please deploy both a source and target proxy, and then select the appropriate proxies in the job configuration and test again. I think you will see a tremendous difference as then the actual data transferred across the WAN link would indeed be only 549.5MB, and this data would be streamed efficiently between the two proxies, not sent/received via network mode (nbd) across the WAN, which is very inefficient.

[gparker]

Hi,

Yes you are right, the "VMware Backup Proxy" is the Veeam server itself. It is located on the target side.

Thanks for explaining how it works and what "transferred" number actually means in my situation. Thanks also for explaining that the proxy has to read the entire data across the 10Mb link and that it's effectivelly pulling the entire 61GB.

I only have a reletively small deployment scenario here. I am only wanting to replicate 3 VM's from source vSphere environment to target vSphere environment. I will be configuring the replication job(s) to schedule the replications every 4 hours. So I'm happy to deploy a proxy server at the source side, but do I really need to deploy a seperate proxy server at the target side? The VM that I've installed Veeam onto at the target side isn't being heavilly loaded. It also does not have direct access to the target VMFS Datastore where the replica's are going to becuase it's on an ESX host that cannot see the target Datastore. Will I see an improvement in the replications if I only deploy a proxy at the source side that can see the source side Datastore(s) of the source VM's?

Regards,
George.

[tsightler]

You can continue to use the Veeam server as the proxy for the target side. Just deploy an additional proxy on the source side, and make sure to select the specific proxies in the job configuration (don't leave them on automatic). I wouldn't worry about the datastore access because you only have a 10Mb link, and that will definitely be the bottleneck.

[gparker]

Thanks very much.

I hope it doesn't make any difference to the replication speed/efficiency etc, but just to let you know that there is no backup repository setup at the source side. I'm not using Veeam for its backup capabilities, I'm only using it for its replication feature. Does the replication job configuration care one way or the other that there's no backup repository on the source side?

George.

[tsightler]

Replication does require a repository to store replication metadata. By default it will use the default repository on the Veeam server. Ideally you should use a repository on the source side. Ideally you should make the source proxy also a repository and select it to store the metadata for the most efficient replication performance.

[gparker]

Thanks again tsightler, you've been of invaluable help

I will go ahead and configure a source-side Veeam proxy/repository server and will let you know what improvements I get when I kick off replication.

Many thanks,

George.

[gparker]

Hi tsightler,

You're a genius

I configured a Veeam proxy and Repositiory on a VM at the source site, reconfigured the replication job to use these setting and an 80GB VM whos seed on the target side is over 2 weeks old took 20 mins to complete the replication

Thanks again.

George.

[zak2011]

In the job reports..could you explain difference between READ and TRANSFERRED mean.
Does Transferred mean the daily change rate?
Thanks!

[jpeake]

What do these fields mean (looking at the report for a job)
1) Total Size - This appears to be total of VM disk provisioned size.
2) Backup Size - Is this the ("Total Size") - (free space) of the VM disks?
3) Data Read - Is this the amount of changed data as reported by CBT?
4) Transferred - I assume this is the amount of data transferred to the repository after dedupe and compression?

[aalders]

Example Backup job contains two small W2K3 VM's each with a 5GB disk (thin provisioned). Job report of the initial (full) backup shows :
- Total size 10GB (2x 5GB ... that's understandable)
- Data read 3,5 GB (which is the amount of data actually used by the VM's ... also makes sense)
- Transferred 1,8GB (so de-dupe and compression reduces 3,5GB to 1,8GB if I understand corerct .. this is approximately a factor 2)
- Compression 2,0x (hmm .. that matches what we see looking at transferred vs data read)
- Backup Size 1,2GB (this is what is actually used on the repository side for storing the backup data ... so some deduplication is being done, but ...)
- Dedupe 4,5x : This is something I don't understand. How is this number being calculated ? Anybody any ideas ?

Andre

[foggy]

Andre, please also see this topic for dedupe factor explanation. Thanks.

[aalders]

OK ... the explanation makes sense, but the numbers did not match. I just upgraded my 6.1 installation to 6.5 and using the same backup job data the report now says that the dedupe factor is 1,5x (in stead of 4,5x with version 6.1).
This makes more sense (transferring 1,8GB and storing 1,2GB matches the dedupe factor of 1,5). I'm happy now

[Paulus]

can any one explain to me the veeam backup job session report? I have a full backup of a SQL server with total used space of 300 GB, but according to the report it states: Read 95.5 GB, Transfered 65.6 GB. Does it mean my whole sever was not backed up?

[veremin]

Does it mean my whole sever was not backed up?

Nope.

I assume that it was an incremental run and CBT mechanism was definitely leveraged. If so, VB&R tried to read only the blocks that were reported by CBT mechanism as changed ones, also performing additional activities, like filtering out overlapping snapshot blocks, zero-data blocks, blocks of swap files, etc. Later the “read” data was compressed and transferred to target location, thus, there was a difference between read and transferred values.

Thanks.

[BackupDude]

Greetings Everyone,

I am looking for a greater in-depth explanation on the job process summary located at the top of a backup job. As seen in the screen shot below, I am trying to understand the data column. Can the collective review my explanations and make adjustments where necessary.

Processed: The amount of data the job has scanned in the VM
Read: The amount of data that is unique
Transferred(x.x%): The amount of data sent over the wire. The percentage is the rate of inline data deduplication.

Much thanks to everyone

[alanbolte]

I've heard that eventually the user guide will be updated to include that level of detail.

In the mean time, please see my post here.

[DanielRibeiro]

Hello guys,

I have a doubt about a job of replication.

My VM has a size of the 56GB, in job log, the size of 'Transferred' is 32,5GB (1,7x), what does that mean?

Other issue, in job log, the "Swap file blocks skipped: 1,6GB" what does that mean?

[foggy]

Daniel, you can find more detailed explanation of the counters in the thread above.

Also please review this user guide section for an answer to your second question.

Thanks.

[baber]

Dear friend
Hi

i have attached a report from veeam can you explain what does means below parameters:
Dara Read
Tranferred
Backup Size
Dedupe

https://www.photobox.co.uk/my/photo/ful ... 0421899734

BR
Babak

[gollem]

Your photo doesn't work.