Comprehensive data protection for all workloads
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Understanding replication job progress information

Post by gparker »

Hi, I'm reletively new to Veeam Backup & Replication. I've got a Prod site and a DR site, each running VMware vSphere. I am setting up Veeam to replicate some VM's from Prod to DR over our WAN link. The WAN bandwith is only 10Mbps, so I'm doing the seeding method and so I've installed Veeam at the prod site, created a full backup of a VM, brought the backup to the DR site, added it as a Backup Repository on the DR Veeam server and restored the VM onto my DR vSphere environment OK.
I then created a replication job with the replica seeding option checked and I checked the Replica Mapping option and mapped the source VM to the restored replica VM. I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% odf the size of the VM, right? The next thing it says is that it's read 61.0GB. What does this mean? The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all). What does the 113.7x in the brackets represent?

Please help me to understand these figures.

George.
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Understanding replication job progress information

Post by Gostev »

gparker wrote:I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% of the size of the VM, right?
Correct. This is overall VM processing pointer.
gparker wrote:The next thing it says is that it's read 61.0GB. What does this mean?
The amount of data that has been read from source VM disk. Less than 68.8GB because some blocks are not being read - as because we do not care about their content, or because the content is already known (zeroed block).
gparker wrote:The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far
Yes. There are 549.5MB of compressed blocks which are different between your seed, and the current state of source VM disks.
gparker wrote:and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all).
Well, your link is only 1 MB/s, so what do you expect when there is 549.4 MB of data to move across. Of course the link will be saturated.
gparker wrote:What does the 113.7x in the brackets represent?
Read divided by transferred. If you are doing seeding, ignore this counter - it only makes sense for "normal" runs.
homerjnick
Expert
Posts: 212
Liked: 35 times
Joined: Feb 20, 2012 4:13 pm
Full Name: Nick Mahlitz
Contact:

Re: Understanding replication job progress information

Post by homerjnick »

Does he have a 10 or 1 Mbs link?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Understanding replication job progress information

Post by Gostev »

gparker wrote:The WAN bandwith is only 10Mbps
homerjnick
Expert
Posts: 212
Liked: 35 times
Joined: Feb 20, 2012 4:13 pm
Full Name: Nick Mahlitz
Contact:

Re: Understanding replication job progress information

Post by homerjnick »

Gostev wrote: Well, your link is only 1 MB/s, so what do you expect when there is 549.4 MB of data to move across. Of course the link will be saturated.
Then this post is not quite right?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Understanding replication job progress information

Post by Gostev »

10 Mpbs = 1.25 MB/s
homerjnick
Expert
Posts: 212
Liked: 35 times
Joined: Feb 20, 2012 4:13 pm
Full Name: Nick Mahlitz
Contact:

Re: Understanding replication job progress information

Post by homerjnick »

Oopps I misread...sorry!
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Re: Understanding replication job progress information

Post by gparker »

Thanks everyone for your posts, answers and comments :)

I can confirm that the WAN link is 10Mbps and the Veeam progress box is reporting transfer rate of 1MB/sec. I don't mind that it says 1MB/sec as opposed to 1.25MB/sec, but what does concern me is that it's constantly saying 1MB/sec while the job is active. The job has been active for something like 13 hours and has tranferred only 549.4MB in all that time while the transfer rate hasn't moved from 1MB/sec. So what it going on on the WAN link?

George.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Understanding replication job progress information

Post by tsightler »

Do you have proxies on both sides of the link? Have you manually selected them in the job? It sounds like you may have a proxy on the target side that is pulling the replica via network mode across the link, instead of proxy-to-proxy. This is the most common configuration error that I see with V6 replication, either having a proxy only on one side, or having a proxy on both sides, in network mode, but not specifically selecting the proxies on each end.
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Re: Understanding replication job progress information

Post by gparker »

Hi tsightler,

You are right, I only have a proxie on the target side. In the job progress information box, it actually says "using source proxy VMware Backup Proxy [nbd]" and "using target proxy VMware Backup Proxy [nbd]". I presume the "[nbd]" means it is pulling the replica via network mode across the link, right?

How does this relate to there being a constant 1MB/sec processing rate when it has only transferred 549.4MB?

George.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Understanding replication job progress information

Post by tsightler »

So the "VMware Backup Proxy" is the Veeam server itself. Is this located on the source or target side?

The number for "transferred" indicates the amount of data that would have been sent between the two proxies, but since you are using the same proxy for both source and target, this number effectively has no meaning at all. Not having a proxy on both ends will lead to very poor replication performance. I'm suspicious that your VMware server is on the target side, and thus the proxy is having to read the entire data across the 10MB link. That would mean it would pull the entire 61GB "read" over the wire which would be about right for a 10Mb link.

Please deploy both a source and target proxy, and then select the appropriate proxies in the job configuration and test again. I think you will see a tremendous difference as then the actual data transferred across the WAN link would indeed be only 549.5MB, and this data would be streamed efficiently between the two proxies, not sent/received via network mode (nbd) across the WAN, which is very inefficient.
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Re: Understanding replication job progress information

Post by gparker »

Hi,

Yes you are right, the "VMware Backup Proxy" is the Veeam server itself. It is located on the target side.

Thanks for explaining how it works and what "transferred" number actually means in my situation. Thanks also for explaining that the proxy has to read the entire data across the 10Mb link and that it's effectivelly pulling the entire 61GB.

I only have a reletively small deployment scenario here. I am only wanting to replicate 3 VM's from source vSphere environment to target vSphere environment. I will be configuring the replication job(s) to schedule the replications every 4 hours. So I'm happy to deploy a proxy server at the source side, but do I really need to deploy a seperate proxy server at the target side? The VM that I've installed Veeam onto at the target side isn't being heavilly loaded. It also does not have direct access to the target VMFS Datastore where the replica's are going to becuase it's on an ESX host that cannot see the target Datastore. Will I see an improvement in the replications if I only deploy a proxy at the source side that can see the source side Datastore(s) of the source VM's?

Regards,
George.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Understanding replication job progress information

Post by tsightler »

You can continue to use the Veeam server as the proxy for the target side. Just deploy an additional proxy on the source side, and make sure to select the specific proxies in the job configuration (don't leave them on automatic). I wouldn't worry about the datastore access because you only have a 10Mb link, and that will definitely be the bottleneck.
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Re: Understanding replication job progress information

Post by gparker »

Thanks very much.

I hope it doesn't make any difference to the replication speed/efficiency etc, but just to let you know that there is no backup repository setup at the source side. I'm not using Veeam for its backup capabilities, I'm only using it for its replication feature. Does the replication job configuration care one way or the other that there's no backup repository on the source side?

George.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Understanding replication job progress information

Post by tsightler »

Replication does require a repository to store replication metadata. By default it will use the default repository on the Veeam server. Ideally you should use a repository on the source side. Ideally you should make the source proxy also a repository and select it to store the metadata for the most efficient replication performance.
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Re: Understanding replication job progress information

Post by gparker »

Thanks again tsightler, you've been of invaluable help :)

I will go ahead and configure a source-side Veeam proxy/repository server and will let you know what improvements I get when I kick off replication.

Many thanks,

George.
gparker
Enthusiast
Posts: 58
Liked: 6 times
Joined: Feb 01, 2012 2:24 am
Full Name: George Parker
Contact:

Re: Understanding replication job progress information

Post by gparker »

Hi tsightler,

You're a genius :)

I configured a Veeam proxy and Repositiory on a VM at the source site, reconfigured the replication job to use these setting and an 80GB VM whos seed on the target side is over 2 weeks old took 20 mins to complete the replication :)

Thanks again.

George.
zak2011
Veteran
Posts: 367
Liked: 41 times
Joined: May 15, 2012 2:21 pm
Full Name: Arun
Contact:

[MERGED] Reports

Post by zak2011 »

In the job reports..could you explain difference between READ and TRANSFERRED mean.
Does Transferred mean the daily change rate?
Thanks!
jpeake
Enthusiast
Posts: 88
Liked: 25 times
Joined: Sep 25, 2012 7:57 pm
Contact:

[MERGED] Job statistics - field definitions

Post by jpeake »

What do these fields mean (looking at the report for a job)
1) Total Size - This appears to be total of VM disk provisioned size.
2) Backup Size - Is this the ("Total Size") - (free space) of the VM disks?
3) Data Read - Is this the amount of changed data as reported by CBT?
4) Transferred - I assume this is the amount of data transferred to the repository after dedupe and compression?
aalders
Novice
Posts: 4
Liked: never
Joined: Oct 26, 2012 9:08 am
Full Name: Andre Aalders
Contact:

[MERGED] How to interpret dedupe factor

Post by aalders »

Example Backup job contains two small W2K3 VM's each with a 5GB disk (thin provisioned). Job report of the initial (full) backup shows :
- Total size 10GB (2x 5GB ... that's understandable)
- Data read 3,5 GB (which is the amount of data actually used by the VM's ... also makes sense)
- Transferred 1,8GB (so de-dupe and compression reduces 3,5GB to 1,8GB if I understand corerct .. this is approximately a factor 2)
- Compression 2,0x (hmm .. that matches what we see looking at transferred vs data read)
- Backup Size 1,2GB (this is what is actually used on the repository side for storing the backup data ... so some deduplication is being done, but ...)
- Dedupe 4,5x : This is something I don't understand. How is this number being calculated ? Anybody any ideas ?

Andre
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Understanding replication job progress information

Post by foggy »

Andre, please also see this topic for dedupe factor explanation. Thanks.
aalders
Novice
Posts: 4
Liked: never
Joined: Oct 26, 2012 9:08 am
Full Name: Andre Aalders
Contact:

Re: Understanding replication job progress information

Post by aalders »

OK ... the explanation makes sense, but the numbers did not match. I just upgraded my 6.1 installation to 6.5 and using the same backup job data the report now says that the dedupe factor is 1,5x (in stead of 4,5x with version 6.1).
This makes more sense (transferring 1,8GB and storing 1,2GB matches the dedupe factor of 1,5). I'm happy now :D
Paulus
Lurker
Posts: 1
Liked: never
Joined: Jun 08, 2011 10:30 am
Full Name: Paulus Tshiningayamwe
Contact:

[MERGED] backup job session report

Post by Paulus »

can any one explain to me the veeam backup job session report? I have a full backup of a SQL server with total used space of 300 GB, but according to the report it states: Read 95.5 GB, Transfered 65.6 GB. Does it mean my whole sever was not backed up?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: backup job session report

Post by veremin »

Does it mean my whole sever was not backed up?
Nope.

I assume that it was an incremental run and CBT mechanism was definitely leveraged. If so, VB&R tried to read only the blocks that were reported by CBT mechanism as changed ones, also performing additional activities, like filtering out overlapping snapshot blocks, zero-data blocks, blocks of swap files, etc. Later the “read” data was compressed and transferred to target location, thus, there was a difference between read and transferred values.

Thanks.
BackupDude
Enthusiast
Posts: 31
Liked: 3 times
Joined: Jan 09, 2014 7:22 pm
Full Name: Fabio Vasco
Contact:

[MERGED] Better Explanation of Job Progress

Post by BackupDude »

Greetings Everyone,

I am looking for a greater in-depth explanation on the job process summary located at the top of a backup job. As seen in the screen shot below, I am trying to understand the data column. Can the collective review my explanations and make adjustments where necessary.
Image
Processed: The amount of data the job has scanned in the VM
Read: The amount of data that is unique
Transferred(x.x%): The amount of data sent over the wire. The percentage is the rate of inline data deduplication.

Much thanks to everyone
alanbolte
Veteran
Posts: 635
Liked: 174 times
Joined: Jun 18, 2012 8:58 pm
Full Name: Alan Bolte
Contact:

Re: Better Explanation of Job Progress

Post by alanbolte »

I've heard that eventually the user guide will be updated to include that level of detail.

In the mean time, please see my post here.
DanielRibeiro
Lurker
Posts: 1
Liked: never
Joined: May 27, 2015 2:31 pm
Full Name: Daniel Ribeiro
Contact:

[MERGED] Doubt - Log Job Replication

Post by DanielRibeiro »

Hello guys,

I have a doubt about a job of replication.

My VM has a size of the 56GB, in job log, the size of 'Transferred' is 32,5GB (1,7x), what does that mean?

Other issue, in job log, the "Swap file blocks skipped: 1,6GB" what does that mean?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Understanding replication job progress information

Post by foggy »

Daniel, you can find more detailed explanation of the counters in the thread above.

Also please review this user guide section for an answer to your second question.

Thanks.
baber
Veteran
Posts: 665
Liked: 4 times
Joined: Nov 21, 2013 12:02 pm
Full Name: Babak Seyedi nejad
Contact:

[MERGED] veeam report parameters

Post by baber »

Dear friend
Hi

i have attached a report from veeam can you explain what does means below parameters:
Dara Read
Tranferred
Backup Size
Dedupe

https://www.photobox.co.uk/my/photo/ful ... 0421899734

BR
Babak
gollem
Enthusiast
Posts: 33
Liked: 7 times
Joined: Jun 16, 2012 7:26 pm
Full Name: Erik E.
Contact:

Re: veeam report parameters

Post by gollem »

Your photo doesn't work.
Post Reply

Who is online

Users browsing this forum: ante_704 and 172 guests