-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Understanding replication job progress information
Hi, I'm reletively new to Veeam Backup & Replication. I've got a Prod site and a DR site, each running VMware vSphere. I am setting up Veeam to replicate some VM's from Prod to DR over our WAN link. The WAN bandwith is only 10Mbps, so I'm doing the seeding method and so I've installed Veeam at the prod site, created a full backup of a VM, brought the backup to the DR site, added it as a Backup Repository on the DR Veeam server and restored the VM onto my DR vSphere environment OK.
I then created a replication job with the replica seeding option checked and I checked the Replica Mapping option and mapped the source VM to the restored replica VM. I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% odf the size of the VM, right? The next thing it says is that it's read 61.0GB. What does this mean? The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all). What does the 113.7x in the brackets represent?
Please help me to understand these figures.
George.
I then created a replication job with the replica seeding option checked and I checked the Replica Mapping option and mapped the source VM to the restored replica VM. I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% odf the size of the VM, right? The next thing it says is that it's read 61.0GB. What does this mean? The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all). What does the 113.7x in the brackets represent?
Please help me to understand these figures.
George.
-
- Chief Product Officer
- Posts: 31768
- Liked: 7271 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Understanding replication job progress information
Correct. This is overall VM processing pointer.gparker wrote:I kicked of the replication job successfully and am monitoring it now and I don't quite understand some of the information it's telling me. The size of the VM is 80GB. The progress screen is saying that it's so far processed 68.8GB (86%) which I presume relates to 86% of the size of the VM, right?
The amount of data that has been read from source VM disk. Less than 68.8GB because some blocks are not being read - as because we do not care about their content, or because the content is already known (zeroed block).gparker wrote:The next thing it says is that it's read 61.0GB. What does this mean?
Yes. There are 549.5MB of compressed blocks which are different between your seed, and the current state of source VM disks.gparker wrote:The next thing it says is it's transferred 549.4 MB (113.7x). What does this mean - has it only had to transfer 549.4MB through the WAN link so far
Well, your link is only 1 MB/s, so what do you expect when there is 549.4 MB of data to move across. Of course the link will be saturated.gparker wrote:and if so, why is the WAN link constantly saturated with traffic (there is no other data going through this WAN link at all).
Read divided by transferred. If you are doing seeding, ignore this counter - it only makes sense for "normal" runs.gparker wrote:What does the 113.7x in the brackets represent?
-
- Expert
- Posts: 213
- Liked: 35 times
- Joined: Feb 20, 2012 4:13 pm
- Full Name: Nick Mahlitz
- Contact:
Re: Understanding replication job progress information
Does he have a 10 or 1 Mbs link?
-
- Chief Product Officer
- Posts: 31768
- Liked: 7271 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Understanding replication job progress information
gparker wrote:The WAN bandwith is only 10Mbps
-
- Expert
- Posts: 213
- Liked: 35 times
- Joined: Feb 20, 2012 4:13 pm
- Full Name: Nick Mahlitz
- Contact:
Re: Understanding replication job progress information
Then this post is not quite right?Gostev wrote: Well, your link is only 1 MB/s, so what do you expect when there is 549.4 MB of data to move across. Of course the link will be saturated.
-
- Chief Product Officer
- Posts: 31768
- Liked: 7271 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Understanding replication job progress information
10 Mpbs = 1.25 MB/s
-
- Expert
- Posts: 213
- Liked: 35 times
- Joined: Feb 20, 2012 4:13 pm
- Full Name: Nick Mahlitz
- Contact:
Re: Understanding replication job progress information
Oopps I misread...sorry!
-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Re: Understanding replication job progress information
Thanks everyone for your posts, answers and comments
I can confirm that the WAN link is 10Mbps and the Veeam progress box is reporting transfer rate of 1MB/sec. I don't mind that it says 1MB/sec as opposed to 1.25MB/sec, but what does concern me is that it's constantly saying 1MB/sec while the job is active. The job has been active for something like 13 hours and has tranferred only 549.4MB in all that time while the transfer rate hasn't moved from 1MB/sec. So what it going on on the WAN link?
George.
I can confirm that the WAN link is 10Mbps and the Veeam progress box is reporting transfer rate of 1MB/sec. I don't mind that it says 1MB/sec as opposed to 1.25MB/sec, but what does concern me is that it's constantly saying 1MB/sec while the job is active. The job has been active for something like 13 hours and has tranferred only 549.4MB in all that time while the transfer rate hasn't moved from 1MB/sec. So what it going on on the WAN link?
George.
-
- VP, Product Management
- Posts: 6033
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Understanding replication job progress information
Do you have proxies on both sides of the link? Have you manually selected them in the job? It sounds like you may have a proxy on the target side that is pulling the replica via network mode across the link, instead of proxy-to-proxy. This is the most common configuration error that I see with V6 replication, either having a proxy only on one side, or having a proxy on both sides, in network mode, but not specifically selecting the proxies on each end.
-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Re: Understanding replication job progress information
Hi tsightler,
You are right, I only have a proxie on the target side. In the job progress information box, it actually says "using source proxy VMware Backup Proxy [nbd]" and "using target proxy VMware Backup Proxy [nbd]". I presume the "[nbd]" means it is pulling the replica via network mode across the link, right?
How does this relate to there being a constant 1MB/sec processing rate when it has only transferred 549.4MB?
George.
You are right, I only have a proxie on the target side. In the job progress information box, it actually says "using source proxy VMware Backup Proxy [nbd]" and "using target proxy VMware Backup Proxy [nbd]". I presume the "[nbd]" means it is pulling the replica via network mode across the link, right?
How does this relate to there being a constant 1MB/sec processing rate when it has only transferred 549.4MB?
George.
-
- VP, Product Management
- Posts: 6033
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Understanding replication job progress information
So the "VMware Backup Proxy" is the Veeam server itself. Is this located on the source or target side?
The number for "transferred" indicates the amount of data that would have been sent between the two proxies, but since you are using the same proxy for both source and target, this number effectively has no meaning at all. Not having a proxy on both ends will lead to very poor replication performance. I'm suspicious that your VMware server is on the target side, and thus the proxy is having to read the entire data across the 10MB link. That would mean it would pull the entire 61GB "read" over the wire which would be about right for a 10Mb link.
Please deploy both a source and target proxy, and then select the appropriate proxies in the job configuration and test again. I think you will see a tremendous difference as then the actual data transferred across the WAN link would indeed be only 549.5MB, and this data would be streamed efficiently between the two proxies, not sent/received via network mode (nbd) across the WAN, which is very inefficient.
The number for "transferred" indicates the amount of data that would have been sent between the two proxies, but since you are using the same proxy for both source and target, this number effectively has no meaning at all. Not having a proxy on both ends will lead to very poor replication performance. I'm suspicious that your VMware server is on the target side, and thus the proxy is having to read the entire data across the 10MB link. That would mean it would pull the entire 61GB "read" over the wire which would be about right for a 10Mb link.
Please deploy both a source and target proxy, and then select the appropriate proxies in the job configuration and test again. I think you will see a tremendous difference as then the actual data transferred across the WAN link would indeed be only 549.5MB, and this data would be streamed efficiently between the two proxies, not sent/received via network mode (nbd) across the WAN, which is very inefficient.
-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Re: Understanding replication job progress information
Hi,
Yes you are right, the "VMware Backup Proxy" is the Veeam server itself. It is located on the target side.
Thanks for explaining how it works and what "transferred" number actually means in my situation. Thanks also for explaining that the proxy has to read the entire data across the 10Mb link and that it's effectivelly pulling the entire 61GB.
I only have a reletively small deployment scenario here. I am only wanting to replicate 3 VM's from source vSphere environment to target vSphere environment. I will be configuring the replication job(s) to schedule the replications every 4 hours. So I'm happy to deploy a proxy server at the source side, but do I really need to deploy a seperate proxy server at the target side? The VM that I've installed Veeam onto at the target side isn't being heavilly loaded. It also does not have direct access to the target VMFS Datastore where the replica's are going to becuase it's on an ESX host that cannot see the target Datastore. Will I see an improvement in the replications if I only deploy a proxy at the source side that can see the source side Datastore(s) of the source VM's?
Regards,
George.
Yes you are right, the "VMware Backup Proxy" is the Veeam server itself. It is located on the target side.
Thanks for explaining how it works and what "transferred" number actually means in my situation. Thanks also for explaining that the proxy has to read the entire data across the 10Mb link and that it's effectivelly pulling the entire 61GB.
I only have a reletively small deployment scenario here. I am only wanting to replicate 3 VM's from source vSphere environment to target vSphere environment. I will be configuring the replication job(s) to schedule the replications every 4 hours. So I'm happy to deploy a proxy server at the source side, but do I really need to deploy a seperate proxy server at the target side? The VM that I've installed Veeam onto at the target side isn't being heavilly loaded. It also does not have direct access to the target VMFS Datastore where the replica's are going to becuase it's on an ESX host that cannot see the target Datastore. Will I see an improvement in the replications if I only deploy a proxy at the source side that can see the source side Datastore(s) of the source VM's?
Regards,
George.
-
- VP, Product Management
- Posts: 6033
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Understanding replication job progress information
You can continue to use the Veeam server as the proxy for the target side. Just deploy an additional proxy on the source side, and make sure to select the specific proxies in the job configuration (don't leave them on automatic). I wouldn't worry about the datastore access because you only have a 10Mb link, and that will definitely be the bottleneck.
-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Re: Understanding replication job progress information
Thanks very much.
I hope it doesn't make any difference to the replication speed/efficiency etc, but just to let you know that there is no backup repository setup at the source side. I'm not using Veeam for its backup capabilities, I'm only using it for its replication feature. Does the replication job configuration care one way or the other that there's no backup repository on the source side?
George.
I hope it doesn't make any difference to the replication speed/efficiency etc, but just to let you know that there is no backup repository setup at the source side. I'm not using Veeam for its backup capabilities, I'm only using it for its replication feature. Does the replication job configuration care one way or the other that there's no backup repository on the source side?
George.
-
- VP, Product Management
- Posts: 6033
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Understanding replication job progress information
Replication does require a repository to store replication metadata. By default it will use the default repository on the Veeam server. Ideally you should use a repository on the source side. Ideally you should make the source proxy also a repository and select it to store the metadata for the most efficient replication performance.
-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Re: Understanding replication job progress information
Thanks again tsightler, you've been of invaluable help
I will go ahead and configure a source-side Veeam proxy/repository server and will let you know what improvements I get when I kick off replication.
Many thanks,
George.
I will go ahead and configure a source-side Veeam proxy/repository server and will let you know what improvements I get when I kick off replication.
Many thanks,
George.
-
- Enthusiast
- Posts: 62
- Liked: 7 times
- Joined: Feb 01, 2012 2:24 am
- Full Name: George Parker
- Contact:
Re: Understanding replication job progress information
Hi tsightler,
You're a genius
I configured a Veeam proxy and Repositiory on a VM at the source site, reconfigured the replication job to use these setting and an 80GB VM whos seed on the target side is over 2 weeks old took 20 mins to complete the replication
Thanks again.
George.
You're a genius
I configured a Veeam proxy and Repositiory on a VM at the source site, reconfigured the replication job to use these setting and an 80GB VM whos seed on the target side is over 2 weeks old took 20 mins to complete the replication
Thanks again.
George.
-
- Veteran
- Posts: 367
- Liked: 41 times
- Joined: May 15, 2012 2:21 pm
- Full Name: Arun
- Contact:
[MERGED] Reports
In the job reports..could you explain difference between READ and TRANSFERRED mean.
Does Transferred mean the daily change rate?
Thanks!
Does Transferred mean the daily change rate?
Thanks!
-
- Enthusiast
- Posts: 88
- Liked: 25 times
- Joined: Sep 25, 2012 7:57 pm
- Contact:
[MERGED] Job statistics - field definitions
What do these fields mean (looking at the report for a job)
1) Total Size - This appears to be total of VM disk provisioned size.
2) Backup Size - Is this the ("Total Size") - (free space) of the VM disks?
3) Data Read - Is this the amount of changed data as reported by CBT?
4) Transferred - I assume this is the amount of data transferred to the repository after dedupe and compression?
1) Total Size - This appears to be total of VM disk provisioned size.
2) Backup Size - Is this the ("Total Size") - (free space) of the VM disks?
3) Data Read - Is this the amount of changed data as reported by CBT?
4) Transferred - I assume this is the amount of data transferred to the repository after dedupe and compression?
-
- Novice
- Posts: 4
- Liked: never
- Joined: Oct 26, 2012 9:08 am
- Full Name: Andre Aalders
- Contact:
[MERGED] How to interpret dedupe factor
Example Backup job contains two small W2K3 VM's each with a 5GB disk (thin provisioned). Job report of the initial (full) backup shows :
- Total size 10GB (2x 5GB ... that's understandable)
- Data read 3,5 GB (which is the amount of data actually used by the VM's ... also makes sense)
- Transferred 1,8GB (so de-dupe and compression reduces 3,5GB to 1,8GB if I understand corerct .. this is approximately a factor 2)
- Compression 2,0x (hmm .. that matches what we see looking at transferred vs data read)
- Backup Size 1,2GB (this is what is actually used on the repository side for storing the backup data ... so some deduplication is being done, but ...)
- Dedupe 4,5x : This is something I don't understand. How is this number being calculated ? Anybody any ideas ?
Andre
- Total size 10GB (2x 5GB ... that's understandable)
- Data read 3,5 GB (which is the amount of data actually used by the VM's ... also makes sense)
- Transferred 1,8GB (so de-dupe and compression reduces 3,5GB to 1,8GB if I understand corerct .. this is approximately a factor 2)
- Compression 2,0x (hmm .. that matches what we see looking at transferred vs data read)
- Backup Size 1,2GB (this is what is actually used on the repository side for storing the backup data ... so some deduplication is being done, but ...)
- Dedupe 4,5x : This is something I don't understand. How is this number being calculated ? Anybody any ideas ?
Andre
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Understanding replication job progress information
Andre, please also see this topic for dedupe factor explanation. Thanks.
-
- Novice
- Posts: 4
- Liked: never
- Joined: Oct 26, 2012 9:08 am
- Full Name: Andre Aalders
- Contact:
Re: Understanding replication job progress information
OK ... the explanation makes sense, but the numbers did not match. I just upgraded my 6.1 installation to 6.5 and using the same backup job data the report now says that the dedupe factor is 1,5x (in stead of 4,5x with version 6.1).
This makes more sense (transferring 1,8GB and storing 1,2GB matches the dedupe factor of 1,5). I'm happy now
This makes more sense (transferring 1,8GB and storing 1,2GB matches the dedupe factor of 1,5). I'm happy now
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Jun 08, 2011 10:30 am
- Full Name: Paulus Tshiningayamwe
- Contact:
[MERGED] backup job session report
can any one explain to me the veeam backup job session report? I have a full backup of a SQL server with total used space of 300 GB, but according to the report it states: Read 95.5 GB, Transfered 65.6 GB. Does it mean my whole sever was not backed up?
-
- Product Manager
- Posts: 20382
- Liked: 2292 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: backup job session report
Nope.Does it mean my whole sever was not backed up?
I assume that it was an incremental run and CBT mechanism was definitely leveraged. If so, VB&R tried to read only the blocks that were reported by CBT mechanism as changed ones, also performing additional activities, like filtering out overlapping snapshot blocks, zero-data blocks, blocks of swap files, etc. Later the “read” data was compressed and transferred to target location, thus, there was a difference between read and transferred values.
Thanks.
-
- Enthusiast
- Posts: 31
- Liked: 3 times
- Joined: Jan 09, 2014 7:22 pm
- Full Name: Fabio Vasco
- Contact:
[MERGED] Better Explanation of Job Progress
Greetings Everyone,
I am looking for a greater in-depth explanation on the job process summary located at the top of a backup job. As seen in the screen shot below, I am trying to understand the data column. Can the collective review my explanations and make adjustments where necessary.
Processed: The amount of data the job has scanned in the VM
Read: The amount of data that is unique
Transferred(x.x%): The amount of data sent over the wire. The percentage is the rate of inline data deduplication.
Much thanks to everyone
I am looking for a greater in-depth explanation on the job process summary located at the top of a backup job. As seen in the screen shot below, I am trying to understand the data column. Can the collective review my explanations and make adjustments where necessary.
Processed: The amount of data the job has scanned in the VM
Read: The amount of data that is unique
Transferred(x.x%): The amount of data sent over the wire. The percentage is the rate of inline data deduplication.
Much thanks to everyone
-
- Veteran
- Posts: 635
- Liked: 174 times
- Joined: Jun 18, 2012 8:58 pm
- Full Name: Alan Bolte
- Contact:
Re: Better Explanation of Job Progress
I've heard that eventually the user guide will be updated to include that level of detail.
In the mean time, please see my post here.
In the mean time, please see my post here.
-
- Lurker
- Posts: 1
- Liked: never
- Joined: May 27, 2015 2:31 pm
- Full Name: Daniel Ribeiro
- Contact:
[MERGED] Doubt - Log Job Replication
Hello guys,
I have a doubt about a job of replication.
My VM has a size of the 56GB, in job log, the size of 'Transferred' is 32,5GB (1,7x), what does that mean?
Other issue, in job log, the "Swap file blocks skipped: 1,6GB" what does that mean?
I have a doubt about a job of replication.
My VM has a size of the 56GB, in job log, the size of 'Transferred' is 32,5GB (1,7x), what does that mean?
Other issue, in job log, the "Swap file blocks skipped: 1,6GB" what does that mean?
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Understanding replication job progress information
Daniel, you can find more detailed explanation of the counters in the thread above.
Also please review this user guide section for an answer to your second question.
Thanks.
Also please review this user guide section for an answer to your second question.
Thanks.
-
- Veteran
- Posts: 665
- Liked: 4 times
- Joined: Nov 21, 2013 12:02 pm
- Full Name: Babak Seyedi nejad
- Contact:
[MERGED] veeam report parameters
Dear friend
Hi
i have attached a report from veeam can you explain what does means below parameters:
Dara Read
Tranferred
Backup Size
Dedupe
https://www.photobox.co.uk/my/photo/ful ... 0421899734
BR
Babak
Hi
i have attached a report from veeam can you explain what does means below parameters:
Dara Read
Tranferred
Backup Size
Dedupe
https://www.photobox.co.uk/my/photo/ful ... 0421899734
BR
Babak
-
- Enthusiast
- Posts: 33
- Liked: 7 times
- Joined: Jun 16, 2012 7:26 pm
- Full Name: Erik E.
- Contact:
Re: veeam report parameters
Your photo doesn't work.
Who is online
Users browsing this forum: Bing [Bot], Gostev, renatorichina, whistler and 92 guests