Replication Timing

[rogerdu]

I am becoming somewhat disillusioned. In your pamphlets about Replication, you state that your replication targets can be every 15 minutes.

I have a job that does replication of one system across a WAN link that can burst to 100 Mb/s that takes an hour to process. the typical transfer amount is under 20 MB.My customer is expecting their replication job for 6 systems to replicate hourly. If I look at the daily backup, these 6 jobs generate around 20-25 GB of data on a daily incremental, so roughly 2-3 GB per hour. Is this a reasonable target? Unless my math is all wrong, I'm thinking that a 100 Mb/s link should be able to transfer 3 GB of data in an hour...

Is there processing that will slow this transfer rate significantly?

Roger

[PTide]

Hi,

What are your bottleneck stats for replication jobs?

[rogerdu]

I have 4 replication jobs 2 just started so I can't see the stats of the last run... Silver and Gold replicate from the source VMs, Bronze from the Backup Copy. Its the gold I'm concerned with.

SiteA-to-B-Gold: Source 100% > Source WAN 0% > Network 0% > Target WAN 95% > Target 25%
SiteA-to-B-Bronze: Source 9% > Proxy 32% > Network 26% > Target 70%

[PTide]

The Gold stats clearly show source being the bottleneck (production storage). How many VMs do you have in Gold? Is the same set of VMs chosen as source for the backup job? If so then could you please share the stats for that job as well so we can compare them?

Thanks

[rogerdu]

The Backup goes to a local Repository, the Replication goes across 2 WAN accelerators that have a 100Mb/s burstable wireless WAN between them

Backup stats from last night's backup:

Code: Select all

11/16/2016 9:15:06 PM :: Job started at 2016-11-16 9:15:04 PM 
11/16/2016 9:15:07 PM :: Building VMs list 
11/16/2016 9:15:19 PM :: VM size: 1000.0 GB (709.5 GB used) 
11/16/2016 9:15:19 PM :: Changed block tracking is enabled 
11/16/2016 9:15:24 PM :: Queued for processing at 2016-11-16 9:15:24 PM 
11/16/2016 9:15:24 PM :: Required backup infrastructure resources have been assigned 
11/16/2016 9:17:13 PM :: Creating storage snapshot 
11/16/2016 9:18:20 PM :: Processing DC01 
11/16/2016 9:18:25 PM :: Processing EXCH 
11/16/2016 9:18:25 PM :: Processing ENT-DB 
11/16/2016 9:18:25 PM :: Processing ENT-APP 
11/16/2016 9:18:26 PM :: Processing WTSrvDB 
11/16/2016 9:18:26 PM :: Processing WTSrvIIS 
11/16/2016 9:31:05 PM :: All VMs have been queued for processing 
11/16/2016 10:15:31 PM :: Deleting storage snapshot 
11/16/2016 10:15:40 PM :: Load: Source 29% > Proxy 20% > Network 25% > Target 74% 
11/16/2016 10:15:40 PM :: Primary bottleneck: Target 
11/16/2016 10:15:40 PM :: Job finished at 2016-11-16 10:15:39 PM 

Stats from the replication job this AM:
11/17/2016 6:00:03 AM :: Job started at 2016-11-17 6:00:02 AM 
11/17/2016 6:00:04 AM :: Building VMs list 
11/17/2016 6:00:19 AM :: VM size: 1000.0 GB (709.5 GB used) 
11/17/2016 6:00:20 AM :: Changed block tracking is enabled 
11/17/2016 6:00:21 AM :: Queued for processing at 2016-11-17 6:00:21 AM 
11/17/2016 6:00:21 AM :: Required backup infrastructure resources have been assigned 
11/17/2016 6:02:17 AM :: Creating storage snapshot 
11/17/2016 6:03:59 AM :: Processing DC01 
11/17/2016 6:04:54 AM :: Processing EXCH 
11/17/2016 6:26:44 AM :: Processing ENT-DB 
11/17/2016 8:46:12 AM :: Processing ENT-APP 
11/17/2016 11:05:37 AM :: Processing WTSrvDB 
11/17/2016 11:56:58 AM :: Processing WTSrvIIS 
11/17/2016 12:18:18 PM :: All VMs have been queued for processing 
11/17/2016 12:47:45 PM :: Deleting storage snapshot 
11/17/2016 12:47:58 PM :: Load: Source 96% > Source WAN 2% > Network 0% > Target WAN 95% > Target 27% 
11/17/2016 12:47:58 PM :: Primary bottleneck: Source 
11/17/2016 12:47:58 PM :: Job finished at 2016-11-17 12:47:57 PM 

EXCH = Exchange, the 2 DB servers are MS SQL, DC01 is an AD server

[foggy]

Unless my math is all wrong, I'm thinking that a 100 Mb/s link should be able to transfer 3 GB of data in an hour...

Is there processing that will slow this transfer rate significantly?

There's of course some processing overhead, but your math looks reasonable (you should be able to transfer that amount even without WAN acceleration). Only actual testing will show whether it is correct though.

[rogerdu]

Damn... should have updated this thread... We removed teh WAN accelerators out of the equation and went direct. We have a low latency connection and the speed nears 100Mb/s.

The replication now takes 10-20 minutes to complete , the backup copy completes in a reasonable timeframe... in other words, the WAN accelerators were slowing us down considerably. Once I went Direct, all my throughput values went up, my time to do any backup copies and replication went down and my customer is happy.