v6 WAN Replication - Wow!

[mcwill]

I've been happily using Veeam for over a year now and in the whole have found it to work very well in producing backups, but one area that has always been troublesome is sending the information off site. It didn't matter if we tried Push or Pull, replication or backup we weren't able to find a viable option to send nightly backups off-site using Veeam at a reasonable speed. We would instead backup locally and use a simple rsync job to replicate to remote storage.

That is all changed now, I've just run some tests on the new v6 replication system and I'm very impressed, after seeding a small VM I made some changes on the VM and re-ran the replication job. I was both suprised and pleased to see the second run still saturated the link to the remote site.

Thanks for a major improvement.
Iain

[Gostev]

Thanks for your feedback, Iain.

When we ran customers' survey half a year ago on tape vs. offsite backup, we have heard it loud and clear that offsite backup and replication is highly desired by our current customer base. However, we also found that most customers are very limited on WAN link bandwidth. Which is why we have put the focus into WAN optimizations with this release.

We are seeing the same improvement in our production too. The performance improvement is almost by an order of magnitude - I had some numbers in my section of weekly forum's digest email couple of weeks ago.

I would be very interested to hear some specifics about your deployment once you get it all going. What WAN link speed you are using, what latency it has, how much are you backing up over WAN, whether you are using any WAN accelerators on top of Veeam (as internally we do not), and of course some numbers.

If anyone else wants to chime in and post their numbers too - please do - I think this information would be interesting to everyone assessing offsite backup and replication.

Thanks.

[vmexpert]

I am having superb results as well with my initial testing. Even funny how veeam never mentions "cloud" backup and replication once in any v6 materials. Most vendors nowadays seem to have all of their marketing stuffed with "cloud" - without doing much to actually facilitate cloud DR. I would expect veeam to be all over"cloud" too, especially now.

[Daveyd]

I am having superb results as well with my initial testing. Even funny how veeam never mentions "cloud" backup and replication once in any v6 materials. Most vendors nowadays seem to have all of their marketing stuffed with "cloud" - without doing much to actually facilitate cloud DR. I would expect veeam to be all over"cloud" too, especially now.

I'm sure it's only a matter of time before they put the "cloud" into their advertisements. Personally, I cant stand the word anymore!!!

[rvopat]

can you please elaborate on the Superb results?

We are currently running veeam 5 in our dr site and pulling our replicas over a 6 mb wan link

last night there was a sql vm that had 10 gb of data changes and it took about 8hrs and 22 min

can you offer some details on your data size and speed and time?

[Gostev]

I would expect veeam to be all over"cloud" too, especially now.

Cloud is not cool.

Seriously though, the word is way too abused by the industry these days. Everyone claims to do something for the cloud. It's hard to find a better way to make the value proposition extremely vague than to use "cloud". We prefer to explain exactly what we do, and in simple terms that you cannot read otherwise. Check out the What's New for v6, you will find very specific and technical explanation of what we did to improve backup and replication over WAN. We worry about building the best technology, and let our customers decide how best to use - whether you want to backup to your remote office (some would say "private cloud"), or replicate to a service provider environment (aka "public cloud").

By the way, DR via backup or replication to a service provider environments definitely seem to become increasingly popular these days... last month, we have reached 1000 of service provider partners using our products - and many of them provide their customers with backup and replication targets. I had meetings with a couple of them in Vegas during VMworld 2011, and it was an eye opener for me.

On a side note - hope this makes you smile - at VMworld 2011, where every other banner had "cloud" written on it, Veeam Backup & Replication v6 won the Best New Technology (sic!) award without mentioning word "cloud" once in the submission

[vmexpert]

can you please elaborate on the Superb results?

We are currently running veeam 5 in our dr site and pulling our replicas over a 6 mb wan link

last night there was a sql vm that had 10 gb of data changes and it took about 8hrs and 22 min

can you offer some details on your data size and speed and time?

I will be sure to share my results here once I go production use, but it is clear that my offsite backups are roughly 5 times faster now.

Did some quick math with your numbers, and they look very reasonable (for v5)... in the end, there is only as much data as you can pipe through the 6mbps link in 8 hours - those 10 gigs of uncompressed data must be physically transported over WAN. By all means, you should upgrade to v6 ASAP - the improvement will be massive! You will thank me later <g>

Also, frankly, I would consider 6mbps as "very slow" even for home use... this is hardly enough to even watch HD via Apple TV. You definitely need to consider investing in your WAN links in this "cloudy" world <g> especially if you are serious about protecting high change rate workloads such as SQL.

By the way, are you using "WAN" optmization in the advanced job settings?

[mike143]

Rep with ATT "Major Account Manager National Business Markets" described ATT provided voice mail as "IN THE CLOUD......". This was in reference of roll over lines and if last line in roll over was dialed directly but busy would it roll to one of the other unused lines. Every time I hear the word CLOUD it makes me cringe. Unless its: "Look at that cloud it looks like a T-Rex".

[Gostev]

We are currently running veeam 5 in our dr site and pulling our replicas over a 6 mb wan link

Comparing to v5 pull replication, you should get up to 10x performance improvement by upgrading to v6 and leveraging the advanced replication.

About 3-4x should come from the network traffic compression (there is no compression with v5 pull replication).
About 2x should come from better utilization of your WAN link (which now looks to be only 50% utilized, unless I failed in math).
Plus some additional improvements from the data transfer protocol optimizations.

Our IT team is backing up a part of our production environment over Atlantic via WAN link throttled to 6Mbps (exactly your scenario). With v6, they are protecting VMs that "v5 could not even touch" (a quote from our backup admin).

[jpaul]

I just posted this on my blog... http://jpaul.me/?p=2718

It describes the layout differences in v6 compared to v5

Gostev... If I don't have anything right in my post let me know

[ctchang]

Anton,

Do we have to upgrade to ESXi 5 in order to enjoy this 10x replication?
Does B&R v6 replication x10 improvement apply to ESX 4.1 as well?

Thx.

[Vitaliy S.]

The new replication engine works no matter of what vSphere version you're running on your source and target servers, or host type (ESX or ESXi)

[Gostev]

Gostev... If I don't have anything right in my post let me know

Hi Justin - this looks to be great v6 replication 101, added this to FAQ as well. Thank you!

[mcwill]

There was a request for some real world figures, we have now performed our first replication after setting up all the replicas at the remote site.

The link is approx 20Mbs (fibre at main site - ADSL at DR site), latency 13-16ms.

The Replication Job was for 24 VM with a total size of 575 GB which took 91 minutes which is less than 4 minutes per VM!
Other stats for the run are Data Read = 46.4GB and Data Transferred = 2.5GB

Compare this to the setup we had before of backing up locally then running rsync to replacte the backups over the same link. Ignoring the time to generate the backups, the rsync job would normally take 3 hours and transfer 14GB of data.

[Gostev]

Thanks Ian, great improvement. Curious if you rsynced backups before, why did you decide to switch to replication with v6 (versus doing backup over WAN)?

[mcwill]

The initial plan for Veeam when purchased was to use it to maintain our DR site, however replication jobs were just too slow in V4/V5. We therefore used rsyncing of backups to maintain backups offsite with the knowledge that if required there would be a delay in restoring backups to DR environment.

With V6 we can now obtain our original goal of having the DR environment kept up to date automatically.

[murdocmk]

Speaking of WAN backups vs. WAN replication ... in v6 should we notice any significant benefit (in terms of backups times and WAN traffic) to using replication vs. a WAN backup? We have customers that are using Veeam for off-site backups and DR and find the recovery time adequate using vPower and restoring VMs. If they simply keep their jobs configured that way in v6 will they be missing out on WAN traffic improvements that replication offers, or are those improvements now essentially the same for replication and backups?

[Gostev]

Great question!

All improvement are a part of core engine, so both backup and replication over WAN in v6 are exactly identical in terms of traffic and performance. If they are using Linux target with v5 for off-site backup (which is really the only way to do it efficiently with v5), then they do not need to change anything in order to take advantage of WAN optimizations v6 engine brings.

However, I don't necessarily agree that vPowering-up a backup is good approach. Sure, it's perfect to handle single VM outages, but what if a big part of the site goes down? vPower NFS will not provide sufficient I/O performance for large scale outages. Also, quite frankly, all the new v6 replication functionality (failback, Re-IP) makes the whole experience much more streamlined. It takes just a few clicks to failover a bunch of VMs and have them running on DR site at full speed not constrained by vPower NFS, and then bring them back over to production with another few clicks... you cannot get anything like this with the backups.

[tsightler]

Just in case it wasn't clear from Anton's post, if they were NOT previously using a Linux target for offsite backups with V5, then with V6 you should add either a Windows or Linux repository on the remote side to optimize the traffic and there will be a huge benefit.

With V6 the traffic compression for replication (proxy->proxy) is identical to backup (proxy->repository), so as long as there is a remote repository of some sort, you get the benefit.

[Gostev]

...and since both proxy and repository use the same data mover agent, all other WAN optimizations (multiple connections and enhanced protocol) also apply no matter of what components are "talking" as per Tom's post.

[VER]

v5 replication: 20 to 24 hours for a 1 TB fileserver with an already excluded drive for swapfile en temp directory.
v6 replication: between 40 and 50 minutes. This v6 job includes the drive with the temp dir's and has only the swapfile excluded.

Please note that the v5 replication was achieved using push replication to an ESXi target which is not recommended.

Other replication jobs show a similar decrease is running time.

[tsightler]

Assuming you system is Windows then you don't even need to exclude the swapfile, V6 does this automatically.

[Gostev]

Here you go, 25 times faster comparing to the "worst case" v5 scenario. Looks like we have over delivered our marketing promise of "10x faster" replication!

[bdoellefeld]

SQL 2008 (our primary)
ver. 5: would take 14 hours
ver. 6: < 1 hour

Impressed!