Very Slow Backup speeds ISCSI, comparison vs. vReplicator

[floss]

Hi

I wonder if any one can help we run ESX 3.5 to LHN ISCSI cluster and backups pointed to Data Domain device , i cannot get backup speed faster than 10mb sec to the DD device or or 16-18 mb sec to ISCSI disk , IOPS are as 36 disk spindles and i have disabled AV ???I get faster performance from Vranger ???

Love the product and VSS func but this is a show stopper

Also can some one explain in depth how the Replication works , i know Vranger opens up a snapshot in hybrid mode that contains changes that are then applied to the destination which is quite quick , it looks to me like Veeam replication scans the source for changes , which is time consuming for lardge files , for example i cannot have small replication schedules of say 3 hours when it takes 4 hours to scan ...

Any one who can help please

regards

[Gostev]

Hello, we recommend to perform integration with DD over NFS. There is currently known performance if you intergrate via CIFS share due to DD CIFS implementation peculiarities, we are working with DD to resolve this, so this should be addressed soon.

As for the reported speed, you should not compare it directly between products due to different calculation approaches. As can be seen from this most recent blog post, with the same speed reported in UI, Veeam Backup is actually faster than vRanger in terms of time taken to complete the job.

To answer your replication question, here is how our approach is different from Vizioncore vReplicator (I assume this is what you mean since vRanger does not do replication).

1. Yes, vReplicator requires snapshot to be open all the time, this is exactly what allows it to work very fast on incremental replication cycles. However, we have found that this requirement of running VM with snapshots is not acceptable for majority of our customers, because running VM in snapshot mode disables VMotion/DRS/DPM for the VM. This is why we have designed Veeam Backup differently, and do not require that VM is running in snapshot mode. So with our approach, while you get slower incremental replication, you don't loose VMotion/DRS/DPM capabilities.

Also note here (looking into the very near future), Veeam Backup 4.0 will leverage some new vSphere functionality and APIs that would allows us near-CDP replication with very fast replication cycles (10-30 seconds according to the current testing), but still WITHOUT requirement for a VM to run in snapshot mode. Release is expected in Q3, and you can participate in beta for this functionality if you'd like (currently aiming end of mid-July for beta).

2. vReplicator only allows failover to the latest state, and does not provide ability to rollback to an earlier state. This functionality is especially needed in case of small replication intervals, where you will almost never be able to spot the possible software corruption (like virus) in time, so your replica will get infected/corrupted as well shortly after original VM. Such replica is totally useless, so in most cases vReplicator will simply not be able to provide what it is designed to provide - recovery in case of disaster. Thus, vReplicator's ability to perform fast incremental cycles often does not give customers anything good in the larger picture.

3. Proprietary VSS integration, as you mention. If you use Small Business Server, Exchange server, SQL servers, or domain controllers; you need to keep in mind that vReplicator does not provide own VSS implementation and relies on VMware one. Which is in turn very simplistic: does not support app-level quiescence for Windows 2008, and does not implement required custom restore logic for the above-mentioned applications.

4. Last but not least, as I am sure you are aware, vReplicator is licensed separately and per VM, while Veeam Backup provides both backup and replication for one price, and is licensed per socket, with same price as vRanger. Big difference here

5. Not sure if this applied to you, but it was key for some other customers:
a) vReplicator requires full root access to opened between ESX hosts, which can be big compliance concern.
b) vReplicator uses SSH to transport the data, and being a secure protocol Expand Networks Accelerator box does not accelerate this traffic. While with Veeam, results are excellent.

[joemagjr]

I know that this is an old post. We are using 4.0, and getting ready to goto 4.1 very soon. Our proposed DR site only has 2 t-1's and just with our normal traffic is slow, so we have not attempted to set up our replicated san and virtual environment in that location as of yet due to this concern.

We are working with the various vendors to discuss WAN accelerators and optimizers to help us work through some latency issues. These companies optimize certain protocol traffic. Does Veeam 4.x use ssl when performing its replication function? If so, many of these companies need to incorporate the SSL certificate into there appliance to optimize this traffic, is this SSL cert obtainable from the box that host Veeam?

Does anyone have any feedback on better performing WAN Accelerators with Veeam?

thanks for your time

[Gostev]

Hello Joe, Veeam 4.x does not use SSL to transfer data, and our customers leveraging WAN accelerators are reporting absolutely great results.

Here are some existing topics about this:
Anyone used WAN accelleration tools to replicate over WAN?
VEEAM with Riverbed

[joemagjr]

Thanks for the fast response. What type of traffic is veeam seen as on the wire? TCP

[tsightler]

Just to post some comments. Veeam does not use SSL. The actual data is sent unencrypted across the wire. You'll want to make sure you size your WAN acceleration box appropriately. WAN acceleration likely won't help a lot for the first replica, but, assuming your product does DRE (basically caches data on the remote side), then I expect you'll see tremendous improvement from any WAAS solution. We're seeing compression ratios of 90% for most of our replication passes.

Just to be clear, that doesn't translate into being 9x faster however. Latency still plays a role with Veeam, so overall performance improvement is more like 2-4x. For example, replicating our Exchange server generates almost 75GB of transfer in a full day. Even with our 10Mb link that would be 17 hours of transfer time using the entire bandwidth. With WAN acceleration, the total time is about 4 hours, but only 8GB of data is moved across the wire so the link is 50% utilized during the replication.

In other words, without WAN acceleration it would take 17 hours and require 100% utilization, with WAN acceleration it takes 4 hours, and only uses 50% of the link during that time. We really wouldn't be able to use Veeam replication without WAN acceleration, at least not to replicate our highly transactional servers like Exchange.

[glennsantacruz]

For example, replicating our Exchange server generates almost 75GB of transfer in a full day. Even with our 10Mb link that would be 17 hours of transfer time using the entire bandwidth.

Curiosity - are you using Exchange 2007? If so, have you contrasted/compared the difference between a full Veeam replica job versus a "native" Exchange SCR (standby continuous replication) deployment?

[tsightler]

Not yet, currently we're only Exchange 2003. I suspect that log shipping (SCR) almost has to be a win in this scenario, but it does have the disadvantage of being more complicated to setup, and yet another "one-off" solution for replication/failover with it's own recovery steps. Our goal was to have a simple, universal "fail-over" plan that could be implemented with as close to a "flip-the-switch" process as possible. Using a single tool like Veeam rather that implementing every applications own failover option makes this possible, although Veeam doesn't offer as good of a RPO window as many of the application level solutions, it's good enough for our needs.

That being said, I'm sure when we move to Exchange 2010 (we're skipping 2007) we will evaluate our DR options for that environment.

[glennsantacruz]

Thanks for the response - I completely agree with your goal to have a "single-switch" for failover - it would be ideal. In our shop, we've found that "native" replication (although initially cumbersome) does provide both better RPO and RTO than an external replication solution (like Veeam), but I certainly concede that all factors should be weighed in a comparison. For us, we've chosen native replication for Exchange 2007 and Oracle; we are still undecided regarding Microsoft SQL Server... I'd be interested in hearing your feedback on evaluating Exchange 2010 in comparison with Veeam replication.

[tsightler]

We actually use native replication for Oracle as well, at least for our most transactional, critical database. Since it's our ERP system the RPO is quite stringent (pretty much, don't loose any transaction). Not only that, but it's a physical server so Veeam's a no go anyway, and we have the DBA talent onboard that understands that solution and are dedicated to that environment in a failover situation. We do use Veeam for some smaller, lightly loaded, but still production Oracle databases where the RPO was less of a concern.

For everything else, it's Veeam, for now. Many of these systems are more general purpose and are running applications that don't have a dedicated support staff for every application. To develop failover solutions for each one individually just seemed like a nightmare, and most have pretty minimal RPO's so we decided Veeam would do. Exchange probably wouldn't have fit in the category, but message archiving makes a big difference. We archive all email immediately upon arrival, and the archive server is at another site, so we can "afford" to loose a few hours of messages in a completely unexpected disaster. If we couldn't recover up to the point in time due to a catastrophic event like a fire or other major facility issue, we'd simply tell users they'd need to check the archive for messages that arrived during the missing time. They could forward message back to their mailbox if they were important. Not perfect, but a minor event in the case of a true "disaster".