Comprehensive data protection for all workloads
Post Reply
dfowler
Influencer
Posts: 11
Liked: never
Joined: Mar 14, 2012 4:10 pm
Full Name: David Fowler
Contact:

Performance expectations with EMC VNXe3100

Post by dfowler »

Hello,

Case ID#5177812

I have recently installed Veeam v6 to backup Vmare VMs hosted on ESXi v5 servers. Veeam is installed on physical server i.e. HP DL380 G5 (2 x Quad core CPUs, 12 GB RAM) with an iSCSI connection to the EMC (VNXe3100) SAN hosting the VMs. The Veeam backup proxy server is on the same physical server and configured to use 'Direct SAN access' as the Transport mode.

First ever backup was run against 3 VMs with 720 GB of thin provisioned disks (203 GB used) took 5 hrs, 52 minutes to complete at 9 MB/s. The Performance/Bottleneck statistics were: Source 99% > Proxy 51% > Network 4% > Target 0%.

As recommended by Veeam support I applied the latest patch (6.0.0.181). A second backup against 3 different VMs with 720 GB of thin provisioned disks (154 GB used) took 57 Minutes at 43 MB/s, which was a huge improvement. The Performance/Bottleneck statistics were: Source 72% > Proxy 42% > Network 37% > Target 5%.

I don't have any prior exposure to Veeam so I am unsure what to expect when it comes to performance; others have reported performance ranging from 60 MB/s to 1 GB/s, which is quite a range. Is 43 MB/s reasonable?

No problems with Veeam support just trying to figure out if the performance behaviour is 'normal'.

Thanks in advance for any responses.

Dave
Jfmoots
Veeam Software
Posts: 215
Liked: 26 times
Joined: Oct 28, 2011 3:26 pm
Full Name: James Moots
Location: Ohio, United States
Contact:

Re: Veeam performance expectations - case ID#5177812

Post by Jfmoots »

That's not bad at all. The number you're quoting sounds like the overall job performance, which is a good reference, but I'm interested in how fast we read the VMs off disk. If you right click on that job, and then select Realtime Statistics, you'll be able to view details on the VMs we backed up. Click on the name of the VM and you'll see even deeper details. How many MB/s are you seeing on each disk on those VMs? I would imagine it's a higher number... but maybe just a little higher.

What types of disks make up that VNX? How many disks are in the array?
tfloor
Veteran
Posts: 270
Liked: 15 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: Veeam performance expectations - case ID#5177812

Post by tfloor »

This setup looks a little bit the same as my: bottleneck source and proxy
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam performance expectations - case ID#5177812

Post by Gostev »

David, 45MB/s is what most people report out of box with 1Gb iSCSI SAN without any iSCSI traffic tuning. As a next step, you may want to read the sticky FAQ topic for some tips on improving iSCSI performance. Some Windows 2008 network stack tweaks, jumbo frames and good switches should bring you all the way up to the line speed. Thanks!
dfowler
Influencer
Posts: 11
Liked: never
Joined: Mar 14, 2012 4:10 pm
Full Name: David Fowler
Contact:

Re: Veeam performance expectations - case ID#5177812

Post by dfowler »

Thanks for the responses.

The VNX solution has a DAP enclosure with 12 x 600 GB 15K SAS drives and a DAE expansion enclosure with 9 x 600 GB 15K SAS drives. Currently we have 2 x datastores at 1TB each. The iSCSI backbone comprises 2 x Brocade FastIron WS624G switches, the ESXi servers are HP DL380 G6s with 2 CPUs (Xeon E5540 6 cores/each) with 96 GB RAM, jumbo frames are enabled.

The VM statistics of the full backup job show read rates between 47 MB/s for the slowest and 56 MB/s for the fastest on the VM disks, which is slightly faster than the overall processing rate of 43 MB/s.

I will take a look at the tuning tips and post my results. Thanks again for the responses.

Dave
cyberwarrior101
Novice
Posts: 6
Liked: never
Joined: May 03, 2012 4:24 pm
Full Name: Ken White
Contact:

Veeam Replication - slowness

Post by cyberwarrior101 »

[merged]

I've just installed VEEAM on an ESXi 5 Server hosted on an EMC VNXe3100. We have additional Volumes for D/R purposes (backups) hosted on a HP LeftHand iSCSI san.

I've started replication and notice that I only get about 30-37MB/S and it continues to show the bottleneck as the source. Any suggestions as to what to change/add that might increase performance? We're trying to replicate core production servers to our colo although at this rate it seems slow.

I should also mention the VM's and the VEEAM R/B server is on the same host both Virtual as I had previously read posts suggesting that the VEEAM server should be a VM also..

We are not using any proxy servers..

Thank You!
Ken
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Replication - slowness

Post by Gostev »

Make sure you are using direct SAN or virtual appliance (hot add) processing mode?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Performance expectations with EMC VNXe3100

Post by dellock6 »

Ken, you wrote you are using replicas and you are not using any proxy servers (besides the Veeam Backup server ifself).
Best practices requires to have at least two proxies for remote replicas like you are doing, one in the production site, connected to the VNX (this can be the Veeam Backup server) and another one at the DR site. Speed differences using two proxies as recommended are usually great.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Performance expectations with EMC VNXe3100

Post by Gostev »

Luca, he mentioned his bottleneck is source, while no-target-proxy issue would cause target being shown as a bottleneck... and he would be getting much slower speed anyway, probably.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Performance expectations with EMC VNXe3100

Post by dellock6 »

You are right, I've gone too fast as soon as I read replica+noproxy, it's becoming a sort of auto-reply :oops:
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
cyberwarrior101
Novice
Posts: 6
Liked: never
Joined: May 03, 2012 4:24 pm
Full Name: Ken White
Contact:

Re: Performance expectations with EMC VNXe3100

Post by cyberwarrior101 »

Sounds good. I'll take a look at the proxy setup and see what's involved in the setup. Documentation available for this online?

One Update: When I originally setup VEEAM, I selected the Enable Application-Aware image processing option - as we're replicating SQL Server and Exchange I wanted to be sure no transactions were lost or corrupted during the copy process. When this was enabled, we saw roughly 20-35MB/Sec. I turned it off per our Vendor's recommendation - speed increased from 20-35MB/Sec to 200+/MB/SEC. In a scenerio where we're replication SQL or Exchange, is it recommended that this be on or off? I want to be sure no transactions are lost during the replication although also need to be sure we can replicate to the Co-location.

Appreciate everyone's input. My apologies if my questions are duplicated, I'm still a newbie :)
Ken
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Performance expectations with EMC VNXe3100

Post by Gostev »

You should always have it on for any Windows VM, and it does not affect backup performance anyway - only how guest quiescence is performed before the VM snapshot is created. While data transfer only starts once the VM snapshot is created.
Post Reply

Who is online

Users browsing this forum: dbaages and 159 guests