Availability for the Always-On Enterprise
Post Reply
dbrett
Novice
Posts: 9
Liked: never
Joined: Oct 29, 2009 9:27 am
Full Name: Dan Brett
Contact:

iSCSI (Equallogic) performance

Post by dbrett » Oct 29, 2009 9:38 am

Like most I was greatly anticpating the release of v4, but, I'm after testing last night it seems I'm having issues with the speed of the full backups. I suspect this is down to setup rather than Veeam but I know a couple of EQL users are having good results so thought I'd post up.

I did a couple of test backups of complete VM's, one via network, and one via SAN. The network backup returned 15MB/sec, the SAN backup returned 17MB/sec without jumbo frames enabled and 19MB/sec with jumbo frames. Based on previous forum posts I'd have thought the SAN backup should've returned two or three times more throughput.

Network connectivity is via 1Gbps connections to a Cisco 2960G, SAN connectivity is via 1Gbps connections to a Cisco 3750.

At the moment I have only cabled up one connection into the SAN to avoid any potential multipathing issues. I can't think of any particular reason why the performance should be so slow, the only thing I can think of is that since the Equallogic group consists of two arrays then I should possibly cable up two connections so the array can load balance.

Can any EQL users offer some tips?

Thanks.

McClane
Enthusiast
Posts: 93
Liked: 10 times
Joined: Jun 20, 2009 12:47 pm
Contact:

Re: iSCSI (Equallogic) performance

Post by McClane » Oct 29, 2009 10:50 am

What compression level was used? I have problems on my rather old and slow backup server with compression set to "Optimal".

dbrett
Novice
Posts: 9
Liked: never
Joined: Oct 29, 2009 9:27 am
Full Name: Dan Brett
Contact:

Re: iSCSI (Equallogic) performance

Post by dbrett » Oct 29, 2009 11:04 am

Compression was set to 'optimal' too. - I'll change that and see what happens.

tsightler
Veeam Software
Posts: 5199
Liked: 2082 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: iSCSI (Equallogic) performance

Post by tsightler » Oct 29, 2009 2:01 pm

I'm an EQL user. We just installed Veeam 4 on an old Dell 2650 so not much CPU horsepower. We're using vStorage API mode and yesterday, around 5PM, I started two backups that contain a total of about 40 VM's that are almost 4TB of total storage. We're using optimal compressions and seeing 25-30MB/sec per backup, so a total of about 60MB/sec. We're backing up to a Linux target at a remote site (connected via 1Gb MetroE) and we're seeing about 70-90% CPU utilization on the Veeam server, and about 50% CPU usage on the Linux host, a Dell 2950 with 2Ghz quad-core processor.

This is obviously a little slow, but we think it will be plenty fast for the incremental runs so we're going to wait and see. With Veeam 4 "Optimal" is much more of a CPU hog than with Veeam 3. My understanding is because, with Veeam 3, "Optimal" meant optimal performance when used in network backup mode as this placed minimal load on the ESX host. With Veeam 4, "Optimal" seems to have a new definition of being "Optimal" for vStorage API mode with backup server that has a modern processor. In our test this means that "Optimal" compression in Veeam 4 preforms similar, although not quite as good, as "Best" compression on Veeam 3, but needs a powerful CPU.

dbrett
Novice
Posts: 9
Liked: never
Joined: Oct 29, 2009 9:27 am
Full Name: Dan Brett
Contact:

Re: iSCSI (Equallogic) performance

Post by dbrett » Oct 29, 2009 2:14 pm

Hmmm, just tried a quick lunchtime run using 'low' or 'best' compression. 'Low' gave us 10MB/sec, 'best' was around the 15MB/sec mark.

Tom, thanks for the information. Are you using a single NIC into your storage? And do you have more than one EQL array in the group? The backup server is a DL380 G4, dual Xeon 3Ghz. Not blindingly fast but similar to a 2650.

I noticed that the network performance was spiking between 25% and 50% almost continuously, that may be indicative of CPU issues (ie drag down some data, process the data and repeat) but equally could be something else. I'll have to dig out perfmon and see if the issue is local or elsewhere. :roll:

mealan
Lurker
Posts: 1
Liked: never
Joined: Oct 29, 2009 2:36 pm
Full Name: Alan Harrington
Contact:

Re: iSCSI (Equallogic) performance

Post by mealan » Oct 29, 2009 2:44 pm

We are getting 50-55MB/s on our ps5000e,

the veeam backup server is a VM living on local storage....


do you have jumbo frames on? are your vm's or vol thin pro?

tsightler
Veeam Software
Posts: 5199
Liked: 2082 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: iSCSI (Equallogic) performance

Post by tsightler » Oct 29, 2009 5:03 pm

I'm actually using a dual port Qlogic iSCSI HBA and Equallogic's host integration tools with load balancing. We have multiple arrays, but LUN's are not spread across them. The two backups I mentioned above were both running from the same PS400E (I guess that's now a PS5000E with 750GB SATA drives). If I just run a single backup we generally see 45-50MB/sec, so I think we're probably topping out the backup hardware.

Our Veeam 3 server was running on an IBM HS20, which has a little more horsepower than the Dell 2650, but not a lot. Using VCB we would typically see 30-35MB/sec, but with optimal compression we could run 6 jobs in parallel and get reasonably close to topping out the 2 HBA ports at about 180MB/sec on that single host.

dbrett
Novice
Posts: 9
Liked: never
Joined: Oct 29, 2009 9:27 am
Full Name: Dan Brett
Contact:

Re: iSCSI (Equallogic) performance

Post by dbrett » Oct 29, 2009 5:10 pm

Thanks guys - I think I've identified the issue but am just about to run another test. Looks like the disks on the backup server are flat out so am going to switch to different storage and re-run the tests.

Jumbo frames are on, as is flow control, all vm's are converted from v4 hardware and thick provisioned.

I'll run some more tests and see how we get on, but I think the backup server is the cause.

dbrett
Novice
Posts: 9
Liked: never
Joined: Oct 29, 2009 9:27 am
Full Name: Dan Brett
Contact:

Re: iSCSI (Equallogic) performance

Post by dbrett » Oct 30, 2009 9:58 am

Changed the storage and now seeing 30MB/sec for full backups with a single nic. :D

Incrementals running at 170MB/sec. :mrgreen:

Post Reply

Who is online

Users browsing this forum: Bing [Bot], Majestic-12 [Bot], PeterCNZ and 46 guests