Bad iSCSI performance when backing up

Hyper-V specific discussions

Bad iSCSI performance when backing up

Veeam Logoby SnapperNL » Sun Aug 25, 2013 7:53 pm

Hi,

I have a problem with the backup of an Hyper-V environment with iSCSI storage. I have an HP MSA2324i G2 with two controllers. These controllers (total of 4 1GB ports) are connected to two GB switches. These switches are only used voor iSCSI traffic. The controllers are crosslinked with these switches so that if one switch or controller fails, the other one will pick up. Also connected to these switches are three Hyper-V hosts. In the HP MSA2324i I have installed SSD disks.

For backup purposes I have an QNAP TS-459U+. The QNAP and the three Hyper-V hosts are connected to a different GB switch (Backup Lan). The connection protocol used for the QNAP is also iSCSI.

I have installed VEEAM on a Virtual Machine on one of these three hosts.

The problem I have is that when the backup starts of lets say a VM on Host-1 the iSCSI connection of that host to the HP MSA2324i is up to the maximum in use. This is a big problem, because the remaning VM's are very very slow because of this. I know that in VEEAM i can throttle the lan to the QNAP but not the source iSCSI network.

Is there a way to throttle the source network or source host to process slower. Lets say 50% or so?

I hope someone can help me. Otherwise i can not use VEEAM anymore. And that would be a shame because furthermore It is working perfectly.

Thank you in advance.

Best Regards,
Ernst
SnapperNL
Lurker
 
Posts: 2
Liked: never
Joined: Tue Aug 13, 2013 11:00 pm
Full Name: E. Bijlsma

Re: Bad iSCSI performance when backing up

Veeam Logoby foggy » Thu Aug 29, 2013 11:04 am

Ernst, looks like built-in Veeam B&R network traffic throttling could help to limit the source agent reading speed also. It seems to be implemented on the source side, as we've just performed a quick test limiting the agent speed and data were read with the specified speed. You could try to enable throttling in Veeam and monitor performance of the corresponding network.

Also, you could play with the 'Use multiple TCP/IP connections per job check box' in the Traffic Throttling window, probably would help.

Btw, what Hyper-V and Veeam B&R versions are you using?
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Bad iSCSI performance when backing up

Veeam Logoby SnapperNL » Fri Aug 30, 2013 10:38 pm

Hi Foggy,

Thank you for your response.

I tried what you suggested but nothing seems to work. The subnets of the MSA are 192.168.100.* for ports A and 192.168.101.* for ports B.
The subnet of the backup lan is 192.168.102.*

In the Global Network Traffic Throttling I configured 4 types.

Source - Target
192.168.100.1-192.168.100.255 - 192.168.102.1-192.168.102.255
192.168.102.1-192.168.102.255 - 192.168.100.1-192.168.100.255
192.168.101.1-192.168.101.255 - 192.168.102.1-192.168.102.255
192.168.102.1-192.168.102.255 - 192.168.101.1-192.168.101.255

I turned "Use multiple TCP/IP connections per job" off

Throttling is set to 50Mbps for all types. But when I start one backup the source iSCSI network is fully in use again. It peaks over the 900Mbps.

I'm using Windows 2012 and Veeam B&R 6.5. For some reason I cannot upgrade to 7. The licenses I have seem to be unsupported.

Thank you,
Ernst
SnapperNL
Lurker
 
Posts: 2
Liked: never
Joined: Tue Aug 13, 2013 11:00 pm
Full Name: E. Bijlsma

Re: Bad iSCSI performance when backing up

Veeam Logoby yizhar » Sat Aug 31, 2013 12:44 pm 1 person likes this post

Hi.

It looks like MPIO isn't configured/working optimally in your SAN ISCI network.

I suggest that you also open case with HP support team to review network configuration.

Some things to note:

* Please check if "flow control" is enabled on all ISCSI switch ports.

* If you can add a fast link between SAN switches (a stacking cable for example), then you can configure all SAN ISCSI ports in the same subnet.
This will provide a simpler configuration and also might improve MPIO (as each NIC on host can access every NIC on storage and not only subset).

* You can condiser deploying a physical Veeam server, in "off-host" mode.

* If Veeam remains as VM, you might benefit try both "on-host" or "off-host" modes to check what works better for you.
The difference is whether proxy is the Hyper-V host itself, or Veeam VM.

Again - I suggest that you also open support case at HP to recheck ISCSI SAN configuration, and see if it can be improved.

Yizhar
yizhar
Expert
 
Posts: 179
Liked: 48 times
Joined: Mon Sep 03, 2012 5:28 am
Full Name: Yizhar Hurwitz

Re: Bad iSCSI performance when backing up

Veeam Logoby foggy » Mon Sep 02, 2013 1:26 pm

SnapperNL wrote:192.168.100.1-192.168.100.255 - 192.168.102.1-192.168.102.255
192.168.102.1-192.168.102.255 - 192.168.100.1-192.168.100.255
192.168.101.1-192.168.101.255 - 192.168.102.1-192.168.102.255
192.168.102.1-192.168.102.255 - 192.168.101.1-192.168.101.255

Throttling rules are applied between two Veeam agents. In your case, the source agent is started on the host (as you do onhost backup) and the target agent is started on the repository. Is QNAP is mounted to Veeam B&R? Then the target agent is started on Veeam B&R server itself. So you should create rules between these two servers within your backup subnet (192.168.102.*).
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Bad iSCSI performance when backing up

Veeam Logoby Vitaliy S. » Sat Sep 07, 2013 12:53 pm

SnapperNL wrote:I'm using Windows 2012 and Veeam B&R 6.5. For some reason I cannot upgrade to 7. The licenses I have seem to be unsupported.

License keys from version 6 are not supported with v7, you should generate a new key via licensing portal on our website.
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov


Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 7 guests