Veeam 8 Performance Issue

VMware specific discussions

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Tue Oct 27, 2015 2:56 am

Hi Chris,

BL490c Gen9
Single dual-port HBA in our blade server. We have a single HP QMH2672 16Gb FC HBA. We have it in Mezzanine Slot 1 in our blades which maps to Virtual Connect Bays 3 and 4.

We are using Fibre Switches.. what makes you suspect that we are not ? :)

Still waiting for the SAN guy to respond. Will let you know as soon as gives me the model and OS of the 3PAR.

Is this what our 3PAR connectivity should look like?

Image
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Tue Oct 27, 2015 3:03 am

emachabert wrote:You should see 4 active path (on a two node system), you should look to your zoning configuration and/or the cabling into the fabrics.
Did you apply the 3par best practices regarding zonning and cabling ? Like the one regarding port persostence for example?


Thanks Eric... I'll check with the Storage Admin.

I'm unsure if they applied the 3Par best practices (I'd like to think so)... but I will need to talk it through with them.
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Tue Oct 27, 2015 2:03 pm

Here is the details for the 3PAR.

3PAR Model = HP_3PAR 7400
Nodes = 4
OS = 3.2.1 (MU3)
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Tue Oct 27, 2015 2:47 pm

Here is the details of the virtual connect (SUS)

Image

I can see that we what we have configured is using Active\Passive and not as you have recommended.
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby emachabert » Tue Oct 27, 2015 4:39 pm 1 person likes this post

Your setup is definitively wrong.
If you have a 7400 with 4 nodes, it is just not configured as it should if you only see two path.

Since you are not using Oneview for configuration, I would suggest you take a look at the VirtualConnect cookbook from HP. It will give you all the information regarding the configuration of the shared uplink sets.

Mixing uplink ports from both VC modules in the samed shared uplink set will always give you an active/passive configuration. You should have one shared uplink set per VC module. Each set carying the same set of vlans.
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Re: Veeam 8 Performance Issue

Veeam Logoby chjones » Thu Oct 29, 2015 1:09 am

I have to agree with Eric. Using ports from both VC modules in the same shared uplink set will result in an Active/Passive configuration. We have two shared uplink sets, one for each VC module.

We are also running 3PAR OS 3.2.1 (MU3), along with Patches 17, 18 and 21.

Again, I agree with Eric that the HP Virtual Connect Cookbook is a great source for information. I used this extensively to design and implement our implementation to ensure an active/active configuration and maximum throughput and redundancy.

From your fibre diagram, there are a few things:

1. You have two links from the 3PAR to each fibre switch. So at a minimum your host would see 4 paths to the 3PAR, but your earlier screenshot only shows 2 paths. How are the zones configured on the fibre switches?
2. If you have 4 Nodes (0, 1, 2 and 3) you should have one uplink from each node to each of the fibre switches. So you'd have 4 uplinks to Fibre Switch 1, and 4 to Fibre Switch 2.
3. The best practice for fabric zoning for the 3PAR is not to create a single alias for the 3PAR and add all controller WWNs to it, but to create a separate alias for each controller WWN and then create a zone which includes your host WWN alias and the four aliases for each of the 3PAR controllers. So you end up with:

Fibre/Fabric Switch 1

Aliases:
Blade Server Host - Host HBA Port 1 WWN)
3PAR Node 0 - Controller Node 0 HBA 0 WWN
3PAR Node 1 - Controller Node 1 HBA 0 WWN
3PAR Node 2 - Controller Node 2 HBA 0 WWN
3PAR Node 3 - Controller Node 3 HBA 0 WWN

Zone:
Host Alias and the 4 3PAR Node Alias

This should be repeated on the second fibre switch, but with the second HBA on the host server and 3PAR controller nodes. There is where you end up with 4 paths to the 3PAR via each HBA port from your blade server, so your blade should see 8 paths.

As I think I mentioned before, if you have 4 nodes in a 3PAR then you have 2 controller shelves in the storage array. One shelf contains Nodes 0 and 1, and the other has 2 and 3. One controller shelf manages half of the disks in the system, and the other controller shelf controls the other half. If you only have zoning from your server to half the 3par controller nodes then you are not accessing the nodes directly that can access all of the disks. If you request data from a disk that is not controlled by the pair of controllers you are zoned to then those controllers have to talk to the other controllers via the 3PAR backplane and request the data from them, which is passed back to the controller your host can see, and then back to your host. I didn't expect this would impact performance too much but my testing shows it has as massive impact.

Overall I'd be doing the following:

1. Create separate shared uplink sets for each VC module
2. Ensure all Shared Uplink sets are active/active
3. Ensure each 3PAR controller node is cabled to each fibre switch
4. Zone each host HBA port to all 4 3PAR controller nodes
5. Ensure you can see 8 paths to the 3PAR
chjones
Enthusiast
 
Posts: 83
Liked: 25 times
Joined: Tue Oct 30, 2012 7:53 pm
Full Name: Chris Jones

Re: Veeam 8 Performance Issue

Veeam Logoby kte » Fri Oct 30, 2015 6:59 am

flex10 configuration is for the networking so nothing to do with the San, but you ned to check the zoning on the sanswitches and also the host configuration on the 3 par that he sees both HBA 's per host, so 4 wwn's
kte
Expert
 
Posts: 172
Liked: 7 times
Joined: Tue Jul 02, 2013 7:48 pm
Full Name: Koen Teugels

Re: Veeam 8 Performance Issue

Veeam Logoby emachabert » Fri Oct 30, 2015 3:14 pm

kte wrote:flex10 configuration is for the networking so nothing to do with the San, but you ned to check the zoning on the sanswitches and also the host configuration on the 3 par that he sees both HBA 's per host, so 4 wwn's


Since he is targeting a SO4900 using Ethernet (FiberChannel Catalyst coming soon on Veeam), we advise him to also enhance his network connectivity by switching from active/passive uplinks to active/active. I don't think we were making him believe flex10 is for FC SAN (it could have been for ISCSI or FCoE. :D)
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Mon Nov 02, 2015 3:37 pm 1 person likes this post

Thats correct... we understand that Flex-10 has nothing to do with SAN.

We have some current restrictions that will prevent us moving to an A\A configuration on the Flex-10 modules. We will get that corrected as soon as those operational restrictions are lifted.
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby kte » Mon Nov 02, 2015 7:12 pm

both flex-10 are active also on the blades , both 10 gbit nics are active only the uplinks are active passive, but 2x10gbit, it guess this is enough for a couple of TB/h
kte
Expert
 
Posts: 172
Liked: 7 times
Joined: Tue Jul 02, 2013 7:48 pm
Full Name: Koen Teugels

Re: Veeam 8 Performance Issue

Veeam Logoby albertwt » Tue Nov 03, 2015 1:21 am

Jack1874 wrote:Thats correct... we understand that Flex-10 has nothing to do with SAN.

We have some current restrictions that will prevent us moving to an A\A configuration on the Flex-10 modules. We will get that corrected as soon as those operational restrictions are lifted.


Hi Jack,

Let us know with the update on your end.
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Tue Nov 03, 2015 9:43 pm

Thanks Guys.. just running through a few more tests and will let you know the outcome.
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby Jack1874 » Thu Nov 05, 2015 7:16 pm 1 person likes this post

Hi Guys, we contacted HP to "double check" our path config and here is what they said ...

I reached out to the HPE Installation and Deployment Services (IDS) group. They provided the attached 3PAR zoning reference. I also attached a 3PAR Windows 2012 Implementation Guide.
The following examples indicate a host connection to all nodes is not required. Please let me know if you need further clarification regarding 3PAR zoning


Image

Image

Image

Thoughts??
Jack1874
Enthusiast
 
Posts: 88
Liked: 4 times
Joined: Sat Oct 17, 2015 3:32 pm
Location: Canada
Full Name: Stuart Little

Re: Veeam 8 Performance Issue

Veeam Logoby emachabert » Fri Nov 06, 2015 9:57 am

Minimum required (for me) for a Host is to make him see each node, therefore one port per node. You should have at least 4 active paths in a 4 nodes system.

But you also want performance (in case of switch failure for example), and spread the load (and the queue depth) accross multiple ports on each node, thanks to the round robin path selection policy.
Usually we use a minimum of 2 frontend ports per node, therefore we end up with 8 active paths.

I have dozens of 3Par arrays configured like that and they are delivering the best availability and performance you could get.
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Previous

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 17 guests