high read latencies when backing up with 3PAR integration

VMware specific discussions

high read latencies when backing up with 3PAR integration

Veeam Logoby yunales » Tue Apr 18, 2017 1:12 pm

Hello Veeam community,

I'm experiencing the following issue with our Veeam + production environment:

we are experiencing high read latencies(100-300ms) when SAN backup is active with our HPE 3PAR integration. The latencies can be seen on ALL production VMs and on ALL esxi Servers(50 +) when using 3par commands vstatlun the latency can be seen also.(frontend latency)
vstatvv on the other hand is at 4-5ms.(backend latency)
We can see it since the upgrade from 9.0 to 9.5u1.(two weeks ago) Our HPE 3PAR was upgraded to the latest release 3 weeks ago and is currently running on version 3.2.2 MU4. all backups are disabled currently because it's breaking production vm availability! is there a known issue or hotfix for that?

SR number #02131145

is this a known issue? anyone else got the same problems?
A SR was opened with HPE since the blades,san switches are all from hpe.

regards
yunales
Influencer
 
Posts: 15
Liked: 4 times
Joined: Wed Aug 20, 2014 7:34 pm
Full Name: P K

Re: high read latencies when backing up with 3PAR integratio

Veeam Logoby jmmarton » Tue Apr 18, 2017 1:47 pm

Have you enabled Backup I/O Control (storage latency control)? That could help, especially in light of our advanced data fetcher introduced in v9.5 which accelerates reading data when using BfSS (your 3PAR integration).

Storage Latency Control
https://helpcenter.veeam.com/docs/backup/vsphere/options_parallel_processing.html?ver=95

Advanced Data Fetcher
https://www.veeam.com/blog/enterprise-scalability-enhancements-coming-veeam-availability-suite.html

Joe
jmmarton
Veeam Software
 
Posts: 937
Liked: 100 times
Joined: Tue Nov 17, 2015 2:38 am
Location: Chicago, IL
Full Name: Joe Marton

Re: high read latencies when backing up with 3PAR integratio

Veeam Logoby Gostev » Thu Apr 20, 2017 2:08 am

Just reduce concurrent tasks on your proxy or repository by half for a start, as it looks like you are simply throwing way too much load at your SAN with current amount of parallel tasks. As Joe already noted, 9.5 backup proxies are way more efficient in data retrieval than in previous versions.
Gostev
Veeam Software
 
Posts: 21603
Liked: 2405 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: high read latencies when backing up with 3PAR integratio

Veeam Logoby yunales » Thu May 04, 2017 10:10 am

hello all,

thanks for all the replies. Just wanted you know that in the end the interswitchlink seemed to be saturated and all kind of latencies occured on all host due to that.

We fixed that and added an additional backup proxy to optimize the environment. We are using the Storage Latency control feature too now, which seems to help a little.

regards
yunales
Influencer
 
Posts: 15
Liked: 4 times
Joined: Wed Aug 20, 2014 7:34 pm
Full Name: P K


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 3 guests