Direct Storage Access FC - slow

VMware specific discussions

Re: Direct Storage Access FC - slow

Veeam Logoby m1kkel » Fri Mar 18, 2016 1:38 pm

A-ha. Interessting! Maybe this is not a storage issue after all, at least, not related to a specific storage system. I just got off the phone with vmware, but unfortunately i do not have SDK Entitlement support, which is where the VDDK library resides, so support can't help me.

As i said before, it seems like the speed is capped somewhere. When we use direct san backup, we are utilizing the VDDK system of vmware, so therefore testing speed on a disk inside a VM may give another result. https://www.vmware.com/support/developer/vddk/VDDK-1.0-Relnotes.html

It is also interessting that emachabert can push much more than the rest of us.
What else can we test / investigate?
m1kkel
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Thu Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen

Re: Direct Storage Access FC - slow

Veeam Logoby lightsout » Fri Mar 18, 2016 3:14 pm

I actually have multiple NetApp SANs with the same issues, and they are all 8Gb FC/Nexus 5ks. So given what you've seen and what others have seen, my immediate thought is to look at the FC switch!
lightsout
Expert
 
Posts: 185
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Direct Storage Access FC - slow

Veeam Logoby emachabert » Sat Mar 19, 2016 9:19 am

Since all the arrays I installed are at different customer locations, I can't give the information as quick as you would like. Next week, I will make a new test on a 7200 using veeam9. I'll give you the results.
Regarding Netapp, I often find the same results as you when I deploy Veeam on that type of array and I haven't been able to push more than 700MB/s (multiple streams) in the biggest setup I have been working with.
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Re: Direct Storage Access FC - slow

Veeam Logoby emachabert » Mon Mar 21, 2016 9:12 am 1 person likes this post

So I did a test on a 7200 with 63*450 10K (RAID 5 5+1)
One VM, with only one 60GB disk with real data Inside (to not read zeroes...).
Processing speed 196MB/s.
Average Read speed: 204 MB/s
Max Read speed: 227MB/s.

Trying with two VM gives me twice those numbers and so on until I cap arround 1GB/s wich seems to be the max for that array.
It is correlated to what I see on another setups (6*200 ~ 1200MB/S)
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Re: Direct Storage Access FC - slow

Veeam Logoby m1kkel » Tue Mar 22, 2016 7:46 am

Allright, that means you are seeing the same results as me, and a lot of other people. It seems capped somewhere.

What we all have in common is Veeam (on different versions) and vmWare on different versions. I honestly think this is a limitation in vmware, and not in Veeam. Veeam is just using whatever api and librarys vmware made available to them..
m1kkel
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Thu Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen

Re: Direct Storage Access FC - slow

Veeam Logoby lightsout » Thu Mar 24, 2016 8:40 pm

I still think there is something odd happening, but yeah, guess it is not clear what is causing it. :( I will post if I ever find out what!
lightsout
Expert
 
Posts: 185
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Direct Storage Access FC - slow

Veeam Logoby kte » Mon Mar 28, 2016 8:50 pm

1,2 GB/s is the 8 GB FC limit , correct? not sure id veeam pulls on multiple paths? Also the 3PAR gives priority to radom IO.
rhe limit per vmdk is maybe a vmware issue,limitation, there are alot of software that have single stream limitations
kte
Expert
 
Posts: 172
Liked: 7 times
Joined: Tue Jul 02, 2013 7:48 pm
Full Name: Koen Teugels

Re: Direct Storage Access FC - slow

Veeam Logoby emachabert » Tue Mar 29, 2016 7:39 am

In theory, you could pull 2400MB/s, since 3par is active/active and you are using round robin at host level. But as you said, the Inform OS is acting to maintain overall performance for all VVs and hosts. I'll do a test on a full flash 8000 next month and I'll let you know.
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Re: Direct Storage Access FC - slow

Veeam Logoby kte » Tue Mar 29, 2016 8:24 pm

8000 is 16 gbit x4 , so source shouldn't be any issue
kte
Expert
 
Posts: 172
Liked: 7 times
Joined: Tue Jul 02, 2013 7:48 pm
Full Name: Koen Teugels

Re: Direct Storage Access FC - slow

Veeam Logoby emachabert » Wed Mar 30, 2016 7:09 am

I was more thinking about the per vmdk speed and not the overall speed :D
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017
emachabert
Veeam Vanguard
 
Posts: 354
Liked: 163 times
Joined: Wed Nov 17, 2010 11:42 am
Location: France
Full Name: Eric Machabert

Re: Direct Storage Access FC - slow

Veeam Logoby csteb » Wed Mar 30, 2016 12:10 pm

HI,

i got the same Problem since we updated to v9.
V8 Backup Time for 6,3 TB data via FC - 5 hours
V9 Backup Time for 6,3 TB data via FC - 8 hours
csteb
Lurker
 
Posts: 2
Liked: never
Joined: Wed Mar 30, 2016 12:05 pm

Re: Direct Storage Access FC - slow

Veeam Logoby Vitaliy S. » Wed Mar 30, 2016 1:30 pm

csteb, the OP is not discussing difference between v8 and v9. It seems like you're facing something different. Did you contact our support team with both job logs for review (from v8 and from v9)?
Vitaliy S.
Veeam Software
 
Posts: 19575
Liked: 1106 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Direct Storage Access FC - slow

Veeam Logoby csteb » Wed Mar 30, 2016 2:51 pm

That's correct, but the FC performance decreased with V9. There maybe relation.
Ive contacted the Support without success. The ticket is still open.
csteb
Lurker
 
Posts: 2
Liked: never
Joined: Wed Mar 30, 2016 12:05 pm

Re: Direct Storage Access FC - slow

Veeam Logoby Vitaliy S. » Wed Mar 30, 2016 2:55 pm

Please post your ID, so that we could always reference to it when discussing this internally.
Vitaliy S.
Veeam Software
 
Posts: 19575
Liked: 1106 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Direct Storage Access FC - slow

Veeam Logoby m1kkel » Thu Apr 07, 2016 2:02 pm

My support case is still open, will post results when i have any.

Looking forward hearing from: emachabert on the 3par 8000 results single threaded.

I agree, that this seems like there is a max throughput pr. single treaded stream..
m1kkel
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Thu Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen

PreviousNext

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 24 guests