Discussions specific to the VMware vSphere hypervisor
m1kkel
Enthusiast
Posts: 47
Liked: 1 time
Joined: Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen
Contact:

Re: Direct Storage Access FC - slow

Post by m1kkel » Mar 18, 2016 1:38 pm

A-ha. Interessting! Maybe this is not a storage issue after all, at least, not related to a specific storage system. I just got off the phone with vmware, but unfortunately i do not have SDK Entitlement support, which is where the VDDK library resides, so support can't help me.

As i said before, it seems like the speed is capped somewhere. When we use direct san backup, we are utilizing the VDDK system of vmware, so therefore testing speed on a disk inside a VM may give another result. https://www.vmware.com/support/develope ... notes.html

It is also interessting that emachabert can push much more than the rest of us.
What else can we test / investigate?

lightsout
Expert
Posts: 207
Liked: 54 times
Joined: Apr 10, 2014 4:13 pm
Contact:

Re: Direct Storage Access FC - slow

Post by lightsout » Mar 18, 2016 3:14 pm

I actually have multiple NetApp SANs with the same issues, and they are all 8Gb FC/Nexus 5ks. So given what you've seen and what others have seen, my immediate thought is to look at the FC switch!

emachabert
Veeam Vanguard
Posts: 355
Liked: 163 times
Joined: Nov 17, 2010 11:42 am
Full Name: Eric Machabert
Location: France
Contact:

Re: Direct Storage Access FC - slow

Post by emachabert » Mar 19, 2016 9:19 am

Since all the arrays I installed are at different customer locations, I can't give the information as quick as you would like. Next week, I will make a new test on a 7200 using veeam9. I'll give you the results.
Regarding Netapp, I often find the same results as you when I deploy Veeam on that type of array and I haven't been able to push more than 700MB/s (multiple streams) in the biggest setup I have been working with.
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017

emachabert
Veeam Vanguard
Posts: 355
Liked: 163 times
Joined: Nov 17, 2010 11:42 am
Full Name: Eric Machabert
Location: France
Contact:

Re: Direct Storage Access FC - slow

Post by emachabert » Mar 21, 2016 9:12 am 1 person likes this post

So I did a test on a 7200 with 63*450 10K (RAID 5 5+1)
One VM, with only one 60GB disk with real data Inside (to not read zeroes...).
Processing speed 196MB/s.
Average Read speed: 204 MB/s
Max Read speed: 227MB/s.

Trying with two VM gives me twice those numbers and so on until I cap arround 1GB/s wich seems to be the max for that array.
It is correlated to what I see on another setups (6*200 ~ 1200MB/S)
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017

m1kkel
Enthusiast
Posts: 47
Liked: 1 time
Joined: Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen
Contact:

Re: Direct Storage Access FC - slow

Post by m1kkel » Mar 22, 2016 7:46 am

Allright, that means you are seeing the same results as me, and a lot of other people. It seems capped somewhere.

What we all have in common is Veeam (on different versions) and vmWare on different versions. I honestly think this is a limitation in vmware, and not in Veeam. Veeam is just using whatever api and librarys vmware made available to them..

lightsout
Expert
Posts: 207
Liked: 54 times
Joined: Apr 10, 2014 4:13 pm
Contact:

Re: Direct Storage Access FC - slow

Post by lightsout » Mar 24, 2016 8:40 pm

I still think there is something odd happening, but yeah, guess it is not clear what is causing it. :( I will post if I ever find out what!

kte
Expert
Posts: 172
Liked: 7 times
Joined: Jul 02, 2013 7:48 pm
Full Name: Koen Teugels
Contact:

Re: Direct Storage Access FC - slow

Post by kte » Mar 28, 2016 8:50 pm

1,2 GB/s is the 8 GB FC limit , correct? not sure id veeam pulls on multiple paths? Also the 3PAR gives priority to radom IO.
rhe limit per vmdk is maybe a vmware issue,limitation, there are alot of software that have single stream limitations

emachabert
Veeam Vanguard
Posts: 355
Liked: 163 times
Joined: Nov 17, 2010 11:42 am
Full Name: Eric Machabert
Location: France
Contact:

Re: Direct Storage Access FC - slow

Post by emachabert » Mar 29, 2016 7:39 am

In theory, you could pull 2400MB/s, since 3par is active/active and you are using round robin at host level. But as you said, the Inform OS is acting to maintain overall performance for all VVs and hosts. I'll do a test on a full flash 8000 next month and I'll let you know.
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017

kte
Expert
Posts: 172
Liked: 7 times
Joined: Jul 02, 2013 7:48 pm
Full Name: Koen Teugels
Contact:

Re: Direct Storage Access FC - slow

Post by kte » Mar 29, 2016 8:24 pm

8000 is 16 gbit x4 , so source shouldn't be any issue

emachabert
Veeam Vanguard
Posts: 355
Liked: 163 times
Joined: Nov 17, 2010 11:42 am
Full Name: Eric Machabert
Location: France
Contact:

Re: Direct Storage Access FC - slow

Post by emachabert » Mar 30, 2016 7:09 am

I was more thinking about the per vmdk speed and not the overall speed :D
Veeamizing your IT since 2009/ Vanguard 2015,2016,2017

csteb
Lurker
Posts: 2
Liked: never
Joined: Mar 30, 2016 12:05 pm
Contact:

Re: Direct Storage Access FC - slow

Post by csteb » Mar 30, 2016 12:10 pm

HI,

i got the same Problem since we updated to v9.
V8 Backup Time for 6,3 TB data via FC - 5 hours
V9 Backup Time for 6,3 TB data via FC - 8 hours

Vitaliy S.
Veeam Software
Posts: 21537
Liked: 1281 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Direct Storage Access FC - slow

Post by Vitaliy S. » Mar 30, 2016 1:30 pm

csteb, the OP is not discussing difference between v8 and v9. It seems like you're facing something different. Did you contact our support team with both job logs for review (from v8 and from v9)?

csteb
Lurker
Posts: 2
Liked: never
Joined: Mar 30, 2016 12:05 pm
Contact:

Re: Direct Storage Access FC - slow

Post by csteb » Mar 30, 2016 2:51 pm

That's correct, but the FC performance decreased with V9. There maybe relation.
Ive contacted the Support without success. The ticket is still open.

Vitaliy S.
Veeam Software
Posts: 21537
Liked: 1281 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Direct Storage Access FC - slow

Post by Vitaliy S. » Mar 30, 2016 2:55 pm

Please post your ID, so that we could always reference to it when discussing this internally.

m1kkel
Enthusiast
Posts: 47
Liked: 1 time
Joined: Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen
Contact:

Re: Direct Storage Access FC - slow

Post by m1kkel » Apr 07, 2016 2:02 pm

My support case is still open, will post results when i have any.

Looking forward hearing from: emachabert on the 3par 8000 results single threaded.

I agree, that this seems like there is a max throughput pr. single treaded stream..

Post Reply

Who is online

Users browsing this forum: DonZoomik, Exabot [Bot] and 16 guests