Network mode performance on vSphere 6.5

Discussions specific to VMware vSphere hypervisor

Network mode performance on vSphere 6.5

Veeam Logoby pmaa » Mon Jul 24, 2017 8:08 am

I've learned that vSphere 6.5 now enforces NBDSSL for network mode. From what I've read, on previous versions you could expect at least about 25 MB/s in network mode on gigabit ethernet and maybe more with tuning.

I'm getting 5 MB/s with vSphere 6.5. Is this the new normal or should I expect a bit more?
pmaa
Novice
 
Posts: 7
Liked: never
Joined: Mon Jul 24, 2017 7:44 am

Re: Network mode performance on vSphere 6.5

Veeam Logoby DGrinev » Mon Jul 24, 2017 10:51 am

Hi pmaa and welcome to the community!

Can you share the bottleneck stats of the job, that should help to understand the cause of the slow performance. Thanks!
DGrinev
Veeam Software
 
Posts: 678
Liked: 73 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

Re: Network mode performance on vSphere 6.5

Veeam Logoby pmaa » Tue Jul 25, 2017 8:35 am

Hi DGrinev,

Bottleneck is source 99% and also target is 98%. Proxy and network are very low with only a few %.

I tested setting up the VMwareNBDCompressionLevel registry key with value 3 and the processing rate went up to 6 MB/s with peaks of 9 and 11 MB/s. Now the bottleneck is target 99%, source 34%, proxy 11% and network 4%.

Edit: This is a replication job between local hosts.
pmaa
Novice
 
Posts: 7
Liked: never
Joined: Mon Jul 24, 2017 7:44 am

Re: Network mode performance on vSphere 6.5

Veeam Logoby pmaa » Tue Jul 25, 2017 9:47 am

Another replication job using VMwareNBDCompressionLevel of 3 is now running with processing rate of 7 MB/s. Bottleneck stats are: Source 7%, Proxy 9%, Network 0%, Target 49%. So with this one it definitely seems that the infrastructure is keeping up easily but perhaps NBDSSL is the bottleneck.
pmaa
Novice
 
Posts: 7
Liked: never
Joined: Mon Jul 24, 2017 7:44 am

Re: Network mode performance on vSphere 6.5

Veeam Logoby DGrinev » Tue Jul 25, 2017 5:00 pm

Hi,

The reduction shouldn't be significant about 10% - 20% than ordinary NBD.
Overall processing rate depends on different factors such as read or write speed, configuration of NICs and so on, ideally you can expect about 25 MB/s in network mode.

During the last run the bottleneck was Target, which means 49% of the time the job spent writing data to the target storage.
Please review the bottleneck analysis post for additional information. Thanks!
DGrinev
Veeam Software
 
Posts: 678
Liked: 73 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

Re: Network mode performance on vSphere 6.5

Veeam Logoby pmaa » Tue Jul 25, 2017 8:06 pm

Ok, I'll open a support case, although this VMware KB isn't too encouraging: https://kb.vmware.com/kb/2147768
pmaa
Novice
 
Posts: 7
Liked: never
Joined: Mon Jul 24, 2017 7:44 am

Re: Network mode performance on vSphere 6.5

Veeam Logoby pmaa » Wed Jul 26, 2017 2:36 pm

Support determined that it's indeed a limitation of NBDSSL in vSphere 6.5. So 5 MB/s is currently to be expected with gigabit ethernet.
pmaa
Novice
 
Posts: 7
Liked: never
Joined: Mon Jul 24, 2017 7:44 am

Re: Network mode performance on vSphere 6.5

Veeam Logoby DGrinev » Wed Jul 26, 2017 4:19 pm

In case you are replicating between two local hosts, why don't you consider to use Hotadd transport mode? Thanks!
DGrinev
Veeam Software
 
Posts: 678
Liked: 73 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

Re: Network mode performance on vSphere 6.5

Veeam Logoby pmaa » Thu Jul 27, 2017 7:44 am

For the moment we are replicating between local hosts but will move to offsite replication later. Hot-add works with offsite if there is backup proxy on both ends right? Currently we don't have enough Windows VM's to do Hot-add, so unfortunately we have to get Windows Server licenses just for that.
pmaa
Novice
 
Posts: 7
Liked: never
Joined: Mon Jul 24, 2017 7:44 am

Re: Network mode performance on vSphere 6.5

Veeam Logoby DGrinev » Thu Jul 27, 2017 9:39 am

pmaa wrote:Hot-add works with offsite if there is backup proxy on both ends right?

Yes, the proxy role should be assigned to a VMs on both sites. Thanks!
DGrinev
Veeam Software
 
Posts: 678
Liked: 73 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

Re: Network mode performance on vSphere 6.5

Veeam Logoby DonZoomik » Mon Jul 31, 2017 7:10 am

How about a Veeam hotfix to include updated VDDK. It seems that this was fixed in April: http://pubs.vmware.com/Release_Notes/en ... notes.html
6.5U1 would be nice but 6.5b has been out for months.
DonZoomik
Enthusiast
 
Posts: 30
Liked: 9 times
Joined: Fri Nov 25, 2016 1:56 pm

Re: Network mode performance on vSphere 6.5

Veeam Logoby foggy » Mon Jul 31, 2017 12:36 pm

pmaa wrote:Currently we don't have enough Windows VM's to do Hot-add, so unfortunately we have to get Windows Server licenses just for that.

You can use any existing VM as hotadd proxy, no need to have a dedicated one.
foggy
Veeam Software
 
Posts: 15394
Liked: 1142 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Network mode performance on vSphere 6.5

Veeam Logoby foggy » Mon Jul 31, 2017 12:40 pm

DonZoomik wrote:How about a Veeam hotfix to include updated VDDK.

You're right that switch to NBDSSL was fixed in a later VDDK version. Chances are it will be included in v9.5 Update 3.
foggy
Veeam Software
 
Posts: 15394
Liked: 1142 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Network mode performance on vSphere 6.5

Veeam Logoby tmtech » Tue Aug 01, 2017 7:47 pm

I'm running vbr 9.5 u2 on server 2016 datacenter with one proxy, VMware esxi 6. After lots of troubleshooting final verdict was to remove cdrom and sata controller from vbr vm's.
tmtech
Lurker
 
Posts: 2
Liked: never
Joined: Tue Aug 01, 2017 1:15 pm
Full Name: Tmtech


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 1 guest