-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
network bottleneck, 2~3gbit traffic on 10gbit network
Hi guys,
I'm trying to find out the performance bottleneck here with a vcenter 6.7 and hosts on ESXi 6. I've got the backup server using a single 10Gbit nic that is also used for iscsi and network traffic so I'm trying to understand if VMware doesn't allow the traffic to go over 2-3gbit becuase I have iscsi on the same nic for example.
My storage can do 600mb/sec sequential writes, and I have proxy server running on each host with a fair amount of CPU to be able to hotadd disks (reading data from SSDs) and send the traffic compressed I believe over the network to the backup repository where Veeam also runs.
I get a very flat network traffic rate at 200MB/sec, this when backing up stuff from several places at the same time using different proxies, so it becomes evident the network or the backup server itself is the bottleneck. Accoording to Veeam B&R report after the backup finishes, it is a network bottleneck.
Is there anything I could do in this situation to get this running a bit faster? It seems I have everything to make it runs faster, but it doesn't due to some software limit somewhere?
I'm trying to find out the performance bottleneck here with a vcenter 6.7 and hosts on ESXi 6. I've got the backup server using a single 10Gbit nic that is also used for iscsi and network traffic so I'm trying to understand if VMware doesn't allow the traffic to go over 2-3gbit becuase I have iscsi on the same nic for example.
My storage can do 600mb/sec sequential writes, and I have proxy server running on each host with a fair amount of CPU to be able to hotadd disks (reading data from SSDs) and send the traffic compressed I believe over the network to the backup repository where Veeam also runs.
I get a very flat network traffic rate at 200MB/sec, this when backing up stuff from several places at the same time using different proxies, so it becomes evident the network or the backup server itself is the bottleneck. Accoording to Veeam B&R report after the backup finishes, it is a network bottleneck.
Is there anything I could do in this situation to get this running a bit faster? It seems I have everything to make it runs faster, but it doesn't due to some software limit somewhere?
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
-
- Veteran
- Posts: 298
- Liked: 85 times
- Joined: Feb 16, 2017 8:05 pm
- Contact:
Re: network bottleneck, 2~3gbit traffic on 10gbit network
Do you have Throttling enabled?
-
- Expert
- Posts: 193
- Liked: 47 times
- Joined: Jan 16, 2018 5:14 pm
- Full Name: Harvey Carel
- Contact:
Re: network bottleneck, 2~3gbit traffic on 10gbit network
Hey Mephisto,
Simple test is just iPerf between the two servers; do you see the same speeds there? Since it's a VM, I'd suspect that it's the VMware network stack between the VMs causing you grief. Windows Copy won't be reliable here due to the mischief that Windows and the VMware Tools will do to improve speeds.
Simple test is just iPerf between the two servers; do you see the same speeds there? Since it's a VM, I'd suspect that it's the VMware network stack between the VMs causing you grief. Windows Copy won't be reliable here due to the mischief that Windows and the VMware Tools will do to improve speeds.
-
- Service Provider
- Posts: 43
- Liked: 15 times
- Joined: May 07, 2013 2:50 pm
- Full Name: James Davidson
- Location: Northeast UK
- Contact:
Re: network bottleneck, 2~3gbit traffic on 10gbit network
Sounds like you're using NBD, rather than hot-add or direct SAN.
NBD limited to around 300MB/s when using 10Gb.
More info here:
https://forums.veeam.com/vmware-vsphere ... 39238.html
And
https://kb.vmware.com/s/article/2147768
NBD limited to around 300MB/s when using 10Gb.
More info here:
https://forums.veeam.com/vmware-vsphere ... 39238.html
And
https://kb.vmware.com/s/article/2147768
@jam_davidson
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Re: network bottleneck, 2~3gbit traffic on 10gbit network
no, I've checked, no throttling enablednitramd wrote:Do you have Throttling enabled?
ipperf between the veeam server and a VM for example?csydas wrote:Hey Mephisto,
Simple test is just iPerf between the two servers; do you see the same speeds there? Since it's a VM, I'd suspect that it's the VMware network stack between the VMs causing you grief. Windows Copy won't be reliable here due to the mischief that Windows and the VMware Tools will do to improve speeds.
It says it is hot adding on each proxy server, it is going way faster than before with the veeam server being the proxy alone using NBD. It seems it is using hotadd across all backup jobs. Any suggestions where else I can look?jdavidson_waters wrote:Sounds like you're using NBD, rather than hot-add or direct SAN.
NBD limited to around 300MB/s when using 10Gb.
More info here:
https://forums.veeam.com/vmware-vsphere ... 39238.html
And
https://kb.vmware.com/s/article/2147768
-
- Veteran
- Posts: 298
- Liked: 85 times
- Joined: Feb 16, 2017 8:05 pm
- Contact:
Re: network bottleneck, 2~3gbit traffic on 10gbit network
For your repository in Veeam B&R, what number is the "Limit maximum concurrent tasks" set to?
Who is online
Users browsing this forum: No registered users and 53 guests