Network Bottleneck

VMware specific discussions

Network Bottleneck

Veeam Logoby lobo519 » Mon Apr 09, 2012 9:01 pm

I am running a backup job over the WAN (VPN). I can clearly see the available bandwidth and there is plenty available that is not being used - yet my Job shows the bottleneck as the Network.

Expected behavior?

My replication jobs usually have no problem filling the pipe.

I am not using any traffic throttling.
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby foggy » Tue Apr 10, 2012 8:19 am

What are the whole bottleneck stats numbers for this job?
foggy
Veeam Software
 
Posts: 14712
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Network Bottleneck

Veeam Logoby Gostev » Tue Apr 10, 2012 1:20 pm

Also, are you in patch #3, because it has enhancements around network load calculations.
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Network Bottleneck

Veeam Logoby lobo519 » Tue Apr 10, 2012 2:09 pm

I am on patch 3. I'll post back when the job is finished. Is running a full and I am in hour 82.
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby lobo519 » Wed Apr 11, 2012 12:39 pm

4/10/2012 11:32:27 PM :: Load: Source 0% > Proxy 11% > Network 98% > Target 0%

This job ran for 96 hours before I just cancelled it. I re-ran it after canceling it and it seems to be running better. It was a full backup
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby Gostev » Wed Apr 11, 2012 1:44 pm

Thank you - seeing these numbers makes troubleshooting so easy! Based on this statistics, my only guess is that there is some QoS mechanism in place on the networking equipment that limits bandwidth per connection. To verify this, try disabling multiple TCP/IP connections option in Veeam settings under Traffic Throttling settings, and see if this reduces the backup performance. I would recommend creating a very small VM for testing this, and trying full backup with and without multiple connections option enabled. And post the results (time to perform full backup) here. Thanks!
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Network Bottleneck

Veeam Logoby lobo519 » Wed Apr 11, 2012 2:04 pm

hmm - well there is defiantly no QOS policies on any of our equipment.
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby lobo519 » Wed Apr 11, 2012 4:15 pm

Would enabling the "Use multiple TCP/IP connections per job" setting help if indeed there was some QOS policy? Do I need to create a throttling rule for this setting to be used?
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby Gostev » Wed Apr 11, 2012 7:22 pm

Actually, this checkbox is enabled by default - are you saying you have it disabled right now?
No, this functionality is not dependent on throttling rules presence.
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Network Bottleneck

Veeam Logoby lobo519 » Wed Apr 11, 2012 7:37 pm

Yes its disabled at the moment - Perhaps I should turn it back on! :oops:


Maybe a UI change is in-order for that check box..
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby Gostev » Wed Apr 11, 2012 7:47 pm

There is a good reason why this feature was added, and is enabled by default ;)
Enabling it back will most definitely help with your WAN link utilization.
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Network Bottleneck

Veeam Logoby lobo519 » Thu Apr 12, 2012 12:43 pm 1 person likes this post

:D :D Its only about 5 times faster!

Thanks!!
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby Gostev » Thu Apr 12, 2012 8:02 pm

Default is 5 TCP/IP connection per job, thus the number you are seeing. This also proves that you have some sort of QoS that limits each connection to the same amount of bandwidth. We actually have the same in our own environment on the WAN link between US and EMEA sites... no matter if I open 1 or 5 FTP sessions for upload, they all see to have the same (and slow) speed.
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Network Bottleneck

Veeam Logoby lobo519 » Thu Apr 12, 2012 8:11 pm

Can you add more connections per job?!??!

I'm going to need to track down what is limiting per connection (not that its really a huge deal). It must be either on our providers end or a default setting in one of our ASAs - I set everything up! :shock:
lobo519
Expert
 
Posts: 292
Liked: 34 times
Joined: Wed Sep 29, 2010 3:37 pm

Re: Network Bottleneck

Veeam Logoby Gostev » Thu Apr 12, 2012 8:24 pm

Yes, by adding this registry value in the B&R registry key on your backup management server:
Code: Select all
DownloadStreamsNumber
DWORD
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Next

Return to VMware vSphere



Who is online

Users browsing this forum: DGrinev, rweis and 32 guests