Comprehensive data protection for all workloads
tsightler
VP, Product Management
Posts: 5283
Liked: 2141 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Speed performance issues copying between VLANs

Post by tsightler » Feb 18, 2010 7:39 pm

So you're saying backup is "SAN mode with NBD/failover" right. So, since the Veeam VM doesn't have direct access to the storage, it will be falling back to NBD mode. NBD mode requires the VM to connect the to SC and you stated that the Veeam is a VM in VLAN 51 so something has to route that traffic from VLAN51 to the SC VLAN, it won't just magically cross the vswitch because it's a layer 2 switch and otherwise it would be bypassing your ACL's.

I think you might be confusing MB/sec with Mb/sec. Those are drastically different terms. Veeam generally reports performance in MB/sec or "Megabytes per second". Networks are generally referred to in Mb/sec or "Megabits per second".

100Mb/sec = ~12MB/sec

So on your best day you could get 12MB/sec, however, since it's a "one-armed" configuration where traffic in and traffic out has to go over the same link, you're looking at a theoretical max of 6MB/sec. Add in the fact that the port is saturated and yes, you'd see higher pings and dropped packets and 5MB/sec is pretty close to what I'd expect you to see.

Options include:

1. Use "Virtual Appliance" mode for your backups, which would access the storage directly via the ESX host
2. Configure your Veeam VM to have at least one interface on the same VLAN as the service console and continue to use NBD mode since it will not longer need to be routed
3. Configure your Veeam VM to have at least one interface on the same VLAN with the storage and do SAN backups.

Since you're running Veeam in a VM, virtual appliance mode would likely be the best option, and the simplest.

n2it-dk
Enthusiast
Posts: 29
Liked: never
Joined: Dec 29, 2009 9:36 am
Contact:

Re: Speed performance issues copying between VLANs

Post by n2it-dk » Feb 18, 2010 9:35 pm

ur absolutely right about your writeup of the various setups, ive tried them all, and still there are bottleneck somewhere... Even though ive made a local iscsi connection to the iscsi box from the Veeam appliance there are still indications of slow speed. Yea ur right about the theoretical speed limits etc when we talk 100Mb or Gb speed and yes the speed is around 5MB wich is as it would be, or I would have wanted it to be highter since no network traffic is going on excepts for the backup window. Ill try and see if I can tweak it and rewiew the setup. Perhaps run a sniffer on the network, cause it seems some of the traffic is running places I dont want it to.

Even though ive tried Virtual Appliance mode, the speed were still the same, read: slow.. and allso the problem with consolidated helper file, that werent released afterwards... Therefore ill stick with SAN mode with NBD/failover.

Im not confused about the Mb and MB terms, my head were just elsewhere and focus werent on terms, but actual events.

Cheers.

rchew
Influencer
Posts: 20
Liked: never
Joined: Dec 16, 2009 7:02 pm
Full Name: Raymond Chew
Contact:

Re: Speed performance issues copying between VLANs

Post by rchew » Feb 18, 2010 9:46 pm

Maybe I don't understand what you're trying to convey. Nothing you've told me so far refutes the premise that if your backups are traversing a router that is not capable of keeping up with the data flow, your backups will be slow. Your ping times will be affected because it also has to traverse the same router that can't keep up. If it can't process your backups fast enough, it can't process your pings either.

Am I missing something here? As indicated by tsightler, you are probably running on the "Network" failover so you must be routed since your SC and SAN networks are on different subnets.

Ray

n2it-dk
Enthusiast
Posts: 29
Liked: never
Joined: Dec 29, 2009 9:36 am
Contact:

Re: Speed performance issues copying between VLANs

Post by n2it-dk » Feb 18, 2010 9:52 pm

One thing though, what would be an idea to try out would be the E1000 or the VMXNET (2) Enhanced or the VMXNET (3)

What would be the suggested solution and what are peeps experiences with the various nics?

tsightler
VP, Product Management
Posts: 5283
Liked: 2141 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Speed performance issues copying between VLANs

Post by tsightler » Feb 18, 2010 10:42 pm

OK, so you're focused on "events" rather than "terms", I get that. But you're asking for help on a forum. If you're going to take time from troubleshooting your "events" to ask for help on a forum, and then expect others to take their time to assist with troubleshooting your "events" then you need to slow down and actually communicate the problem. Communicating the problem involves using the correct terms so that the people attempting to assist you aren't wasting their time and yours because of misunderstanding of the setup.

I'd suggest, if you actually want to get to the bottom of your situation with assistance from people on this forum, that you start over again, explain your environment, and then do your best to respond to the best of your ability to the questions people ask and the suggestions they make. So far people have offered many helpful suggestions, yet your answers are always "I've tried them all" or you turn around and just ask another question. This implies that you either don't understand your environment enough to explain it, don't understand enough about the suggestions to know why they might help, or both.

rchew
Influencer
Posts: 20
Liked: never
Joined: Dec 16, 2009 7:02 pm
Full Name: Raymond Chew
Contact:

Re: Speed performance issues copying between VLANs

Post by rchew » Feb 18, 2010 11:58 pm

n2it-dk wrote:One thing though, what would be an idea to try out would be the E1000 or the VMXNET (2) Enhanced or the VMXNET (3)

What would be the suggested solution and what are peeps experiences with the various nics?
If you had a 10 lane highway with a toll-booth in each lane, the traffic would probably flow fairly well. Now imagine if you had that same 10 lane highway but only 2 toll-booths. The infrastructure would be overloaded and cars would have to merge into the 2 toll-booths causing traffic jams. What do you think would happen if you expanded your highway to 100 lanes but did not increase the number of toll-booths? You are applying the same logic with your network cards. Until you resolve your router (aka toll-booth) problem, you will continue to have issues.

Ray

n2it-dk
Enthusiast
Posts: 29
Liked: never
Joined: Dec 29, 2009 9:36 am
Contact:

Re: Speed performance issues copying between VLANs

Post by n2it-dk » Feb 19, 2010 12:30 am

a workaround here is that ive added several nics to the veeam backup station and then added those nics to individual vlans and its progressively better in performance. Ive allso installed iscsi sw on the veeam backup station aswell and added the Luns to the backup station to try and make SAN backups.

So far results are like below.

Total size Processed size Processing rate Duration
20,00 GB 20,00 GB 45 MB/s 0:07:39

Im getting there, just not there yet :-)

n2it-dk
Enthusiast
Posts: 29
Liked: never
Joined: Dec 29, 2009 9:36 am
Contact:

Re: Speed performance issues copying between VLANs

Post by n2it-dk » Feb 19, 2010 12:33 am

btw rchew and tsightler your suggestions are more than welcome... guess we all troubleshoot differently.

Gostev
SVP, Product Management
Posts: 23879
Liked: 3213 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Speed performance issues copying between VLANs

Post by Gostev » Feb 19, 2010 12:37 am

Multiple NICs cannot make difference with Veeam Backup working in network mode. I believe what helped here is the fact that the product switch to SAN mode backups after you installed iSCSI. Backup traffic now goes direct from Veeam Backup VM to your storage, instead of going through the service console.

But I don't think you can get more speed via software iSCSI initiator... 45-50MB/s this is pretty much best you can get for full backups in ideal situation, with Veeam Backup running in VM and using software iSCSI initiator.

n2it-dk
Enthusiast
Posts: 29
Liked: never
Joined: Dec 29, 2009 9:36 am
Contact:

Re: Speed performance issues copying between VLANs

Post by n2it-dk » Feb 19, 2010 12:41 am

Yes Gostev, your might right, but if I choose SAN mode only the backup job gives an error, so I have to go for the default settings to achieve the results above. Ill try rebooting the Veeam backup station and see if the SAN mode will behave afterwards.

Thanks ;)

n2it-dk
Enthusiast
Posts: 29
Liked: never
Joined: Dec 29, 2009 9:36 am
Contact:

Re: Speed performance issues copying between VLANs

Post by n2it-dk » Feb 19, 2010 7:38 am

Somehow this issue is solved, there are nomore unusual traffic coming in through the iscsi vlan, ping session times is absolutely fine. Its a combination between a semi intervlan routing setup and then the sw based iscsi locally installed on the veeam backup station beeing able to create san backup.

Cheers and have a nice friday. :roll:

Post Reply

Who is online

Users browsing this forum: J.Giroux, Majestic-12 [Bot], michaelobrien and 43 guests