Host-based backup of VMware vSphere VMs.
Post Reply
Heiko
Service Provider
Posts: 19
Liked: 2 times
Joined: Apr 15, 2010 8:34 am
Contact:

Veeam B&R 6 slow on ESXi 4.1

Post by Heiko »

Good Day

This forum is to find out how the community experience is when running Veeam B&R version 6 Patch3 to protect a vSphere 4.1 environment.
Veeam support ref ID#5173604

Configuration:
iSCSI SAN
ESXi 4.1 U2
Veeam B&R server is Virtual

Problem: After upgrading to Veeam 6 backup performance are very very slow. 1MB/s to local attached storage. (Attached are VMDK files which stores the backup)

What has been done:
Upgrade to from Veeam B&R 5.0.2 to Veeam B&R 6.0 Patch 3.
Uninstalled and fresh installed Veeam B&R 6.0 Patch3 and same performance problems. We even created proxy servers and ran a single 50GB Virtual Machine job.
After uninstalling version 6 and install version 5 all is back to normal.

Explanation on the problem: If it works fine in v5 probably it is issue with VDDK version. VDDK is VMware libraries that allow is get access to the VMFS datastore. We are using VDDK 1.2.1 in v5. In v6 we are using VDDK 5.0 I suspect VDDK 5.0 may have some performance issue with reading data from older VMFS version.

Any ideas suggestions?

Thank you
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by Gostev »

VDDK 5.0 has no known performance issues. What are the 4 numbers from bottleneck stats in v6? Most likely, your v5 jobs are working in the different processing mode than with v5, that's the only thing I can think of...
Heiko
Service Provider
Posts: 19
Liked: 2 times
Joined: Apr 15, 2010 8:34 am
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by Heiko »

Says the source.
Did take a snapshot of the Veeam server before uninstalling all. Easy now to revert back and test.
FYI - tried Virtual mode etc and no differance
Heiko
Service Provider
Posts: 19
Liked: 2 times
Joined: Apr 15, 2010 8:34 am
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by Heiko »

Hi

For Feedback purposes.
All backups are running 100% fine on Veeam 5. We even tried all various settings in Veeam 6 without any success.
Veeam support is investigating the problem further. The call has been escalated to a higher tier.
paps40
Enthusiast
Posts: 31
Liked: 13 times
Joined: Dec 12, 2011 4:10 pm
Full Name: Peter Pappas
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by paps40 »

Heiko,
Did you ever find a solution to get version 6 to work correctly? I am also experiencing performance issues after I upgraded ESX from 4.0 to 4.1 and upgraded to veeam version 6 patch 3.

Just Curious.

Thanks.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by foggy »

Peter, as far as I can tell from the Heiko's support case, it's still under investigation (by VMware at this time).

What are your bottleneck stats numbers and what backup mode is actually used by your jobs?
tomuk5
Lurker
Posts: 1
Liked: never
Joined: Nov 16, 2011 3:16 pm
Full Name: Thomas Wing
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by tomuk5 »

Hi

im also having the same issues now, just started a week ago out of the blue.

hosts running 4.1, separate physical veeam box with local storage running veeam 6 patch 3

the bottleneck stats are: Load: Source 99% > Proxy 9% > Network 0% > Target 0%

the backup mode is network.
paps40
Enthusiast
Posts: 31
Liked: 13 times
Joined: Dec 12, 2011 4:10 pm
Full Name: Peter Pappas
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by paps40 »

Alexander,
Sorry, I had forgot about this post. My issues have been resolved. My issues were related to a Dual Port Intel NIC and a failing switch. I'm not quite sure why but when using a dual port nic when you set one ethernet port to jumbo frames and the other one to not use jumbo frames your backup speed decreases. Once I used a different nic (the Dell Server Motherboard NIC) my speed increased. Once I replaced the failing switch it increased even further.

We are using backup network mode and backing up to our old SAN MD3000i. Our bottleneck stats are: Source 93% > Proxy 93% > Network 28% > Target 18%. On full vm backups over the weekend we are getting speeds of around 14 MB/s. Our reverse incremental speeds during the week are anywhere between 70 MB/s and 150 MB/s. We are also running veeam 6 patch 3. We we do full backups to local disk 15K SAS on the Physical Veeam Server our Full VM backups speeds jump to 48 MB/s.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by foggy »

Peter, glad to hear that your issues are now resolved. Thanks for coming back to update the topic!
Heiko
Service Provider
Posts: 19
Liked: 2 times
Joined: Apr 15, 2010 8:34 am
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by Heiko »

Hi

Apology for the late replay.
The problem has been resolved after it has been escalated to VMware.
Cause: We saw in the logs each 1-3ms connection loss to the iSCSI target.
Resolution: Upgraded to the latest SAN software revision and all problems have been resolved. We do get about 40-50MB/s iSCSI backups speed in production times (Virtual Appliance mode) to an CIFS target.

Need to mention how good VMware is. Even with connection drops we did not experience any problems during production times. Well done VMware. :D

Thank you for all your support. If there are any queries let me know.

Heiko
OWTech
Novice
Posts: 7
Liked: never
Joined: Jan 16, 2012 11:32 am
Full Name: James Richmond
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by OWTech »

Alexander, fyi, I don't know if it's switch specific, but for my procurve if you set jumbo frames on a port, all the devices and ports in that vlan must also be set to jumbo frames.
Heiko
Service Provider
Posts: 19
Liked: 2 times
Joined: Apr 15, 2010 8:34 am
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by Heiko »

OWTech ur 100% correct but if a port is set to 1500MTU it will be able to communicate with a port that set to 9000MTU. But thats a reason why he might experience performance problems.
My experience if you do not use Jumbo frames from start to the end you will have performance problems and its always a good idea to make sure all hardware is updated to the latest firmware.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Veeam B&R 6 slow on ESXi 4.1

Post by dellock6 »

You always have to have jumbo frames enable along the whole chain, from start to end. If you do not do so, you will have for sure performance problems, more than not using jumbo at all, since for every packet there is always a check to see if jumbo can be used, so before using 1500mtu there are some re-transmits.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Post Reply

Who is online

Users browsing this forum: No registered users and 74 guests