Hyper-v vs VMWare Backup Performance

Hyper-V specific discussions

Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Fri Aug 10, 2012 2:40 am

I started looking further into why I can backup 80 VMWare vm's in 5-6 hours but 23 Hyper-v VM's take me 14 hours. Both job types show oan avg transfer rate of 60 mb/s. But what I am seeing is that on the VMWare jobs, it will list the hard drive sizes as let's say [100GB] and Read size as let's say 2GB. On the Hyper-v jobs, most of the time it is having to read the entire Hard Drive. So if the hard drive size is [100GB] the read size will also show 100GB. A few of the [100GB] hard drives showed as 84GB read size. I clicked on manage volumes on the Hyper-v hosts and verified that CBT was enabled on the Hyper-v jobs.

Is this just a difference in technologies or is something wrong with my config? If so, I can open up a support case.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby Gostev » Fri Aug 10, 2012 8:15 am

Hi Collin, are you comparing full or incremental backup here?
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Fri Aug 10, 2012 5:57 pm

I am seeing this on both incremental and fulls. I think what happened is there were hyper-v snapshots applied on all the vm's. The Veeam email status usually alerts me when there are vm snapshots on vmware with a warning message, but the same alerts aren't happening with hyper-v backups.

Isn't it true that vm snapshots essentially disable CBT for that particular vm?
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby Gostev » Fri Aug 10, 2012 8:01 pm

What you are thinking about is true only with VMware (you cannot enable CBT on a VM that has snapshot). Our Hyper-V CBT can be enabled even if VM has snapshots.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby Gostev » Fri Aug 10, 2012 8:04 pm

Try removing snapshots from one of the VMs and see if it helps... at least, it's plausible. Could be either by design, or bug (will confirm with devs next week).
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Thu Aug 16, 2012 2:58 pm

Hyper-v snapshots seem to have no bearing on this behavior. I guess I don't know how you can have an environment where you are backing up 100's of Hyper-v VM's successfully within a night with Veeam when each backup has to read the entire .vhd. Some of the VMWare backups take seconds if nothing has changed. Whereas the Hyper-v backups take much much much longer. Yes, I am using the hardware vss provider and am running multiple jobs at the same time.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby tsightler » Thu Aug 16, 2012 4:23 pm

You should open a case with support. It seems obvious that the Veeam Hyper-V CBT filter driver is not functioning correctly in your environment. Are you perhaps using a filesystem such as MelioFS which doesn't allow our CBT driver to function?
tsightler
Veeam Software
 
Posts: 4775
Liked: 1742 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby tietzjd » Tue Aug 21, 2012 3:04 am

Collinp- I always seen this issue with Hyper-V and backups.
tietzjd
Influencer
 
Posts: 16
Liked: 2 times
Joined: Wed Nov 09, 2011 3:00 pm
Full Name: Joe Tietz

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Tue Aug 28, 2012 5:06 pm

I have a ticket open for this. Rebooting the Hyper-v hosts fixes it temporarily and then CBT works on Hyper-v vm's on all jobs. Then after a few days CBT stops working for specific vm's, one at a time until CBT stops working again for the entire job. Another interesting symptom when this is happening is the dedup ratio falsely shows a high ratio of 18 to 1 when CBT isn't working correctly on the Hyper-v backups. Will let you know the outcome. CBT on the vmware backups seems to me much more reliable.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby Gostev » Wed Aug 29, 2012 7:25 pm

Hi Collin, what is the ticket - I want to draw lead dev's attention to this issue. Thanks!
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Thu Aug 30, 2012 5:59 am

Gostev,

Thank you! The ticket # is 5211474. I purchased 12 veeam hyper-v cpu licenses and there is no way I am going make my backup windows without CBT working. It is sporadic and I am seeing it across different Hyper-v clusters and different underlying storage. The one Hyper-v job that CBT always worked on, finally has reverted back to not working. Sounds like others have had this issue to. Don't know how widespread it is. I also had a special hotfix installed a couple months ago for the Hyper-v host BSOD issues. Don't know if this special hotfix affected anything. I have upgraded since then.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby Gostev » Fri Aug 31, 2012 11:59 am

OK, so to answer the original statement of this topic - no, there is not performance difference in backup performance between VMware and Hyper-V. Likewise, all original ideas of why performance may be different (such as VM snapshot presence) were incorrect.

According to the logs, slow backup was caused by the fact that changed block tracking driver was not working, possibly because Hyper-V hosts required reboot after product update, or due to some 3rd party conflict, or something else. As logs show, drivers did start working couple of weeks later (without any changes on product side - likely, after host reboot). This particular case will be investigated further.

If you have similar issue, especially after product update - try rebooting your Hyper-V hosts first.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Fri Aug 31, 2012 10:14 pm

Yes, technical support had me reboot the hosts a week ago and CBT started working again for a few days but now it isn't working again. They are investigating further.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby tsightler » Sat Sep 01, 2012 1:03 am

Gostev wrote:OK, so to answer the original statement of this topic - no, there is not performance difference in backup performance between VMware and Hyper-V. Likewise, all original ideas of why performance may be different (such as VM snapshot presence) were incorrect.

OK, not "all original ideas" since my post specifically indicated that CBT was obviously not working in his environment and that the reason needed to be investigated. :mrgreen:
tsightler
Veeam Software
 
Posts: 4775
Liked: 1742 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Hyper-v vs VMWare Backup Performance

Veeam Logoby collinp » Sat Nov 03, 2012 5:11 pm 3 people like this post

Just an update on this post. It looks as though there were major improvements in the Veeam Filter driver in version 6.5. I had issues before in multiple Hyper-v environments where CBT would not work reliably on every vm. All of these problems are fixed and backups are now working perfectly! With both CBT and the Netapp hardware VSS provider working, we are now able to back up a large number of Hyper-v vm's within our backup window without any issues.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Next

Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 11 guests