Host-based backup of VMware vSphere VMs.
Post Reply
Dacuy
Novice
Posts: 5
Liked: never
Joined: Nov 12, 2013 5:00 am
Full Name: Dennis Uy
Contact:

ESX 5.5 Update 1

Post by Dacuy »

We upgraded our vmware hosts to ESX 5.5 Update 1 (from 5.5) over the weekend. Since then, some of our backups notably one with high I/O VM's have had uncommited snapshots at the onset, and then for the most part, backup times have increased from 40 mins - 1 hour to 3- 7 hours mostly. ALSO yesterday and today, my mid-morning backups are just hanging with 0% completed - these are supposed to go to 2 repositories, one a Dell Poweredge server and another a Dell Dedup applicance. We use Dell Equallogics for pimary storage, Dell for Host servers. I have logged a support case with Veeam support. Has anyone else who who has updated to ESX 5.5 Update 1 had issues similar to these?
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: ESX 5.5 Update 1

Post by foggy »

Dennis, what are the bottleneck stats for the jobs that have degraded performance and what transport mode is being used? Could you please also double check that CBT is used? And what is your support case ID number?
Dacuy
Novice
Posts: 5
Liked: never
Joined: Nov 12, 2013 5:00 am
Full Name: Dennis Uy
Contact:

Re: ESX 5.5 Update 1

Post by Dacuy »

case # 00629031. Last night and today, all problem symptoms have disappeared despite the fact that we havent done anything on Veeam, VCentre, SAN...absolutelty nothing. The high I/O vm's in that one job (SAP Production VM's) which backs up every 3 hours have returned to its normal < 3 hour duration. CBT is used, and historically Target was the usual bottleneck even prior to these issues starting probably because its a reverse incremental.
This particular Job would have its load shared like this: Source 75% Proxy 5% Network 35% Target 50%. It appears the load is mostly shared between Source and Target with the Source bearing the brunt. Occasionally Source and Target would be 50-50. The values for Proxy and Network hardly fluctuate.
veremin
Product Manager
Posts: 20406
Liked: 2298 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: ESX 5.5 Update 1

Post by veremin »

I'm wondering whether there had been any tasks taking place inside the said VMs before backup job ran. Those tasks, such as antivirus, defragmentation activities or some other maintenance tasks might have affected number of changed blocks and led to backup jobs taking longer and VMs becoming unresponsive during snapshot commit operation (or even "leftover" snapshots). Thanks.
ggrice
Influencer
Posts: 10
Liked: 3 times
Joined: Aug 16, 2013 8:19 am
Full Name: Geoff Grice
Contact:

Re: ESX 5.5 Update 1

Post by ggrice »

Not sure if it's related, but I also had problems when upgrading from esx 5.1 to 5.5. My Veeam backup jobs would not run correctly and they would not detect the VM sizes correctly. But the issues were not limited to Veeam alone, I also saw latency spikes and network counter problems within esx itself.

After some hunting around, I found that the problem was caused by an incompatible NIC driver in the ESX installer (downloaded from VMware's site). We are running Dell servers, I downloaded a Dell VMware 5.5 .iso, reloaded my hosts with that and hey presto we were back in business. Did you install ESX from a web download or a Dell image?
edfops
Enthusiast
Posts: 37
Liked: 6 times
Joined: Feb 13, 2014 8:17 pm

Re: ESX 5.5 Update 1

Post by edfops »

ggrice wrote:Not sure if it's related, but I also had problems when upgrading from esx 5.1 to 5.5. My Veeam backup jobs would not run correctly and they would not detect the VM sizes correctly. But the issues were not limited to Veeam alone, I also saw latency spikes and network counter problems within esx itself.

After some hunting around, I found that the problem was caused by an incompatible NIC driver in the ESX installer (downloaded from VMware's site). We are running Dell servers, I downloaded a Dell VMware 5.5 .iso, reloaded my hosts with that and hey presto we were back in business. Did you install ESX from a web download or a Dell image?
We also have Dell hosts and have not seen these issues with 5.5U1, but I will say we also updated any and all firmware alongside the upgrade.
Dacuy
Novice
Posts: 5
Liked: never
Joined: Nov 12, 2013 5:00 am
Full Name: Dennis Uy
Contact:

Re: ESX 5.5 Update 1

Post by Dacuy »

Thanks for the responses. Strangely enough the symptoms have ceased without us doing any further changes, but I am still closely monitoring. We didn't have issues migrating from pre-5.5 to 5.5, and that was quite a while ago.
ggrice
Influencer
Posts: 10
Liked: 3 times
Joined: Aug 16, 2013 8:19 am
Full Name: Geoff Grice
Contact:

Re: ESX 5.5 Update 1

Post by ggrice »

Just to clarify, the problems I had were because we were using a 10gb NIC, the 1gb NIC's ran ok with the standard VMware .iso
Post Reply

Who is online

Users browsing this forum: No registered users and 46 guests