Host-based backup of VMware vSphere VMs.
Post Reply
svenh
Enthusiast
Posts: 30
Liked: 5 times
Joined: Jan 21, 2016 12:56 pm
Full Name: Sven Hartge
Location: Germany
Contact:

ESXi 6.0 Express Patch 6 breaks CBT

Post by svenh »

Hi!

Just a little heads up:

http://www.running-system.com/take-care ... p-cbt-bug/

Short synopsis: "There is an issue with Change Block Tracking (CBT) in Express Patch 6 for ESXi 6 (Build 3825889) that can affect your backup if your backup software relies on CBT."

Affected are Windows VMs on HW version 7 to 11 and with any VMware Tools version on both thin and thick provisioned disks while doing backups with quiesced snapshots.

Solutions so far:
  • don't use quiesced, i.e. application-aware snapshots
  • or downgrade to a older ESXi6 version
If you have been hit, you have to reset your CBT data after fixing the problem (as usual with CBT problems).
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by mkretzer »

Does somebody know already if this means corrupted backups or only slow backups because CBT is just not working? From the article it sounds like CBT is just not working and the backups should be ok...
justind
Enthusiast
Posts: 32
Liked: 4 times
Joined: Nov 09, 2014 4:20 am
Location: Australia
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by justind »

What on earth has happened to VMware? Every patch they release either fixes or breaks CBT then requires a CBT reset.

I don't recall a time (since ESX 3 anyway) where i have had to revert patches. Or downgrade VM tools mind you either.

The numpties even have the hide to send out an email asking for beta testers for vSphere.Next... maybe they should focus on vSphere.Now!!!
dkrimmer
Lurker
Posts: 2
Liked: never
Joined: May 12, 2015 3:21 pm
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by dkrimmer »

mkretzer wrote:Does somebody know already if this means corrupted backups or only slow backups because CBT is just not working? From the article it sounds like CBT is just not working and the backups should be ok...
Can anybody please confirm that (other than the last CBT bug) this newly introduced bug doesn't mean corrupt backups which B&R has no way to detect? While extended backup time due to unavailability of CBT is bad, unnoticed backup corruptions are of course much worse. Thanks.
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by lando_uk »

I'm still running patch 3380124 (Update 1b) on my newer hosts, as I think this was the only patch that had no real issues.

I've updated vcenter to U2, but this whole upgrade project seems to be dragging on forever as there's no new build that is good and I'm too wary of upgrading my good 5.1 hosts.
ashman70
Expert
Posts: 203
Liked: 12 times
Joined: Dec 04, 2012 2:18 pm
Full Name: Both
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by ashman70 » 1 person likes this post

With the mess that was 5.5 and its patches and updates that caused purple screens, most of my customers are still running 5.1 and I see no reason to move them until VMWare sorts out this mess. Version 6 was looking promising until these updates and patches started causing havok.
DaveWatkins
Veteran
Posts: 370
Liked: 97 times
Joined: Dec 13, 2015 11:33 pm
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by DaveWatkins »

This bug only appears to affect Veeam if you're using VMWare quiescense. If you're not (and I'm guessing most people aren't because it's terrible) and just using normal AAP then it supposedly doesn't affect you.

That's pointed out in the link but is probably worth putting here. So for the most part we should all be fine.
kjstech
Expert
Posts: 160
Liked: 16 times
Joined: Jan 17, 2014 4:12 pm
Full Name: Keith S
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by kjstech »

3 of my hosts are on this patch level of 6.0 and 5 hosts are still on the latest patch level for 5.0.

I'm not updating my 5 other hosts until a new patch comes out (expected late july) that fixes ESXi 6.0 and vSphere replication 6.1 hostd crashes with "Panic: Assert Failed: "!repDiskInfo->GetDiskReplicationId().empty()" @ bora/vim/hostd/hbrsvc/ReplicationConfig.cpp:769" if you try to vMotion a replicated VM where only 1 (of many) disks are configured to replicate.
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by Gostev »

Tibo
Influencer
Posts: 10
Liked: never
Joined: May 20, 2011 8:15 am
Full Name: Tibo
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by Tibo »

Can anyone confirm that this bug does not affect Veeam when using Veeam Application Aware Processing ?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by Vitaliy S. »

Since Veeam AAIP does not use VMware Tools Quiescence, AAIP-enabled Veeam backup jobs are not affected by this.
mcvosi
Enthusiast
Posts: 65
Liked: 8 times
Joined: Jun 14, 2011 1:55 pm
Full Name: Matthew Vaughan
Contact:

[MERGED] Incremental virtual machine backups effectively run

Post by mcvosi »

mcvosi
Enthusiast
Posts: 65
Liked: 8 times
Joined: Jun 14, 2011 1:55 pm
Full Name: Matthew Vaughan
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by mcvosi »

Guess I was late to the party on this...

But yes, QA has really slipped at VMware these days. Every time I think I patch to a stable build something breaks with each new "fix."
Zew
Veteran
Posts: 365
Liked: 80 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by Zew » 2 people like this post

mcvosi wrote:Guess I was late to the party on this...

But yes, QA has really slipped at VMware these days. Every time I think I patch to a stable build something breaks with each new "fix."
Funny I had an argument with someone on IRC, the cisco channel I believe I was in at the time, (I'm in many of them), to which he kept saying ESXi6 is solid and suggested to move to it. Which I replied back with endless KB's from VMware on how terrible QA has been on it and that CBT in general has broke multiple times. The amount of links i had to reference has near endless... I asked for any references on its stability which I received none.
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by albertwt »

So does this problem finally fixed by VMware in the latest VMware vCenter Server 6.0 Update 3 https://docs.vmware.com/en/VMware-vSphe ... notes.html ?

or this is something that is addressed in Veeam Backup & Replication Update 3 ?
--
/* Veeam software enthusiast user & supporter ! */
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: ESXi 6.0 Express Patch 6 breaks CBT

Post by foggy »

According to the official VMware KB mentioned above, this issue was resolved in ESXi 6.0 Patch 03.
Post Reply

Who is online

Users browsing this forum: No registered users and 101 guests