Discussions specific to the VMware vSphere hypervisor
Locked
Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Nov 28, 2015 12:22 pm

Hi Sergey, please read my previous post.

jamerson
Expert
Posts: 307
Liked: 17 times
Joined: May 01, 2013 9:54 pm
Full Name: Julien
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by jamerson » Nov 30, 2015 12:44 am

Hi Guys,
as I understand the CBT bug is only on ESX 6.0 not on the 5.5 ?
After the patch 3247720 is applied.
do I have to run the script only if I still need the old backups ? otherwise no need ?
if I don't need the old backups ,do I have to do something so the VEEAM will start taking care of the new backup ?

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Nov 30, 2015 1:00 am

Hi, Julien. Veeam KB article included in the first post of this topic answers all your questions. Also, read my previous post in case you need more details. Thanks!

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Nov 30, 2015 11:38 am 1 person likes this post

All, please appreciate that this topic has many subscribers, so do keep all posts here strictly on topic. I've deleted all off-topic posts regarding environment-specific issues with performing CBT reset.

CBT reset is a common task, and there are existing topics discussing various procedures and scripts you can refer to. You can also create a new topic, if you'd like. Additionally, we have the following support KB articles providing a few different methods of performing CBT reset > KB1113, KB1940.

If you still have issues performing CBT reset, feel free to open a support case for assistance.

Thanks!

thuizenga
Influencer
Posts: 22
Liked: never
Joined: May 08, 2010 2:00 am
Full Name: Travis h
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by thuizenga » Nov 30, 2015 6:09 pm

Thanks for the information, all my backup jobs have been reset.

What about replication jobs? Do i just run the CBT reset on the source VM's or is there a different procedure that needs to be done to make sure the destination machines aren't corrupted? Am i also suppose to run the CBT against the replicated VM's and if so do i need to commit existing snapshots first. Is there a script that handles powered off machines, or do i have to do the "Primary Reset CBT (Requires Power-Off)".

Thanks,

teknomage
Service Provider
Posts: 23
Liked: 2 times
Joined: Jul 21, 2010 8:55 pm
Full Name: Mike
Location: Fargo, ND
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by teknomage » Nov 30, 2015 6:34 pm

Can I ask a question regarding the resting of the CBT data?

I understand that resetting CBT is required and I've run those scripts before, but I'm curious to the difference between running the scripts and toggling the "Changed block tracking" option in individual backup jobs. Does this only tell the job to use/ignore CBT data, but do nothing with the setting on the individual machines? Also, is the disabling CBT in the Veeam job itself not sufficient to avoid possible issues?

-Mike

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Nov 30, 2015 7:41 pm 1 person likes this post

thuizenga wrote:What about replication jobs? Do i just run the CBT reset on the source VM's or is there a different procedure that needs to be done to make sure the destination machines aren't corrupted? Am i also suppose to run the CBT against the replicated VM's and if so do i need to commit existing snapshots first.
Yes, you are supposed to run the CBT against the VMs you are replicating as well. There is no need to commit any snapshots though, as you only need to reset CBT for "source" (production) VMs, which should not have any snapshots. You don't need to reset CBT for "target" (replica) VMs which you are probably talking about here.
teknomage wrote:I understand that resetting CBT is required and I've run those scripts before, but I'm curious to the difference between running the scripts and toggling the "Changed block tracking" option in individual backup jobs.
Great question, Mike! The difference is that resetting CBT recreates and re-populates CBT map file, which is essential to do when they may contain inconsistency due to the bug discussed in this topic. The option you are talking about simply makes the job not use CBT data - and while disabling this option solves the issue, this is not a type of setup that most people will be happy to live with forever due to the associated performance penalty of "full scan" incremental runs.

thuizenga
Influencer
Posts: 22
Liked: never
Joined: May 08, 2010 2:00 am
Full Name: Travis h
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by thuizenga » Nov 30, 2015 8:37 pm

Thanks Gostev, you are correct i am strictly talking about the target systems. From a technical perspective when CBT is reset on a replica machine does this invoke the process where source and destinations blocks are compared which would be similar to the initial replica seeding process?

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Nov 30, 2015 10:29 pm

That is correct, as this is the only way to perform an incremental pass without knowing which blocks have changed. Thanks!

prabhash.jena
Enthusiast
Posts: 35
Liked: 2 times
Joined: Jan 16, 2015 1:32 pm
Full Name: Prabhash Kumar Jena
Contact:

[MERGED] ESXi 6.0 CBT issue

Post by prabhash.jena » Dec 01, 2015 2:05 pm

Hi,

I have a general question regarding CBT issue on ESXi 6.0.

Is it same to presume that my environment is not affected with this bug at all if I have not get the message "CBT data is invalid, failing over to legacy incremental backup. No action is required, next job run should start using CBT again. If CBT data remains invalid, follow KB1113 to perform CBT reset. Usual cause is power loss." during the days the bug existed. I have around 30 jobs and I don't see this message on any job, I have checked job history as well.
My colleague has now applied the patch, do we need to carry out some tasks to ensure we have correct backups?

Thanks.
Prabhash Jena

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Dec 01, 2015 2:14 pm

Hi. Assuming you are talking about KB2136854, this particular issue causes silent backup data corruption without any error messages logged. Every environment running vSphere 6 is affected, and must perform CBT reset on all existing VMs after installing the patch from KB3247720. Thanks!

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by cffit » Dec 02, 2015 4:23 pm

The VMware KB article mentions that the individual VMs need to be migrated or restarted. I ran the suggested script that disables CBT, then takes and deletes a snapshot, then re-enables CBT. I did not reboot the VMs. Is it still necessary to restart the individual VMs in this case?

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by cffit » Dec 02, 2015 5:44 pm

This is from the VMware KB per my last post saying the VMs have to be rebooted:

Release Date: Nov 25, 2015

Download Filename:
ESXi600-201511001.zip

Build:
3247720

Download Size:
358.9 MB

Host Reboot Required: Yes
Virtual Machine Migration or Shutdown Required: Yes

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by Gostev » Dec 02, 2015 10:41 pm

cffit wrote:VMs have to be rebooted
Sorry, but can you quote that, because I must be blind :shock:

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: [KB2136854] There's a new CBT bug in ESXi 6

Post by cffit » Dec 02, 2015 11:15 pm

In this KB:
http://kb.vmware.com/selfservice/search ... Id=2137545

"Virtual Machine Migration or Shutdown Required: Yes"

Locked

Who is online

Users browsing this forum: Bing [Bot] and 8 guests