[KB2136854] There's a new CBT bug in ESXi 6

VMware specific discussions

[KB2136854] There's a new CBT bug in ESXi 6

Veeam Logoby ngagne » Thu Nov 12, 2015 6:49 pm

Source: https://miketabor.com/another-cbt-bug-f ... -esxi-6-0/ and http://kb.vmware.com/kb/2136854. I'm not affected by this, but wanted to make sure everyone knows about it!

[EDIT] Veeam KB > KB2075
ngagne
Novice
 
Posts: 8
Liked: never
Joined: Tue Sep 22, 2015 4:01 pm

Re: There's a new CBT bug in ESXi 6

Veeam Logoby nullifi » Thu Nov 12, 2015 8:01 pm

Annoying, the VMware KB link doesn't appear to be working.

How are you not affected, do you not use CBT?
nullifi
Influencer
 
Posts: 23
Liked: 5 times
Joined: Wed Aug 05, 2015 1:41 pm
Full Name: Jason Taylor

Re: There's a new CBT bug in ESXi 6

Veeam Logoby ngagne » Thu Nov 12, 2015 8:03 pm

The VMware link took a while to load for me too; the first article says pretty much the same thing that the KB did though. We're not affected because we're still on 5.5 :)
ngagne
Novice
 
Posts: 8
Liked: never
Joined: Tue Sep 22, 2015 4:01 pm

Re: There's a new CBT bug in ESXi 6

Veeam Logoby nullifi » Thu Nov 12, 2015 8:04 pm 2 people like this post

Ah. Wonderful. Hurray for embracing the future!
nullifi
Influencer
 
Posts: 23
Liked: 5 times
Joined: Wed Aug 05, 2015 1:41 pm
Full Name: Jason Taylor

Re: There's a new CBT bug in ESXi 6

Veeam Logoby ngagne » Thu Nov 12, 2015 8:06 pm

Hooray for my predecessor not keeping VMware SnS active so I could upgrade. Between this and the plethora of other v6 bugs, I'm actually thankful that he didn't!
ngagne
Novice
 
Posts: 8
Liked: never
Joined: Tue Sep 22, 2015 4:01 pm

[MERGED] MAJOR: new CBT issue on vsphere6.

Veeam Logoby ashleyw » Thu Nov 12, 2015 11:34 pm

Hi,. We've just been made aware of this shocker;

http://www.running-system.com/attention-new-cbt-bug-in-vsphere-esxi-6-0/

To what extent are Veeam customers exposed to this issue (looking at the problem it impacts all products running snapshot backups).
oh deary me - a bad end to a good week!
ashleyw
Service Provider
 
Posts: 137
Liked: 16 times
Joined: Thu Oct 28, 2010 10:55 pm
Full Name: Ashley Watson

Re: MAJOR: new CBT issue on vsphere6.

Veeam Logoby ashleyw » Fri Nov 13, 2015 12:28 am

and we've just brainstormed this through;-
it's our opinion that;
- surebackup won't save you either as just because the VM starts up - it doesn't mean that there hasn't been corruption on the image.
- Replication is dependent on CBT so that is potentially bad as well.
- Synthetic fulls merge the incrementals so they are potentially bad.
- If we have to run active fulls on our 70TB or primary storage every day, well need to move our Veeam host to Venus becauase a day on Venus is 243 Earth days - it's the only way we could get through our backup cycle in a day!

I'm trying to see the positive side here - but unfortunately I can't - and the extent of the impact here is unknown.
ashleyw
Service Provider
 
Posts: 137
Liked: 16 times
Joined: Thu Oct 28, 2010 10:55 pm
Full Name: Ashley Watson

Re: There's a new CBT bug in ESXi 6

Veeam Logoby andyg » Fri Nov 13, 2015 1:32 pm

this looks bad. any comment from Veeam ?
-= VMCE v9 certified =-
andyg
Service Provider
 
Posts: 56
Liked: 5 times
Joined: Wed Apr 23, 2014 9:51 am
Full Name: Andy Goldschmidt

Re: MAJOR: new CBT issue on vsphere6.

Veeam Logoby nullifi » Fri Nov 13, 2015 2:57 pm

I've decided to continue to use incrementals (Maybe because I tried fulls last night and I had 14 failures due to the backup window... ahem..) since I am mostly concerned about files, and no files are generally modified during the backup window, including the time the snapshots are being consolidated.

I'll just take the risk that some OS files may be damaged if I need to restore the entire VM.
nullifi
Influencer
 
Posts: 23
Liked: 5 times
Joined: Wed Aug 05, 2015 1:41 pm
Full Name: Jason Taylor

[MERGED] : Incremental without CBT?

Veeam Logoby nullifi » Fri Nov 13, 2015 4:03 pm

With the recent CBT bug, I was wondering if unchecking the "use CBT" option in job properties would alleviate the issue caused by that bug?

All my VM's have CBT enabled, what would happen if I uncheck that option? How are incrementals taken?
nullifi
Influencer
 
Posts: 23
Liked: 5 times
Joined: Wed Aug 05, 2015 1:41 pm
Full Name: Jason Taylor

Re: There's a new CBT bug in ESXi 6

Veeam Logoby Gostev » Fri Nov 13, 2015 4:12 pm 5 people like this post

Hi, all. There is very little info available from VMware on this issue, so we will put together some tests to try and reproduce it in-house. KB article wording prompted me to think that there is a chance B&R may not be affected, but to be sure we need to confirm where this bug actually sits.

For those who don't want to take chances, the easiest workaround is to disable the use of CBT data in the advanced backup job settings. Your backups will remain incremental, but they will take longer because the job will need to read the entire VM to determine changes.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: There's a new CBT bug in ESXi 6

Veeam Logoby Gostev » Fri Nov 13, 2015 4:24 pm 1 person likes this post

ashleyw wrote:surebackup won't save you either as just because the VM starts up - it doesn't mean that there hasn't been corruption on the image

Actually, even "stock" SureBackup job will immediately detect impact from such an issue at least on some VMs in your environment, thanks to application test scripts. This is because applications like AD, Exchange or SQL will refuse to mount corrupted application databases.

And for general purpose VMs, you could set up a custom SureBackup test script to remotely execute chkdsk, and collect its results via an exit code.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: There's a new CBT bug in ESXi 6

Veeam Logoby larry » Fri Nov 13, 2015 8:23 pm 1 person likes this post

I think I will Un read this until Monday as 5 oclock on Friday is almost here. :D
larry
Expert
 
Posts: 372
Liked: 89 times
Joined: Wed Mar 24, 2010 5:47 pm
Full Name: Larry Walker

Re: There's a new CBT bug in ESXi 6

Veeam Logoby chrmol » Sat Nov 14, 2015 11:45 am

Hi Veeam (Gostev :-) )
Would Active full backups also be suspect to corruption (as last time we saw CBT problems on 5.5) ?
/Christian
chrmol
Enthusiast
 
Posts: 36
Liked: 2 times
Joined: Mon May 17, 2010 7:41 pm
Location: Denmark
Full Name: Christian Moeller

Re: There's a new CBT bug in ESXi 6

Veeam Logoby Gostev » Sat Nov 14, 2015 4:41 pm

Potentially yes, they may also be suspect to corruption, because they leverage CBT data as well.
Instead of taking chances, at this point it is best to simply disable the use of CBT data completely.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Next

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 11 guests