ESXi 6 CBT Bug reset

VMware specific discussions

ESXi 6 CBT Bug reset

Veeam Logoby B.F. » Tue Dec 15, 2015 3:29 pm

Both Veam and Vmware are up to date to get past the CBT bug issue.

Now going through the VM's per https://www.veeam.com/kb1113 to reset CBT. I'm not scripting but just going through the motions via the client. An interesting symptom I've noticed while doing step 5 (deleting *-CTK.vmdk files). When I browse to the datastore of the powered off VM, select the *-CTK.vmdk files, delete, they are gone but then a few seconds later, they reappear?? The modified time has changed to the current time however.

I've tried this on both vSphere Client and Web Client on 4 different VM's spanning 2 different hosts, symptom is the same. Is this normal?

Thanks!
B.F.
Expert
 
Posts: 107
Liked: 6 times
Joined: Tue Jan 28, 2014 5:41 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby PTide » Wed Dec 16, 2015 2:23 pm

Hi,

a few seconds later, they reappear??
Were those files of the same size as before deletion? If so then please check if files are locked by something.

Thank you.
PTide
Veeam Software
 
Posts: 3019
Liked: 246 times
Joined: Tue May 19, 2015 1:46 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby B.F. » Wed Dec 16, 2015 2:27 pm

The size did appear to be the same. I will have to do another VM CBT reset to confirm if they were exact. How do I check if they are locked and by what? I get no message when I delete the files that they are locked.

So I'm assuming that this is not normal. :(

Thanks!
B.F.
Expert
 
Posts: 107
Liked: 6 times
Joined: Tue Jan 28, 2014 5:41 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby PTide » Wed Dec 16, 2015 2:40 pm

I get no message when I delete the files that they are locked.
I just want to make sure it's not a UI bug and that files are not locked and are actually get deleted. You can use the vmkfstools -D command. Please see this guide for detailed description.

Most likely the reason for files to reappear is that in ESXi and vCenter v5.1 and later missing ctk.vmdk files are recreated automatically.

Thank you.
PTide
Veeam Software
 
Posts: 3019
Liked: 246 times
Joined: Tue May 19, 2015 1:46 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby B.F. » Wed Dec 16, 2015 4:36 pm

When deleting the -CTK files, the new ones are the exact same size. The difference I see is the modified time/date is now current. Did the vmkfstools -D and there are no locks on the -CTK files.

Everything is awesome?
B.F.
Expert
 
Posts: 107
Liked: 6 times
Joined: Tue Jan 28, 2014 5:41 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby Gostev » Wed Dec 16, 2015 8:16 pm

I find it strange that they reappear on a VM that has CBT disabled (assuming you followed KB).
Can you try power cycling the VM before deleting those -CTK files?
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: ESXi 6 CBT Bug reset

Veeam Logoby B.F. » Wed Dec 16, 2015 9:56 pm

That is my exact feeling as well. If the VM is powered down, the parameters described in the KB are switched to "false", and the files are deleted, for them to pop right back seemed odd. Note: They don't actually show up on the list view on their own. I either have to do a refresh of the view or close and re-open the viewing window and they are then back.

So for fun I SSHed into one of the hosts and went into the last test VM's directory where I attempted to remove the -CTK. I deleted them via the CLI and they stayed gone! I popped right back to the client and sure enough they were still gone.

I then powered up the VM, did a few MS security patches, then powered it back down. -CTK files are still gone.

Powered up the VM again but this time did a replication. Completed, powered down the VM. -CTK files are back and the parameters were set back to "true" (as expected). I then attempted to the same CBT reset process again via the client and the -CTK files pop right back just like before. The task list on the client even shows "Delete file" competed for each one I selected.

For even more fun, I decided to see what would happen if I were to delete the -CTK files from the Files section in Veeam B&R. Browsed to the folder and then deleted. The -CTK files were gone from the list in B&R. (Don't see any refresh option). This time the client's Recent Tasks list shows "Deleted datastore file" instead of "Delete file" like before. I change folders in B&R to try and make it refresh, files still appear to be gone. Hop back over on the client and the files are indeed still there. I confirmed via the SSH session as well. So I closed B&R, re-opened and then they showed up again.

My question: Doing the -CTK file deletion via either B&R or the client where they magically come back is good enough or do I need to force my hand via an SSH CLI?

Thanks
B.F.
Expert
 
Posts: 107
Liked: 6 times
Joined: Tue Jan 28, 2014 5:41 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby B.F. » Fri Dec 18, 2015 2:08 pm

I just so happened to attend a VMUG meeting yesterday afternoon. No one had any good answers there as well. Most of them ran the script to reset the CBT (which I have not) but now they are wondering if the -CTK files were actually deleted as well.
B.F.
Expert
 
Posts: 107
Liked: 6 times
Joined: Tue Jan 28, 2014 5:41 pm

Re: ESXi 6 CBT Bug reset

Veeam Logoby jamerson » Sat Dec 19, 2015 4:33 pm

B.F. wrote:I just so happened to attend a VMUG meeting yesterday afternoon. No one had any good answers there as well. Most of them ran the script to reset the CBT (which I have not) but now they are wondering if the -CTK files were actually deleted as well.

In one of our customer host had the same issue.
the only solution was to move the VMs with vMotion to another host and reboot the host .
jamerson
Expert
 
Posts: 257
Liked: 16 times
Joined: Wed May 01, 2013 9:54 pm
Full Name: Julien

Re: ESXi 6 CBT Bug reset

Veeam Logoby damadhatter » Sun Dec 20, 2015 11:50 pm

I wondered this same thing, talking to both VMware & Veeam support they seemed to think this was normal. The next time the Veeam job ran it did mention invalid CBT data and the run after that no mention of anything other than the use of CBT.
damadhatter
Influencer
 
Posts: 24
Liked: never
Joined: Fri Feb 21, 2014 5:15 am
Full Name: Chris A


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 8 guests