ESXi 6 CBT Bug reset

[B.F.]

Both Veam and Vmware are up to date to get past the CBT bug issue.

Now going through the VM's per https://www.veeam.com/kb1113 to reset CBT. I'm not scripting but just going through the motions via the client. An interesting symptom I've noticed while doing step 5 (deleting *-CTK.vmdk files). When I browse to the datastore of the powered off VM, select the *-CTK.vmdk files, delete, they are gone but then a few seconds later, they reappear?? The modified time has changed to the current time however.

I've tried this on both vSphere Client and Web Client on 4 different VM's spanning 2 different hosts, symptom is the same. Is this normal?

Thanks!

[PTide]

Hi,

a few seconds later, they reappear??

Were those files of the same size as before deletion? If so then please check if files are locked by something.

Thank you.

[B.F.]

The size did appear to be the same. I will have to do another VM CBT reset to confirm if they were exact. How do I check if they are locked and by what? I get no message when I delete the files that they are locked.

So I'm assuming that this is not normal.

Thanks!

[PTide]

I get no message when I delete the files that they are locked.

I just want to make sure it's not a UI bug and that files are not locked and are actually get deleted. You can use the vmkfstools -D command. Please see this guide for detailed description.

Most likely the reason for files to reappear is that in ESXi and vCenter v5.1 and later missing ctk.vmdk files are recreated automatically.

Thank you.

[B.F.]

When deleting the -CTK files, the new ones are the exact same size. The difference I see is the modified time/date is now current. Did the vmkfstools -D and there are no locks on the -CTK files.

Everything is awesome?

[Gostev]

I find it strange that they reappear on a VM that has CBT disabled (assuming you followed KB).
Can you try power cycling the VM before deleting those -CTK files?

[B.F.]

That is my exact feeling as well. If the VM is powered down, the parameters described in the KB are switched to "false", and the files are deleted, for them to pop right back seemed odd. Note: They don't actually show up on the list view on their own. I either have to do a refresh of the view or close and re-open the viewing window and they are then back.

So for fun I SSHed into one of the hosts and went into the last test VM's directory where I attempted to remove the -CTK. I deleted them via the CLI and they stayed gone! I popped right back to the client and sure enough they were still gone.

I then powered up the VM, did a few MS security patches, then powered it back down. -CTK files are still gone.

Powered up the VM again but this time did a replication. Completed, powered down the VM. -CTK files are back and the parameters were set back to "true" (as expected). I then attempted to the same CBT reset process again via the client and the -CTK files pop right back just like before. The task list on the client even shows "Delete file" competed for each one I selected.

For even more fun, I decided to see what would happen if I were to delete the -CTK files from the Files section in Veeam B&R. Browsed to the folder and then deleted. The -CTK files were gone from the list in B&R. (Don't see any refresh option). This time the client's Recent Tasks list shows "Deleted datastore file" instead of "Delete file" like before. I change folders in B&R to try and make it refresh, files still appear to be gone. Hop back over on the client and the files are indeed still there. I confirmed via the SSH session as well. So I closed B&R, re-opened and then they showed up again.

My question: Doing the -CTK file deletion via either B&R or the client where they magically come back is good enough or do I need to force my hand via an SSH CLI?

Thanks

[B.F.]

I just so happened to attend a VMUG meeting yesterday afternoon. No one had any good answers there as well. Most of them ran the script to reset the CBT (which I have not) but now they are wondering if the -CTK files were actually deleted as well.

[jamerson]

I just so happened to attend a VMUG meeting yesterday afternoon. No one had any good answers there as well. Most of them ran the script to reset the CBT (which I have not) but now they are wondering if the -CTK files were actually deleted as well.

In one of our customer host had the same issue.
the only solution was to move the VMs with vMotion to another host and reboot the host .

[damadhatter]

I wondered this same thing, talking to both VMware & Veeam support they seemed to think this was normal. The next time the Veeam job ran it did mention invalid CBT data and the run after that no mention of anything other than the use of CBT.