CBT Reset in v9

[btmaus]

Will simply restarting the VM cause the CBT to be reset, or do they have to be powered off?

[foggy]

Simply restarting the VM will not reset CBT. Ways to do that are described in the KB article.

[the_mentor]

I think what you can do for now is to create a PowerShell script and specify it as a pre-job activity. The script will check whether it's a full backup day and reset CBT for VMs chosen as a source for a backup job. Thanks.

Any suggestions on how to do such a thing?

[jamerson]

Wow cant believe this CBT bug on V9 too. and also on the latest ESXI 6,0 Build 3380124.
had to reset the CBT again.
is this a VMware or Veeam bug ? or both ?

[Gostev]

VMware bug, but it is actually fixed in the ESXi build you are using.

[jamerson]

VMware bug, but it is actually fixed in the ESXi build you are using.

after I've updated to the latest build 3380124. checked the ctkEnabled and scsi0:x.ctkEnabled both were on true.

[Gostev]

...and?

[jamerson]

I believe they should be false ?

[Gostev]

They should be false on VMs with changed block tracking disabled. I assume all your VMs have CBT enabled previously by Veeam or whatever other backup solution you are using (or have used)?

[jamerson]

We are using only VEEAM at our customers. that why I found it strange.
Now we cant trust VEEAM / VMWARE anymore .
every week we have to check all our Customers backups if they have CBT enabled.

[Gostev]

Based on this statement, you seem to think that the requirement to test backups for recoverability is an optional activity that depends on platform or backup technology in use? This is not the case. No backup can be relied upon unless they were tested.

[jamerson]

we are using Veeam for over 5 years now, never ever been worried about the back-up as the last month with the CBT bug.
what worries me is the data not the VM. if we can restore the data in a disaster we are ok.

[Gostev]

Perhaps you simply were not following these forums closely until lately, as the last month bug with CBT was not the first one in history. VMware had a few bugs with CBT in the past 5 years which resulted in data corruption.

[jamerson]

Perhaps you simply were not following these forums closely until lately, as the last month bug with CBT was not the first one in history. VMware had a few bugs with CBT in the past 5 years which resulted in data corruption.

I am glad I am following it now
thank you Anton

[stewsie]

I am starting to think about upgrading from V8 to V9 and noticed in the installation notes that CBT needs to be reset. Is this an absolute must do? I saw some mention of using a script to reset this. Is it an all or nothing approach or is it possible to do this on a number of VMs at a time?

This is just a question. Mo support call has been raised.

[Pat490]

the script worked great for me. you can do multiple VMs at once or single VM every time you run the script. I guess it depends on the amount of VMs that you have in your environment. But cbt reset has to be done!

[stewsie]

I am not a PowerCLI expert so please bear with me.

I have run the script against one VM and although it appears to work I see some red text with the following.

Proceed with resetting CBT for all 1 listed VMs? [Y/n] : y

Code: Select all

-------------------------------------------------------------------------------
Processing server
===============================================================================
Reconfiguring server to set SCSI0:0.ctkEnabled FALSE
Reconfiguring server to set SCSI0:1.ctkEnabled FALSE
Reconfiguring server to set ctkEnabled FALSE
Creating snapshot
[color=#FF0000]New-Snapshot : 11/02/2016 09:09:22    New-Snapshot        An item with the same key has already been added.
At C:\Files\Scripts\CBTReset\ResetCBT.ps1:149 char:33
+     Get-VM -name $vm | New-Snapshot <<<<  -Name Snapshotcbt -Description "snapshot for cbt reset" | Out-Null
    + CategoryInfo          : NotSpecified: (:) [New-Snapshot], VimException
    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewSnapshot[/color]

Deleting all snapshots on server
Reconfiguring server to set SCSI0:0.ctkEnabled TRUE
Reconfiguring server to set SCSI0:1.ctkEnabled TRUE
Reconfiguring server to set ctkEnabled TRUE
===============================================================================
CBT has been reset for  server 
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
Press any key to exit...

The snapshot process appears to complete in 2 seconds. I need to understand why the snapshot process errors in PowerCLI and if CBT Reset has actually worked.

Thanks

[geraldb]

Hi,

in the veeam b&r upgrade document (version 9) there is a point with:
Reset CBT for all VMs in the environment. For more information, refer to Veeam support article KB1940.
this point is listed after the upgrade is done ..
but the linked KB-entry is just for Version: 5.x, 6.x, 7.x, 8.x
https://www.veeam.com/kb1940

so do we have to do it ?

in this KB-entry it says also:
The following commands can be used to reset CBT for all VMs where it is presently enabled.
(Run these one line at a time after connecting to the vCenter Server via VMware PowerCLI.)

we only have standalone hosts without vcenter server .. is this a problem ?
ESXi hosts are version 5.1.0

best regards
Gerald

[geraldb]

P.S.: if we have never extended any vmdk disk we don't have to reset CBT ?
Am I right ?

[geraldb]

ok .. we have done the reset ob CBT already on all VMs ..

just for my understanding ..
After reseting CBT .. there should be full backups done from veeam of all VMs ?
replicas should also be "fully recreated" because CBT is reseted ?

[PTide]

Hi,

After reseting CBT .. there should be full backups done from veeam of all VMs ?

Resetting CBT will cause Veeam to read the whole VM disks. However if you suspect that your backup chain might have been corrupted then yes - you should do an active Full.

replicas should also be "fully recreated" because CBT is reseted ?

No, they should not. After CBT reset the whole VM disk will be read in order to find what blocks has chenged since last backup. The same applies to replication - the same amount of data to be transferred however initial virtual disk read might take significantly longer.

Please see the thread you've been merged to, escpecially this post and let us know if any questions persist.

Thank you.

[geraldb]

ok, thx!
sorry, I didn't saw that there is an existing thread for this

[Gostev]

Actually it is the same thread as you posted your question in, just the first page of it

[lukejf]

Hi Guys, I have ran the script in this post across some of our 5.5 hosts as come across an issue in a test restore. How do you determine if the CBT reset? The next veeam job doesn't seem to indicate any differences.

Edit
Looks like the read data in the veeam job before and after the reset indicates it has worked. Am i correct that the lastest version of veeam will now pickup HD changes and reset the CBT as required?

[Vitaliy S.]

Veeam backup server itself does not reset CBT data for the VMs you protect. If you did this manually and the entire virtual disk was read, then all subsequent runs will be using CBT data again.

[lukejf]

Veeam backup server itself does not reset CBT data for the VMs you protect. If you did this manually and the entire virtual disk was read, then all subsequent runs will be using CBT data again.

Reading the KB it indicates that future disk resizing will be detected by veeam and will reset the CBT. Is this information not accurate - See below.
Veeam Backup and Replication 8 has a built-in solution for this issue. Veeam Backup & Replication resets CBT for any resized VMware disk to prevent corruption.

[Vitaliy S.]

Yes, that quote is correct, I was just saying that issues discussed in this topic are not about disk resizing, but rather about ESXi updates that affect CBT data. This is not tracked by a built-in solution of the Veeam backup server.

[Peejay62]

At last, finally I will be able to upgrade to V9.. Just to talk about this CBT reset again. We are running V8.0.0.2084 en VMware 6.0 u2. Isn't it a viable option to reset cbt prior to upgrading to V9? I do not feel like cbt-resetting 200TB worth of VMs.
As it isn't a Veeam issue anyway and V8 seems to be handling any cbt bug this could be a convenient way for me to go to V9. I do some phased CBT resets in the coming weeks, after all completed I then upgrade to V9.
That aside, we have done numerous restores and surebackups and verifications the last year and we haven't encountered any cbt issues till today.

Thanks, Peter

[m.novelli]

I run the "Mass reset cbt.ps" script that you can find here https://www.veeam.com/kb1940 at every vSphere upgrade, I've never had issues out of dozens of customers

[Gostev]

I run the "Mass reset cbt.ps" script that you can find here https://www.veeam.com/kb1940 at every vSphere upgrade

I totally agree with this approach, especially if the environment size is not very large (otherwise, you can do this in portions).