CBT Data is Invalid - Shouldn't this be an Error/Warning?

Availability for the Always-On Enterprise

CBT Data is Invalid - Shouldn't this be an Error/Warning?

Veeam Logoby Cragdoo » Thu Dec 21, 2017 2:56 pm

While investigating a replication job taking longer than expected I cam across this in the job statistics

Image

Code: Select all
18/12/2017 20:08:01 :: CBT data is invalid, failing over to legacy incremental backup. No action is required, next job run should start using CBT again. If CBT data remains invalid, follow KB1113 to perform CBT reset. Usual cause is power loss.

I would have thought this would flag up as a warning rather than green tick? Given it will signficantly increase the duration of you backups, possibly breaching any backup windows you may have set.
Cragdoo
Veeam Vanguard
 
Posts: 489
Liked: 181 times
Joined: Tue Sep 27, 2011 12:17 pm
Location: Scotland
Full Name: Craig Dalrymple

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby Gostev » Thu Dec 21, 2017 4:16 pm

Our paradigm for events is a follows:

Success/Info event > Backup is created, and no action is required from end user.
Warning event > Backup is created, but action is required from end user.
Error event > Backup creation fails.

So this case is clearly Success/Info event one, no doubt.

But even when in doubt, we always give preference to Success over Warning. This is after dozens of situations in the past 10 years where a warning in some newly added functionality would instantly result in a multi-page topic of unhappy customers who don't like to receive this new warnings, forcing us to revert one to Info event in the immediate update. For what it's worth, everyone wants to get nothing but green emails unless something happens when they really need to act.
Gostev
Veeam Software
 
Posts: 22400
Liked: 2675 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby tsightler » Thu Dec 21, 2017 6:34 pm 1 person likes this post

This is a tough one because this problem is usually transient and self-corrects on the next run, however, if it fails for multiple runs, action will almost certainly be required from the end user to correct the issue. I just had this exact situation come up twice in the last week, both customers were seeing very long backup times, running into their production day and causing issues (in one case aborting due to backup window), but there's "nothing wrong" according to Veeam. Investigating turned up mulitple VMs with the invalid CBT issue every night but the customer scanned right over the message in the event log because it was green. Resetting CBT for the impacted VMs corrected the issue after one additional run.

It would be really nice if we could potentially turn this into a warning only in cases where the previous sessions had also experienced the failure. Perhaps another option would be to use a blue "info" status for such a message (I know, we don't really have that status today, but I could see it's use in other cases as well), something that doesn't change the overall status of the job/task, but would stand out when scanning the task log. Just throwing out some ideas, I certainly understand our current reasoning, but it's somewhat misleading when this failure happens to be persistent rather than transient.
tsightler
Veeam Software
 
Posts: 5090
Liked: 2011 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby Cragdoo » Fri Dec 22, 2017 10:49 am

I'm with Tom on this one, having just been in an almost identical situation. I'll add a +1 for the info/blue icon, for the exact same reasons as Tom.

everyone wants to get nothing but green emails unless something happens when they really need to act.

I have to disagree, I'd rather have a warning, check it out for it be ok, that get a green email, assume it was ok and find out it wasn't ok.
Cragdoo
Veeam Vanguard
 
Posts: 489
Liked: 181 times
Joined: Tue Sep 27, 2011 12:17 pm
Location: Scotland
Full Name: Craig Dalrymple

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby nitramd » Fri Dec 22, 2017 2:43 pm

I concur also with Tom for these situations - I've seen several over the past few months.

+1 for the blue icon/info status.
nitramd
Enthusiast
 
Posts: 93
Liked: 16 times
Joined: Thu Feb 16, 2017 8:05 pm

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby MSweetman » Wed Feb 14, 2018 11:04 pm

We are also seeing this causing issues due to extended backup times.

Please add one more request for a Blue icon/info status.

Thank you.
MSweetman
Service Provider
 
Posts: 21
Liked: never
Joined: Thu Oct 29, 2015 3:21 am
Full Name: Mark Sweetman

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby csydas » Sat Feb 17, 2018 10:22 am

Not a bad idea, but at least in the case of older Hyper-V Backups it wouldn't be useful with the proprietary CBT tech, right? Cause from the Hyper V CBT article, it's expected that after resetting CBT that you'd need multiple runs before the CBT is properly built.

Honestly, the more I think about it the more it seems that HyperV is just the wrench for this - there are currently a lot of situations where it's normal and expected for Hyper V to not get any CBT (SMB3 shares on Nutanix for example...though I guess that goes away in v10, or mixed host levels). Maybe there's a way to just apply it to VMware backups?
csydas
Enthusiast
 
Posts: 82
Liked: 19 times
Joined: Tue Jan 16, 2018 5:14 pm
Full Name: Harvey Carel

Re: CBT Data is Invalid - Shouldn't this be an Error/Warning

Veeam Logoby MSweetman » Mon Feb 19, 2018 10:06 pm

Hi Product Mgt.
Yet another reason to have a warning. A customer's ESX 6.0 cluster was not patched for a CBT issue which caused a number of VM backups to be corrupt. (see https://kb.vmware.com/s/article/2116127 ... ateRoute=1)
Even though Daily SureBackups run which appeared to be successful, there was still an issue.

Thank you.... Mark
MSweetman
Service Provider
 
Posts: 21
Liked: never
Joined: Thu Oct 29, 2015 3:21 am
Full Name: Mark Sweetman


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], getontoit99, Google Feedfetcher, Majestic-12 [Bot] and 83 guests