Hidden corruptions, synthetic fulls?

[lando_uk]

Hi
I've had a frustrating week with Veeam...
We recently had a disk issue on a large 33TB repository volume, it was fixed, but left some of our copies and backups corrupt.
The health check on the copy jobs detected errors, and we dealt with them. But the reports do tend to get lost with all the other warning and error logs that the copy jobs tend to spam out. It would be good if a failed health check produced custom (more critical) error that could be filtered out from the normal noise.
Most of the jobs on this corrupted volume were short retention copy jobs that can be easily re-seeded without too many headaches, but we also had some longer retention backup jobs on there that didn't show any issues until we ran the cmd line validator. There is no health check feature for Backup Jobs, just an Integrity check which didn't detect any problems.
I know you'll say that we should now run a surebackup, but is that actually feasible on all VM's in a job and also all jobs, each week? The validator takes about 15hrs to process each backup job, would surebackup work quicker, would it be able to process all VM's in a 5TB VBK in a few hours? (whilst offsite copy jobs are also accessing the same VBK).

Maybe it would be a good thing if there was an feature for a background low priory disk validator, like a RAID controller patrol check that constantly checks all your VBKs when the system is idle and finds any corruptions.

Or do I just need to spend more money on faster disks, dedupe devices etc, so I can run and store proper full backups rather that synthetic with rollbacks?

[Gostev]

There is no health check feature for Backup Jobs

Hi, Mark! Good news, we added Health Check functionality to primary jobs in v9.

Funny you should ask about this right now, when I had the corresponding dialog open in Visual Studio... what are the chances with hundreds of dialogs I need to go through for my UI review?

[lando_uk]

That's good! I don't suppose v9 will also add another 2 days to the week so I have a window to run the health checks !

When a health check / validation is running, does it lock for all other activity?

[foggy]

Backup files are locked for read during health check, so you should be able to restore from them and run backup copy with this job as a source.

[lando_uk]

Do you mean locked for Write?

What happens if the health check doesn't finish for when the next backup runs? Does it cancel and run the backup, or will the backup fail? And will the health check then continue where it left off next time?

[foggy]

Health check is considered a part of the job, so it does not finish until health check is complete (and hence, cannot start the next cycle until that).