VBM backup validator?

[mdiver]

How are you validating the consistency of your long-term stored VBM backups?

We just had a customer for which it turned out that storage problems made his VBM backups partly useless.
Also I fear that not every current on-prem S3 implementation is totally bug free right now.

So wouldn't it be nice if we could do a full check of a repos CRC consistency?
Quite like backup validator in VBR.

Thanks.

[Mildur]

Hi Michael

There is currently no health check built into our product to check older objects.
We use the object storage native controls to verify that our object was written away correctly at the time of writing. We do not perform regular checks on objects after they were written away.

Some object storages appliances provide functions to check the integrity of an object. If an object is damaged, they automatically replace it with a copy on another node. This can be helpful to detect damaged objects. I found at least three vendors which provides such functionality.
Does the appliance from this customer provide such functionality?

Thanks
Fabian

[mdiver]

Hi Fabian.

At least for S3 internal checks might be part of a solution.

For this specific customer it even was file based storage (JetDB).
So in general a CRC from within Veeam against the data would be useful I guess. You might want to consider it a feature request...
A workaround could be to do a backup copy of the full repository maybe? This would detect inconsistencies, wouldn't it?

Thanks
Michael

[Mildur]

Hi Michael

For this specific customer it even was file based storage (JetDB).

I thought we are talking about "not every current on-prem S3 implementation".

For jet db based databases, the customer can use a manual approach with esentutl.exe. It can be used to check the database integrity.
VB365 services have to be stopped while doing the integrity check.

You might want to consider it a feature request...

Chances are small, that there will be new features around jet db based repositories.
But for object storage-based repositories it would make sense to have some checks in place.

A workaround could be to do a backup copy of the full repository maybe? This would detect inconsistencies, wouldn't it?

Backup copies won't be a solution.. They require object storage as the source repository.

Thanks
Fabian

[oflores]

Hi everyone, I recently had a problem with my S3 storage, so I asked the support team how to check the status around my snapshots, and the answer was that I should open my restore points randomly, is this enough? Someone has been in the same case to be sure of the integrity of the information, how do they verify the integrity of restore points in an object storage?

Case ID #07086398

[Mildur]

Hi Oscar

Welcome to the forum.
Our customer support is correct on this matter.
There is currently no integrity check in VB365. We know about the request and we plan todo research how it can be done for a vNext version.
At the moment, manual restore testing is required to check if the restore points are healthy.

Best,
Fabian

[oscarm]

We are a couple of years further now, is there any news on integrity checks voor VB365?

[Polina]

Hi Oscar,

There's been meaningful progress here, even if it hasn't been loudly announced.

RND has shipped several verification and remediation mechanisms, which live with Support right now and get used as part of cases where integrity concerns come up.

One thing worth knowing: rather than building one big "run this on everything" integrity check, we've gone with targeted, scenario-specific tools. It's more work to build, but it means checks are actually fit for purpose rather than generic.

The reason we're not releasing them publicly (yet) is straightforward — they touch repository data directly. Getting that wrong is worse than not running the check at all, so we want Support in the loop to make sure the right tool gets used the right way.

If you've got a specific situation you're trying to address, a support case is the best path — they'll know exactly what to run.

[oscarm]

Hi Polina,

Thanks for clearing this. Does this mean that if I would like to do an integrity check for backups I can open a support case and they will help me out?