SureBackup for only certain VMs in object storage

[cgsm]

Is there a way to create a SureBackup job for an Object Storage repo?

I have a Backup Copy job transferring backups to Wasabi. I would like to test some of the VMs to make sure I can restore them successfully. I do not want to use the standard health check because this will process all of my VMs resulting in A LOT of download traffic from Wasabi (and it takes too long to complete, if ever, too). Therefore, I would like to run a SureBackup job on a subset of my VMs.

However, when I try to set up a SureBackup job, I cannot configure it as needed. When I go to choose a job (either for a selected application group), I cannot choose the Backup Copy job nor can I choose the object storage repo in anyway.

[Gostev]

SureBackup from Backup Copy is not currently supported. SureBackup is designed to test backups on primary backup repository, before they are copied elsewhere (normally to some storage that is much slower).

@veremin are we adding "health check lite" option for backups on object storage repositories as to avoid the above-mentioned download traffic?

[veremin]

Aren't you thinking about Capacity Tier health check by any chance?

We are adding it in version 12a. Initially, it will be provided in the light version, meaning it will merely ensure that the required objects are still in place without checking the actual content. Fully-blown Capacity Tier health check will come after v12a release.

In the case of standalone object storage repositories, you can do the full health scans only (managed agents are the exception). However, you can avoid downloading content locally by deploying a helper appliance in the cloud - this way traffic will not leave the cloud during the health check process.

Light health check for standalone object storage repositories (for everything besides managed agents) is in the long-term plans.

Thanks!

[cgsm]

@gostev
Yes, I understand I can perform Sure Backup on my local data. I currently I do this once a day. However, this doesn't prevent some sort of corruption issue occurring while the data is in transit to the object storage or an issue with the object storage itself (or really any Backup Copy destination repo).

It sounds like the "lite" health check will be helpful since it will at least confirm that "what VBR thinks is in the object storage is actually there". I still think it would be helpful to use a Backup Copy repo (object storage or not) as the repo to run a Sure Backup job from/on.

@veremin
We cannot use a helper appliance with Wasabi, they only provide storage! It is the download volume of data that is the real killer to the full health checks set up with each job.

[veremin]

Can you provide more information regarding the corruption issue? Maybe a ticket number or something so we can double-check the investigation and see whether there is a problem to address.

v12a Capacity Tier with its light health check might be a solution for your needs, but it will require configuration changes (replacing backup copy and standalone repositories with Scale-Out Backup Repository and Capacity Tier).

Thanks!

[cgsm]

@veremin
I don't have a corruption issue now. But I would like to test by object storage stored repo in case of this. You can never be too sure! As of right now, verifying object storage restore points is a manual "restore VM" process which is a bit annoying to do so often and thus gets pushed down the to-do list.

I moved away from SOBR (capacity tier) to direct-to-object so I could control the number of restore points and GFS separately for my main VBR repo and the object storage repo. I am not looking at going back to SOBR.

***

I have an idea, not sure if this has been considered: Instead of downloading all the blocks to reconstruct and start a VM, why not only download a random subset (say 10%, configurable) and compute a checksum for each downloaded block and compare it to a checksum of each block before it was uploaded to the object storage? You might not even have to download the blocks since S3-compatible object storage set an "Etag" header for each block, therefore only a HEAD request would be needed drastically reducing downloaded data.

[veremin]

But in most cases, it wouldn't identify backup file corruption, so we might as well implement the light health check where we verify that blocks the backup chain consists of are present. This should decrease traffic to a minimum.

We have this requirement tracked internally for post-v12a major releases.

Thanks!

[cgsm]

@veremin

I assume nothing has changed here in v12.1 regarding "lite" healthcheck/SureBackup for standalone object storage repos? As in, there is still no good way to run "storage level corruption guard" against a Wasabi repo without downloading all the data.

[Gostev]

Standalone objects storage repos supported "lite" health check since inception? 12.1 adds the same capability to SOBR Capacity Tier because it was missing there.

[cgsm]

@Gostev
I am not sure this is true... I only moved from SOBR to standalone object repo as of v12. I had to turn off the health check because it took too long (downloading every block of data), hence this forum thread.

I will turn on the "storage level corruption guard" again for the copy-to-object job and see if something has changed.

Am I missing something here or misunderstanding?

[Gostev]

Do you perhaps have the Verify content of each object in backup option selected? This is the option that controls whether to download the content of every object, or just check for object's existence.

[cgsm]

@Gostev
I am not sure if the image at the link you posted is general, or specific to this question/thread... Regardless, I do not see the "verify content of each object in backup" checkbox.

To clarify, I am using the object storage repo as the storage location for a Backup Copy job for VMs.

[Gostev]

Oh, OK. I'm not sure if Backup Copy jobs include the same capabilities as primary backup jobs as it comes to object storage. @veremin will know for sure of any current limitations and our plans around them.

Our recommendation for your scenario of offloading local backups to cloud object storage specifically is of course to use SOBR Capacity Tier, which provides health check lite in 12.1

[cgsm]

@Gostev
Yes, I know using SOBR is a more typical/recommended route. We used to use a SOBR. We switched to a separate Copy Job when v12 came out so that we could manage the retention policy on the local storage separate from the cloud-based object storage, mostly to save on cloud storage costs (as you note here: post477770.html#p477770). We retain backups on local storage for 30 days, with 4 weekly, and 12 monthly; on the cloud object storage we store 7 days, 4 weekly, 3 monthly.

I will await a response from veremin. Thanks.

[veremin]

Currently, the lite health check is available only for agent backup jobs pointed to object storage repositories (the one illustrated on Anton's screen) and Capacity Tier. Thanks!

[cgsm]

@veremin
Thanks for the confirmation. It would be great if the "lite" option was available for standalone object repos, or even the new "perform scan of a random subset of VMs in backup" feature.

[veremin]

Understood, we will keep this request tracked for future product versions. Thanks!

[jazzoberoi]

+1 just checking if health check lite is available in the latest version of veeam for standalone s3 repo used in backup to copy jobs?