Deleting specific backups from imported object storage

[ctg49]

Quick Q... is there a way to delete individual backups from an imported object storage? From disk I'd normally just delete the .vbk, rescan, and remove from configuration... can't really do that with object storage though without deleting the entire backup. Thoughts?

[Gostev]

Correct, you can't really delete individual restore points from backup in object storage like that. First, that would require deleting millions of individual objects, which is hardly manageable manually. But even if you create a script to automate this, you simply can't know which objects you can delete and which you cannot. Keep in mind that the same object is usually a part of many different restore points, including the most recent ones.

With file storage, you can just delete individual VBKs specifically because they are completely standalone. Every VBK is fully independent because it contains of independent copies of each block, resulting in many redundant copies of the same block within a volume (unless the file system supports block cloning).

While in object storage, backups are effectively "deduped". So you simply cannot delete blocks belonging to the specific restore point without rendering all other restore points useless.

[ctg49]

Right... so... how does one delete individual restore points then? Or is it just not possible? If it's not possible, can we submit for it to become possible? I suspect this isn't the first time a request to make restore points purgeable has come up.

[veremin]

But are you ok with this feature being purely cosmetic? Meaning specific restore point deletion will not result in consumed space reduction, cause:

While in object storage, backups are effectively "deduped".

Thanks!

[Gostev]

Bad choice of words. While it's not purely cosmetic, you won't see any significant disk space consumption changes as a result, like you would when nuking a VBK from a regular file system. As almost all objects behind the deleted restore point will still stay in place due to being a part of more recent restore points.

May be you can start from clarifying what exactly are you trying to achieve. What is the actual need behind purging individual restore points? For example, if you're deleting oldest backups because your retention policy is longer than needed, then you should simply change the retention policy in the job settings - and it will clean up oldest backups automatically.

[ctg49]

So there's a relatively complicated background to this but I'll give it a swing.

We are migrating our data from one Wasabi bucket to another for some rather silly accounting reasons. As such, I'm downloading the files in question, physically moving them to another SOBR, then letting them re-upload into the new repository. That part actually isn't working correctly, and I've got a ticket in for it. The deleting part comes in from the old object repository which is, at present, an imported object storage. As I've been downloading those vbk's from the object storage, I'd like to be able to delete them out a) for tracking purposes and b) to save space for billing reasons while we go through this process. Now if you say it doesn't actually utilize much space that latter part stings a bit less. Worst case I guess I can just excel it up, and delete the whole bulk at once. I still say it's a relatively simple feature request for the product though, and at minimum would have required less work from a couple people in this thread today

[Gostev]

So why deleting them all it once is a "worst case" option? Is it because you expect the full migration to take weeks/months and thus you'd like to remove already migrated data in portions?

[veremin]

Additionally, may we ask you to share the ticket number, so that we can check what the issue was with the migration procedure? Thanks!

[ctg49]

Basically yes, it's a long-term migration since we're working with several hundred TB of data. We'd definitely prefer not to lose the backups, but not sure if management has the appetite for maintaining them for months until we reach the retention window for them, and we don't have the space to store them on-site.

Ticket number: 05273428