Discussions related to using object storage as a backup target.
Post Reply
jazzoberoi
Enthusiast
Posts: 96
Liked: 24 times
Joined: Oct 08, 2014 9:07 am
Full Name: Jazz Oberoi
Contact:

How does deletion with governance work?

Post by jazzoberoi »

Hi Steve.
Appreciate the reply and great article :)

I’m starting from scratch with a new VBR server.
Could you please also explain how the delete process works in governance mode? Is this done from the S3 bucket end or from VBR server end?

Wondering if there is a happy compromise that can be made so we can set long GFS retention without fearing lock-in if things start getting out of hand with costs.

Unless, veeam introduce a more streamlined and granular immutability where we can say select daily/weekly/monthly as immutable but not yearly etc..

#MOD: split from Backup Copy to Object Storage vs. Scale-Out Backup Repository Copy Mode
jazzoberoi
Enthusiast
Posts: 96
Liked: 24 times
Joined: Oct 08, 2014 9:07 am
Full Name: Jazz Oberoi
Contact:

Re: Backup Copy to Object Storage vs. Scale-Out Backup Repository Copy Mode

Post by jazzoberoi »

Hi Steve,
Just following up to see if you can please explain how to delete a immutable backup in governance mode please..
Mildur
Product Manager
Posts: 8735
Liked: 2296 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: Backup Copy to Object Storage vs. Scale-Out Backup Repository Copy Mode

Post by Mildur »

Hi Jazz

Any user with the s3:BypassGovernanceRetention permission can delete such objects.
https://docs.aws.amazon.com/AmazonS3/la ... ing-bypass

It's not possible from within the Veeam console. You need to logon directly to the storage (object storage UI, object storage browsing tools, ...) and delete such objects from there.

Best,
Fabian
Product Management Analyst @ Veeam Software
jazzoberoi
Enthusiast
Posts: 96
Liked: 24 times
Joined: Oct 08, 2014 9:07 am
Full Name: Jazz Oberoi
Contact:

Re: Backup Copy to Object Storage vs. Scale-Out Backup Repository Copy Mode

Post by jazzoberoi »

Hi Mildur,
Thanks for your reply.

Wouldn’t that mess up the retention points in Veeam though? As these is no concept of VBK / VIB files in S3, how will we know which objects belong to which GFS point, and whether or not deleting them will cause issue with other restore points?

It would be great if we could get some Governance bypass within Veeam to manage this?
Mildur
Product Manager
Posts: 8735
Liked: 2296 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: Backup Copy to Object Storage vs. Scale-Out Backup Repository Copy Mode

Post by Mildur »

Hi Jazz
Wouldn’t that mess up the retention points in Veeam though? As these is no concept of VBK / VIB files in S3, how will we know which objects belong to which GFS point, and whether or not deleting them will cause issue with other restore points?
Of course. Deleting objects used by Veeam directly in object storage is not supported.
Governance mode is meant for service providers to delete "EVERY OBJECT" when a customer has cancelled his contract with the service provider.
It gives the service provider an option to completely clean out the bucket and then delete it. A service provider needs that functionality or he may need to keep ex-customers backups for years.
It would be great if we could get some Governance bypass within Veeam to manage this?
Every attacker or malicious insider would love that feature as well :)
The primary reason for Immutability is to protect your backups against deletion. Having a bypass option directly in the Veeam console is not a good idea. Any attacker will be able to make use of the bypass as well.
Also make sure that the S3 credentials you use to connect from the backup server to the bucket doesn't have this bypass permission. An attacker can export those credentials from the Veeam server and start deleting your backups.

s3:BypassGovernanceRetention must only be given to dedicated users which are only used for emergency access.

Best,
Fabian
Product Management Analyst @ Veeam Software
jazzoberoi
Enthusiast
Posts: 96
Liked: 24 times
Joined: Oct 08, 2014 9:07 am
Full Name: Jazz Oberoi
Contact:

Re: Backup Copy to Object Storage vs. Scale-Out Backup Repository Copy Mode

Post by jazzoberoi »

Mildur wrote: Mar 15, 2024 11:33 am Hi Jazz

Any user with the s3:BypassGovernanceRetention permission can delete such objects.
https://docs.aws.amazon.com/AmazonS3/la ... ing-bypass

It's not possible from within the Veeam console. You need to logon directly to the storage (object storage UI, object storage browsing tools, ...) and delete such objects from there.

Best,
Fabian
Hmm.. looks like Steve, the author of this article has a different view on where it should be deleted from! The only question is how is it done from Veeam?
https://community.veeam.com/blogs-and-p ... art-7-6757

Image
Mildur
Product Manager
Posts: 8735
Liked: 2296 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: How does deletion with governance work?

Post by Mildur »

Steves comment is correct. Deleting restore points of selected machines is only supported from the backup console. As you mentioned earlier, there are no backup files. A backup administrator will not be able to know which objects has to be deleted. Therefore the backup console has to be used.
Which I confirmed already in my last comment:
Of course. Deleting objects used by Veeam directly in object storage is not supported.
The backup console wouldn‘t work for backups which are still immutable. The console would throw an error that those restore points are still immutable. Doesn‘t matter if governance or compliance mode was used.

I also explained why we have the governance mode. Service Provider who need to delete all backups, because a customer has left. The entire repository. Deleting all objects directly on the object storage is ok and totally supported, because the backup server won‘t use this repository anymore. Just remove the repository from the backup server and delete everything on it.
Governance mode is meant for service providers to delete "EVERY OBJECT" when a customer has cancelled his contract with the service provider.
It gives the service provider an option to completely clean out the bucket and then delete it. A service provider needs that functionality or he may need to keep ex-customers backups for years.

To summarize this topic:
- you can use governance mode as an exit strategy to delete an entire bucket with immutable backups
- you cannot delete backups from single machines directly on object storage (outside of Veeam Backup & Replication)
- we most likely will never introduce an option to delete immutable backups within the backup console when governance mode was used. This opens a huge security hole

Best,
Fabian
Product Management Analyst @ Veeam Software
Post Reply

Who is online

Users browsing this forum: No registered users and 10 guests