Using object storage as a backup target
Post Reply
lowlander
Service Provider
Posts: 443
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Deletion Markers

Post by lowlander »

Hi,

I recently ran into an issue with offloading to object storage with immutability enabled on the bucket. Some research resulted in a known issue with deletion markers not being deleted properly. This seems to be an issue with V11, and there is a fix available.

Not being aware of this issue the fix replaces the agents on the satelites / gateway servers, and a script is provided to remove the deletion markers. I am quite intereste since when this issue has been determined to be an issue ;)

Anyone else having experience with this workaround, and can the importance of deletion markers be explained in relation to immutability on bucket level.

sfirmes
Veeam Software
Posts: 147
Liked: 82 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Deletion Markers

Post by sfirmes »

@lowlander the issue was the actual creation of the deletion markers and not the removing of them. We didn’t intend to create them, so they were of no importance.

The combination of the patch to v11 and the script that removed the deletion markers will resolve everything.
Thanks,

Steve

lowlander
Service Provider
Posts: 443
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: Deletion Markers

Post by lowlander »

Hi Steve,

thanks for you quick response. As I understood from the AWS documentation :

A delete marker in Amazon S3 is a placeholder (or marker) for a versioned object that was named in a simple DELETE request. Because the object is in a versioning-enabled bucket, the object is not deleted. But the delete marker makes Amazon S3 behave as if it is deleted.

As I read this, the objects will still exists and the delete marker behave as if the object is deleted.

Can you explain more in detail if I am missing something ;) ?

sfirmes
Veeam Software
Posts: 147
Liked: 82 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Deletion Markers

Post by sfirmes »

You are correct. This is a feature when using versioning, which is required to use object lock.

When you request an object to be deleted that has an active lock on it, a new version of the object with a deletion marker is created. The original object is still intact. If you remove the deletion marker, you “un-delete” the original object.
Thanks,

Steve

lowlander
Service Provider
Posts: 443
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: Deletion Markers

Post by lowlander »

So basically deleting the deletion marker will get back the object, being able to delete this object is as it is not being locked/immutable anymore, and being able to delete. Would this result in a lot of objects being locked when the deletion marker is created unintendedly, resulting in objectstorage being overused ?

veremin
Product Manager
Posts: 19861
Liked: 2141 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Deletion Markers

Post by veremin »

Not sure whether I completely understood the question, but even if you remove a delete market from a locked object, you will not be able to delete the object (as the object lock period has not expired yet, so the object is still immutable).

sfirmes
Veeam Software
Posts: 147
Liked: 82 times
Joined: Jul 24, 2018 8:38 pm
Full Name: Stephen Firmes
Contact:

Re: Deletion Markers

Post by sfirmes »

Not entirely sure what your question is, but let me try to answer what I think your asking.

Yes, removing the deletion marker will “get back” the original object. But this won’t “result in a lot of objects being locked”, nor will it result on overuse of the object storage.

If you are referring to the script we use to delete the markers we didn’t intend to create, the objects were already correctly deleted permanently and all the script is doing is removing the deletion markers. Those markers are the only version of the object so deleting them didn’t “un-delete” any objects. Deleting them simply deleted them.

Hope this answers your questions and/or clarifies things.
Thanks,

Steve

lowlander
Service Provider
Posts: 443
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: Deletion Markers

Post by lowlander »

Thanks Steve,

this clarifies my question indeed 😉

Post Reply

Who is online

Users browsing this forum: No registered users and 9 guests