background retention probably not working?

[mcz]

Hi everyone,

I'm using a SOBR with an immutable object storage (90 days). The GFS BCJ also has a retention of 90 days. Now I had to export a certain restore point and was very close to the retention, so in order not to loose that restore point I've increased the retention for the GFS-job up to 180 days. What I saw was an increase in consumed space on the object storage from that point on, as the incrementals weren't deleted/merged anymore. After 3 months I was able to finish the export and reversed the retention on the BCJ back to 90 days.

I would have expected the background retention to kick in and clean the object storage from all the unneded objects, but it didn't happen as we're meanwhile did run out of disk space (the 3-4 TB I had to increase in that period didn't get reclaimed) and also the object storage provider (wasabi) doesn't show a significant amount of deleted objects in the corresponding period in the portal.

In the properties of the backup job on the object storage I do not see the increments half a year ago, so at least the picture is what I'd expect - now I'd love to get the space back.

I haven't created a case yet, but will if it is unexpected. Am I just too unpatient or is this a expected situation? Thanks!

[veremin]

Aren't those objects still immutable and thus still cannot be deleted, by any chance? Thanks!

[mcz]

well, immutability is and was set to 90 days. and we have reached that period more than twice. AFAIK the immutability (period) is set once the object is being written - and that was in January, when I change the retention on the BCJ, it shouldn't change the immutability, right?

[tyler.jurgens]

When you set GFS policies, those GFS restore points are set immutable for the duration of that GFS settings.

Eg: If you set GFS to keep 6 monthly backups, those monthly backups will be immutable for 6 months, even if you set the repository immutability to 10 days. 10 days is the minimum immutability period, the GFS settings will set their own immutability period.

[mcz]

I'm not really talking about the GFS, I mean the "normal" increments. I had many increments in January and they had a 90 days retention. I've increased it in march to 180 days and decreased it in June to 90 days again. Are you really sure that these increments got a new retention of 180 days?

[mcz]

Meanwhile I've created a ticket - # 06178762. So far, it's a bit hard to get some progress...

[mcz]

OK, so allthough not fully validated, I'm going to post what the engineer has sent me:

I also checked internally and it seems that the block deletion logic has changed when combined with immutability.
Blocks are deleted when they are no longer referenced by any metadata file.
Any object on the bucket is immutable, including the checkpoint files.

So looks like that the objects of the backup itself might not be immutable, but the checkpoints are. That's why nothing got deleted yet and will be processed at a later stage. This should happen in my case in the next couple of days - so curious if that was the case...