Partial S3 offloads when full expires

[efd121]

I need to find out how Veeam handles the offloaded blocks for a system that doesn’t fully get offloaded before the full backup expires.

Here is the scenario.
The system has a 5TB full backup.
Aug 4th – 100GB is offloaded.
Aug 5th – 100GB is offloaded.
Aug 6th – 100GB is offloaded.
Aug 7th – 100GB is offloaded.
Aug 8th – 100GB is offloaded.
Aug 9th – 100GB is offloaded.
Aug 10th – 100GB is offloaded.
Aug 11th the full that started offloading on Aug 4th has expired and has been removed.
Aug 11th – this process starts over again offloading 100GB/day but never fully gets caught up for weeks/months.

Fast forward to today and that original 5TB VM is fully offloaded.

What happens to the 100GB that was offloaded on Aug 4th? Is it removed right away on Aug 11th or does it remain on the S3 appliance for 70 days (30-day retention, 30 days immutable +10)

If my assumption is correct that the partial data offloaded is kept for 70 days this could lead to a significant amount of data consumed on the S3 appliance. It eventually goes away but only 70 days after the system finally completes the full system offload and is keeping up each day.

Dave

[Mildur]

Hi Dave

Backup objects are immutable at the time when they are written to object storage. Or an attacker could remove objects during the backup job session.

Aug 11th – this process starts over again offloading 100GB/day but never fully gets caught up for weeks/months.

I would expect the session to continue until the first full backup has been created. But if this is a primary backup job, then the VSS maximum lifetime of 7 days could become an issue and the job gets cancelled on the 7th day. That should not happen with backup copy jobs.

this could lead to a significant amount of data consumed

Agree if there is a high change rate in your production machines. Our backup jobs with object storage as a target are forever incremental backup chains. In theory, unchanged blocks should reuse already present objects for unchanged data from object storage. Do you see something else?

Best,
Fabian

[efd121]

That makes sense that its immutable at the time the block is written.

This is my scenario, the primary backup is expiring and being removed from the performance tier before it has been fully offloaded to the immutable capacity tier. The piece that I'm still not sure of is those partial offloads are fully abandoned (to be deleted after immutability expires) or how Veeam would know if those blocks are unchanged from the latest full that now started to offload.
This is not occurring with my Backup Copy jobs, its just the offload from the primary backup from the performance tier to S3 for the immediate copy.

I think my best indication will be if I start seeing space free up on the capacity tier in the next few weeks after it gets a full offloaded. The partials should all fall off in ~70 days (30 retention + 10 block generation + 30 immutable period)

Dave