Integration of Forever Forward Incremental Backup with S3 Bucket: Merging, Removal, and Immutability Considerations

[Danh3]

Hello,

Questions in regards to integration of Forever Forward Incremental Backup with S3 Bucket: Merging, Removal, and Immutability Considerations

We've implemented a "Forever forward incremental approach" for our primary backup, eliminating the need for active full or synthetic full backups, and these backups are stored locally. Additionally, we've configured a backup copy that runs daily at 12 AM to copy data to an S3 bucket.

During the primary backup, if the allowable number of restore points be exceeded, Veeam will trigger the merging of data blocks from the incremental backup file into the full backup file.

So how this newly merged full backup will replace the one in the S3 bucket?
I'm curious whether Veeam will automatically remove the full backup job from s3 bucket once the merged full backup has been successfully transferred to it, or if the merging process takes place directly within the S3 bucket when Veeam detects an excess of restore points.

If the merging process does occur within the S3 bucket, does this imply that S3 bucket immutable doesnot support the "Forever forward incremental approach"

thanks

[Gostev]

Hello,

Object storage has no file system and so there are no files in principle. So naturally, the easiest for you would be to just forget everything you knew about backup FILE maintenance (like merges), because backup files simply don't exist on object storage. So if you keep trying to make ANY connections to the world with classic backup repositories like you do above, you will only keep confusing yourself.

What object storage repository holds is just a bunch of objects representing data blocks from the protected disk images. Each such object can be a part of 1..N restore points, which can be full backup, increments etc. And the restore point is nothing but a bunch of pointers to these objects.

These objects can also be freely managed individually, for example an object that is not a part of any restore points can then be freely deleted by a retention processing engine, can be make immutable for the required number of days - according to how long the latest restore point it belongs to needs to remain immutable, etc.

Thanks

[Danh3]

thanks @Gostev

it's more clear for me now.

if backup copies store in block storage (e.g. Nimble Storage) and XFS on top of block storage? a merge occur on the backup copy repository when the backup copy detects that the allowed number of restore points has been exceeded - correct ?

[Gostev]

Correct. It's not specific to backup copy, this is just how forward forever incremental backup mode works.