Backup chandling with S3

[redfasdfasf]

Hi everyone,

I am struggling to wrap my head around on how incremetal forward forever backup chains are handled on public S3 target and how it work with immutability. Hopefully you can help me.
1. If I use direct backup to S3 (no SOBR) and I use incremetal forward forever, at the time retention is exceed, how is the merge of the of the full and the oldest inc done? Usually on a block device, the retention +1 is written and then the merge happens, the full moves up the ladder and the oldest inc is deleted. With S3 I would image that the full is downloaded and merged locally before the upload. This is important for calculating traffic costs with a public provider because this creates outgoing traffic.
2. I read in the Object Storage FAQs veeam-backup-replication-f2/read-this-f ... ml#p338749 about the 10 day full backup optimization with immutability. I know that 10 days are added because of Block Generation, but again I dont understand how a forever inc chain can have a second generation since its written forever? Or is there some kind of synthenic full happening every 10 day to reduce chain merges in S3.

Thanks for your help

[MarkBoothmaa]

It's completely different to block in so far as there are no full or incremental files.
Now you have objects that are stored and the immutability is applied to the object. Once the immutability period has passed the object will be removed (Once retention is met)

[Mildur]

Hello redfasdfasf

1. If I use direct backup to S3 (no SOBR) and I use incremetal forward forever, at the time retention is exceed, how is the merge of the of the full and the oldest inc done? Usually on a block device, the retention +1 is written and then the merge happens, the full moves up the ladder and the oldest inc is deleted. With S3 I would image that the full is downloaded and merged locally before the upload. This is important for calculating traffic costs with a public provider because this creates outgoing traffic.

There are no backup files to merge on object storage. Each restore point is split in unique blocks. All blocks are initially uploaded as objects to the S3 storage. When you run an incremental backup, we upload new blocks to the S3 storage. The backup server will store the required objects for each restore point in it's database and metadata.
When a restore point has aged out, we just delete all objects which are not reference by any other restore point. No downloading of a full backup is required.

2. I read in the Object Storage FAQs about the 10 day full backup optimization with immutability. I know that 10 days are added because of Block Generation, but again I dont understand how a forever inc chain can have a second generation since its written forever? Or is there some kind of synthenic full happening every 10 day to reduce chain merges in S3.

We add 1-10 days to the configured immutability. Without block generation, we would have to extend immutability each day by +1 for all objects belonging to the new restore point. This would lead to many S3 API calls to prolong object lock. By setting 1-10 days longer than configured, we only have to extend it for unchanged backup objects every 10 days.

Best,
Fabian

[redfasdfasf]

When a restore point has aged out, we just delete all objects which are not reference by any other restore point. No downloading of a full backup is required.

Hi Fabian,
if I get this correct, you dont have single backup files anymore but the restore points are split up and the metadata contains map which blocks belong to which restore point? Therefore no merge is needed but the retention applies to blocks rather than the restore point because the restore point might have data required for other backups.

Thanks, this explains a lot

[Mildur]

Correct. Retention still applies to restore points.
If a block/object belongs to a not yet retired restore point, that object won't be deleted.

Best,
fabian