Rotate Backup Job

[cowhow]

I've got two file copy jobs pushing backups off site to AWS S3. One runs on the first Saturday of the month and a second job runs on the third Saturday. Has anyone came up with a way of rotating backups out of S3 rather than just allowing them to pile up until manually deleted? I'm not that creative or smart.

[Gostev]

How do you get them to S3?

[cowhow]

Gostev, I use a third party app to map my S3 bucket as a drive and then use the file copy job to push them up. But, I don't have a way of cleanly overwriting or rotating old backup out.

[Gostev]

That's what I thought. What backup mode are you using on your primary backup job? And does this third party app support block-level incrementals?

[cowhow]

I'm using reverse incrementals with synthetic full once a week. The third party app is an app called TntDrive. Basically, all it does is allow me to mount the bucket as a network drive so the file copy job can see it. The app is actually not in the loop as far as backup jobs are concerned. I'm trying to use this approach rather than Cloudberry so I can keep my backup jobs self contained in the Veeam console.

[dellock6]

What about mounting different buckets and use them as different drives? AWS charges you by bytes and consumed bandwidth, not the number of buckets...

[cowhow]

I've kind of done something similar. I created two copy jobs for each backup job I push to AWS and route them to different folders in the same bucket to help sort out dates. The copy jobs only run every every other week into their own specific folder. It would be nice to be able to delete previous week's files prior to uploading the current job. That way I'd only have each week's current backup and my bucket storage quota would stay essentially the same rather than grow as time goes by.

[dellock6]

You can just create a pre-job script that deletes the files in a given bucket, easy since it's seen as a folder from the local Veeam server.

[veremin]

I'm a bit against using pre-job script to delete previous backups. Because if the backup job run fails for some reason, not only would you have no up-to-date restore point, but also you'd loose the previous one. Post-job command might be a better option.

[cowhow]

I'm a bit against using pre-job script to delete previous backups. Because if the backup job run fails for some reason, not only would you have no up-to-date restore point, but also you'd loose the previous one. Post-job command might be a better option.

That's a good point. I'm still scratching my head about this one. Every possible solution I can come up with has more downsides than benefits. You're right about possibly deleting your only good backup so I guess for now at least I'll just have to manually do bucket housekeeping periodically. Of course, S3 isn't terribly expensive to upload to, it's downloading that gets you.

The journey continues...