Recompress old backups for only file versions

[SeanGraham]

Hello Veeam Team!

I'm sure that this has been discussed before, but I couldn't find a "recent" discussion on this.

We've been using Veeam to do monthly full + daily incremental backups and it works great. Our data volume is about 150GB, and we have maybe a few hundred MB of changes on a daily basis. However, the daily incrementals are 2-10GB. We have long accepted that Veeam is not space-efficient when looking at volume-level backups, but I'm wondering if there is any script that could be used to effectively convert our ~20TB of backups from the past 2 years into something that just looks at file level changes and stores the versions? I don't care how inefficient or long it might take, as long as there is some way.

I.e. analyze the entire set, and for every file identify "this file changed on Jan 12, April 30, Sep 1, Dec 22" and just keep those 4 versions of the file. We really only need the latest month as a volume-level backup for DR restore purposes.

We do have a legal need to keep backups for up to 10 years, but we really only need file versions.

[HannesK]

Hello,

the daily incrementals are 2-10GB.

I guess, that includes the operating system (Windows)?

I'm wondering if there is any script that could be used to effectively convert our ~20TB of backups from the past 2 years into something that just looks at file level changes and stores the versions?

no, not really.

Veeam is doing block-based backup. There are no "files". The blocks we back up have changed (whether intentionally or not, for example by "defrag"). It's not, that we make the backups "large", just for the sake of it

For the future: you can also do file-based backup / NAS backup

Technically, you can even mount your backups (with the data integration API) and do NAS backup from that mountpoint. Overall, that sounds like a lot of work for no real benefit.

Best regards,
Hannes

[SeanGraham]

Appreciate the reply HannesK.

I know that block level backups are larger, but Veeam is particularly bad compared to others. I know that this has been discussed ad nauseum in the past, and that's not the point at the moment.

The benefit in my request is that I have file changes contained within 20TB of block-based backups for 2 years that need to be kept for 10 years. That's a lot of effort to keep those when there is likely less than 100GB of actual needed data within there.

I was hoping that, since I'm sure that I'm not the only one in this situation, that someone might have written a script that would allow delta-based file-level extracts from the block level backups. I.e. just extract files that have changed since the last incremental or full backup. Or at the very least discard the non-file-level data from the block-level backups, or convert from block-level to file-level backups.

It might not even be that hard to script. For example - if I could mount a backup set from command line, I could iterate through successive backup increments and use 7zip to create file-level incremental backups pretty easily. It looks like this might be possible on linux, is it possible on Windows?

[soncscy]

> I was hoping that, since I'm sure that I'm not the only one in this situation, that someone might have written a script that would allow delta-based file-level extracts from the block level backups. I.e. just extract files that have changed since the last incremental or full backup.

To be honest Sean, this is far more difficult and complex than you're making it out to be.

If the goal is to thin out your historical backups, I'd recommend the following:

1. Instant Recovery one of the backups to production in an isolated network and with a new name - it should be "spaceless"
2. Use something like sdelete within the guest to thin out the used space: https://kb.vmware.com/s/article/2014832
3. Backup that Instant Recovered VM and see if the space is any better for your purposes

If you have a fileserver that is so egregious you need to slim it down, consider doing a File Level recovery from the first increment, and do some rsync with the minimum lastmodifiedDate value you want and that should slim it down a bit.

But ultimately, it kind of sounds like what you needed was NAS Backup before it existed. Check your available instances and see if it can cover the cost of backing up your existing backups. All File Level Restores on Veeam mount to C:\VeeamFLR, and you can quickly write something to auto-mount the backup and then do a NAS Backup of the same directory. Should be sufficient to "convert" your backup to a NAS Backup, but mind the instance costs.