Forever Incremental? No more merges

[mmonroe]

I know that Tivoli and a couple of other backup solutions can have a backup data format that allows for a single full backup to take place and then can do forever incremental backups without ever needing to do merges or subsequent active full backups.

I am not exactly sure how Tivoli does it, however, they are able to "remove" data from previous backups and "add" new data without having to merge and move forward massive VBK-like files. We have seen a Tivoli system run years and years and never do full backups other than the first initial one.

One of the biggest issues for us and apparently others is with both main backups and backup copy jobs is the merge. All of the Veeam merge techniques are heavy on the disk and when they are store remotely, this means heavy on the network be it local or remote.

It would be really great if the Veeam base backup data could be stored in a way like Tivoli does so that blocks/chunks/whatever are added and removed without having to "move forward or backward" a massive VBK file. Creating a Veeam backup data structure that would eliminate the various merges on various process would be a huge plus for Veeam customers and would really open up more flexibilty on storage options and network (remote and local) options.

Random thoughts...

[Gostev]

I would guess Tivoli does not store backups as a files then, but rather as one giant chunk store that you can't really do anything with?

[mongie]

Gostev, do you guys ever think about creating some sort of Windows app or appliance that brings this functionality to Veeam? I understand the benefits of backups being stored as files, but there are several benefits to the other way of doing things too...

[McClane]

That's one thing I also liked with TSM. And the conservative way of tape management. We needed far less tapes with a way longer retention period as with Veeam. And you needed less space for tape restores. Only catch was a restore when older data was scattered over many tapes. Main reason to ditch TSM was the management and GUI (if you can call that a GUI) and that we would have needed a 3rd party backup für Sharepoint.

[Gostev]

I understand the benefits of backups being stored as files, but there are several benefits to the other way of doing things too...

What are those benefits? As far as I am concerned, I see nothing but drawbacks... you just need to dig a bit deeper on what the perceived "benefit" means for reliability of the solution, as well as recoverability implications. But lets discuss those benefits and see how we can address them without parting with storing backups as files.

I very much don't like alternative approaches simply because it is impossible for metadata-dependent, forever-incremental data blob to compete with self-contained, standalone full backup files in terms of reliability for archiving purposes specifically. Sure, it costs you more to store those - but thanks to that, a single bad block will not render your entire archive useless. In the end, we don't do archiving to save money, we do it to ensure recoverability...