Discussions related to using object storage as a backup target.
Post Reply
wa15
Veteran
Posts: 323
Liked: 25 times
Joined: Jan 02, 2014 4:45 pm
Contact:

What happens when backup chain is corrupted?

Post by wa15 »

Let's say we have an 8 year retention. Since backups to S3 are deduplicated/compressed, what happens when one of the backups in the chain is corrupted? Does that mean that all impacted backups are nullified? Are there any mechanisms that work to prevent corruption?

Example:

Jan 2019 Full Backup (800GB)
Feb 2019 Full Backup (100GB, dedupe'd with Jan)
...
Dec 2019 Full Backup (100GB, dedupe'ed with existing files)

If Jan 2019 is corrupted, does that mean that everything that comes after it is no good?
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What happens when backup chain is corrupted?

Post by Gostev »

Depends on what S3 you're talking about.

Amazon S3 provides built-in redundancy with multiple copies of each block, so in case one copy gets corrupted - the restore will be transparently served by Amazon from another copy of the block. In fact, if not this feature, we would not implement forever-incremental archiving at all.

And for other S3-compatible cloud providers and on-prem storage solutions, you need to check with the vendor to be sure. I personally am not aware of a single object storage solution without the built-in redundancy that uses either mirroring or erasure coding.
wa15
Veteran
Posts: 323
Liked: 25 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Re: What happens when backup chain is corrupted?

Post by wa15 »

Good to know. Thanks for your response!
wa15
Veteran
Posts: 323
Liked: 25 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Re: What happens when backup chain is corrupted?

Post by wa15 »

On this topic, how is corrupted data discovered? Does Veeam run an integrity check upon each SOBR S3 offload? Or is this all dependent on the S3 provider?
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What happens when backup chain is corrupted?

Post by Gostev »

It is dependent on the S3 provider. Usually, for disk based storage, the technique known as "patrol reads" is used.
In Amazon S3, every object has MD5 checksum attached to it, which enables identification of corrupted blocks.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 12 guests