Let's say we have an 8 year retention. Since backups to S3 are deduplicated/compressed, what happens when one of the backups in the chain is corrupted? Does that mean that all impacted backups are nullified? Are there any mechanisms that work to prevent corruption?
Example:
Jan 2019 Full Backup (800GB)
Feb 2019 Full Backup (100GB, dedupe'd with Jan)
...
Dec 2019 Full Backup (100GB, dedupe'ed with existing files)
If Jan 2019 is corrupted, does that mean that everything that comes after it is no good?
-
- Veteran
- Posts: 323
- Liked: 25 times
- Joined: Jan 02, 2014 4:45 pm
- Contact:
-
- Chief Product Officer
- Posts: 31814
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: What happens when backup chain is corrupted?
Depends on what S3 you're talking about.
Amazon S3 provides built-in redundancy with multiple copies of each block, so in case one copy gets corrupted - the restore will be transparently served by Amazon from another copy of the block. In fact, if not this feature, we would not implement forever-incremental archiving at all.
And for other S3-compatible cloud providers and on-prem storage solutions, you need to check with the vendor to be sure. I personally am not aware of a single object storage solution without the built-in redundancy that uses either mirroring or erasure coding.
Amazon S3 provides built-in redundancy with multiple copies of each block, so in case one copy gets corrupted - the restore will be transparently served by Amazon from another copy of the block. In fact, if not this feature, we would not implement forever-incremental archiving at all.
And for other S3-compatible cloud providers and on-prem storage solutions, you need to check with the vendor to be sure. I personally am not aware of a single object storage solution without the built-in redundancy that uses either mirroring or erasure coding.
-
- Veteran
- Posts: 323
- Liked: 25 times
- Joined: Jan 02, 2014 4:45 pm
- Contact:
Re: What happens when backup chain is corrupted?
Good to know. Thanks for your response!
-
- Veteran
- Posts: 323
- Liked: 25 times
- Joined: Jan 02, 2014 4:45 pm
- Contact:
Re: What happens when backup chain is corrupted?
On this topic, how is corrupted data discovered? Does Veeam run an integrity check upon each SOBR S3 offload? Or is this all dependent on the S3 provider?
-
- Chief Product Officer
- Posts: 31814
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: What happens when backup chain is corrupted?
It is dependent on the S3 provider. Usually, for disk based storage, the technique known as "patrol reads" is used.
In Amazon S3, every object has MD5 checksum attached to it, which enables identification of corrupted blocks.
In Amazon S3, every object has MD5 checksum attached to it, which enables identification of corrupted blocks.
Who is online
Users browsing this forum: No registered users and 4 guests