-
- Novice
- Posts: 3
- Liked: never
- Joined: Feb 09, 2023 1:59 pm
- Full Name: Nexsan
- Contact:
Extract Hashes for integrity verification?
Is there a way to extract the hashes from a VEEAM backup file after completion for file integrity verification?
-
- Product Manager
- Posts: 10062
- Liked: 2675 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: Extract Hashes for integrity verification?
Hi Nexsan
Which Hashes do you need? Can you elaborate on your goal?
If you want to check File Integrity of your backup files, you can use Health Checks, Sure-Backup Jobs or the Veeam Backup Validator.
You can use any third party Hash tool or PowerShell (Get-FileHash) to get the hash value of a single backup file. But I'm not sure if that would help your use case.
Best,
Fabian
Which Hashes do you need? Can you elaborate on your goal?
If you want to check File Integrity of your backup files, you can use Health Checks, Sure-Backup Jobs or the Veeam Backup Validator.
You can use any third party Hash tool or PowerShell (Get-FileHash) to get the hash value of a single backup file. But I'm not sure if that would help your use case.
Best,
Fabian
Product Management Analyst @ Veeam Software
-
- Novice
- Posts: 3
- Liked: never
- Joined: Feb 09, 2023 1:59 pm
- Full Name: Nexsan
- Contact:
Re: Extract Hashes for integrity verification?
Hi Fabian!
First of all, thank you for the reply. We have many 30 TB VEEAM backups that we are trying to process which are painful to manipulate. I was hoping that during the backup process, VEEAM could or does, hash (take a fingerprint) the file in order to insure file integrity once the job is completed. So basically it verifies the final file to make sure the fingerprint is what it's expecting. What I would like is to, if that verification is taking place, or could take place during the backup process, is pass this information over to another vendor in order for them to run their own checks and make sure when it's backed up, that the fingerprint in the final location matches the fingerprint from the original backup on the production storage.
Basically the issue is that the backup software is taking extremely long to hash a 36 TB file, so if this process could be done in VEEAM during the backup process and then passed to the vendor directly, it would say a massive amount of time, and then they can verify the integrity once the asset is stored using that finger print from VEEAM instead of calculating it after VEEAM places the file in the original repository. Trying to think outside the box to deal with massive files and make them more manageable.
I hope that's clear, but it might not be, lol. I'm pretty knew to VEEAM, so you'll have to bare with me.
Thanks for the help regardless!
First of all, thank you for the reply. We have many 30 TB VEEAM backups that we are trying to process which are painful to manipulate. I was hoping that during the backup process, VEEAM could or does, hash (take a fingerprint) the file in order to insure file integrity once the job is completed. So basically it verifies the final file to make sure the fingerprint is what it's expecting. What I would like is to, if that verification is taking place, or could take place during the backup process, is pass this information over to another vendor in order for them to run their own checks and make sure when it's backed up, that the fingerprint in the final location matches the fingerprint from the original backup on the production storage.
Basically the issue is that the backup software is taking extremely long to hash a 36 TB file, so if this process could be done in VEEAM during the backup process and then passed to the vendor directly, it would say a massive amount of time, and then they can verify the integrity once the asset is stored using that finger print from VEEAM instead of calculating it after VEEAM places the file in the original repository. Trying to think outside the box to deal with massive files and make them more manageable.
I hope that's clear, but it might not be, lol. I'm pretty knew to VEEAM, so you'll have to bare with me.
Thanks for the help regardless!
-
- Veeam Software
- Posts: 2312
- Liked: 552 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Extract Hashes for integrity verification?
heya @Nexsan,
I get what you're going for, but for a final hash value like that it wouldn't really be any faster doing it during the backup, as the hash won't be valid until the file is written and finalized; so it would just be shifting the time-sink to the backup operation itself.
Out of curiosity, how long is "extremely long" in a quantifiable number? https://www.veeam.com/kb2014#restoreperf If you run this on the repository, what is your _read_ random IO + speed?
if the performance is pretty slow, you might consider seeing if there's a way to present the storage so that it can be used by XFS/ReFS, which have their own scrubbing to catch corruption pretty fast.
> so if this process could be done in VEEAM during the backup process and then passed to the vendor directly, it would say a massive amount of time, and then they can verify the integrity once the asset is stored using that finger print from VEEAM instead of calculating it after VEEAM places the file in the original repository. Trying to think outside the box to deal with massive files and make them more manageable.
I'm not actually sure this makes a ton of sense, but maybe I misunderstand you here; even if a hash is calculated during backup, you still need to hash the file entirely in order to compare it, so you're still doing the same calculation twice? Already backup files have an internal structure that tracks this (this is how Health Check knows if data is alright in the backup), so this just comes down to how/when do you want to spend the resources to validate your backups.
If it were me, I'd just use XFS/ReFS and their respective native tooling for scrubbing. If the storage performance is not so hot, let the filesystem handle it opportunistically, don't try to do it in one go.
I get what you're going for, but for a final hash value like that it wouldn't really be any faster doing it during the backup, as the hash won't be valid until the file is written and finalized; so it would just be shifting the time-sink to the backup operation itself.
Out of curiosity, how long is "extremely long" in a quantifiable number? https://www.veeam.com/kb2014#restoreperf If you run this on the repository, what is your _read_ random IO + speed?
if the performance is pretty slow, you might consider seeing if there's a way to present the storage so that it can be used by XFS/ReFS, which have their own scrubbing to catch corruption pretty fast.
> so if this process could be done in VEEAM during the backup process and then passed to the vendor directly, it would say a massive amount of time, and then they can verify the integrity once the asset is stored using that finger print from VEEAM instead of calculating it after VEEAM places the file in the original repository. Trying to think outside the box to deal with massive files and make them more manageable.
I'm not actually sure this makes a ton of sense, but maybe I misunderstand you here; even if a hash is calculated during backup, you still need to hash the file entirely in order to compare it, so you're still doing the same calculation twice? Already backup files have an internal structure that tracks this (this is how Health Check knows if data is alright in the backup), so this just comes down to how/when do you want to spend the resources to validate your backups.
If it were me, I'd just use XFS/ReFS and their respective native tooling for scrubbing. If the storage performance is not so hot, let the filesystem handle it opportunistically, don't try to do it in one go.
David Domask | Product Management: Principal Analyst
-
- Novice
- Posts: 3
- Liked: never
- Joined: Feb 09, 2023 1:59 pm
- Full Name: Nexsan
- Contact:
Re: Extract Hashes for integrity verification?
David,
Thanks for the info, I'm out of the office but will reply to this next week.
Thanks for the info, I'm out of the office but will reply to this next week.
Who is online
Users browsing this forum: No registered users and 8 guests