PowerShell script exchange
Post Reply
Nexsan
Novice
Posts: 3
Liked: never
Joined: Feb 09, 2023 1:59 pm
Full Name: Nexsan
Contact:

Extract Hashes for integrity verification?

Post by Nexsan »

Is there a way to extract the hashes from a VEEAM backup file after completion for file integrity verification?
Mildur
Product Manager
Posts: 10062
Liked: 2675 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian K.
Location: Switzerland
Contact:

Re: Extract Hashes for integrity verification?

Post by Mildur » 1 person likes this post

Hi Nexsan

Which Hashes do you need? Can you elaborate on your goal?
If you want to check File Integrity of your backup files, you can use Health Checks, Sure-Backup Jobs or the Veeam Backup Validator.

You can use any third party Hash tool or PowerShell (Get-FileHash) to get the hash value of a single backup file. But I'm not sure if that would help your use case.

Best,
Fabian
Product Management Analyst @ Veeam Software
Nexsan
Novice
Posts: 3
Liked: never
Joined: Feb 09, 2023 1:59 pm
Full Name: Nexsan
Contact:

Re: Extract Hashes for integrity verification?

Post by Nexsan »

Hi Fabian!

First of all, thank you for the reply. We have many 30 TB VEEAM backups that we are trying to process which are painful to manipulate. I was hoping that during the backup process, VEEAM could or does, hash (take a fingerprint) the file in order to insure file integrity once the job is completed. So basically it verifies the final file to make sure the fingerprint is what it's expecting. What I would like is to, if that verification is taking place, or could take place during the backup process, is pass this information over to another vendor in order for them to run their own checks and make sure when it's backed up, that the fingerprint in the final location matches the fingerprint from the original backup on the production storage.

Basically the issue is that the backup software is taking extremely long to hash a 36 TB file, so if this process could be done in VEEAM during the backup process and then passed to the vendor directly, it would say a massive amount of time, and then they can verify the integrity once the asset is stored using that finger print from VEEAM instead of calculating it after VEEAM places the file in the original repository. Trying to think outside the box to deal with massive files and make them more manageable.

I hope that's clear, but it might not be, lol. I'm pretty knew to VEEAM, so you'll have to bare with me.

Thanks for the help regardless!
david.domask
Veeam Software
Posts: 2312
Liked: 552 times
Joined: Jun 28, 2016 12:12 pm
Contact:

Re: Extract Hashes for integrity verification?

Post by david.domask » 1 person likes this post

heya @Nexsan,

I get what you're going for, but for a final hash value like that it wouldn't really be any faster doing it during the backup, as the hash won't be valid until the file is written and finalized; so it would just be shifting the time-sink to the backup operation itself.

Out of curiosity, how long is "extremely long" in a quantifiable number? https://www.veeam.com/kb2014#restoreperf If you run this on the repository, what is your _read_ random IO + speed?

if the performance is pretty slow, you might consider seeing if there's a way to present the storage so that it can be used by XFS/ReFS, which have their own scrubbing to catch corruption pretty fast.

> so if this process could be done in VEEAM during the backup process and then passed to the vendor directly, it would say a massive amount of time, and then they can verify the integrity once the asset is stored using that finger print from VEEAM instead of calculating it after VEEAM places the file in the original repository. Trying to think outside the box to deal with massive files and make them more manageable.

I'm not actually sure this makes a ton of sense, but maybe I misunderstand you here; even if a hash is calculated during backup, you still need to hash the file entirely in order to compare it, so you're still doing the same calculation twice? Already backup files have an internal structure that tracks this (this is how Health Check knows if data is alright in the backup), so this just comes down to how/when do you want to spend the resources to validate your backups.

If it were me, I'd just use XFS/ReFS and their respective native tooling for scrubbing. If the storage performance is not so hot, let the filesystem handle it opportunistically, don't try to do it in one go.
David Domask | Product Management: Principal Analyst
Nexsan
Novice
Posts: 3
Liked: never
Joined: Feb 09, 2023 1:59 pm
Full Name: Nexsan
Contact:

Re: Extract Hashes for integrity verification?

Post by Nexsan »

David,

Thanks for the info, I'm out of the office but will reply to this next week.
Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests