-
- Novice
- Posts: 6
- Liked: never
- Joined: Aug 07, 2009 7:35 pm
- Full Name: Justin burdine
- Contact:
File Integrity: diffing backups
So I've searched around a bit on the forums and have not run across this request or need but wanted to know if this was possible. Part of our requirements for the data we host is to be able to detect unwanted file changes. (currently we are looking at something like Tripwire) But as I got to thinking about it, I wondered if Veeam Backup might be able to do the same thing. (or perhaps already does)
The idea is that if I have a backup from yesterday, and I make another backup today, Veeam Backup could show me what files were changed (or even better) compare it to a list of files I want to monitor and send me an alert if those files were changed. It's basically doing a diff between the two backups, and testing the files (or directories) to see if they are in the watch list and send an alert if they are on the watch list.
I figured I'd throw this out there because we have been toying with spending $80k on Tripwire (yet another system to keep track of) and if this could be implemented or if it exists we are willing to seriously consider paying for it!
Does anyone else out there have a need for this or are you solving this issue in a different way?
-justin
The idea is that if I have a backup from yesterday, and I make another backup today, Veeam Backup could show me what files were changed (or even better) compare it to a list of files I want to monitor and send me an alert if those files were changed. It's basically doing a diff between the two backups, and testing the files (or directories) to see if they are in the watch list and send an alert if they are on the watch list.
I figured I'd throw this out there because we have been toying with spending $80k on Tripwire (yet another system to keep track of) and if this could be implemented or if it exists we are willing to seriously consider paying for it!
Does anyone else out there have a need for this or are you solving this issue in a different way?
-justin
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: File Integrity: diffing backups
Justin, do you mean comparing files inside image-level backups?
-
- Novice
- Posts: 6
- Liked: never
- Joined: Aug 07, 2009 7:35 pm
- Full Name: Justin burdine
- Contact:
Re: File Integrity: diffing backups
yeah, I could see it happening several different ways during or post backup or in a different application altogether.
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: File Integrity: diffing backups
This could be possible to script with PowerShell and v4:
- Open Backup1.vbk for guest FLR restore (VM drives will be mounted as volume on Veeam Backup server)
- Enum all required guest files and calculate their hashes with script, store data in file1
- Stop FLR for Backup1.vbk (volume dismounted)
- Open Backup2.vbk for guest FLR restore (VM drives will be mounted as volume on Veeam Backup server)
- Enum all required guest files and calculate their hashes with script, store data in file2
- Stop FLR for Backup2.vbk (volume dismounted)
- Compare file1 and file2 with fc
Not so simple to script, but hey... you save 80K
- Open Backup1.vbk for guest FLR restore (VM drives will be mounted as volume on Veeam Backup server)
- Enum all required guest files and calculate their hashes with script, store data in file1
- Stop FLR for Backup1.vbk (volume dismounted)
- Open Backup2.vbk for guest FLR restore (VM drives will be mounted as volume on Veeam Backup server)
- Enum all required guest files and calculate their hashes with script, store data in file2
- Stop FLR for Backup2.vbk (volume dismounted)
- Compare file1 and file2 with fc
Not so simple to script, but hey... you save 80K
-
- Novice
- Posts: 6
- Liked: never
- Joined: Aug 07, 2009 7:35 pm
- Full Name: Justin burdine
- Contact:
Re: File Integrity: diffing backups
yet again why I love this company... we will indeed give this a shot... now, to get a copy of 4.0
-
- Novice
- Posts: 6
- Liked: never
- Joined: Aug 07, 2009 7:35 pm
- Full Name: Justin burdine
- Contact:
Re: File Integrity: diffing backups
Your suggestion worked like a charm. I have not automated the mounting of the vbk, which should be fairly trivial with Autoit. But I ran the process manually using fsum.exe and now have it exporting a list of changed files when there is a failure. Thanks again for the process!
-justin
-justin
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: File Integrity: diffing backups
Great! Actually, I've been browsing some autotest logs for the latest build, and found that one of the tests is doing exactly this as well (mounting backup, obtaining file hashes and them comparing them with file hashes from original VM). This is done nightly as part of new build testing... so definitely 100% suitable and working approach.
Our autotests are 100% PowerShell based, so you should not need to deal with Autoit either (with v4).
Our autotests are 100% PowerShell based, so you should not need to deal with Autoit either (with v4).
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: File Integrity: diffing backups
Here's some pieces of code from that autotest. To clarify, we are using specially generated VM image with *.dat files with same name as their MD5 hash, to make autotest simpler. So basically we are comparing actual file hash with the hash taken from file name. You would need to dump hashes to the output file, and compare the two output files instead.
"Main" script (Backup 4.0 PowerShell Extension required)
File checker (as I explained, this is specific to our autotest, so just FYI)
Function that calculates MD5 hash of the supplied file
Hope this helps
"Main" script (Backup 4.0 PowerShell Extension required)
Code: Select all
Add-JobBackup MD5 VDDK "san;nbd" "My Computer" "C:\VmBackups" "MD5" -Objects MD5
Start-Job MD5
$rest = Start-FileRestore (Get-RestorePoint MD5 MD5)
CheckDatFile($rest.Drive)
Stop-FileRestore ($rest)
Code: Select all
function CheckDatFile ($drive) {
ls ($drive)*.dat |% {
$fileName = $_.Name
$fileMD = Hash-MD5($drive + $fileName)
$fileRMD = $fileName.Replace(".dat", [System.String]::Empty)
Write-Host "File:" $fileName " MD5:" $fileMD " Size: " $_.
if ($fileMD -ieq $fileRMD) {
write-host " Pass " -backgroundcolor green -foregroundcolor darkblue
}
else {
write-host "*FAIL*" -backgroundcolor red
}
}
}
Code: Select all
function Hash-MD5 ($file) {
$hasher = [System.Security.Cryptography.MD5]::Create()
$inputStream = New-Object System.IO.StreamReader ($file)
$hashBytes = $hasher.ComputeHash($inputStream.BaseStream)
$inputStream.Close()
$builder = New-Object System.Text.StringBuilder
$hashBytes | Foreach-Object { [void] $builder.Append($_.ToString("X2")) }
$output = New-Object PsObject
$output | Add-Member NoteProperty HashValue ([string]$builder.ToString())
$output.hashvalue
}
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: File Integrity: diffing backups
Please note that Veeam cmdlets will be renamed slightly in the release build to conform with PowerShell naming specification.
Who is online
Users browsing this forum: Google [Bot] and 49 guests