Comprehensive data protection for all workloads
Post Reply
cyburdine
Novice
Posts: 6
Liked: never
Joined: Aug 07, 2009 7:35 pm
Full Name: Justin burdine
Contact:

File Integrity: diffing backups

Post by cyburdine »

So I've searched around a bit on the forums and have not run across this request or need but wanted to know if this was possible. Part of our requirements for the data we host is to be able to detect unwanted file changes. (currently we are looking at something like Tripwire) But as I got to thinking about it, I wondered if Veeam Backup might be able to do the same thing. (or perhaps already does)

The idea is that if I have a backup from yesterday, and I make another backup today, Veeam Backup could show me what files were changed (or even better) compare it to a list of files I want to monitor and send me an alert if those files were changed. It's basically doing a diff between the two backups, and testing the files (or directories) to see if they are in the watch list and send an alert if they are on the watch list.

I figured I'd throw this out there because we have been toying with spending $80k on Tripwire (yet another system to keep track of) and if this could be implemented or if it exists we are willing to seriously consider paying for it!

Does anyone else out there have a need for this or are you solving this issue in a different way?

-justin
Gostev
Chief Product Officer
Posts: 31766
Liked: 7266 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: File Integrity: diffing backups

Post by Gostev »

Justin, do you mean comparing files inside image-level backups?
cyburdine
Novice
Posts: 6
Liked: never
Joined: Aug 07, 2009 7:35 pm
Full Name: Justin burdine
Contact:

Re: File Integrity: diffing backups

Post by cyburdine »

yeah, I could see it happening several different ways during or post backup or in a different application altogether.
Gostev
Chief Product Officer
Posts: 31766
Liked: 7266 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: File Integrity: diffing backups

Post by Gostev »

This could be possible to script with PowerShell and v4:

- Open Backup1.vbk for guest FLR restore (VM drives will be mounted as volume on Veeam Backup server)
- Enum all required guest files and calculate their hashes with script, store data in file1
- Stop FLR for Backup1.vbk (volume dismounted)
- Open Backup2.vbk for guest FLR restore (VM drives will be mounted as volume on Veeam Backup server)
- Enum all required guest files and calculate their hashes with script, store data in file2
- Stop FLR for Backup2.vbk (volume dismounted)
- Compare file1 and file2 with fc ;)

Not so simple to script, but hey... you save 80K :mrgreen:
cyburdine
Novice
Posts: 6
Liked: never
Joined: Aug 07, 2009 7:35 pm
Full Name: Justin burdine
Contact:

Re: File Integrity: diffing backups

Post by cyburdine »

yet again why I love this company... we will indeed give this a shot... now, to get a copy of 4.0 :D
cyburdine
Novice
Posts: 6
Liked: never
Joined: Aug 07, 2009 7:35 pm
Full Name: Justin burdine
Contact:

Re: File Integrity: diffing backups

Post by cyburdine »

Your suggestion worked like a charm. I have not automated the mounting of the vbk, which should be fairly trivial with Autoit. But I ran the process manually using fsum.exe and now have it exporting a list of changed files when there is a failure. Thanks again for the process!

-justin
Gostev
Chief Product Officer
Posts: 31766
Liked: 7266 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: File Integrity: diffing backups

Post by Gostev »

Great! Actually, I've been browsing some autotest logs for the latest build, and found that one of the tests is doing exactly this as well (mounting backup, obtaining file hashes and them comparing them with file hashes from original VM). This is done nightly as part of new build testing... so definitely 100% suitable and working approach.

Our autotests are 100% PowerShell based, so you should not need to deal with Autoit either (with v4).
Gostev
Chief Product Officer
Posts: 31766
Liked: 7266 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: File Integrity: diffing backups

Post by Gostev »

Here's some pieces of code from that autotest. To clarify, we are using specially generated VM image with *.dat files with same name as their MD5 hash, to make autotest simpler. So basically we are comparing actual file hash with the hash taken from file name. You would need to dump hashes to the output file, and compare the two output files instead.

"Main" script (Backup 4.0 PowerShell Extension required)

Code: Select all

Add-JobBackup MD5 VDDK "san;nbd" "My Computer" "C:\VmBackups" "MD5" -Objects MD5
Start-Job MD5
$rest = Start-FileRestore (Get-RestorePoint MD5 MD5)
CheckDatFile($rest.Drive)
Stop-FileRestore ($rest)
File checker (as I explained, this is specific to our autotest, so just FYI)

Code: Select all

function CheckDatFile ($drive) {
      ls ($drive)*.dat |% {
            $fileName = $_.Name
            $fileMD = Hash-MD5($drive + $fileName)
            $fileRMD = $fileName.Replace(".dat", [System.String]::Empty)
      
            Write-Host "File:" $fileName "  MD5:" $fileMD " Size: " $_.
            if ($fileMD -ieq $fileRMD) {
                  write-host " Pass " -backgroundcolor green -foregroundcolor darkblue
            }
            else {
                  write-host "*FAIL*" -backgroundcolor red
            }
      }
}
Function that calculates MD5 hash of the supplied file

Code: Select all

function Hash-MD5 ($file) {
      $hasher = [System.Security.Cryptography.MD5]::Create()
      $inputStream = New-Object System.IO.StreamReader ($file)
      $hashBytes = $hasher.ComputeHash($inputStream.BaseStream)
      $inputStream.Close()
      $builder = New-Object System.Text.StringBuilder
      $hashBytes | Foreach-Object { [void] $builder.Append($_.ToString("X2")) }
      $output = New-Object PsObject
      $output | Add-Member NoteProperty HashValue ([string]$builder.ToString())
      $output.hashvalue
}
Hope this helps ;)
Gostev
Chief Product Officer
Posts: 31766
Liked: 7266 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: File Integrity: diffing backups

Post by Gostev »

Please note that Veeam cmdlets will be renamed slightly in the release build to conform with PowerShell naming specification.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 49 guests