Backup weekly full to tape with DPM

[Aurock]

We've just started using veeam, and I need to set something up to backup the weekly full to tape using DPM. I'm trying to piece together something that will work for this, but it seems like it's more complicated than it might need to be. Here's roughly what I'm thinking I need to do, does this look right?

Create a post job script in Veeam to run on saturdays, after the weekly synthetic full.
Initial script will run using the method described in this thread, so that the main script executes on the repository server: http://forums.veeam.com/viewtopic.php?f ... +scripting

Main script:
Move the most recent .vbk file to a temp directory (Reusing some of the scripts in this thread: http://forums.veeam.com/viewtopic.php?f ... 7&start=15)
Trigger DPM Backup job to backup the temp directory
Somehow monitor status of the backup job
After dpm job completes, move the .vbk file back to the original directory


As far as I know, there is no way to backup a single file in DPM, just the contents of a directory, thus the need to isolate the 1 file to be backed up. I think moving the file to a temp directory on the same local drive should be near instant, where copying the file would take some quite a bit of time. Plus copying it to the temp directory would require double the disk space for this process.

Am I over-thinking things, or missing something? Or am I re-inventing the wheel when someone else already been through this and created/shared a script for this purpose?

[veremin]

I’m not that familiar with DPM, so it’s rather hard for me to make any speculations with regards to it.

However, I do believe that the easiest way you have is to wait till the VB&R version 7 with tape functionality is released to general public. From my perspective, it’s more convenient to orchestrate all backup/replication activities through one pane of glass. Also, with the upcoming version your tape backup data will be tracked successfully: you will know on which tape medias particular VMs reside on, what medias are required in order to restore certain VM, how copying/restore process should be handled, etc.

As to your current situation, it might be worth creating a permanent folder, not a temporary one, to which weekly-worth .vbk files will be copied and specifying it as a source for DPM tape job. Then, write a custom script that will execute DPM tape job and schedule it to run on day after the active full has taken place. Not only will it reduce the overall complexity, but also it will guarantee an additional copy of backup data (assuming, of course, that corresponding .vbk files will be copied to a different drive).

Nevertheless, if you find this approach inacceptable, you might implement the scenario you described; should do the trick.

Thanks.

[Aurock]

Unfortunately, our tape library is directly attached to our dpm server via scsi, so I don't think Veeam 7 will help in this case.

Copying the vbk to a different drive and backing it up from there would be slightly simpler, but at the cost of another TB or so of disk space. The whole point of buying veeam was to reduce disk and tape consumption, we were quite happy using DPM to backup VM's other than the lack of dedup/compression.

[tsightler]

Instead of moving the VBK file why not simply use a hardlink? This requires no extra space at all and there are already scripts out there to do exactly what you want (create a hardlink to the most recent VBK in another directory). Then you point DPM to backup the directory containing the hardlinks, which point only at the most recent VBK files. Here's a script written by a customer for BackupExec but the same idea should work fine with DPM:

http://forums.veeam.com/viewtopic.php?f ... 718#p67738

[Aurock]

DPM doesn't process reparse points. Is a hard link considered a reparse point?

[tsightler]

I'm not a DPM expert but now that you mention that it seems like I do remember reading somewhere that DPM would not backup hardlinks. Should be very easy to test though. In the end, it's not that different from your move and move back option, I just thought it might be easier since the script is already there and proven to work with other backup products.

[Aurock]

Thanks. The hard link option sounds like a good way to go, if it would work.

I also posted on the DPM forums and got a reply suggesting I use pre/post job scripts in dpm instead. Thinking about it, that might be a lot easier. That way I don't need to monitor the dpm job and move the file back when it stops, the post script can do it. And the DPM pre-script runs on the server to be backed up, not on the dpm server, so that avoids having to double layer the veeam pre-script to get it running on the repository.

[veremin]

As far as I know, previously existing reparse point limitation was overcome with the introduction of DMP 2012:

When you select a data source that contains a reparse point (mount points and junction points are data sources that contain reparse points), DPM prompts you to specify whether you want to include the target of the reparse point in the protection group. The reparse point itself is not replicated; you must manually re-create the reparse point when you recover the data.

Thanks.

[Aurock]

Wow, that's great to know. Reading that, I'm not sure how it would handle a reparse point added after the protection group is created, but it's worth looking into.

[veremin]

Yep, it definitely stands to reason to test it. In fact, I'd like to hear about the results you get; kindly, keep us updated. Thanks.

[Aurock]

It appears that adding a hardlink to a folder already included in a DPM Protection group doesn't work. I'll try modifying the PG and see if that makes a difference.

[Aurock]

For what it's worth, when I couldn't reproduce the behavior described in that technet article, I posted to the DPM forums asking about it. The response I received from MS was that the only time DPM prompts for whether to backup the target of a reparse point is when it's a mount point in the root of a volume, linked to another volume. Other than adding support for Server 2012 Deduplication reparse points, nothing has changed in how reparse points are handled between DPM2010 and DPM2012SP1.

http://social.technet.microsoft.com/For ... rse-points

[Aurock]

Okay, here's what I ended up doing, in case it's helpful to someone else in the future.

I didn't end up using anything from the example thread's link on how to Offload to tape as a post job script, since the repository server doesn't have the Veeam Powershell Snap-In installed.

I created a pre-job script and Post job script attached to the dpm backup.

DPMPreScript.ps1:

Code: Select all

# This Pre-script runs before DPM backs up anything on the D: drive.
# The script moves the most recent full backup (.vbk) file from Veeam to a separate folder backed up by dpm.
# After the DPM job is complete, a separate script (dpmpostscript.ps1) is executed to move the file back to the Veeam Backup Folder

# VeeamBackupFolder Should be the the directory containing the output files (.vbk, .vib, .vrb) from the Veeam backup job
# DPMProtectedFolder should be the directory being backed up by a DPM Protection Group

$VeeamBackupFolder = "D:\Backups\Main VM Backup\"
$DPMProtectedFolder = "D:\Backups\ToDPM\"
$FullBackupFile = "*.vbk"
$FileExists = (Test-Path $VeeamBackupFolder\$FullBackupFile -Pathtype Leaf)

If ($FileExists)
{
    $VeeamBackupFile = get-childitem $Veeambackupfolder *.vbk | Sort-Object {$_.LastWriteTime} -Descending | select-object -First 1
    move-item $VeeamBackupFolder\$VeeamBackupFile $DPMProtectedFolder
}
ElseIf (!($FileExists))
{
    Write-Host "No full backup (.vbk) files found in " $VeeamBackupFolder
}

DPMPostScript.ps1

Code: Select all

# This Post-Job script runs after DPM backs up anything on the D: drive.
# The script reverts the file(s) moved by the Pre-Job Script back to their original location, the Veeam backup folder. 

# VeeamBackupFolder Should be the the directory containing the output files (.vbk, .vib, .vrb) from the Veeam backup job
# DPMProtectedFolder should be the directory being backed up by a DPM Protection Group

$VeeamBackupFolder = "D:\Backups\Main VM Backup\"
$DPMProtectedFolder = "D:\Backups\ToDPM\"
$FullBackupFile = "*.vbk"
$FileExists = (Test-Path $DPMProtectedFolder\$FullBackupFile -Pathtype Leaf)

If ($FileExists)
{
    move-item $DPMProtectedFolder\$FullBackupFile $VeeamBackupFolder
}
ElseIf (!($FileExists))
{
    Write-Host "No full backup (.vbk) files found in " $DPMProtectedFolder
}

To set DPM to run these scripts, I had to modify "c:\Program Files\Microsoft Data Protection Manager\DPM\Scripting\ScriptingConfig.xml" on the Veeam repository server. The file already exists, I just had to add the "DatasourceScriptConfig" section.

ScriptingConfig.xml

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<ScriptConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
 xmlns="http://schemas.microsoft.com/2003/dls/ScriptingConfig.xsd">
 <DatasourceScriptConfig DataSourceName="D:">
     <PreBackupScript>"C:\Scripts\DPMPreScript.ps1"</PreBackupScript>
     <PostBackupScript>"C:\Scripts\DPMPostScript.ps1"</PostBackupScript>
     <TimeOut>30</TimeOut>
   </DatasourceScriptConfig>

</ScriptConfiguration>

[veremin]

Thanks, Steve, for sharing valuable information with the community; highly appreciated.

[Aurock]

Small correction. DPM won't directly run a powershell script, so the scriptingconfig.xml file has to point to a shell script, which calls powershell:

ScriptingConfig.xml

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<ScriptConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
 xmlns="http://schemas.microsoft.com/2003/dls/ScriptingConfig.xsd">
 <DatasourceScriptConfig DataSourceName="d:">
     <PreBackupScript>"c:\scripts\startprescript.cmd"</PreBackupScript>
     <PostBackupScript>"c:\scripts\startpostscript.cmd"</PostBackupScript>
     <TimeOut>30</TimeOut>
   </DatasourceScriptConfig>

</ScriptConfiguration>

StartPrescript.cmd

Code: Select all

Powershell.exe -executionpolicy remotesigned -noninteractive -noprofile -file C:\Scripts\DPMPreScript.ps1

StartPostScrpt.cmd

Code: Select all

Powershell.exe -executionpolicy remotesigned -noninteractive -noprofile -file C:\Scripts\DPMPostScript.ps1

[adrian916]

If I understand you correctly, are you just trying to backup the .vbk files with DPM and ignore the .vrb ones?

If so, when setting up the protection group within DPM, there is a link in the bottom right of the selected items screen to 'exclude files'. Just exclude any with the .vrb extension. We do this to copy the full VEEAM backup from disk to tape within DPM without the additional space overhead of all the delta files.

[veremin]

Also, nowadays you can just put into use new type of job called “backup to copy” which, by default, skips .vrb from processing and copy only .vib/.vbk files. Thanks.