script to copy recent .vbk to tape

[markhumphreys]

We have multiple backups that we require backing up to tape. We only want the last .vbk to be backed up. We currently have a 14 day retention to disk. We have tried reverse incremental but backup performance is too slow.
We have also tried to create hard-links via a PowerShell script but this filled the backup repository disk.

Does anyone have a script to do this....

[veremin]

Hi, Mark, what do you mean by "filled backup repository"? It shouldn't be that hard to just add some lines responsible for clearing previous hard-links. For instance, this script uses the described approach.

Thanks.

[markhumphreys]

Hi,

By 'filled backup repository' I mean by using the hard-link option the backend storage gets consumed with the files created by the hard-link script

[veremin]

Not sure whether I completely follow you on that, but anyway. If the hard link scenario doesn't work for you, what about using some "staggering" area - the folder (called "Latest .vbk", for instance) that is specified as a source for file to tape job. Using PS commands, you will find and copy the latest full backup to it, and, then, execute the corresponding tape job. Once the tape job is finished, the backup files will be removed from the "staggering" area.

Thanks.

[beovax]

Can a script be created which basically takes the backup jobs and finds the latest vbk for each backup job and adds this to an array? Then when the backup to tape runs the backup to tape job runs using the file location pointers in the array rather than using hard links or a separate staging area. Previously I have used a script to find the latest backup and write it to tape but this was the latest backup from one job rather than multiple jobs. Someone with the PowerShell script experience could surely be able to do a script which can do this with just using a pointer to the files and back them up without hard links or staging first?

[veremin]

Can a script be created which basically takes the backup jobs and finds the latest vbk for each backup job and adds this to an array?

Yes, it's possible.

Then when the backup to tape runs the backup to tape job runs using the file location pointers in the array rather than using hard links or a separate staging area.

Backup to tape job operates with backup jobs/repositories, rather than with particular files. Thus, it's impossible to point it to the specific .vbks/.vibs.

In order to copy given files you should use file to tape job, instead. However, currently it's not possible to change "source" settings of file to tape job, so, you have to use either "hardlinks" or "staggering" area.

You can provide previously used script here. I believe it wouldn't be that hard to modify it for your current needs.

Thanks.

[beovax]

Here is the script. Someone provided me this previously that works on the job name and takes the last successful job and writes the last backup to tape. What I would like to be able to do is for multiple job names scan the repository and copy the last successful vbk full backup to tape for each job that has been run. If their is anyone that has a piece of code that can do this task would appreciate it.

Code: Select all

   if ( (Get-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue) -eq $null )
   {
          Add-PsSnapin VeeamPSSnapIn

   }

    $VeeamJobName = "jobname"
    $backupExecJob   = "jobname" 


    if ((Get-VBRJob | where {$_.Name -eq $VeeamJobName}).GetLastResult() -eq "Success")
    {
       $latestOib = Get-VBRBackup | where {$_.JobName -eq $VeeamJobName} | Get-VBRRestorePoint | sort CreationTime -Descending | select -First 1
       $latestOib
       $storage = $latestOib.GetStorage()
       $storage.FilePath

       $file = get-item $storage.FilePath


       if ($file.Attributes -band ([System.IO.FileAttributes]::Archive))
       {      
         & "d:\Program Files\Symantec\Backup Exec\bemcmd.exe" -o2 -j:$backupExecJob -s:$storage.FilePath -mTapes-Weekdays -r -w

    $file.Attributes = 'Archive'
       } 
    }

[veremin]

Just to clarify situation before we proceed to script modification - you're going to use Backup Exec, not VB&R, to copy latest backup data to tape, right? I'm wondering because the script would differ in two cases. Thanks.

[markhumphreys]

No that was an old script, we need to use VB&R thanks

[veremin]

As mentioned above, the only possible way to archive specific file (latest .vbk) with current tape implementation is to use combination of file to tape job and hardlinks/staggering area. Both examples are provided in the referenced thread. Thanks.

[markhumphreys]

We have found this script for backing up with Backup Exec to tape. Surely this can be done within Veeam for backups to tape direct?

http://www.symantec.com/connect/forums/ ... ry-bue2012


Mark

[veremin]

As it's been said above, the backup to tape job has to do with backup jobs/repositories, rather than with individual backup files. So, it's not the option here, and you have to use file to tape job, if you're after archiving certain files.

The problem with file to tape job is that Symantec PS snapin allows passing certain file/folders to tape jobs. Meanwhile, with our current PS implementation, once the "source" object is specified for file to tape job, it cannot be changed no matter what. So, you have to resort to hardlinks/staggering area approaches.

Hope this makes sense.
Thanks.

[beovax]

Looking into the Add-VBRTapeFilesJob it accepts multiple files. So I would have assumed the hardlink script that collects the filenames could write these to an array which could be passed to the add-VBRTapeFiles command or is that a limitation of the available command set?

Code: Select all

# create hardlinks from latest vbk files, erase online tape media and start backup job
# joerg riether
# august 2013
# NOT for ANY production use!
# only for experimetal purposes in testlabs
# please do NOT use this if you are not 100% sure what you are doing, this will erase your tape media!

add-pssnapin veeampssnapin

# Settings

$linkpath = "d:\veeam\veeam-hardlinks\"
$backuppath = "d:\veeam\veeam-backup\"

# kill previous hard links

del $linkpath*.*

# use with caution
# this will kill everything it finds in the drives

$tape = get-vbrtapemedium | Where-Object {$_.IsOnline -like "True"}
Erase-VBRTapeMedium -Medium $tape -Confirm:$false

# this will find the latest vbk files for you recursively starting from your backup path.

$files = dir $backuppath -filter "*.vbk" -rec
$filteredfiles = $files | group directory | foreach {@($_.group | sort {[datetime]$_.creationtime} -desc)[0]}

# this loop will separate path and name and it will create your hard links

ForEach ($File in $filteredfiles)
        {
        $FilePath = $File.DirectoryName
        $FileName = $File.Name
        cmd /c mklink /H $linkpath$Filename $Filepath\$Filename
        }

# call your backup to tape from file job pointing to your hard link path

Start-VBRJob -Job "FileToTapeL" -FullBackup

Add-VBRTapeFilesJob -Name "Files to tape backup job 1" -Server $server -Path "C:\backup\Backup Job 01\VM01 Backup Job 01.vbm" -MediaPool $mediapool

[veremin]

I'm not sure whether I follow you on that.

The script provided by you was referenced in the second post. It finds latest .vbk file, creates a hard link for it, and, then, executes the corresponding file to tape job. This script will work, if the file to tape job has been already created and pointed to the "hardlink" directory. The last line you've added creates a new file to tape job named "Files to tape backup job 1" that has backup metadata as it source ("C:\backup\Backup Job 01\VM01 Backup Job 01.vbm").

Thanks.

[beovax]

Apologies, ignore the last line. What I was asking was can the script rather than build up a separate folder location with hard links. Run through the loop writing the file location into a variable that could be passed to the backup job to run.

So for example I have 3 backup jobs (Backup1,Backup2, Backup3) these jobs have 14 days retention on disk. I then run a script that looks in each of these locations and just gets the latest full backup job. Then it writes that location to a variable I.e

C:\backuplocation\full1.vbk

It runs through the loop a second time and writes

C:\backuplocation2\full2.vbk to an array or another variable

Then once all locations are scanned I can pass this to the VBR-Tape command $full1,$full2 to write these to the tape.

Is this possible? I have seen people do similar things to this when using Symantec Backup Exec passing the files as a selection list. So was hoping this would be an option via the Veeam powershell command set.

[veremin]

Apologies, ignore the last line. What I was asking was can the script rather than build up a separate folder location with hard links. Run through the loop writing the file location into a variable that could be passed to the backup job to run.

This is essentially what the script referenced in the second post does. The only prerequisite is that the "hardlink" directory should exist, and file to tape job should be pointed to it(either in the GUI or in the initial script that creates job). Currently, there is no other way, apart from "staggering area" (that virtually uses the same concept as hardlink approach does).

Is this possible?

Kindly, see the answers provided above.

Thanks.

[infrateluq]

Hi, I try this script and I always have folders item in the job objects instead of file.

Code: Select all

#This script backup to tape the most recent vbk file from all of our backup repositoy
if ( (Get-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue) -eq $null )
{
 Add-PsSnapin VeeamPSSnapIn
}

#Get the most recent vbk from a folder
function GetNewestVBKFile($folder) {
    $items = Get-ChildItem $folder -File -Filter *.vbk;
    if($items -is [Array]) {
        return $items | Sort-Object -Property LastWriteTime -Descending | Select-Object -First:1
    }
    return $items;
}


#Clean the old jobs that has been created from this script
foreach($job in (Get-VBRTapeJob | Where-Object {$_.Name -match "^backutpToTape-Hebdo"})) {
    Remove-VBRJob -Job $job -Confirm:$false
}
#Seeting variables
$hebdoMP = Get-VBRTapeMediaPool -Name "Hebdomadaire"
$veeamCred = Get-VBRCredentials -Name "tlquebec1\svc-veeam"
$svrs = @();
#ServerName is used for the backup repository server name
#Folders2Backup is used for the backup folders to use as local folder name in the repository server
#Folders is used by Get-ChildItem from the server where the script is executed (same number of folders as Folders2Backup
#JobName is the job name
$svr = "" | select ServerName,Folders,Folders2Backup,JobName;
$svr.ServerName = "This server";
$svr.Folders = "E:\Backups\backup SVR1";
$svr.Folders2Backup = "E:\Backups\backup SV1";
$svr.JobName = "backutpToTape-Hebdo (SVR1)"
$svrs += $svr;
$svr = "" | select ServerName,Folders,Folders2Backup,JobName;
$svr.ServerName = "SVR2";
$svr.Folders2Backup = "F:\Backups\backup SVR2","F:\Backups\Backup SVR-2";
$svr.Folders = "\\SV2\f$\Backups\backup SVR2","\\SV2\f$\Backups\Backup SVR2-2";
$svr.JobName = "backutpToTape-Hebdo (SVR2)"
$svrs += $svr;
$previousJob = $null;
$currentJob = $null;
$firstJob = $null;
#For each server, create a backup job if any file has to be backuped
foreach($svr in $svrs) {
    #Get the backup files that has not been backuped
    $files2Backup = $null;
    foreach($folder in $svr.Folders) {
        $item = GetNewestVBKFile $folder;
        $file = Find-VBRTapeCatalog -Name $item.Name
        #if null, then the file has not been backuped
        #so, the file has to be backuped
        if($file -eq $null) {
            $fileName = "$($svr.Folders2Backup[$idx])\$($item.Name)";
            $files2Backup += $fileName;
        }
    }
    #If no file to backup, then continue to other server
    if($files2Backup.Count -eq 0) {
        continue;
    }
    $veeamSvr = Get-VBRServer -Name $svr.ServerName;
    $currentJob = Add-VBRTapeFilesJob -Name $svr.JobName -Server $veeamSvr -Path $files2Backup -MediaPool $hebdoMP -Credentials $veeamCred; #-Masks $mask 
    if($previousJob -ne $null) {
        #Set the job's schedule as after the previous
        $sched = Set-VBRJobSchedule -Job $currentJob -After -AfterJob $previousJob
    }
    if($firstJob -eq $null) {
        $firstJob = $currentJob;
    }
    $previousJob = $currentJob;
}
Start-VBRJob -Job $firstJob

What is the problem with the way I use the Add-VBTTapeFilesJob cmdlet?

Christian Poirier
TELUQ

[veremin]

Hi, Christian,

It must be a bug with the way "Add-VBRTapeFilesJob" commandlet works with individual files. If individual file is present to the said commandlet, it will be automatically added as a folder, and file to tape job will eventually fail with "nothing to backup" message.

According to the plan, this problem along with many others related to tape PS functionality will be fixed in the next release. For now, you have to use certain workarounds to bypass existing limitation.

Thanks.

[menthol]

Hello,

I've installed the latest patch but this functionality hasn't been fixed?

Please can you advise on a workaround?

Here's my script (the affected part)

Code: Select all

# Time to get our latest full backup file and store it into a variable, ready for duplication


$logger.AppendLine("Selecting latest full backup of SRV60")
$SRV60_Latest = Get-ChildItem $SRV60_b2d -filter *.vbk | Sort-Object LastWriteTime -descending | select-object -first 1


# Now we have the latest file in a variable, create the job that will duplicate the backup


try
{
    $logger.AppendLine("Creating job for duplicating $SRV60_Latest")
    Add-VBRTapeFilesJob -Name "SRV60 to Tape" -Server "This server" -Path "$SRV60_b2d\$SRV60_Latest" -MediaPool "DSG Tape Media"
    $logger.AppendLine("Successfully created job")
}
catch
{
    $logger.AppendLine("Error while creating duplication job: $error")
    $logger.AppendLine("Script execution FAILED")
    $smtp.Send($emailFrom, $emailTo, $subject_failure, $logger.ToString())
    exit
}

As the OP says, it tries to duplicate it as a folder and fails.

[veremin]

As I've said, the problems with tape PS functionality will be addressed in the next release, not patch. For now, you can use the script referenced in the second post. Thanks.

[menthol]

Thanks.

However it still doesn't work, it's still trying to copy a folder even with hard links.

Can anyone who has successfully done this advise?

[veremin]

The Add-VBRTapeFilesJob currently doesn't allow you to pass a single file to the newly created file to tape job. So, you have to create a folder called "Latest .vbk" or something, specify it as a source for a file to tape job via GUI, and, then, use the referenced script that creates a hard link to the latest .vbk file and execute file to tape job in accordance. Thanks.

[menthol]

That did it!

Thank you.

When I get some time I will add my script to the forum, hopefully it will help people.

[veremin]

Glad to hear that you've nailed it. According to the plan, the next version of PS snap-in will provide more flexibility in terms of working with tapes, so, there would be no need for using creative workarounds such hardlinks and similar. Thanks.

[menthol]

Thanks Eremin - and that's in B&R v8 yes? I look forward to it.

[veremin]

Correct. Stay tuned.