Comprehensive data protection for all workloads
Post Reply
souperstar
Enthusiast
Posts: 38
Liked: 4 times
Joined: Dec 30, 2011 10:26 pm
Full Name: Chris

Copying backups to offsite (daily)

Post by souperstar »

We've been using Veeam B&R since version 5, and have just migrated our entire VMware infrastructure to new hosts and ~40TB of storage.

Our data usage has grown, immensely. I'm looking at throwing out our old methods of backups with Veeam and start completely fresh. I was using Reverse Incrementals, but the amount of time required to copy the full .vbk's to our offsite daily disks is growing. The backup jobs themselves still complete timely.

I've been reading about Version 7 all day, looking at the beta .dll's, a spreadsheet on the best backup job methods, and the features of Backup Copy Jobs (there seems to be an awful lot of confusion around that topic, it took me some time to become familiar myself). I've got V7 in place on Server 2012.

So, my questions are:
  1. What's the FASTEST way I can a copy of a .vbk to offsite disks daily. I was hoping to utilize the new Backup Copy Job, as I understand it only copies changed blocks of the backup just as the actual backup job does. This would greatly reduce the amount of time it would take me to have a working copy on the rotated disks. But, since the disks are in rotation every day, it looks like Veeam will destroy the previous .vbk on the rotated disk and perform a new full copy. This is the same behavior as the regular backup job, and nets me no gain over my robocopy/powershell scripting.
  2. Is there any way with Veeam to reuse an X day old backup file on one repository and compare it to the backup in another, and copy changes?
  3. Are there any 3rd party tools I could be looking at to copy only changed blocks of the .vbk's without corrupting them?
Yizhar basically summed up what I'm trying to get at: http://forums.veeam.com/post94266.html#p94266
I suggest the following as feature request for next Veeam release:
...
An option to select if we want to use existing data (a VBK from prior backup) as seed for the a new synthetic one, or simply to overwrite it.
...
Does it exist?
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Copying backups to offsite (daily)

Post by foggy »

Chris, am I understanding right that the main problem is that you need to have a full VBK file offloaded to rotated drives in offsite location daily? The currently implemented rotated drives scenario does not allow to transfer only changes using the older backup as a seed (however we are looking into adding this functionality down the road). However, I assume you still can benefit from the backup copy jobs.

Unlike regular backup jobs, backup copy jobs do not hit production VMs twice but rather synthetically create required restore points in target location from VM data in source backup repositories. Plus, built-in WAN acceleration allows for up to 50x faster data transfer than a traditional file copy across the WAN (using 3d party tools like rsync or robocopy). What you could probably do is to create several backup copy jobs for the number of rotated drives and schedule them to run correspondingly, each writing exclusively on its own drive. Thus you will have a complete backup copy job chain on each drive, including VBK file, and transfer offsite only changes each time.
veremin
Product Manager
Posts: 20405
Liked: 2298 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Copying backups to offsite (daily)

Post by veremin »

Hi, Chris,

As an additional option, you can utilize something like RSync, that is built for block-level file synchronization. Rsync will allow not transferring the whole .vbk-file, but instead just updating the .vbk file that resides on the given drive.

In case of Rsync, you have to enable previous naming convention first. Once it's done, only block level changes will be transferred by rsync and not the backup file as a whole. The regkey responsible for enabling previous naming convention can be found here:

http://cp.veeam.com/knowledgebase/article/KB1076

Thanks.
souperstar
Enthusiast
Posts: 38
Liked: 4 times
Joined: Dec 30, 2011 10:26 pm
Full Name: Chris

Re: Copying backups to offsite (daily)

Post by souperstar »

foggy wrote:Chris, am I understanding right that the main problem is that you need to have a full VBK file offloaded to rotated drives in offsite location daily? The currently implemented rotated drives scenario does not allow to transfer only changes using the older backup as a seed (however we are looking into adding this functionality down the road). However, I assume you still can benefit from the backup copy jobs.

Unlike regular backup jobs, backup copy jobs do not hit production VMs twice but rather synthetically create required restore points in target location from VM data in source backup repositories. Plus, built-in WAN acceleration allows for up to 50x faster data transfer than a traditional file copy across the WAN (using 3d party tools like rsync or robocopy). What you could probably do is to create several backup copy jobs for the number of rotated drives and schedule them to run correspondingly, each writing exclusively on its own drive. Thus you will have a complete backup copy job chain on each drive, including VBK file, and transfer offsite only changes each time.
Correct, I need to have full VBK's go offsite via a pair of esata COTS drives every day. The thought crossed my mind to do what you're talking about in the second paragraph, I was just really hoping something was going to be built in. I remember hoping that's what the "File Copy" was back in version 5 when we first setup Veeam. This would be a great feature!

My initial thoughts are to:
  1. Create a separate repository for each backup disk.
  2. Expand on my powershell script to check the date (day), and mount the drive that should be for that day.
  3. Create a separate Backup Copy job for each backup disk, with a continuous schedule, or a set schedule, something!
  4. Figure out how to also do this for our monthly offsite disks (2 sets of those, so the current month and last month are also on disk).
This should get me faster differential copies and the full .vbk's I need. When my script is finished, I'll probably have to post it in here. I can't be the only one trying to figure out how to get a .vbk on removable media daily. The script is already fairly long, because I'm not a programmer (no functions here!) and it scans all physical drives attached to the server, locates the appropriate backup drives based on size (though it could be many other attributes such as vendor, this is the best for us), changes their drive letter(s), then mounts them in TrueCrypt. The people doing the actual swapping of the drives sure did like not having to do this manually...

As far as rsync is concerned, I'm really just using Windows (only a few Linux virtual servers/appliances). I know there are Windows implementations of rsync, and I hear the algorithm behind it is great, but I for some reason don't want to mess with it.

Finally, just one last question: Should I call support and ask for the feature to be able to have Veeam recognize and use old .vbk's?
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Copying backups to offsite (daily)

Post by foggy »

souperstar wrote:Finally, just one last question: Should I call support and ask for the feature to be able to have Veeam recognize and use old .vbk's?
No need to contact support with that, since you've already made request on the forums (and this is the most right place for such requests). Moreover, this is already on our list.

And of course, sharing the details of your final implementation with the community will be tremendously appreciated!
souperstar
Enthusiast
Posts: 38
Liked: 4 times
Joined: Dec 30, 2011 10:26 pm
Full Name: Chris

Re: Copying backups to offsite (daily)

Post by souperstar »

Can the Veeam Powershell snap-in be used to call a BackupCopy Job? I'm hoping to chain my Backup jobs and start the BackupCopy Jobs immediately after.

If there is a better way, I'm open to it!
Vitaliy S.
VP, Product Management
Posts: 27377
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Copying backups to offsite (daily)

Post by Vitaliy S. »

Hi Chris,

Yes, you can use PowerShell script to trigger a backup copy job. See this topic for the example > Backup copy "Sync Now" with powershell

Thanks!
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Copying backups to offsite (daily)

Post by foggy »

Please note that jobs chaining is not among our best practices as it could have undesired consequences. Using max concurrent number of tasks assigned to proxy/repository servers is recommended instead.
souperstar
Enthusiast
Posts: 38
Liked: 4 times
Joined: Dec 30, 2011 10:26 pm
Full Name: Chris

Re: Copying backups to offsite (daily)

Post by souperstar »

Gotcha, I think I will try and avoid using Powershell with Veeam altogether and chain schedule the jobs. I'll explore more about the BackupCopy job scheduling. I'm still a little fuzzy on it. I hadn't completely finished the script, so I'm happy. You guys need to take some days off too, posting on the weekends... :x
souperstar
Enthusiast
Posts: 38
Liked: 4 times
Joined: Dec 30, 2011 10:26 pm
Full Name: Chris

Re: Copying backups to offsite (daily)

Post by souperstar »

Well, I spent way too much time trying to kick off a Backup Copy Job to rotated media. I accept defeat. After creating a repository for two daily disks, for 8 days worth of backups, and their corresponding jobs, I'm deleting all of it.

The scheduler is downright useless for this purpose, and unless you disable the job you'll receive ~10,000,000 errors before a disk comes back the following week, because the repo is offline. What I tried was using a PS script to enable the Backup Copy Job after my Backup Jobs as a post-job script, and using a post-job script on the Backup Copy Job to disable itself. It was fairly easy to incorporate logic to only process on a "Success"ful job completion. But it wasn't netting me much time, if any, due to it needing to "transform" backups at the Backup Copy Job destination. This would often take hours...

Anyway, here is a script I use to mount my offsite drives, in case anyone may find it useful. I'm not a programmer, the best I can do is sometimes write functions. I don't bother with parameters and the like :) . This script get's called by the script below it, and I'm also going to schedule it in Task Manager in case they forget to run the script.

Code: Select all

<#
 Chris R | MOSERS | 4/10/2014

 This script will find a drive based on the size
 specified, mount the drive to the specified
 letter, and utilize encryption.

 Red text indicates failure, Green is success.

 This script requires PowerShell 3.0 on Server
 2012 or Windows 8!
#>
Clear-Host
#
# Stuff you may edit:
#
## Disk Stuff
$pPath = "Q:\" 				        #This will be the target drive letter (default "Q:\")
#$pSize = 2000398934016
$pSize = 4000787030016           #The size of your drive in bytes
## Encryption Stuff
$pEncPath = 'C:\Program Files\dcrypt\dccon.exe'
$pEncPass = "password"            #Encryption password
##################################################
########## DO NOT EDIT BELOW THIS LINE ###########

# Check to see if path exists
Function PathTest {Test-Path -Path $pPath}

# Mount the encrypted partition
Function MountEncryption {
    & $pEncPath -mount $pPath.Substring(0,2) -p $pEncPass > $null
}

# Online the disk if it is offline
Function OnlineDisk {
    If ($pDisk.OperationalStatus -ne "Online") {
        Write-Host "Disk" $pDisk.Number "is not online, trying to online it..."
        Set-Disk -Number $pDisk.Number -IsOffline $false
        Start-Sleep 2
    }
}

################ START OF SCRIPT #################

# Use PathTest func to find if disk is already mounted
If (PathTest -eq $True) {
	Write-Host "Drive is already mounted! Exiting in 5 seconds..." -ForegroundColor Green -BackgroundColor DarkGreen
	Start-Sleep -s 5
	exit
}

#Introduce tolerance for drive size, it varies
$pSizeL = ($pSize * .99)
$pSizeH = ($pSize * 1.01)

#Find a disk based on $pSize in user edit section
$pDisk = Get-Disk | Where-Object {$_.Size -ge $pSizeL -and $_.Size -le $pSizeH}
               
#Check if we found a disk
If ($pDisk) {

    Write-Host ("Found disk *" + $pDisk.Model + "*, size " + $('{0:N0}' -f ($pDisk.Size / 1GB)) + "GB, as disk number " + $pDisk.Number + ".")
    
    OnlineDisk #check if disk is online

    #Find the right partition on the disk
    $pPart = Get-Partition -DiskNumber $pDisk.Number | Where-Object {$_.Size -ge $pSizeL -and $_.Size -le $pSizeH}

    #Simple empty check, then give the disk the correct drive letter if incorrect
    if ($pPart.DriveLetter -ne $pPath.Substring(0,1)) {
        Set-Partition -PartitionNumber $pPart.PartitionNumber -NewDriveLetter $pPath.Substring(0,1) -DiskNumber $pDisk.Number
    }

    #Use the MountEncryption func and wait
    MountEncryption
    Start-Sleep 5

    #Check to see if it really mounted
    If (PathTest -eq $True) {
        Write-Host "Disk" $pPath.Substring(0,1) "is now mounted." -ForegroundColor Green -BackgroundColor DarkGreen
    }
    Else {
        Write-Host "Disk" $pPath.Substring(0,1) "didn't mount." -ForegroundColor Red -BackgroundColor DarkRed
    }
}
#We didn't find a disk
Else {Write-Host "Suitable disk not found! Exiting..." -ForegroundColor Red -BackgroundColor DarkRed}
Here is a script we will be using to make sure the drives mount, remotely:

Code: Select all

Clear-Host
Write-Host "*** REMOTE MOUNT DRIVE SCRIPT ***"
Write-Host "Attempting to mount Q Drive..."
invoke-command { powershell.exe -noprofile -executionpolicy Bypass "C:\Scripts\MountDrives\MountQ.ps1" } -computername Veeam
Write-Host "."
Write-Host "Attempting to mount R Drive..."
invoke-command { powershell.exe -noprofile -executionpolicy Bypass "C:\Scripts\MountDrives\MountR.ps1" } -computername Veeam
Write-Host "."

$pQ = Test-Path \\Veeam\Q$
$pR = Test-Path \\Veeam\R$

If ($pQ -and $pR -eq "True"){
	Write-Host "SUCCESS: Finished mounting drives! You may close this window." -ForegroundColor Green -BackgroundColor DarkGreen
}
Else {
	Write-Host "FAILURE: One or more drives failed to mount, you're on your own!" -ForegroundColor Red -BackgroundColor DarkRed
}

Start-Sleep 5
I'm back to using RoboCopy to copy my .vbk's.

Veeam, I strongly encourage you to implement built-in encryption, and a real way of incorporating rotating disks. The registry edit for ForceCreateMissingVBK (and the other one) isn't nearly enough... I feel there is a lot of "potential" in the Backup Copy Jobs reusing previous backup chains. I feel like I was trying to push Veeam a little too far. Looking forward to a next version, that's for sure!
Tommy Armando
Influencer
Posts: 14
Liked: never
Joined: Apr 16, 2021 9:02 am
Full Name: Tommy Armando
Contact:

Re: Copying backups to offsite (daily)

Post by Tommy Armando »

Oh,
We plan to use the backup copy feature to store data on the Local NAS and then replicate or send the backup results to our NAS at the remote site. can anyone help recommend a good NAS model for us to use or can anyone recommend another method for doing offsite backups.
We have 40 TB of data that needs to be backed up offsite every day.

Thanks
veremin
Product Manager
Posts: 20405
Liked: 2298 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Copying backups to offsite (daily)

Post by veremin »

We typically avoid recommending NAS devices, especially low-end ones, since they are causing too many data corruption issues.

You would be in better shape with physical server with bunch of JBODs attached. Couple it with backup copy job or Capacity Tier of Scale-Out Backup Repository and you are good to go.

Thanks!
Post Reply

Who is online

Users browsing this forum: Paul.Loewenkamp and 263 guests