Powershell script for Offsite Disk storage

[craigcurtin]

Guys,

We have migrated a customer from a previous product to Veeam (11)

With the old software we basically stored a 4 weekly full backups, each of which went to their own subdirectory per VM so the structure would look like

\VMBackups\VMName1\Date1
\VMBackups\VMName1\Date2
\VMBackups\VMName1\Date3
\VMBackups\VMName1\Date4

each of these would hold the Full backup for that week and then incrementals for that week would be stored in a subdirectory under the Date

So to perform offsite storage for our small customers - they have 20 USB drives - labelled Week1 - Monday through Friday

We then hada series of bash scripts that would prompt for the correct drive for the day to be mounted and would then copy the appropriate subdirectory and all of its files (so the full backup and incrementals since the last full backup.

As the customers have a large investment (and are simple and comfortable with this_ we want to replicate the same functionality in VBR

We have sperate jobs for each server and these then store the backups per server in their own directories so that is a good start and lets us manage what we are copying

What we need is a powershell script to list the latest VBK in a directory, and then list any VIB files that are newer than that VBK and present them results in such a way that we can either copy them or store them to a file and feed this to robocopy etc

We have found plenty of scripts to find the latest VBK as per this

Code: Select all

$SourcePath=y:\W2K16-VM1
$TargetPath=Z:\USB-1\W2K16-VM1

$LastVBK = Get-ChildItem $SourcePath\*.vbk| Sort-Object CreationTime -Descending | Select-Object -First 1| Get-ChildItem -name;
Robocopy $SourcePath $TargetPath $LastVBK;

But have not been able to find a way to then identify the VIBs that are newer than this file and feed those to the same Process

So something like (in pseudo code)
Get Latest VBK (name and path)
Copy Latest VBK to USB Drive
Get Creation Date of Latest VBK
Get VIB files with creation date newer than Latest VBK
Copy VIB files to USB Drive

ANy pointers would be much appreciated ?

[soncscy]

Hey Craig,

To be honest, this feels over-engineered, especially when an in UI solution exists with Backup Copies + immediate mode and rotated drives. I get that you don't want to rock the boat, but if you can do it without scripts and get the same result, why not? I would propose the workflow:

1. Backup Copy Job targeting the repository backed by rotated drives: https://helpcenter.veeam.com/docs/backu ... ml?ver=110
2. Swap the drives on the week desired, and it will follow the retention scheme as you like

Remember, since the retention is across __all__ drives for Windows, not per drive, you will need to accommodate accordingly with your planning.

But I think this is a lot simpler.

If you really must go down this route, I'd use the Veeam Powershell console itself to get your storages paths (vbks + vibs) and use native sorting.


Get the Backup Chain and Storages (vbk + vibs). I sort by CreationTime here for convenience for me mentally:

Code: Select all

PS C:\Users\Administrator> $backup = Get-VBRBackup -name 'vmware-pervm-ffi
PS C:\Users\Administrator> $storages = $backup.GetAllChildrenStorages() | sort -Property CreationTime

I pull out the fulls with the following (we'll use them for their CreationTime as part of the filter later on):

Code: Select all

$Fulls = $storages |Where-Object {$_.IsFull -eq $true}

Then we can add the chain to some array $Chain like so (of course, you will have to design logic to figure out which item from the index to call, so the below is an __example__ of a backup with two chains (full + increments)):

Code: Select all

$Chain += $Fulls[-1]
$Chain += $storages | Where-Object {$_.IsFull -eq $false -and $_.CreationTime -gt $Fulls[-1].CreationTime -and $_.CreationTime -lt $Fulls[0].CreationTime}

Your end result is:

Code: Select all

PS C:\Users\Administrator> $Chain |select PartialPath, CreationTime

PartialPath                                 CreationTime
-----------                                 ------------
vmware-pervm-ffiD2021-04-03T183823_31F5.vbk 4/3/2021 18:36:35
vmware-pervm-ffiD2021-04-09T183637_1B21.vib 4/9/2021 18:36:37
vmware-pervm-ffiD2021-04-08T183640_841F.vib 4/8/2021 18:36:40
vmware-pervm-ffiD2021-04-07T183655_108A.vib 4/7/2021 18:36:55
vmware-pervm-ffiD2021-04-06T183646_5091.vib 4/6/2021 18:36:46
vmware-pervm-ffiD2021-04-05T183654_D525.vib 4/5/2021 18:36:54
vmware-pervm-ffiD2021-04-04T183650_F7A9.vib 4/4/2021 18:36:50

Finally, you can use the .FilePath.ToString() combo to get a file path you can send to Robocopy. (I changed the backup I used here because the first one was on a scale-out repo, which has special handling -- see below)

Code: Select all

PS C:\Users\Administrator> $storages2[0].filePath.Tostring()
E:\offloadbackup\TinyVM_migrated.vm-6756D2021-04-15T220107_5E10.vib

You can do this to get all paths and write to something like $Paths

Then you can pass the $Paths array to Robocopy somehow (I'm not sure how robocopy will handle the array), and be done with it.

Note there are a few caveats -- with scale-out repositories, you have to first get the extent and the file path for the storage, so this gets a little tricky. I repurposed this a bit for v11 from a post I found earlier on the forums (I think it was Tom Sightler who wrote it?), but this can work if you pass a Backup that is on a scale out repo:

Code: Select all

$Repository = $Backup.FindRepository()[0]
	$StoragePathsandBlocksize  = @()
	if($Repository.Type -eq "ExtendableRepository"){
		foreach($Storage in $Storages){
			$Extent = $Repository.FindRepositoryForExistingStorage($Storage.Id)
			$job = $Backup.FindJob()
			$StoragePathsandBlocksize += New-Object -TypeName psobject -Property @{Extent=$Extent.Name;Path=$($Extent.Path.ToString(),$job.Name.ToString(),$Storage.filepath -join "\");BlockSize=$Storage.BlockAlignmentSize;CreationTime=$Storage.CreationTime}
			}

this should produce the desired item; you will need to do some handling though if you have Linux based repositories, as you can see that the -join uses a Windows path delimiter.

Anyways, hope this helps (it was a fun afternoon puzzle), but all in all, I recommend avoid this and just use Backup Copies.

[craigcurtin]

Thanks for taking the time on this.

I initially tried to backup copy routine and it just did not work.

What happened previously (say a wednesday for example) the scripts would copy the latest full backup (taken on a Friday evening) and the incrementals since then (Monday and Tuesday in this example) - they would all be copied to the Wednesda disk - so in the event of a failure/fire etc etc we would have day by day restore capabilities from any single drive as well as access to the full backup no later than a week old.

When i tried this with Veeam - i found a number of issues (maybe i was too new to understand and need to go back and look again) but if i remember - only a single backup copy job can reference a rotated disk repository and we were running into problems with running out of disk space on the external drives as Veeam was not correctly tracking backups and kept trying to put full backups onto the disks.

I will give this another go in the next couple of weeks and in the meantime will check out the code you so kindly posted.

Many thanks

Craig

[soncscy]

Sure, you're welcome; it was a nice refresher actually since I've not touched some of that stuff since v10 released, so was a fun exercise.

>only a single backup copy job can reference a rotated disk repository

No such limitation really, you can do as many as you like.

>kept trying to put full backups onto the disks.

Sounds like maybe a support case would be needed? On each drive you will need a full backup the first time you rotate it, but on rotated drives connected to Windows repos, the chains should continue. I think the Immediate Copy has a way of doing all points for the entire chain btw? I seem to recall one of my clients telling me support helped them with this but I would have to ask a bit to see how they did it.

Anyways, hope the above helps and if you end up with something functional, please share would be curious what you end up with.