Export-VBRRestorePoint cmdlet as a background task

[juanmachado]

Is it possible to use the Export-VBRRestorePoint in a script in a way that it just sends it to the VBR server? Same as you can do it in the UI Console where you can start multiple Exports at the same time (if you want).

Let me explain:

We have a situation where we need to export many restore points to a new repository. I have the complex logic in a script that find the restore points that we need to export.
The issue is, powershell will pause until each Export-VBRRestorePoint is completed before is moves to the next one.

I tried script blocks, parallelism, you name it, and nothing works.


Export-VBRRestorePoint -RestorePoint $toRestore -Repository $repositoryTarget -RetentionPeriodType Days -RetentionNumber $expiringInDays

Does anybody know a way?
Hopefully is something as simple as a non-documented parameter

Thanks

[oleg.feoktistov]

Hi Juan,

Have you tried Invoke-Command with -AsJob parameter?

Code: Select all

$scriptBlock = {

$rp = Get-VBRRestorePoint -Name '*Debian*'
Export-VBRRestorePoint -RestorePoint $rp[0] -RetentionPeriodType Days -RetentionNumber 1 

}
$session = New-PSSession -ComputerName localhost

Invoke-Command -Session $session -AsJob -JobName "Export Job" -ScriptBlock $scriptBlock

This will start a background job in a separate powershell session that you can then monitor with Get-Job. If you need to invoke more commands, you just pass them to the same session.

Best regards,
Oleg

[juanmachado]

Thank you, @oleg.feoktistov.
I ended up doing something similar, but thank you for the tip on Invoke-Command -Session, that helps being able to monitor the jobs in the queue.

[Lewpy]

I ended up doing something similar, but thank you for the tip on Invoke-Command -Session, that helps being able to monitor the jobs in the queue.

Can I ask what you ended up doing?
I am in a very similar situation: I want to export very particular GFS restore points from one repository (ReFS, so block cloned) to another repository (NTFS DeDup, so needs uncompressed data), and the Export-VBRRestorePoint seems to be the best way of achieving this.
The Restore Points only need kept for a finite period of time, so "exporting" them is acceptable (they show differently in the GUI, etc. but this is fine for us).
VeeaMover would be ideal, however that is for moving entire backup chains around, not "cherry picking" individual GFS restore points.
Using the File Copy job is also simple to set up, but it doesn't take in to consideration the source/target repositories, so it doesn't "inflate" the backups in to the DeDup repository which impacts the ability of the deduplication process.
As the DeDup is a background task, I need to be able to stagger the data moves, so I need a way to be able to trigger moves of particular GFS Restore Points between the repositories every day or so over the next week or two (or three).
Using the Export function in the GUI is great as you can bulk select all the VMs in the repository, however selecting the required Restore Point per VM is a bit tedious but it does schedule all the VMs to Export and only runs them based on Repository Resource throttling, which is nice
So I've dropped to PowerShell, and constructed a method to select the required Restore Points, but I've hit the issue that you did that it runs "interactively". I can cancel the PowerShell command when the first one in the pipeline is running, but that then cancels the whole thing, however the one that was running when I cancel stays running in the background in the Backup Console
Other commands have the "RunAsync" parameter that the Export-VBRRestorePoint is missing
Here is the crude code as it stands:

Code: Select all

$mySourceBackupJob = "My Archive"
$myDestinationRespoitoryName = "My Repository"
$myYear = 2024
$myMonth = 2
$myStartDate = Get-Date -Year $myYear -Month $myMonth -Day 1 -Hour 0 -Minute 0 -Second 0
$myEndDate = Get-Date -Year $myYear -Month $myMonth -Day 10 -Hour 0 -Minute 0 -Second 0
$mySourceBackupJobId = (Get-VBRBackup | Where-Object {($_.Name -eq $mySourceBackupJob) -and ($_.JobId -ne "00000000-0000-0000-0000-000000000000")} | Sort-Object -Property CreationDate | Select-Object -Last 1).Id
$myDestinationRespoitory = Get-VBRBackupRepository -Name $myDestinationRespoitoryName
Get-VBRBackup -Id $mySourceBackupJobId | Get-VBRRestorePoint | Where-Object {($_.Type -eq "Full") -and ($_.CreationTime -ge $myStartDate) -and  ($_.CreationTime -le $myEndDate)} | Export-VBRRestorePoint -Repository $myDestinationRespoitory

Notes:

• Selection of the correct BackupJob Id is convoluted, because Veeam appears to remember deleted backup jobs of the same name (that don't show in the GUI), so I try to filter them out by the JobId and then selecting the most recently created one.
• I want GFS restore points that were created at the beginning of the month, so I have some crude checks to see if it falls in the first 10 days of the month, because it might have been created on the 2nd day of the month due to processing time, other backups scheduled beforehand, etc. 10 days is wide enough to catch them all, but not too wide to grab the next months restore points, and I don't have to fiddle around with the varying lengths of months

I have also noticed that Export-VBRRestorePoint ignores any Traffic Throttling rules between agents when going from Repository to Repository [even when unticking "Never throttle any restore activity"], whereas File Copy jobs do respect the Traffic Throttling rules.

[Lewpy]

Okay, I've gone a bit ugly, but it is to work around the interactive "issue" with the Export-VBRRestorePoint
I've adapted my code to start a new PS Session per Export-VBRRestorePoint, because the command stops any further command being issued to that PS Session, even when RunAs Job. I assume this is down to the nature that it is continually sending back progress data to the session, or along those lines.

Code: Select all

$mySourceBackupJob = "My Archive"
$myDestinationRespoitoryName = "My Repository"
$myYear = 2024
$myMonth = 2
$myStartDate = Get-Date -Year $myYear -Month $myMonth -Day 1 -Hour 0 -Minute 0 -Second 0
$myEndDate = Get-Date -Year $myYear -Month $myMonth -Day 10 -Hour 0 -Minute 0 -Second 0
$mySourceBackupJobId = (Get-VBRBackup | Where-Object {($_.Name -eq $mySourceBackupJob) -and ($_.JobId -ne "00000000-0000-0000-0000-000000000000")} | Sort-Object -Property CreationDate | Select-Object -Last 1).Id
Get-VBRBackup -Id $mySourceBackupJobId | Get-VBRRestorePoint | Where-Object {($_.Type -eq "Full") -and ($_.CreationTime -ge $myStartDate) -and  ($_.CreationTime -le $myEndDate)} | ForEach-Object {$myRestorePointName = $_.Name; $myRestorePointId = $_.Id; $myPSSession = New-PSSession -ComputerName localhost; Invoke-Command -Session $myPSSession -AsJob -JobName $myRestorePointName -ScriptBlock {$myDestinationRespoitory = Get-VBRBackupRepository -Name $Using:myDestinationRespoitoryName; Get-VBRRestorePoint -Id $Using:myRestorePointId | Export-VBRRestorePoint -Repository $myDestinationRespoitory}; Disconnect-PSSession $myPSSession}

This kind of works: it is running correctly for each Restore Point, however only 8 of the expected 24 Restore Points appeared in the GUI as exporting

The other issue is this requires Administrator privileges to run the Remote PS Session I believe, because it required me to run my PS Console elevated before it worked.

UPDATE
Things have got uglier in my code: Adding a Sleep for 10 seconds between each PS Session call seems to have resolved the issue
I am guessing something didn't like the flurry of rapid commands, and slowing it down a bit seems to have appeased it.
But not knowing why there was an issue, the choosing of 10 second delay is arbitrary and just may not work all the time (hence not good coding).

Code: Select all

$mySourceBackupJob = "My Archive"
$myDestinationRespoitoryName = "My Repository"
$myYear = 2024
$myMonth = 2
$myStartDate = Get-Date -Year $myYear -Month $myMonth -Day 1 -Hour 0 -Minute 0 -Second 0
$myEndDate = Get-Date -Year $myYear -Month $myMonth -Day 10 -Hour 0 -Minute 0 -Second 0
$mySourceBackupJobId = (Get-VBRBackup | Where-Object {($_.Name -eq $mySourceBackupJob) -and ($_.JobId -ne "00000000-0000-0000-0000-000000000000")} | Sort-Object -Property CreationDate | Select-Object -Last 1).Id
Get-VBRBackup -Id $mySourceBackupJobId | Get-VBRRestorePoint | Where-Object {($_.Type -eq "Full") -and ($_.CreationTime -ge $myStartDate) -and  ($_.CreationTime -le $myEndDate)} | ForEach-Object {$myRestorePointName = $_.Name; $myRestorePointId = $_.Id; $myPSSession = New-PSSession -ComputerName localhost; Invoke-Command -Session $myPSSession -AsJob -JobName $myRestorePointName -ScriptBlock {$myDestinationRespoitory = Get-VBRBackupRepository -Name $Using:myDestinationRespoitoryName; Get-VBRRestorePoint -Id $Using:myRestorePointId | Export-VBRRestorePoint -Repository $myDestinationRespoitory}; Start-Sleep -Seconds 10; Disconnect-PSSession $myPSSession}

[juanmachado]

I did it a little bit differently because I needed them to run in parallel, but you get the gist.

Here is the code I use now:

Code: Select all

#Example: ./ExportRestorePoint_Criticals.ps1 -vmName "ansible.xxx"

param (
    [Parameter(Mandatory=$true)]
    [string]$vmName
)

# Start transcript to begin writing all output to a text file
Start-Transcript -Path ("C:\scripts\Exports\ExportedVMs\ExportRestorePoints-" + $vmName + ".txt")

#Clear-Host

# Disconnect from the previous Veeam server session, if one exists
Write-Host "Disconnecting from previous Veeam server session, if one exists..."
Disconnect-VBRServer -ErrorAction SilentlyContinue

# Connect to Veeam server with a different user account
Write-Host "Connecting to Veeam server with a different user account..."
Connect-VBRServer -Server "veeam.xxx" -User "xxxx" -Password "xxxx"
Write-Host "" # Add a blank line for separation
Write-Host "" # Add a blank line for separation


# Initialize a hashtable to count occurrences of BackupID
$backupIDCount = @{}

# Initialize an array to store custom objects
$reportObjects = @()

# Iterate over each VM name
Write-Host "Restore points found, Type FULL:"
Write-Host "-------------------------------"

# Retrieve the Full restore points for the specified VM
$restorePoints = Get-VBRRestorePoint -Name $vmName | Where-Object { $_.Type -eq "Full" }

# Iterate over each restore point
foreach ($restorePoint in $restorePoints) {
    # Print the VMName, CompletionTimeUTC, and OriginalOibId for each restore point
    Write-Host "VMName: $($restorePoint.VMName)"
    Write-Host "CompletionTimeUTC: $($restorePoint.CompletionTimeUTC)"
    Write-Host "OriginalOibId: $($restorePoint.OriginalOibId)"
    Write-Host "" # Add a separator for clarity
    
    # For each restore point, create a custom object, update the count of BackupID occurrences, and add it to the report objects array
    $reportObjectProperties = @{}
    foreach ($property in $restorePoint.PSObject.Properties) {
        $reportObjectProperties[$property.Name] = $property.Value
    }
    $reportObject = [PSCustomObject]$reportObjectProperties
    $backupIDCount[$restorePoint.BackupID]++
    $reportObjects += $reportObject
}

# Filter the report to only include entries with BackupIDs found more than 4 times
$filteredReportObjects = $reportObjects | Where-Object { $backupIDCount[$_.BackupID] -gt 4 }

# Group the report by VMName and select the last 5 entries for each group
$filteredReportObjectsGrouped = $filteredReportObjects | Group-Object VMName | ForEach-Object {
    $_.Group | Select-Object -Last 5 | Select-Object -Property *
}



# Specify the target repository


$repositoryTarget = Get-VBRBackupRepository -Name "XXX-BackblazeB2-DR-Critical"
# Print the properties of $repositoryTarget
Write-Host "Repository Target for exports:"
Write-Host "-----------------------------"
Write-Host "Name: $($repositoryTarget.Name)"
Write-Host "ID: $($repositoryTarget.Id)"
Write-Host "Group: $($repositoryTarget.Group)"
Write-Host "TypeDisplay: $($repositoryTarget.TypeDisplay)"
Write-Host ""
Write-Host ""




# Iterate over each entry in $filteredReportObjectsGrouped to print the ones in the needed Repository with just the youngest FULL backups
$currentDate = Get-Date
Write-Host "Current Date is: $currentDate "
Write-Host "" # Add a blank line for separation

Write-Host "" # Add a blank line for separation
Write-Host "" # Add a blank line for separation
# Iterate over each entry in $filteredReportObjectsGrouped to print the ones in the needed Repository with just the youngest FULL backups
Write-Host "The following restore points for this VM (and ONLY their youngest 5 FULL backups found in Backblaze "
Write-Host "----------------------------------------------------------------------------"
Write-Host "B2 Object storage repository) will be exported to the new repository: "
Write-Host "---------------------------------------------------------------------"
Write-Host "" # Add a blank line for separation
Write-Host "" # Add a blank line for separation

foreach ($entry in $filteredReportObjectsGrouped) {
    # Extract the UID from the entry
    $UID = $entry.UID
    
    # Output the UID to the screen
    Write-Host "UID: $UID"
    Write-Host "UID Type: $($UID.GetType().FullName)"
    $toRestore = Get-VBRRestorePoint -Id $UID.ToString()
    
    Write-Host "Name: $($toRestore.VmName)"
    Write-Host "OibID: $($toRestore.OriginalOibId)"
    Write-Host "CompletionTimeUtc: $($toRestore.CompletionTimeUtc)"
    
    # Calculate how many days ago the restore point was taken
    $daysAgo = ($currentDate - $toRestore.CompletionTimeUtc).Days
    Write-Host "Age in Days: $daysAgo"
    
    $expiringInDays = 180 - $daysAgo
    Write-Host "Exported restore point will expire in $expiringInDays days"
    Write-Host "" # Add a blank line for separation

    # Call Veeam to Export those restore points
    Export-VBRRestorePoint -RestorePoint $toRestore -Repository $repositoryTarget -RetentionPeriodType Days -RetentionNumber $expiringInDays 
}

# Stop transcript to finish writing to the file
Stop-Transcript

I run the export in chunks of 20 groups.

Code: Select all

# Example: .\RunExportParallel.ps1 -filename "Non_Critical_VMs.txt"

param (
    [Parameter(Mandatory=$true)]
    [string]$filename
)

Clear-Host
# Read the content of the text file
$vmNames = Get-Content $filename

# Define the chunk size
$chunkSize = 20

# Split the VM names into chunks
$chunks = @()
for ($i = 0; $i -lt $vmNames.Count; $i += $chunkSize) {
    $chunks += ,$vmNames[$i..($i + $chunkSize - 1)]
}

# Loop through each chunk of VM names
foreach ($chunk in $chunks) {
    # Start a background job for each VM in the chunk
    $jobs = @()
    foreach ($vmName in $chunk) {
        $jobs += Start-Job -ScriptBlock {
            param($vm)
            Write-Output "Exporting restore points for VM: $vm"
			Write-Host "B2 Object storage repository) will be exported to the new repository: "
			Write-Host "--------------------------------------------------------------------"
			Write-Host "" # Add a blank line for separation
			Write-Host "" # Add a blank line for separation
            $result = & "C:\scripts\Exports\ExportRestorePoint_Criticals.ps1" -vmName $vm
            Write-Output $result
        } -ArgumentList $vmName
    }

    # Wait for all jobs in the current chunk to complete
    $jobs | Wait-Job

    # Retrieve the output from each job and display it
    $jobs | ForEach-Object {
        $jobOutput = Receive-Job $_
        Write-Output $jobOutput
    }

    # Remove the finished jobs
    $jobs | Remove-Job
}

[Lewpy]

Hi Juan,

Thanks for coming back and posting your code
Yeah, our use-cases are different, but it is interesting to see the way you navigated around it.
For me, I see about an extra 200MB of RAM usage per VM Export using my method, as each PowerShell Remote session creates

• wamprovhost.exe [the most RAM]
• Veeam.Backup.Satellite.exe [in the middle]
• conhost.exe [the least RAM]

The PS Session expires 2 hours after I "disconnect" from it, so the processes tidy up by themselves. I could "remove" the PS Session quicker, but I'd want to wait until the job appears running in the Veeam GUI, which takes about 30 seconds from when I create the Job.
Obviously, there are VeeamAgent.exe processes running for each active Export, but that is expected behaviour.

I just batched about 36 VMs, so that is an overhead of 7GB of RAM (for 2 hours) to launch them all: not insignificant, but acceptable to me for now.

Lewis.

Update
Okay, just noticed the -IdleTimeoutSec parameter for Disconnect-PSSession, so I'll use that with the minimum value of 1 minute and see if that still works