-
- Novice
- Posts: 8
- Liked: 1 time
- Joined: May 13, 2010 7:01 am
- Full Name: Richard Ramshaw
- Contact:
Job Chaining / Maximum concurrent jobs - Fix!
Hi All
There have been many people wanting a job chaning solution for Veeam Backup or a way of running a maximum number of concurrent jobs. Many reasons for wanting to do this (ours was that we have a WAN link and 2 jobs saturates it, 3 make it run badly)
For all of those out there that wanted it here is a powershell script that does exactly that.
It accepts 2 command line parameters, 1 for a batch of jobs to run, this supports wild cards so you can run the script with *daily* to run all backups with Daily in their name. The second parameter is the number of concurrent jobs that you want to have running e.g. 2. The script will find all the jobs that match, list them and then wait untill less than the maximum backups are running, it will then kick off the next backup and wait until another 'slot' becomes avaliable before starting another backup.
As the script checks with Veeam as to which backups are running you can run the script more than once at the same time and it will still keep the total number of backups running accross all scripts to less than the maximum.
To run the script from the command line use something like this:
powershell -command ".\Runjobs.ps1 *daily* 4"
If you need to put spaces in it then you need to get funky with the powershell command line like this:
powershell -command "& 'c:\program files\Veeam\Backup and FastSCP\Runjobs.ps1' '*six hourly' 2"
The contents of the script is below, it will need saving as a powershell script i.e. with a .ps1 extension. If you cannot run scripts in powershell (this is the default) you will need to open powershell and type Set-ExecutionPolicy Unrestricted
This is my first powershell script so any comments/improvements would be welcomed .
Cheers
Richard
###############################################
If ($args.Count -ne 2 -or $args[0] -eq "" -or $args[1] -eq 0) {
Write-Host "Incorrect arguments, syntax: runjobs JobContains MaxJobs";
Write-Host "eg: runjobs *Daily* 2";
Exit;
}
add-pssnapin veeampssnapin;
$JobList=@();
$runningjobs=@();
$JobMatch = $args[0];
$Maxjobs = $args[1];
Write-Host "Looking for jobs that match" $JobMatch;
Get-VBRJob | where {$_.Name -like $JobMatch} | foreach {$JobList += $_.name};
Write-Host "Found" $joblist.count "jobs";
If ($joblist -eq $null -or $joblist.count -eq 0) {
Write-Host "No Jobs match that string... exiting";
Exit;
}
$joblist = $joblist | Sort-Object
Write-Host "Jobs to run...";
ForEach ($job in $JobList) {Write-Host " " $job;}
ForEach ($job in $JobList) {
Do {
$runningjobs=@();
Get-VBRJob | where {$_.Control -eq "Start"} | foreach {$RunningJobs += $_.name};
If ($runningjobs.count -ge $Maxjobs) {
Write-Host "Waiting to start '" $job "' as " $runningjobs "are still running";
Start-Sleep -s 60;
}
}
While ($runningjobs.count -ge $Maxjobs)
Write-Host "Starting " $job;
Start-Job -scriptblock {add-pssnapin veeampssnapin; Start-VBRJob $args[0]} -ArgumentList $job;
Write-Host "Waiting for veeam to start the job";
Start-Sleep -s 60;
}
#############################
There have been many people wanting a job chaning solution for Veeam Backup or a way of running a maximum number of concurrent jobs. Many reasons for wanting to do this (ours was that we have a WAN link and 2 jobs saturates it, 3 make it run badly)
For all of those out there that wanted it here is a powershell script that does exactly that.
It accepts 2 command line parameters, 1 for a batch of jobs to run, this supports wild cards so you can run the script with *daily* to run all backups with Daily in their name. The second parameter is the number of concurrent jobs that you want to have running e.g. 2. The script will find all the jobs that match, list them and then wait untill less than the maximum backups are running, it will then kick off the next backup and wait until another 'slot' becomes avaliable before starting another backup.
As the script checks with Veeam as to which backups are running you can run the script more than once at the same time and it will still keep the total number of backups running accross all scripts to less than the maximum.
To run the script from the command line use something like this:
powershell -command ".\Runjobs.ps1 *daily* 4"
If you need to put spaces in it then you need to get funky with the powershell command line like this:
powershell -command "& 'c:\program files\Veeam\Backup and FastSCP\Runjobs.ps1' '*six hourly' 2"
The contents of the script is below, it will need saving as a powershell script i.e. with a .ps1 extension. If you cannot run scripts in powershell (this is the default) you will need to open powershell and type Set-ExecutionPolicy Unrestricted
This is my first powershell script so any comments/improvements would be welcomed .
Cheers
Richard
###############################################
If ($args.Count -ne 2 -or $args[0] -eq "" -or $args[1] -eq 0) {
Write-Host "Incorrect arguments, syntax: runjobs JobContains MaxJobs";
Write-Host "eg: runjobs *Daily* 2";
Exit;
}
add-pssnapin veeampssnapin;
$JobList=@();
$runningjobs=@();
$JobMatch = $args[0];
$Maxjobs = $args[1];
Write-Host "Looking for jobs that match" $JobMatch;
Get-VBRJob | where {$_.Name -like $JobMatch} | foreach {$JobList += $_.name};
Write-Host "Found" $joblist.count "jobs";
If ($joblist -eq $null -or $joblist.count -eq 0) {
Write-Host "No Jobs match that string... exiting";
Exit;
}
$joblist = $joblist | Sort-Object
Write-Host "Jobs to run...";
ForEach ($job in $JobList) {Write-Host " " $job;}
ForEach ($job in $JobList) {
Do {
$runningjobs=@();
Get-VBRJob | where {$_.Control -eq "Start"} | foreach {$RunningJobs += $_.name};
If ($runningjobs.count -ge $Maxjobs) {
Write-Host "Waiting to start '" $job "' as " $runningjobs "are still running";
Start-Sleep -s 60;
}
}
While ($runningjobs.count -ge $Maxjobs)
Write-Host "Starting " $job;
Start-Job -scriptblock {add-pssnapin veeampssnapin; Start-VBRJob $args[0]} -ArgumentList $job;
Write-Host "Waiting for veeam to start the job";
Start-Sleep -s 60;
}
#############################
Re: Job Chaining / Maximum concurrent jobs - Fix!
Hi Richard,
Thanks you very much for sharing it with the community!
Thanks you very much for sharing it with the community!
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Oct 27, 2010 1:51 pm
- Full Name: Rasmus Toft
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
Anyone that have got this script running on Veeam version 5?
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Mar 01, 2010 2:53 pm
- Full Name: Chris Butler
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
I used this script under version 4.1 and loved it. After v5 I have been updating the script to work with the new cmdlets. Everything works except for checking on the concurrently running jobs. I was playing with this but I need to figure out all the possible states of a job and haven't had time to do this yet;
$runningjobs = Get-VBRJob | where {$_.FindLastSession().State -eq "Working"}
Does anyone know the best way to check for number of active jobs?
Thanks,
Chris
$runningjobs = Get-VBRJob | where {$_.FindLastSession().State -eq "Working"}
Does anyone know the best way to check for number of active jobs?
Thanks,
Chris
-
- Novice
- Posts: 8
- Liked: 1 time
- Joined: May 13, 2010 7:01 am
- Full Name: Richard Ramshaw
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
They helpfully changed several of the structures and methods in the PowerShell commandlets in v5.0 without telling anyone and without changing the names of the constructs
However here is a new version of the script, that works on v5.0 *** IT ONLY WORKS on v5.0 ***
It is very slightly different on the command line as now it also support logging so you will need a command line like this...
The script is here, copy and paste to notepad, save as replicate.ps1
However here is a new version of the script, that works on v5.0 *** IT ONLY WORKS on v5.0 ***
It is very slightly different on the command line as now it also support logging so you will need a command line like this...
Code: Select all
powershell -command "& 'c:\program files\Veeam\Backup and Replication\Replicate.ps1' '*daily*' 2 DailyBackups.Log"
Code: Select all
$Global:logfile = $args[2];
If ($logfile -ne $null) {Out-File $logfile}
Function WriteLog ([string]$Entry) {
Write-Host $Entry;
If ($logfile -ne $null) {Out-File $logfile -append -inputobject $Entry;}
}
If ($args.Count -le 1 -or $args[0] -eq "" -or $args[1] -eq 0) {
WriteLog "Incorrect arguments, syntax: runjobs JobContains MaxJobs LogFile";
WriteLog "eg: runjobs *Daily* 2 LogFile.txt";
Exit;
}
add-pssnapin veeampssnapin;
$JobList=@();
$runningjobs=@();
$JobMatch = $args[0];
$Maxjobs = $args[1];
WriteLog ("Looking for jobs that match " + $JobMatch);
Get-VBRJob | where {$_.Name -like $JobMatch} | foreach {$JobList += $_.name};
WriteLog ("Found " + $joblist.count + " jobs");
If ($joblist -eq $null -or $joblist.count -eq 0) {
WriteLog "No Jobs match that string... exiting";
Exit;
}
$joblist = $joblist | Sort-Object
WriteLog "Jobs to run...";
ForEach ($job in $JobList) {WriteLog (" " + $job);}
ForEach ($job in $JobList) {
Do {
$runningjobs=@();
Get-VBRBackupSession | where {$_.State -eq "Working"} | foreach {$RunningJobs += $_.name};
If ($runningjobs.count -ge $Maxjobs) {
WriteLog ("Waiting to start '" + $job + "' as " + $runningjobs + " are still running");
Start-Sleep -s 60;
}
}
While ($runningjobs.count -ge $Maxjobs)
WriteLog ("Starting " + $job);
Get-VBRJob | where {$_.Name -eq $job} | foreach {start-vbrjob $_ -RunAsync};
WriteLog ("Waiting for veeam to start the job");
Start-Sleep -s 60;
}
-
- Chief Product Officer
- Posts: 31793
- Liked: 7295 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
Actually, this change is documented in the Release Notes. It would be pretty hard to stick to existing cmdlets structures after adding 4x more functionality to the product than previous version had...RichardR wrote:They helpfully changed several of the structures and methods in the PowerShell commandlets in v5.0 without telling anyone
Thank you very much for posting an updated version of your script.
-
- Novice
- Posts: 8
- Liked: 1 time
- Joined: May 13, 2010 7:01 am
- Full Name: Richard Ramshaw
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
Hi Gostev, Sorry didn't read the release notes
But it is normal programming behaviour to never change existing constructs in an externally accessible API, if there is a new way of doing something then it should be given a different name such as Get-VBRJobs_EX for extended. If you change a construct to give completely different information back (as in the case of Get-VBRJobs) then all hell could break loose in other peoples extensions, in this case it just caused more concurrent jobs to run than it should, but what if I had written a cleanup script based on the same function any thing could have hapened!
I understand that you want to add new functions to the API, and thanks for them, but as it is an externally published API then Veeam should follow correct coding standards when changing it.
Richard
But it is normal programming behaviour to never change existing constructs in an externally accessible API, if there is a new way of doing something then it should be given a different name such as Get-VBRJobs_EX for extended. If you change a construct to give completely different information back (as in the case of Get-VBRJobs) then all hell could break loose in other peoples extensions, in this case it just caused more concurrent jobs to run than it should, but what if I had written a cleanup script based on the same function any thing could have hapened!
I understand that you want to add new functions to the API, and thanks for them, but as it is an externally published API then Veeam should follow correct coding standards when changing it.
Richard
-
- Chief Product Officer
- Posts: 31793
- Liked: 7295 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
I agree, in ideal world... in reality, because of resource/timelines considerations (just way too many other features to develop and test), our options were either not to provide new capabilities via PowerShell at all, or switch to new cmdlets without keeping compatibility. Testing and maintaining to set of cmdlets was not an option (we would have to re-write and re-test all old cmdlets to work with new API model as well). So, we chose the latter - switch to new cmdlets without keeping compatibility.
-
- Novice
- Posts: 4
- Liked: never
- Joined: Jul 20, 2009 1:32 am
- Full Name: Marc
- Contact:
Re: Job Chaining / Maximum concurrent jobs - Fix!
I have tried implementing the updated script on our v5.0 install, however all the jobs run at once. Testing "Get-VBRBackupSession" gives the following output:
PS C:\> Get-VBRBackupSession
Get-VBRBackupSession : Specified cast is not valid.
At line:1 char:21
+ Get-VBRBackupSession <<<<
+ CategoryInfo : InvalidOperation: (Veeam.Backup.Po...BRBackupSession:GetVBRBackupSession) [Get-VBRBackup
Session], InvalidCastException
+ FullyQualifiedErrorId : Backup,Veeam.Backup.PowerShell.Command.GetVBRBackupSession
What have I done wrong?
PS C:\> Get-VBRBackupSession
Get-VBRBackupSession : Specified cast is not valid.
At line:1 char:21
+ Get-VBRBackupSession <<<<
+ CategoryInfo : InvalidOperation: (Veeam.Backup.Po...BRBackupSession:GetVBRBackupSession) [Get-VBRBackup
Session], InvalidCastException
+ FullyQualifiedErrorId : Backup,Veeam.Backup.PowerShell.Command.GetVBRBackupSession
What have I done wrong?
-
- Novice
- Posts: 4
- Liked: 1 time
- Joined: Jan 27, 2011 7:14 pm
- Full Name: Clint Beilman
Re: Job Chaining / Maximum concurrent jobs - Fix!
I've added this code to retry any jobs that have failed up to $retryJobsCount times per failed job.
Code: Select all
# Retry Failed Jobs
$retryJobsCount = 3
$retryCount = 0
$failedJobs = @{}
[Boolean] $retrydone = $false
Do
{
Get-VBRJob | Where {$_.Name -like $JobMatch -and $_.Info.LatestStatus -eq "Failed"} | % {
if ($failedJobs.ContainsKey($_.Name)) {
if ($failedJobs.get_Item($_.Name) -le $retryJobsCount)
{
Start-VBRJob $_ -RetryBackup -RunAsync
$failedJobs.set_Item($_.Name,$failedJobs.get_Item($_.Name) + 1)
Start-Sleep -s 60
}
} else {
$failedJobs.Add($_.Name,1)
WriteLog ("Retrying " + $_.Name)
Start-VBRJob $_ -RetryBackup -RunAsync
Start-Sleep -s 60
}
}
$lowestFailedCount = 0
$failedJobs.Values | % {if ($_ -gt $lowestFailedCount) {$lowestFailedCount = $_}}
if ($lowestFailedCount -ge $retryJobsCount) {$retrydone = $true}
}
While (Get-VBRBackupSession | Where {$_.State -eq "Working"}) -ne $null -and `
(Get-VBRJob | Where {$_.Name -like $JobMatch -and `
$_.Info.LatestStatus -eq "Failed"}).Count -ne $null -and -not $retryDone
Re: Job Chaining / Maximum concurrent jobs - Fix!
Thanks for posting, Clint. Good script for KB.
-
- Novice
- Posts: 4
- Liked: 1 time
- Joined: Jan 27, 2011 7:14 pm
- Full Name: Clint Beilman
Re: Job Chaining / Maximum concurrent jobs - Fix!
Whoops, I one mistake in that script. Change the "*Daily" in the While statement at the bottom to $JobMatch.
Code: Select all
# Retry Failed Jobs
$retryJobsCount = 3
$retryCount = 0
$failedJobs = @{}
[Boolean] $retrydone = $false
Do
{
Get-VBRJob | Where {$_.Name -like $JobMatch -and $_.Info.LatestStatus -eq "Failed"} | % {
if ($failedJobs.ContainsKey($_.Name)) {
if ($failedJobs.get_Item($_.Name) -le $retryJobsCount)
{
Start-VBRJob $_ -RetryBackup -RunAsync
$failedJobs.set_Item($_.Name,$failedJobs.get_Item($_.Name) + 1)
Start-Sleep -s 60
}
} else {
$failedJobs.Add($_.Name,1)
WriteLog ("Retrying " + $_.Name)
Start-VBRJob $_ -RetryBackup -RunAsync
Start-Sleep -s 60
}
}
$lowestFailedCount = 0
$failedJobs.Values | % {if ($_ -gt $lowestFailedCount) {$lowestFailedCount = $_}}
if ($lowestFailedCount -ge $retryJobsCount) {$retrydone = $true}
}
While (Get-VBRBackupSession | Where {$_.State -eq "Working"}) -ne $null -and `
(Get-VBRJob | Where {$_.Name -like $JobMatch -and `
$_.Info.LatestStatus -eq "Failed"}).Count -ne $null -and -not $retryDone
-
- Service Provider
- Posts: 103
- Liked: 9 times
- Joined: Sep 12, 2011 11:49 am
- Full Name: jmc
- Location: Duisburg - Germany
- Contact:
Backup 50 VM's at the same time! - Max Tasks
[merged]
hello all,
i have the problem, that i have up to 50 vm's to backup separatly. i have no idea how long each vm need and sometimes it differ so much. noe i have set all backup slots to the same start time and i hope that veeam has a max task parameter and not all 50 backups will starts together.
i'm right about that or doesn't veeam has this limiter and i got a lot of trouble this night.
ca i set this limit somewhere?
thx and greetings from germany
ps:
it is not possible to group this vm's together in one job ( business reasons )
hello all,
i have the problem, that i have up to 50 vm's to backup separatly. i have no idea how long each vm need and sometimes it differ so much. noe i have set all backup slots to the same start time and i hope that veeam has a max task parameter and not all 50 backups will starts together.
i'm right about that or doesn't veeam has this limiter and i got a lot of trouble this night.
ca i set this limit somewhere?
thx and greetings from germany
ps:
it is not possible to group this vm's together in one job ( business reasons )
Who is online
Users browsing this forum: Bing [Bot], robnicholsonmalt and 70 guests