Comprehensive data protection for all workloads
veremin
Product Manager
Posts: 17903
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by veremin »

Are there sample scripts that show how to do all this in powershell? I.E. check status of jobs and then running them when predefind other jobs are finished?
In fact, the commands in PowerShell are processed one by one. So, if you just specify three commands responsible for starting corresponding jobs, everything will be ok. In other words, not until the first job is completed, will the PowerShell proceed to the second one, etc.:

Code: Select all

Asnp VeeamPSSnapin
Get-VBRJob -name "Backup Job 1" | Start-VBRJob 
Get-VBRJob -name "Backup Job 2" | Start-VBRJob
Get-VBRJob -name "Backup Job 3" | Start-VBRJob
If you're still willing to implement auditorial part, then something like this should meet your expectations:

Code: Select all

Add-PSSnapin VeeamPSSnapin
$StartInfo = new-object System.Diagnostics.ProcessStartInfo
$StartInfo.FileName = "$pshome\powershell.exe"
$StartInfo.Arguments = ""
$StartInfo.WorkingDirectory = "C:\"
$StartInfo.LoadUserProfile = $true
$StartInfo.UseShellExecute = $true
$StartInfo.CreateNowindow = $False
$firstjob = Get-VBRJob -name "Backup Job 1"
$secondjob = Get-VBRJob -name "Backup Job 2"
$thirdjob = Get-VBRJob -name "Backup Job 3"
$StartInfo.Arguments = "-Command  asnp VeeamPSSnapin; Get-VBRJob -name 'Backup Job 1' | Start-VBRJob"
[System.Diagnostics.Process]::Start($StartInfo)
Start-sleep -s 60
$status = $firstjob.GetLastState()
do 
{
Start-sleep -s 30
$status = $firstjob.GetLastState()
}while ($status -eq "Working")
$StartInfo.Arguments = "-Command  asnp VeeamPSSnapin; Get-VBRJob -name 'Backup Job 2' | Start-VBRJob"
[System.Diagnostics.Process]::Start($StartInfo)
Start-sleep -s 60
$status = $secondjob.GetLastState()
do 
{
Start-sleep -s 30
$status = $secondjob.GetLastState()
}while ($status -eq "Working")
$StartInfo.Arguments = "-Command  asnp VeeamPSSnapin; Get-VBRJob -name 'Backup Job 3' | Start-VBRJob"
[System.Diagnostics.Process]::Start($StartInfo)


Hope this helps.
Thanks.

habibalby
Expert
Posts: 379
Liked: 24 times
Joined: Jul 18, 2011 9:30 am
Full Name: Hussain Al Sayed
Location: Bahrain
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by habibalby »

Veeam should introduce a schedule with different time/day feature for the full back up, this would be changed the input method of the Weekly Backups as CeckBox rather than Radio .

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by cffit »

Ok, got it. Thanks guys. It would be nice if this functionality was available in the GUI though as it is a common task for an administrator. Perhaps it is in v7.

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by cffit »

v.Eremin wrote: In fact, the commands in PowerShell are processed one by one. So, if you just specify three commands responsible for starting corresponding jobs, everything will be ok. In other words, not until the first job is completed, will the PowerShell proceed to the second one, etc.:

Code: Select all

Asnp VeeamPSSnapin
Get-VBRJob -name "Backup Job 1" | Start-VBRJob 
Get-VBRJob -name "Backup Job 2" | Start-VBRJob
Get-VBRJob -name "Backup Job 3" | Start-VBRJob

So if in the GUI I have the jobs setup to run in order, (i.e. when Job1 finishes, Job2 starts, when Job2 finishes, Job3 starts, etc), and in powershell I run the command to start Job1, will I need to run Job2, Job3, etc via powershell? Or will the rest of the jobs just follow Job1 once I kick Job1 off with powershell?

tsightler
VP, Product Management
Posts: 5689
Liked: 2509 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by tsightler »

Perhaps I'm over-simplifying the issue here, but it sounds like you are possibly chaining jobs, which is causing jobs to run over. If you want to run jobs in sequence, but not worry about problems like this, simply set the proxy or repository to run only one job at a time, schedule all of your jobs to start 5 minutes apart. All jobs will start, and queue up, running one after the other, but since their start time was all on the same day they will run the synthetic process.

habibalby
Expert
Posts: 379
Liked: 24 times
Joined: Jul 18, 2011 9:30 am
Full Name: Hussain Al Sayed
Location: Bahrain
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by habibalby »

Create scripts to run daily via Task Schedule in windows keeping in mind the time each job takes to complete incremental backup. Same with weekly, create scripts to run the same jobs but they should runs only on Friday or your weekends, keep also in mind the time it takes for each job as Full. I have jobs they finish full backups in 30 minutes and some jobs they finish in 3 - 4 hours. Schedule tasks configured accordingly.

you can write in the same script to loop through the $Result searching for $Success, if not Loop with ReTry till you get $Success.

habibalby
Expert
Posts: 379
Liked: 24 times
Joined: Jul 18, 2011 9:30 am
Full Name: Hussain Al Sayed
Location: Bahrain
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by habibalby »

This script will help:

Code: Select all

Add-PSSnapin VeeamPSSnapin


# Add the name of the backup jobs to be included here. The order in which they are entered is the order in which they will run

$chainedjobs = ("Critical VMs-1")

foreach ($jobname in $chainedjobs){
$job = Get-VBRJob -name $jobname
$jobtry = 0
start-VBRJob -job $job

$job.GetLastResult()
if($job.GetLastResult() -eq "Failed"){
    do{
        Start-Sleep 480
        Start-VBRJob -job $job -RetryBackup
        $jobtry++
    }
    while(($jobtry -lt 3) -and ($job.GetLastResult() -eq "Failed"))
}
}


ek

veremin
Product Manager
Posts: 17903
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by veremin »

Will I need to run Job2, Job3, etc .via powershell?
No, you don’t have to do it. The other jobs will be started in accordance to chain schedule.

My point was that if the jobs have different schedule and you’re willing to execute them in chain manner, all you need to do is to write three lines responsible for executing given job. Since, lines are processed sequentially, the second job will be executed after the first one is finished, etc.

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by cffit »

tsightler wrote:Perhaps I'm over-simplifying the issue here, but it sounds like you are possibly chaining jobs, which is causing jobs to run over. If you want to run jobs in sequence, but not worry about problems like this, simply set the proxy or repository to run only one job at a time, schedule all of your jobs to start 5 minutes apart. All jobs will start, and queue up, running one after the other, but since their start time was all on the same day they will run the synthetic process.

That would be ideal if it really works like this. I will give it a shot. Thanks!

tsightler
VP, Product Management
Posts: 5689
Liked: 2509 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by tsightler »

It really works does work like that, or at least it should. Please let me know if you try this and have problems. There were a few known issues with this in earlier 6.x releases, for example, the repository wouldn't take synthetic processing into account as a parallel task, but I'm pretty sure that was addressed in current versions. Whether or not to build a synthetic full or not depends primarily on the day the job started. If you have selected synthetic fulls to run on Sunday, then you need all of the jobs to actually start on Sunday, it doesn't really matter when they finish, but if you're using job chaining, where one job starts another job, some jobs will start on Sunday, and others on Monday, and obviously lead to the problem you're seeing.

This is why I'm not a fan of job chaining, and recommend against it as a best practice, it just causes too many corner cases from a scheduling perspective. How are failed jobs handled? What happens when I delete/add a job? What about synthetic fulls when chain jobs are started after midnight? The list goes on. If you don't want jobs running in parallel, I can understand that, but in general it's still better to limit the available resources (repository ingest rate, proxy and repository parallelism) and let the scheduler do what it's designed to do as it's pretty good at it.

I'm am curious as to your reason for wanting to run jobs sequentially in the first place though, if you wouldn't mind sharing that.

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by cffit »

I just have one Veeam server that does all the roles, so just one proxy. I'm not sure if running multiple jobs at one time would get things done faster or run slower and ultmately take the same amount of time as running them one at a time. I could experiment with that. I noticed the bottleneck on the jobs running is always the source which is nearly at 100%, and all our VMs are on the same SAN and cluster of ESXi hosts, so I figured with that being the case it wouldn't help to run more than one job at a time. Or do I have that wrong?

I will post back with my findings on trying your method tsightler. One other question on that method is do the VEEAM jobs fail if they are not able to start after X amount of time?

tsightler
VP, Product Management
Posts: 5689
Liked: 2509 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by tsightler » 1 person likes this post

Most environments, even small ones, will see some improvement with running 2 jobs just because there's less wasted time (all the time spent preparing a VM, and switching between disks and VMs when nothing else is going on), but I'm not necessarily suggesting that you have to run multiple jobs concurrently. Starting the jobs concurrently and having them run concurrently are different things. Even if you start 10 jobs at the same time it will only actively run the number of jobs that there are proxy/repository resources to handle. Even though you only have one Veeam server, you still technically have a proxy and repository, it just happens to be your Veeam server, and you can still set the concurrency on it. By default the proxy will be set to half the number of cores, but you can lower this to one and only one job will run concurrently. Also, in the repository properties you can set the number of concurrent jobs as well so this is another place you can set concurrency to one to limit it.

Regarding the timeout, yes they will fail eventually if the resources don't come available, but it's a pretty long timeout, and I don't think you'll be in any danger of running over that based on your other post.

cffit
Expert
Posts: 338
Liked: 34 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by cffit » 1 person likes this post

Just reporting back now. Setting the proxy to just allow one job at a time and then setting all my backups to run 10 minutes apart took care of my issue. On Sunday night I have jobs starting at 8PM, 8:10, 8:20, 8:30, 8:40 and 8:50. The later jobs don't start processing until after midnight, so technically on Monday, but since they were kicked off on Sunday they end up running as the synthetic fulls.

Thanks everyone for your suggestions and options. I'm sure the other options with using powershell would have worked too, but the suggestion from tsightler worked best for me.

I will say once again that it would be nice if VEEAM would have a more flexible scheduler built into the GUI.

foggy
Veeam Software
Posts: 19552
Liked: 1783 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by foggy »

Glad you've successfully addressed the issue. Thanks for the feedback!

habibalby
Expert
Posts: 379
Liked: 24 times
Joined: Jul 18, 2011 9:30 am
Full Name: Hussain Al Sayed
Location: Bahrain
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by habibalby »

Hope to see the different scheduler for full back up in version 7.

SLM
Lurker
Posts: 2
Liked: never
Joined: Nov 05, 2014 11:34 pm
Full Name: ICT Helpdesk
Contact:

[MERGED] Veeam B&R Synthetic Full Scheduling Feature

Post by SLM »

Is there a chance or future plans that a scheduling should be added for Synthetic Full backup? At the moment only can have different days but not the time for synthetic full is inherited from the backup job scheduling. This should be good to have function and works quite efficiently.

I'm sure I'm not the only one who needs this feature!

Gostev
SVP, Product Management
Posts: 26905
Liked: 4366 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by Gostev »

Yes, this feature is planned.

SLM
Lurker
Posts: 2
Liked: never
Joined: Nov 05, 2014 11:34 pm
Full Name: ICT Helpdesk
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by SLM »

Is there a work around for it though until the feature is released?

veremin
Product Manager
Posts: 17903
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by veremin »

Using the script provided in this thread in conjunction with Windows Task Scheduler, you can perform synthetic full on demand. Thanks.

newfirewallman
Enthusiast
Posts: 35
Liked: 2 times
Joined: Jan 20, 2015 12:08 pm
Full Name: Blake Forslund
Contact:

[MERGED] Feature Request - Schedule Backup Jobs and Times

Post by newfirewallman »

It would be nice to have the ability to set the schedule differently depending on the day to account for windows of time for other services or backup jobs.
Example:
Daily backup at 8pm except saturday would can start at 2pm (say for an active full backup with enough time to complete)

*Keep the backup jobs clean and organized, save IO and diskspace, while allowing for different backups windows for the same job.
Could easily layout the options S,M,T,W,T,F,S at the top with start time below it.

veremin
Product Manager
Posts: 17903
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Feature Request - Schedule Backup Jobs and Times

Post by veremin »

I think that it can be achieved with the use of post-job command. All you need is to specify there a script that checks the upcoming day of week and sets job execution time in accordance. Thanks.

newfirewallman
Enthusiast
Posts: 35
Liked: 2 times
Joined: Jan 20, 2015 12:08 pm
Full Name: Blake Forslund
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by newfirewallman »

I agree it probably could, but lets make it simple and easy for IT to manage in a nice visual manor.

veremin
Product Manager
Posts: 17903
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by veremin »

Ok, I was just trying to give your an idea of potential workaround, because not all of features requested by customers get implemented finally. We tend to prioritize them, based on amount of requests and sheer value it'll bring to our product. Thanks.

newfirewallman
Enthusiast
Posts: 35
Liked: 2 times
Joined: Jan 20, 2015 12:08 pm
Full Name: Blake Forslund
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by newfirewallman »

I would like to do it without having to script/powershell it.

veremin
Product Manager
Posts: 17903
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by veremin »

Your voice has been heard. But, please, don't expect this feature delivered in the nearest future due to the reasons provided above. Thanks.

habibalby
Expert
Posts: 379
Liked: 24 times
Joined: Jul 18, 2011 9:30 am
Full Name: Hussain Al Sayed
Location: Bahrain
Contact:

Re: Separate Scheduling for Daily Incremental and Weekly Ful

Post by habibalby » 1 person likes this post

habibalby wrote:Hope to see the different scheduler for full back up in version 7.
Two years from this post and still customers don't see feature where I think it requires only little time to invest and get it done by Veeam Developers / Integrate Veeam with Windows Task Schedulers and have some sort of Built-In Job Scripts to take care of this integration..

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 63 guests