Different schedule for incr vs full disk backups

[cffit]

I couldn't agree more with ArkHosp. It boggles my mind that this type of scheduling hasn't been implemented long ago. It's a huge pain for us and seems to be a fairly simple feature to implement. Please, please address this VEEAM

[ArkHosp]

Thanks alanbolte! That's exactly what I needed. It's a little clumsy, but it should allow me to do what I need. After reading about that feature, the one thing to remember will be that this is merely when it's allowed to START, not an availability window for it to run (that's actually buried in the same submenu). That's an important distinction, since if I were to configure it like a window, the darn job would kick off every hour during it (which I guess there might be actual cases where that would be useful too).

For now, I'll set the first job using this hourly/schedule blocking method for the various start times and then just chain the other jobs off that first one. As long as I don't have a failure in the chain, that should work nicely.

As I use this product, I keep finding missing features, but then someone shows me where they are hidden somewhere in the GUI. I'm really glad that these features are in there, but wow, there's a real learning curve. Maybe the management console will become a little easier as the product matures. I'm really happy with it so far.

Guess the next thing I'll be tackling will be trying to figure out how to test the backups. Got about another week of configuring/testing before I get to that point.

Thanks for those who responded to my questions! Seems like a really great forum community.

[foggy]

For now, I'll set the first job using this hourly/schedule blocking method for the various start times and then just chain the other jobs off that first one. As long as I don't have a failure in the chain, that should work nicely.

Just keep in mind that the chained jobs do not respect the schedule specified for their parent, so should have their own backup windows also specified (if required).

Also, we typically do not recommend chaining jobs at all, due to some undesired consequences it incurs.

[cffit]

I was crossing my fingers that maybe this was coming in v9, but I've found out it's not. At least not in the initial release.

I do realize there are workarounds, but at the same time, the reason I chose VEEAM for our backup product was because it simplified things. The minute scripting is necessary and then task scheduling through the OS vs doing it all within the backup application, it becomes less simple. If I was a backup admin and backups were all I did, this might be ok, but I do everything where I'm at and backups are a sliver of my time each day.

For the first few years we used VEEAM, I had to use Backup Exec for getting our VEEAM backups to tape. I requested the tape feature 100x via our sales rep, the forums and tech support. Everyone told me for those years that there was no demand and tape is dead, as though we were the only place on earth with a tape drive. Then a few versions later, tape was eventually added and I think it blew VEEAM's mind how much tape still has a strong presence in the backup world. I kind of feel like this is the same thing. I know the perception on VEEAM's end is that there isn't much demand for more flexible scheduling, but my own thought is that for every one person taking the time to post about this in the forum, there are hundreds if not thousands that would like to use it, but just accept that it's not there and deal with it. I don't see this as VEEAM being lazy or stubborn at all, but I have to believe they aren't aware of how many people would benefit from this and how anyone coming from Backup Exec or any other previous backup application is confused why this feature isn't part of VEEAM.

Now knowing this feature won't be in v9, I'm giving in and forcing myself to try a workaround. If someone could suggest a detailed way to accomplish the following (which is very similar to others), I would be grateful. I'd appreciate if the response is to use powershell, a detailed response with what the exact script would be and other details about how it works would be. I assume one has to install the VEEAM snap-in for PowerShell? Or is that installed with the product? Would the scheduled task need to be run as the VEEAM service account I assume? There are more details to setting this up than just saying use PowerShell. I wish I was better at scripting, but it's a weak area of mine.

I have 10 jobs. We'll call them JOB1, JOB2, etc.
All jobs will have the same schedule below:
Monday through Friday - Incremental - JOB1 starts at 5PM and the rest are staggered to start at times after 5PM
Saturday - Mostly Synthetic Fulls, Periodic Active Fulls - JOB1 starts at 1AM and the rest are staggered to start at times after 1AM

As I understand, I would need to leave Saturday checked on each job in the VEEAM GUI. This question was asked earlier, but I'm a little confused. When JOB1 starts via PowerShell on Saturday at 1AM, will it also run again at 5PM as scheduled in the VEEAM GUI?

I honestly do appreciate how the VEEAM moderators get to every post in these forums. I always know I can come on here and get help which is something that isn't common with software vendors. Thank you guys for all your hard work and attention you give to these posts.

[Sicey26]

Come on Veeam, why is this still being talked about ? If your customers are actually taking time to tell you a feature is wanted (and have been for a couple of years now) there must be a strong case for it.

Believe me, I love the product but this request and lack of action is one that does get my back up.

[Gostev]

Come on Veeam, why is this still being talked about?

Because this feature has lower priority than other pending features. For example, we have put a lot of work into v9 to remove the very need for doing those periodic full backups to start with

Not having to do Active Fulls at all >>> Ability to schedule one separately

[cffit]

Gostev, can you elaborate more on how this will work with v9 for those who have a need to use a different schedule on the weekends for their backups? Thank you.

[Gostev]

What I am saying is that with v9, you can simply use forever incremental backup, and don't worry about doing periodic full backups at all (and so scheduling them separately). What is your use case for doing periodic full backups?

[cffit]

In the past, our synthetic backups would take longer and longer over time. I was told by support that to fix this I should do periodic active fulls as doing forever synthetic fulls can cause this issue.

Even with forever incrementals, we'd still need to do weekend synthetic fulls. While the incremental backups to go towards this synthetic full go fast, the creation of the synthetic fulls from the incremental backups take a long time and when they run on our single schedule (limited by the VEEAM GUI) we end up with 5 synthetic fulls being run at one time and it completely overwhelms the I/O of our backup SAN. I wish we had an expensive high end SAN for our backups, but we have a simple HP MSA SAN with 10K drives. If I could do my backups on the scheduling I'd like to do, our SAN would keep up just fine.

My need to create weekly synthetic fulls is because we archive our weekly backups to tape for long term retention.

I think my goal, and the goal of others here, is this (or a close variant). We do incremental backups Monday through Friday late evening or early morning. They go fast so we can schedule them to run closely together, say from 6PM to midnight or 2AM to 6AM. Something like that. We do full backups on Saturday (synthetic usually and active once a month). We want to be able to space out our full backups so that we can start early morning on Saturday or Sunday and space our weekend full backup schedules apart so they can run ALL DAY on a weekend day vs all trying to run between our much shorter weekday schedule. I can easily do my weekend full backups if I start them at 2AM on Saturday and schedule them throughout Saturday morning, afternoon and evening vs them all trying to run in the evening only.

I really do understand that there are so many feature requests and such. I know I don't understand the work or complexity of how the product is developed. So please understand that I say this with that understanding. It "seems" like incorporating the ability to create a different backup schedule for weekend jobs would be a very small feature and take minimal effort to help out a lot of people. I understand that there are other features that you maybe have more requests for and from larger customers, but things like integrating your product with EMC and other vendors has to be a huge effort and cost vs a very small modification to your own product in scheduling. I think people's frustrations in this thread aren't only that multiple schedules aren't available (and are from previous backup applications they've used), but also because it seems like such a simple and low cost, low effort feature to incorporate. Again, I certainly don't claim to know this is the case, but it's just a perception from a customer point of view.

I love this forum and I use it extensively because it really helps me. I'm sure though that only a tiny percentage of VEEAM customers actually make the effort to be involved in these forums. I can assure VEEAM that this feature request is so much larger than the handful on people in this thread. I guarantee that most people coming from a different backup software to VEEAM wonder how this feature isn't there. To be honest, I don't even consider it something that should be a "feature". It should be the most basic part of the product. I have used an enormous number of backup application in my 15 years of IT and this is the only backup application that I've not been able to set different schedules in.

[Gostev]

My need to create weekly synthetic fulls is because we archive our weekly backups to tape for long term retention.

In that case, you don't need synthetic fulls at all - as starting with v8, Backup to Tape job is able to synthesize full backup file on-the-fly from a forever incremental chain... it does not have to physically exist on disk.

[cffit]

I need the fulls on tape and disk per our data retention policy. I appreciate the info and suggestion, but I hope what I wrote above makes sense and explains what others are hoping to accomplish too.

[Gostev]

Can you please explain why your data retention policy requires to have fulls on the disk? Just for my understanding, as you may not need those either with v9.

[foggy]

While the incremental backups to go towards this synthetic full go fast, the creation of the synthetic fulls from the incremental backups take a long time and when they run on our single schedule (limited by the VEEAM GUI) we end up with 5 synthetic fulls being run at one time and it completely overwhelms the I/O of our backup SAN.

Haven't you considered limiting the number of concurrent tasks that can be assigned to repository?

[cffit]

FOR PEOPLE ON HERE WANTING TO SETUP DIFFERENT SCHEDULES FOR WEEKENDS VS WEEKDAYS, HERE'S HOW I DID IT WITHOUT TOO MUCH PAIN:

It takes some time to setup, but after that it should work ongoing with minimal effort. However, I'm still frustrated this isn't in the GUI as things that run as scripts and scheduled tasks tend to get forgotten about when changes are made.

VEEAM B&R installs the VEEAM PowerShell snap-in so you don't have to do anything to get that. You can access it by going into the B&R software and clicking on the blue box with the dropdown arrow in the top left and choosing PowerShell.

I used the service account for VEEAM B&R to create the scheduled tasks and run the PowerShell scripts. For me, this account is a local admin on the server. If yours isn't, you might have to adjust as necessary.

I created two text files named something like "WeekdayScheduleChange.txt" and "WeekendScheduleChange.txt". I then edited them to read like this..

The WeekdayScheduleChange.txt file looks like this:

Code: Select all

Add-PSSnapin VeeamPSSnapin
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job01" -At "17:00" -Daily -DailyKind Weekdays
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job02" -At "17:15" -Daily -DailyKind Weekdays
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job03" -At "17:30" -Daily -DailyKind Weekdays
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job04" -At "18:00" -Daily -DailyKind Weekdays

The WekendScheduleChange.txt file looks like this:

Code: Select all

Add-PSSnapin VeeamPSSnapin
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job01" -At "4:00" -Daily -DailyKind SelectedDays -Days Saturday
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job02" -At "6:30" -Daily -DailyKind SelectedDays -Days Saturday
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job03" -At "10:00" -Daily -DailyKind SelectedDays -Days Saturday
Set-VBRJobSchedule -Job "Exact_Name_of_Backup_Job04" -At "14:00" -Daily -DailyKind SelectedDays -Days Saturday

Obviously change the name of the backup jobs to what yours are named, exactly. Rename both files from .txt to .ps1

Setup two scheduled tasks, one to run each one of these files. I run the WeekendScheduleChange.ps1 script at 2AM Saturday morning and the WeekdayScheduleChange.ps1 script at noon on Monday.

In order to setup scheduled tasks to run PowerShell scripts there are a few ways to do this. I set the task to run as the VEEAM service account (which is a local admin). I set it to run whether logged in or not and to use highest privileges and also configure for the version of Windows I'm using. The trigger is the schedule which I mention a few lines above. For the Action, I find it's easiest to create a new action of "Start a program" and then for Program/script I enter "powershell" (without quotes) and then under "Add arguments" I enter "-file C:\PathToScript\WeekendScheduleChange.ps1" (again without the quotes). If your path has spaces in it you will need to put quotes around the path only like -file "C:\PathToScript\WeekendScheduleChange.ps1". That's all I set. Enter the password for the account you are using for the task and you are good. I then tested by right clicking the task and running it.

Doing this will in essence just change the job schedule within the job. All the other parts of the job will remain the same such as whatever day you set for the full backup and methods. It only changes the checkmarks/time on the scheduling section. This way you can still have all your options set in the GUI and ONLY change when the job runs. I didn't want to have to specify all the job options via PowerShell as I'm not super great with PowerShell and the VEEAM syntax added in makes it even more of a challenge for me.

Hope this makes sense and helps some people out. When I realized v9 wasn't going to resolve this for me, I decided to put the time into figuring out how to do it with PowerShell.

Thanks to Tom and Vladimir from VEEAM for helping me with the syntax of PowerShell. Good luck everyone!

[kte]

if the sheduler in gui could allow this it would be alot better, select on the sheduler if it is a incremental or an active full or a synthetic full,....

[PHBG]

One more vote for this feature.
I think that nobody who is aware of Change Block Tracking issues will relies on forever incremental backup. Veeam recommend 3-2-1 rules, but if the source is corrupted (CBL) the fact that you have 3 copy on 2 different media on two sites will not warm you. Yes you have SureBackup, but it will not guarantee 100% that everything is ok, especially when you have corrupted data, which is not stopping starting OS and application. So for me best practice is - time to time to have active full backup. And obviously it will take much more time than incremental backup and it is not optimal to have same scheduled as incremental.

[cffit]

So I got everything setup using PowerShell and scheduled tasks and now I have a different schedule for my weekend jobs vs my weekday jobs. It works how I need it to. I posted for others to see and hopefully help some people out, and it seems like it did beings it got the post of the week. I'm glad others found this helpful and it feels good to share my experience and help others in the same fashion that I've found help on here from others.

So I thought I was golden after doing this as everything looked to run like I wanted. However, now I've run into another issue. I want my backup copies to run right after my backup jobs finish. The solution for that before was to set them to run 5 minutes after the backup job, so they would wait and run once the backup job finishes. That's great, but now with me setting up my weekend jobs to run at a different time via PowerShell and scheduled tasks, these backup copy jobs stay at the same weekday times and then they don't run when I want them to. So now my options are:

1. Also manually script the backup copy jobs to run at different times (anyone from VEEAM can you send me the syntax for this?)
2. Wish that there was an option to run the backup copy jobs following the completion of the regular backup job (you can only set times, not say "after the backup job completes")
3. Wish that VEEAM had the ability to schedule weekend jobs to run at different times than weekday (which is what this post is all about and the root of all these problems)

The reason I post this and gripe is that if #3 was a feature in the product, everything would be fine. But due to the lack of having different scheduling options in the GUI is seeming to cause me a chain of problems that continues beyond the original issue.

[gosnells]

+1 FeatureRequest
(active) full backups take longer. It would be nice to have flexible scheduling within an individual job to cater for this.

[mma]

another +1 feature request

[mkaec]

I too enjoyed the rich scheduling features in the backup application we migrated from. Too bad, it did so many other things so poorly.

[ekisner]

So, I've run into a bit of a snag.

We have jobs with different RPOs, like most organizations I'm assuming. The active full job is intended to run after business hours at the end of the week - so really, any time after 6pm on Friday.

Some jobs are set to run every 4 hours, others every 12, and others daily at 6:xx (I stagger them to assist in avoiding contention issues). Endpoint backups of physical servers happen at midnight.

Where I run into an issue is in the ones which run every x hours. The active full must be scheduled for Saturday, in which case it starts at midnight on Saturday. I'd much rather it start on Friday evening and not spend 6 hours doing nothing during what is an absolutely prime backup window.

As a feature request, I'm asking to add a time (or a time inclusion/exclusion window such as in the hourly schedule settings) to the active full selection. Or add a time inclusion/exclusion window to the schedule selection for the active full.

[ekisner]

Sorry, I forgot to mention one other complication.

The tape job also starts at midnight on Saturday. Which makes me concerned that it could back up the wrong week.

[kkuszek]

I think this could be of benefit,
You can schedule when the job can run, but you can't really schedule alternate times for active fulls, only acceptable times. I know if I could actually schedule active fulls with the same flexibility of the normal incrementals I would do so under alternate scheduling. I would probably kick off my fulls saturday afternoon where it does not make sense to schedule my nightly's during the afternoon.

[Shestakov]

Hello Erik!
Thank you for the feedback, we`ve taken it into account.
Please review this post for a workaround.

[squirrel]

I would also like to also request the feature for flexible scheduling in the GUI for different times for the full and synthetic backups than the incremental backups.

[omerlin98000]

Hello all.
Thanks for the idea of scheduling differently week and week-end jobs!
I have a little question about that:
I have schedule a job to do a full backup on Saturday 19pm, followed (scheduled after - I know, it's the devil) by another which has also a full on Saturday.
The first job is now much longer in time (growing VMs), so the second starts in early Sunday... thus no more full, only incréments (it's no more Saturday).
If I let the first scheduled job at 19pm, and put let's say the second at 22pm, I see a danger: the second job starts on Saturday 22pm (so it should know it's a full backup job)... but due to the limitation about proxy/repository, it will be held until resources have been freed, let's say Sunday at 3am: does it remember that it starts on Saturday (then have full to do) or only see it releases on Sunday 3am, that it's a Sunday and do again a simple incremental?

[foggy]

If the job is actually started on Saturday, even though it is waiting for available resources, it will make a full once it gets them.

[omerlin98000]

Splendid, so it works like I wanted; thanks foggy

[nfma]

Hello there,

we have a customer with a specific ruleset for backups. It boils down to this:
-daily backups of the most important VMs (Monday through Sunday)
-weekly backup of less important VMs (usually Saturday)
-active full backups on Saturday of ALL VMs
-last full backup chain goes to tape - each day (after the daily jobs are done)

Each VM has its own backup job.
The daily tape job has to finish before work hours the next day. To achieve this, I had to set the daily jobs to start after each other to reduce the whole backup window.
So instead of this:
-mailserver1 - 6pm
-mailserver2 - 7pm
-dc1 - 8pm
-dc2 - 9pm
...

...I chained the backup jobs like this:
-mailserver1 - 6pm
-mailserver2 - after mailserver1
-dc1 - after mailserver2
-dc2 - after dc1

This reduces the backup time considerably so that the tape job has more than enough time to do its job.

Saturday is Full Backup Day, so every backup job has the Active Full Backup flag set to Saturday.

Here's the problem:
An Active Full makes a job finish considerably later, which means that for example mailserver1 starts at 6pm on Saturday and finishes around ~8pm (instead of ~6.15pm), mailserver2 now starts at ~8pm and finishes around 10pm, dc1 starts around 10pm and finishes around midnight - and dc2 now starts on Sunday and won't actually do an Active Full because its AF flag is set for Saturday. It'll do an incremental and will completely leave out a restore point for Saturday.

My question now is: Is there any way to tell a backup job to start an Active Full on a different time than its usual schedule? I'd prefer to start the Active Full for Saturday at around 9am, which gives all jobs plenty of time to finish their AF on Saturday.

Or is there an alternative that I've overlooked? Like, creating a dedicated AF backup job for all VMs on Saturday at 9am and leaving the daily jobs as they are? But then how would the daily incrementals "know" about the full backup on Saturday?

Timing out the jobs that would start on Sunday and then setting Sunday as AF day for these jobs seems a bit finicky to me, since I might end up with two full backups for these jobs for Sunday...

Thanks for your ideas in advance!

[foggy]

You can review this thread for some hints regarding your matter. Btw, why are you trying to run only a single job at a time? A typical recommendation is to start all the jobs simultaneously (or with a short intervals) and limit the maximum number of proxy/repository concurrent tasks, if the resources load is your concern. This will allow you to reduce the backup time even more.