Hi all,
I have put in a new hardened Linux repo and am consolidating my backups into two jobs to disk plus a tape job.
- Job1 starts 5:30pm M-F
- Job2 follows Job1
- Tapejob follows Job2
Both disk jobs run incrementals with synthetic fulls on Friday.
My concern is that the Friday job1 synthetic full may run past midnight, which would mean job2 would not run a synthetic full because it never ran on a Friday. What would happen if I set Job2 to run synthetic fulls on Fri & Sat? Would I end up with two synthetic fulls per week, or just one from whichever day the backup ran?
Any other suggestions? I've seen quite a few posts from many years ago suggesting not to chain backup jobs, but I want Job1 to run as quickly as possible, and finish before Job2 starts, and I don't want the tape job to run until both job1 and job2 have finished.
Edit: This is a small environment, with only 60VMs backed up across both jobs and about 25TB total. Incremental backups are taking about 2.5hr to backup to disk and tape, but I haven't run a full synth backup yet. First time will be tonight, and I'm not sure if the synth backup of Job1 will finish before midnight.
-
- Influencer
- Posts: 20
- Liked: 7 times
- Joined: Jul 27, 2022 5:31 am
- Contact:
-
- Veeam Software
- Posts: 2346
- Liked: 556 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Job scheduling and synthetic fulls
Hi bretth,
If Job 2 is able to produce a Synthetic Full (SF) on Friday while being scheduled for both Friday and Saturday, you will end up with 2 full backups (assuming the saturday one doesn't get interrupted)
I understand your position on job chaining and your goal here, but I would like to offer a few points that might help switch away from job chaining, as this situation is purely caused by that.
I would actually not worry about the Tape job and when it runs, as Veeam's Backup to Tape is designed to assume that the tape job needs to play nicely with the source jobs -- that is, the scheduler for Veeam will understand which resources each job wants and ensures that if there is a contestation between jobs trying to grab the same resource, the scheduler will handle it gracefully; tape job by default will surrender its backup file locks unless you enable the option Prevent this job from being interrupted by source backup jobs in the tape job; this option doesn't prevent the source job from running, it just allows the tape job to maintain its resource lock and the primary job would have its request queued and move on to other tasks while it waits.
In otherwords, the primary jobs and tape jobs overlapping are not an issue, Veeam can and will handle it; similarly, the tape job "knows" which backup files on the repository it hasn't written to tape yet and should write to tape, so if it gets missed on one tape backup session due to like file lock or longer running full creation, it will find that backup on the next tape job run.
Can you elaborate a bit more on why you're preferring to avoid overlap with Jobs 1 and 2? Again, the scheduler should handle this pretty well, but maybe it's about snapshots in production or another concern?
If Job 2 is able to produce a Synthetic Full (SF) on Friday while being scheduled for both Friday and Saturday, you will end up with 2 full backups (assuming the saturday one doesn't get interrupted)
I understand your position on job chaining and your goal here, but I would like to offer a few points that might help switch away from job chaining, as this situation is purely caused by that.
I would actually not worry about the Tape job and when it runs, as Veeam's Backup to Tape is designed to assume that the tape job needs to play nicely with the source jobs -- that is, the scheduler for Veeam will understand which resources each job wants and ensures that if there is a contestation between jobs trying to grab the same resource, the scheduler will handle it gracefully; tape job by default will surrender its backup file locks unless you enable the option Prevent this job from being interrupted by source backup jobs in the tape job; this option doesn't prevent the source job from running, it just allows the tape job to maintain its resource lock and the primary job would have its request queued and move on to other tasks while it waits.
In otherwords, the primary jobs and tape jobs overlapping are not an issue, Veeam can and will handle it; similarly, the tape job "knows" which backup files on the repository it hasn't written to tape yet and should write to tape, so if it gets missed on one tape backup session due to like file lock or longer running full creation, it will find that backup on the next tape job run.
Can you elaborate a bit more on why you're preferring to avoid overlap with Jobs 1 and 2? Again, the scheduler should handle this pretty well, but maybe it's about snapshots in production or another concern?
David Domask | Product Management: Principal Analyst
-
- Influencer
- Posts: 20
- Liked: 7 times
- Joined: Jul 27, 2022 5:31 am
- Contact:
Re: Job scheduling and synthetic fulls
Hi David, thanks for the reply.
I want Job1 to run as quickly as possible because the job needs a pre- and post-job script to pause replication for the VMs in it. I want to minimise the time the replication is paused, so don't want other jobs running at the same time.
The complication with the timing of the jobs is when to run the synthetic full - with Job1 running at 5:30pm and Job2 following it, if Job1 finishes before midnight then Job2 should run its synthetic full backup on the same day, otherwise it should run the synth full the day after Job1 does. Incrementals only take about an hour, but a synth full takes about 5-6hr, which is getting a bit close to midnight for my taste. With my luck the one that runs long would be a yearly tape, and it would just have incrementals on it.
In the absence of being able to say Job2 should run a synthetic full on its next run after Job1 does, it all became a bit of a pain so I've gone with a different option. I wanted to move the synth full from Friday night to Monday night so each tape has a full backup on Monday night & incrementals Tue-Fri, so this scheduling change covers these points and also gives me a 6pm-midnight server maintenance window:
Job1 runs at 00:05 Tue-Sat with synth full on Tue
Job2 follows Job1 with synth full on Tue
Tapejob follows Job2
On the Monday night Job1 will run about 5-6hr, job2 about 3-4hr, and tapejob about 10-11hr, so this should all finish a few hours before the Tue night (Wed 00:05) kicks off. The Tue-Fri night jobs should have finished backing up to both disk and tape by about 6am.
What are the reasons for avoiding using job chaining rather than scheduling the jobs independently?
I want Job1 to run as quickly as possible because the job needs a pre- and post-job script to pause replication for the VMs in it. I want to minimise the time the replication is paused, so don't want other jobs running at the same time.
The complication with the timing of the jobs is when to run the synthetic full - with Job1 running at 5:30pm and Job2 following it, if Job1 finishes before midnight then Job2 should run its synthetic full backup on the same day, otherwise it should run the synth full the day after Job1 does. Incrementals only take about an hour, but a synth full takes about 5-6hr, which is getting a bit close to midnight for my taste. With my luck the one that runs long would be a yearly tape, and it would just have incrementals on it.
In the absence of being able to say Job2 should run a synthetic full on its next run after Job1 does, it all became a bit of a pain so I've gone with a different option. I wanted to move the synth full from Friday night to Monday night so each tape has a full backup on Monday night & incrementals Tue-Fri, so this scheduling change covers these points and also gives me a 6pm-midnight server maintenance window:
Job1 runs at 00:05 Tue-Sat with synth full on Tue
Job2 follows Job1 with synth full on Tue
Tapejob follows Job2
On the Monday night Job1 will run about 5-6hr, job2 about 3-4hr, and tapejob about 10-11hr, so this should all finish a few hours before the Tue night (Wed 00:05) kicks off. The Tue-Fri night jobs should have finished backing up to both disk and tape by about 6am.
What are the reasons for avoiding using job chaining rather than scheduling the jobs independently?
Who is online
Users browsing this forum: Ahrefs [Bot], Bing [Bot], Semrush [Bot] and 73 guests