Full "Monthly" Tape job cannot complete in time

Everything about backing up to tape

Full "Monthly" Tape job cannot complete in time

Veeam Logoby BenoitCote » Tue May 02, 2017 1:25 pm

Hi,

since we moved to Veeam a couple of years back, we have seen some great performance and also gained granularity for our Exchange, SQL,etc.. restores, but sadly we lost on the tape front. Maybe it is the way we have always done it or it simply isn't Veeam's cup of tea, but I do need this to work properly. Here is my issue ;

Our schedule is quite simple, Full active every Friday night, incremental Monday to Thursday(so no backups Saturday/Sunday). Every last Saturday of each month(except April,August and December), I move one of the jobs(22 VM's, very important data) to tape for a 4 month retention). Also, every end of Semester, that Monthly job becomes a Semester job which runs on the last Saturday of the month of with a retention of ''forever'' for those mentioned months. The Tape Job configuration is to take the latest Full backup job in the repository

Sadly, since the size of the VM's and the data + the fact the information has to be rehydrated from the Dedupe Appliance to the Tape Library, the job cannot complete before the Monday night and once the incremental starts, the Tape job stops saying backup files are unavailable and all sorts of bad things happen.

I opened a case a couple of weeks back Case # 02123121 which indicated the initial Full active job hadn't completed(at that time it was on Saturday night) before the Tape job started which explained the ''unavailable backup files. We addressed that by moving the Full Active to the Friday night and making sure the job was completed before the Tape job started. But now, I still have the issue of the Tape job stopping as soon as the incremental from the Monday night starts.

What are my options? I absolutely need the Long term retention for those critical VM's, and the only thing I just thought of is to create a new backup job that only does Full Active's once a month with no incremental of those same 22 VM's and create the Tape job for that backup job so it will have basically 29 days on average to finish the tape backup without any interruptions. Would that be my best option?

If anyone has any other ideas, please let me know. I would greatly appreciate it as this needs to be resolved for good.
BenoitCote
Novice
 
Posts: 7
Liked: never
Joined: Mon Dec 01, 2014 8:31 pm
Full Name: Benoit Cote

Re: Full "Monthly" Tape job cannot complete in time

Veeam Logoby lyapkost » Tue May 02, 2017 2:02 pm

Hi, Benoit! There's a new option in 9.5 in schedule tab of backup to tape job "Prevent this job from being interrupted by primary backup jobs" (see user guide). When ckecked, the option will make source job to wait before deleting/merging/transforming backups before tape job finishes.
lyapkost
Veeam Software
 
Posts: 86
Liked: 15 times
Joined: Fri Nov 27, 2015 2:26 pm
Location: Saint Petersburg
Full Name: Konstantin

Re: Full "Monthly" Tape job cannot complete in time

Veeam Logoby BenoitCote » Tue May 02, 2017 3:18 pm

Hi Lyapkost,

thanks for the quick reply. I just noticed that option and it could potentially work, but if I understand what this means, the Tape job which could potentially take 4-5 days(guesstimating), will tell Veeam to not do the Incrementals for that specific job which means I won't have backups for X amount of days? If my last full is from Friday night and while the Tape job is running, I lose something on the Wednesday morning, it means I lost all the info from Monday,Tuesday and will be forced to go back to the Active Full on the last friday? Am I understanding this correctly?

Thanks for confirming.

Cheers,

Benoit
BenoitCote
Novice
 
Posts: 7
Liked: never
Joined: Mon Dec 01, 2014 8:31 pm
Full Name: Benoit Cote

Re: Full "Monthly" Tape job cannot complete in time

Veeam Logoby lyapkost » Tue May 02, 2017 3:43 pm

In fact, as mentioned, only merge/transform/delete operations may interfere with tape job. So if said checkbox is checked, source job will create increment on monday and wait on 99% as long as needed for tape job to delete old chain. I think in your case a powershell pre-job script in backup to tape job which increases source job's retention and post-job script which decreases it back should solve the problem. When tape job is running, source job will not delete any points due to a long retention and will finish, then start on Tuesday, then on Wednesday and (for example) if the tape job's finised by that time, retention period is shortened back and old backup chain might be deleted from repository at the end of Wednesday's run.
lyapkost
Veeam Software
 
Posts: 86
Liked: 15 times
Joined: Fri Nov 27, 2015 2:26 pm
Location: Saint Petersburg
Full Name: Konstantin

Re: Full "Monthly" Tape job cannot complete in time

Veeam Logoby BenoitCote » Mon May 08, 2017 1:12 pm

Hi Lyapkost,

I tried as you mentioned to activate the "Prevent this job from being interrupted by primary backup jobs" and started the job this weekend. It ran for 12 hours and then stopped with the same error ''Changes in source backup files detected, rescan required'', yet the timestamp of the files from the source backup have not changed and there are no other jobs running during the weekend. I am a bit baffled as to why my Veeam tape job just stops after 12 hours stating there were changes, yet there wasn't any. I will open a case once more as this is not a scheduling issue between the source job and tape job, it is simply an issue I need resolved. If you have any other ideas, please let me know in the meantime.

Ben
BenoitCote
Novice
 
Posts: 7
Liked: never
Joined: Mon Dec 01, 2014 8:31 pm
Full Name: Benoit Cote

Re: Full "Monthly" Tape job cannot complete in time

Veeam Logoby larry » Wed May 10, 2017 1:06 pm

What I did for this was to get a cheap USB 3 drives (10tb) place on Veeam server, create backup job to the USB drive with all options wanted, then use backup files to tape. We use SAN snap shots so the VM is only in use for seconds. The tape drive runs much faster because no putting data back together, I have all the restore points I want, I can rerun the tape job if needed or need a copy of the tape. It has worked so well that I stage all my tape jobs now.
larry
Expert
 
Posts: 380
Liked: 89 times
Joined: Wed Mar 24, 2010 5:47 pm
Full Name: Larry Walker

Re: Full "Monthly" Tape job cannot complete in time

Veeam Logoby lyapkost » Wed May 10, 2017 1:49 pm

Hi Ben.
BenoitCote wrote:It ran for 12 hours and then stopped with the same error ''Changes in source backup files detected, rescan required''

Are you sure the tape job stopped and has status 'failed'? The message indicates that a new backup which may be copied to tape has appeared in a repository. In that case tape job should wait till source job finishes and copy new incremental backup. The stop message is different and sounds like 'stopped by job job_name'.
lyapkost
Veeam Software
 
Posts: 86
Liked: 15 times
Joined: Fri Nov 27, 2015 2:26 pm
Location: Saint Petersburg
Full Name: Konstantin


Return to Tape



Who is online

Users browsing this forum: Yahoo [Bot] and 4 guests