GFS tape job not finding GFS candidates on first run

Everything about backing up to tape

GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Tue Mar 08, 2016 1:38 pm

We send our backups of our critical VMs to tape to send offsite at the beginning of each month. I was excited for V9 because it would let me make a GFS job and not have to create a new job every month to allow me to just grab the last chain from disk instead of every full since the last time the job ran.

This morning was the first time the GFS job was to run, starting at 1am. When I arrived at 7:15 I looked and all but one of the backups that are scheduled in the job are "pending" and have the following note when I click on them in the GUI:

3/8/2016 12:00:24 AM :: Building source backup files list started at 3/8/2016 12:00 AM
3/8/2016 12:00:25 AM :: No GFS candidate for Monthly media set was found for today, waiting
3/8/2016 12:00:25 AM :: Source backup files detected. VBK: 0, VBK map: 0, VIB: 0

The source is both Backup Copy jobs that sync once a week, as well as jobs that are forward incremental with weekly synthetic fulls run on Saturday. The only backup that got copied successfully to tape is a job that runs a weekly active full on Saturday.

I will be putting in a support ticket soon and will update with the case number but I wanted to see if this is behavior others have seen.

EDIT: Support case ID 01720942
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby Dima P. » Tue Mar 08, 2016 2:40 pm

Hi Steve,

Can you clarify, does the source backup job has a restore point for the set day (it might also be in the running state as well)?
Dima P.
Veeam Software
 
Posts: 6263
Liked: 441 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Tue Mar 08, 2016 5:35 pm

None of the jobs were in a running state (Well copy jobs were running but idle waiting for their next interval, but there should be no file locks in that state). I didn't see anywhere in the setup to have a set date for the restore points to be grabbed by GFS. I guess I assumed it would grab the last restore point and create a synthetic restore point for that day.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby Dima P. » Tue Mar 08, 2016 6:30 pm

I didn't see anywhere in the setup to have a set date for the restore points to be grabbed by GFS


The date is set in the GFS tape job. When you specify the date to start a tape job it will search for the restore point on disk. If the source job is not running – it should wait till the end of the scheduled day with periodic checks for a restore points. If no new restore points appear from the source jobs it by the end of the scheduled day it searches for the most recent restore point and place it on tape.
Dima P.
Veeam Software
 
Posts: 6263
Liked: 441 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Tue Mar 08, 2016 7:23 pm

Ahh ok. So since the job that it grabbed ran at 3am, the tape job grabbed it and it should grab everything else tomorrow. Thanks for the info on how it works.

I will check it in the morning and see if it works.

Thanks!
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Wed Mar 09, 2016 3:04 pm

So all of the backups whose source was not a backup copy job were grabbed overnight, but the ones with copy jobs as their source (including two which ran yesterday) are still showing as pending.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby Dima P. » Thu Mar 10, 2016 1:19 am

It may still be waiting for restore point for the set to be synced by backup copy. Can you see the new restore points for this pending source jobs?
Dima P.
Veeam Software
 
Posts: 6263
Liked: 441 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Fri Mar 11, 2016 2:10 pm

The copy jobs only run once a week. When one of the copy jobs ran the second day, once it finished transferring the tape job seemed to start working on that backup but ended with an error because the file merge was going on and when it finished, the VBK file was no longer present (due to the rename when the transform completed).

I had to cancel the job and set up an individual tape job to copy the files from those copy jobs to tape as we needed to send the tapes offsite.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Mon Mar 14, 2016 1:15 pm

Support said this looked like a bug that there was a hotfix for. I have not had a chance to install/test it yet.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby Dima P. » Tue Mar 15, 2016 12:45 am

Steve,

Thanks for sharing. By the way Update 1 is just around the corner and I assume that this fix is included, will check with the team though.
Dima P.
Veeam Software
 
Posts: 6263
Liked: 441 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Tue May 03, 2016 6:26 pm

Ok, so it has taken a bit of time for me to get around to writing this but now I have the information I need to know what is going on.

First, Update 1 fixed the issue with the job failing when a backup copy job merge began: Excellent.

But I also ran into something which caused me chagrin:

When a backup copy job is the source for a GFS tape job, the tape job waits until the end of the current copy interval to process the file. So in our case, as our copy jobs are on a 7 day interval, means that the tape job takes up to 1 week to run. Not fun for a job that is intended to make an archival copy of the latest state of everything at a point in time. I figured, well, ok I can just run the tape on the first Sunday of the month which will take a couple of days but will finish with all of the copies of the state nearest to the end of the month which while not ideal would be acceptable.

But, I discovered this morning looking at the status of the monthly GFS job another thing that makes me confused:
Even though it holds off on sending the data on tape until the current copy interval is completed, it doesn't take the restore point that it is "waiting for". As soon as the copy interval starts, it takes the restore point that was present at the time the GFS tape job began and copys it to tape then shows that source as completed successfully. In my case, that is a restore point from a week ago which further makes it harder to get an "end of month" tape for all of our critical VMs.

Why, if it is just going to use the previous restore point already on disk, does the tape job need to wait until the current copy interval is finished?

When I heard about GFS tape jobs I thought all of my tape scheduling headaches would go away but it seems like it has actually made it worse.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause

Re: GFS tape job not finding GFS candidates on first run

Veeam Logoby skrause » Wed May 04, 2016 6:50 pm

And it has gotten even weirder, on one of the source jobs included in this GFS job, it ignored the 4 most recent full backups on tape and went back a full month (the source job has 35 restore points with an active full every week) to grab a .vbk to send to disk. On another source job (backup copy) it ignored all of the .vib restore points and used only the .vbk that is the start of the backup copy chain, which is from March 29th.

Since there is no rhyme or reason to how the restore points are being flagged to be included or not with the GFS tape job, I have had to manually create a job to include all of the backups that are in the GFS job and kicked it off so I can send the tapes offsite like I did in version 8.
Steve Krause
Veeam Certified Architect
skrause
Expert
 
Posts: 296
Liked: 45 times
Joined: Mon Dec 08, 2014 2:58 pm
Full Name: Steve Krause


Return to Tape



Who is online

Users browsing this forum: No registered users and 6 guests