Backup to Tape isn't working as accepted and tapes are empty

[burak]

Hello,

a backup to tape job of a customer isn't working as accepted and today I saw that the weekly backup tapes of the last three weeks are empty.

The configuration is:

- One LTO-6 drive
- 20 Tapes
- One GFS media pool, where all 20 tapes are included: 4x weekly tapes, 12x monthly tapes, 4x yearly tapes
- GFS media set settings:
- Weekly media set overwrite protection 4 weeks, append, do not export
- Monthly media set overwrite protection 11 months, append, do not export
- Yearly media set overwrite protection 10 years, append, do not export
- Tapes should be removed and inserted at Monday only!

Backup to tape settings:
- One backup to tape job
- Backup files of three backup jobs
- GFS Media pool as destination assigned for full backup
- Eject media upon job completion; no export
- Schedule: Weekly Tuesday, Monthy: First Tuesday of the month: Yearly: Second Tuesday of the year
- Prevent this job from being interrupted by source backup jobs is enabled

Backup to disk settings:
- Three jobs, Incremental backups with active full every Sunday at 9pm
- Backup size is almost 200GB


I will try to describe the current situation. Today, I checked the tapes and the weekly tapes are empty. I inserted the monthly media tape "March" to do the monthly backup, which I started manually. It worked correctly with no problems (I did this at the installation with the yearly and February tape too). Here is the log:

07.03.2018 11:10:04 :: Building source backup files list started at 07.03.2018 11:10:04
07.03.2018 11:10:04 :: Synthetic full backup from Backup DCsD2018-03-06T180224.vib (06.03.2018 18:02:24) will be placed into the monthly media set
07.03.2018 11:10:06 :: Source backup files detected. VBK: 0, VBK map: 1, VIB: 0
07.03.2018 11:10:06 :: Queued for processing at 07.03.2018 11:10:06
07.03.2018 11:10:06 :: Required backup infrastructure resources have been assigned
07.03.2018 11:10:47 :: Drive 1 (Server: SRV-PM-HV01, Library: IBM ULTRIUM-HH6, Drive ID: Tape0) locked successfully
07.03.2018 11:10:47 :: Current tape is March
07.03.2018 11:11:36 :: New tape backup session started, encryption: disabled
07.03.2018 11:12:27 :: Backup to media set monthly started at 03/07/2018 11:10:04
07.03.2018 11:12:31 :: Creating full backup map for Backup DCsD2018-03-06T180224.vib
07.03.2018 11:12:52 :: Processing Backup DCsD2018-03-06T180224.vbk
07.03.2018 11:18:30 :: 0 directories and 1 files backed up
07.03.2018 11:18:30 :: Busy: Source 52% > Proxy 4% > Network 36% > Target 77%
07.03.2018 11:18:30 :: Primary bottleneck: Target
07.03.2018 11:18:30 :: Network traffic verification detected no corrupted blocks
07.03.2018 11:18:30 :: Processing finished at 07.03.2018 11:18:30

However, after the job was finished I get the mail "[Warning] Insert a tape to continue":

Job Backup All to Tape requires a tape to be added to media pool GFS Media Pool 1 (Library: IBM ULTRIUM-HH6). See the corresponding job session in the management console for details.

The backup to tape job is running in WaitingStatus and the details shows me this information:

Building source backup files list started at 07.03.2018 12:05:43
Synthetic full backup from Backup DCsD2018-02-27T180223.vib (27.02.2018 00:00:00) will be placed into the weekly media set
Source backup files detected. VBK: 0, VBK map: 1, VIB: 0
Queued for processing at 07.03.2018 12:05:45
Required backup infrastructure resources have been assigned
Add tape to media pool GFS Media Pool 1 (IBM ULTRIUM-HH6 (SRV-PM-HV01)). Oldest retained tape: Week 4 42:57

At this point I have to ask: why is it waiting for a new tape and why is the date 2018-02-27 and not like above 2018-03-06 ?

I told the customer, to remove the March tape after it is ejected and insert the "Week 1" tape today. The backup job started and finished successfully:

07.03.2018 12:05:43 :: Building source backup files list started at 07.03.2018 12:05:43
07.03.2018 12:05:43 :: Synthetic full backup from Backup DCsD2018-02-27T180223.vib (27.02.2018 00:00:00) will be placed into the weekly media set
07.03.2018 12:05:44 :: Source backup files detected. VBK: 0, VBK map: 1, VIB: 0
07.03.2018 12:05:45 :: Queued for processing at 07.03.2018 12:05:45
07.03.2018 12:05:45 :: Required backup infrastructure resources have been assigned
07.03.2018 12:07:22 :: Add tape to media pool GFS Media Pool 1 (IBM ULTRIUM-HH6 (SRV-PM-HV01)). Oldest retained tape: Week 4
07.03.2018 12:55:44 :: Drive 1 (Server: SRV-PM-HV01, Library: IBM ULTRIUM-HH6, Drive ID: Tape0) locked successfully
07.03.2018 12:55:44 :: Current tape is Week 1
07.03.2018 12:56:32 :: New tape backup session started, encryption: disabled
07.03.2018 12:57:18 :: Backup to media set weekly started at 03/07/2018 12:05:42
07.03.2018 12:57:22 :: Creating full backup map for Backup DCsD2018-02-27T180223.vib
07.03.2018 12:57:44 :: Processing Backup DCsD2018-02-27T180223.vbk
07.03.2018 13:02:44 :: 0 directories and 1 files backed up
07.03.2018 13:02:44 :: Busy: Source 60% > Proxy 4% > Network 43% > Target 72%
07.03.2018 13:02:44 :: Primary bottleneck: Target
07.03.2018 13:02:44 :: Network traffic verification detected no corrupted blocks
07.03.2018 13:02:44 :: Processing finished at 07.03.2018 13:02:44

The next tape backup job should start at Tuesday 00:00. I am going to wait for this and will check the logs and tapes after this. But I want to ask you, if my configuration is correct. I am not sure if the backup to disk job, which is at Sunday 9pm will trigger the backup to tape job too. Even if the full backup is created, will it just wait for the tape schedule at Tuesday 00:00?

[Dima P.]

Hello burak.

At this point I have to ask: why is it waiting for a new tape and why is the date 2018-02-27 and not like above 2018-03-06 ?

Behavior looks expected: if the schedule for backup periods overlap, only the backup for the media set with the longest retention will be archived. That’s why you see this backup file being written to monthly but do not see it in weekly media set. Check this article to know more about the GFS schema. This one is also good read to learn more about GFS limitations.

[burak]

Hello Dima,

the first link doesn't work and I read the second link before I created the tape job. If the behaviour is working as excepted, then it is okay.

But now I have the next problem. I waited to the next weekly backup to tape schedule today at 00:00. An active full backup was created at Sunday 9pm and yesterday Monday at 9pm was one incremental backup. This active full wasn't backuped to tape yet.

Today at 00:00 the backup to tape job starts but right now it is running in a waiting state and the log only shows me this:

13.03.2018 00:00:31 :: Building source backup files list started at 13.03.2018 00:00:31
13.03.2018 00:00:31 :: No GFS candidate for Weekly media set was found for today, waiting
13.03.2018 00:00:31 :: Source backup files detected. VBK: 0, VBK map: 0, VIB: 0

This can't be right, because there has to be a GFS candidate, namely the active full backup from Sunday 03/11. This should be copied to tape with this job.

Should I contact the support for this or is something wrong in my assumption?

[cgrebing]

Hello Burak,

this seems OK because you don't have a backup of today (because the Job starts at 0:00 this is always the case). If I am right the Jobs waits up to 24 hours if a new backup exists. to get rid of this you should search for the registry keys "TapeGFSBackupWaitTimeout" and "UsePreviousStoragesForTodaySynDate". this may help you.

[veremin]

This can't be right, because there has to be a GFS candidate, namely the active full backup from Sunday 03/11.

In this case, you should either schedule GFS tape job to also start on Sunday or implement one of the mentioned regkeys to make GFS tape job take previous restore point as GFS candidate. Thanks.