GFS to tape and how peroidic fulls to disk affect the setup

[Dark-Sider]

Hey guys,

I use a quantum Superloader 3 with 15 LTO-9 tapes and 1 Cleaning tape in conjunction with our 2-Tier backup2disk architecture.

The backup-infrastructure is nothing too complicated, just a few ESXi hosts and a couple of bare metal machines that have independent primary backup jobs. Depending on the size and criticality of the machines we have:
- One forever forward incremental chain (18 restore Points) for a ~40 TB storage tank
- Several ESXi hosts with 20 restore points, normal incremental backups with, peroidic fulls each week.
- Bare metal mimics the ESXi hosts settings

Those backups are then copied to a separate NAS building a GFS archive (7 daily backups, 4 weekly, 12 monthly, 10 yearly)

To make the backups portable for off-site storage we added above mentionend tape library and I configured a GFS Tape-Pool. The idea is to have 7 tapes for daily / weekly backups and a growing amount of tapes for monthly and yearly backups (both monthly and yearly get stored off-site)

What I noticed when the job ran the first time, it created a synthetic full backup as a monthly set (using ~3 Tapes). Then daily increments were written to the daily tapes (makes sense, saves tapes as they reference the monthly set). I assume the weekly set will do the same.

Since the GFS job works on our pirmary backup source, most backups don't have 31+ restore points and some of them even only have 6 .vibs for each .vbk. Then a new full backup is created. How will this affect my GFS pool? I assume that once a new full backup is created by the primary job, it needs to be written to tape, since the following .vibs are incompatibel with the monthly synthetic vbk, potentially wasting a lot of tape space.

What's with the forever forward 40 TB incremental backup that "only" has 18 restore points? Will result to a mid-month .vbk backup once some of the increments are no longer present on disk? In theory all the newer increments should still be compatibel with the synthetic .vbk. because only older increments got merged into the .vbk on disk, not affecting the tape stuff. But I noticed some odd behaviour with other tape jobs when the tape's rention period exceeded the amount of increments of the underlying forever forward incremental backups.

What is the best practice to not waste tape space? Switch to monthly .vbks instead and time them so the monthly tape job runs only after the monthly .vbk was created?

thanks,
Dark-Sider

[david.domask]

Hi Dark-Sider,

I would first give a review of the How it Works section on the User Guide.

Tape GFS jobs always work with the most recent restore point; if it's a Full on disk, it's simply copied to tape, if it's an increment, a Virtual Synthetic Full is created directly on tape.

Daily increments indeed need to link to a full, typically this is in the Weekly media set; the default behavior with GFS media pools is that if there is a schedule overlap (e.g. it's a Daily and Weekly day), write the current candidate to the eldest media set. So if you're having source jobs with periodic fulls, probably best to set the Weekly GFS day to same day that the Full backup is created.

What's with the forever forward 40 TB incremental backup that "only" has 18 restore points?

With Forever Forward Incremental, the Tape GFS job will only focus on the most recent restore point. Increments will go do Daily and will be linked to the Full that goes into the Weekly media set.

But I noticed some odd behaviour with other tape jobs when the tape's rention period exceeded the amount of increments of the underlying forever forward incremental backups.

Can you elaborate a bit more? That setup is in line with our documentation, so wouldn't expect issues if Tape Retention is longer than disk retention.

In short, your setup should be fine; just set the Weekly GFS day to same day that the job which makes Active Fulls produces its full and it will work. The Forever Forward Incremental (no periodic fulls) will be handled normally by the Tape GFS job and put increments in the Daily and will make a Virtual Synthetic Full for the Weekly media set that the increments in the Daily link to.

[Dark-Sider]

Can you elaborate a bit more? That setup is in line with our documentation, so wouldn't expect issues if Tape Retention is longer than disk retention.

sure I'm referring to this behaviour:

When the source job produces a forever forward incremental backup chain, you can set any retention period for the tape archive. To back up such chains, the virtual full mechanism is enabled automatically. Note that the virtual full backup must be always newer than the full backup on the disk. Otherwise, the tape job will copy not only the full backup from the disk, but also the whole chain of incremental backups. That may result in excessive use of additional tapes and significant increase in the tape job processing time. For more information, see Virtual Full Backup.

Retention was not the correct expression. What I menat was that if you have a tape job, that does a daily backup and creates a new media set every month and also creates a new synthetic full backup every month, then your forever incremental disk job needs to cover the whole month of restore points otherwise you will end up with a new full backup on tape "mid-month". There might be a reason for that, although I don't really understand it from a technical point of view - how merging say an 18 day old increment on disk into the vbk on disk affects today's vib in that chain that is copied to tape. I understand that if you would restore from tape in such a scenario, you would need to copy the synthetic full from tape as well as up to 30 increments but why wouldn't that work?

[david.domask]

Ah, okay I see what you're talking about.

In this case you won't need to worry, Tape GFS has a slightly different mechanism as the increments are always linked to a GFS point (in rare cases it may not be possible to link a new increment to an existing GFS backup, so in that case a Virtual Full will be placed in the Daily media set)

What you found was more a warning about Simple Media pools where the source job is Forever Forward Incremental and the tape job was seldomly run (think like a DIY GFS but with simple media pool), so for you it won't be a concern.