Monthly full daily incremental

[trrobinson]

We have been doing monthly fulls to tape for years and we ship them off site forever.

We would like to start shipping incremental daily to a second media pool that we rotate after the monthly full goes off site or maybe after a month (set a and b or something). I see the GFS option but that doesn't seem to quite fit what I'm trying to do (unless I'm misunderstanding it).

The active fulls are spread out over the last week of the month to reduce load on the infrastructure. As such, any daily tape job would have a mix of fulls and incremental.

Maybe I'm overthinking it and simply running the job daily and setting fulls to go to one media pool and the incremental to a different media pool and letting retention on the incremental pool allow overwrites is the answer?

[soncscy]

Heya Trent,

One of the cool things Veeam does with GFS is that when the actual GFS periods need a backup, it must be a full, and it will just check the on-disk jobs and see what eligible backups there are -- if there's a full, the tape job just takes it to take. If it's an increment, they synthesize a full on-the-fly directly to tape (no need for on-disk provisioning, the entire process is written straightaway to the tape)

So you will need weekly GFS points with this schema, but it should work to track your dailies.

Else, with a simple media pool, you can of course just turn on the incremental option for those tape jobs and get the incremental points between your monthly fulls. Personally I kind of like the GFS route just for management purposes (it's easier to group/manage the tapes imo), but either works I guess.

[Dima P.]

Hello Trent,

We would like to start shipping incremental daily to a second media pool that we rotate after the monthly full goes off site or maybe after a month (set a and b or something). I see the GFS option but that doesn't seem to quite fit what I'm trying to do (unless I'm misunderstanding it). The active fulls are spread out over the last week of the month to reduce load on the infrastructure. As such, any daily tape job would have a mix of fulls and incremental.

That's correct, daily media set tapes out what is stored on the disk but relies on the weekly media set (so if you already have a full in weekly media set, daily will use is to link the dependent incrementals). You can try to sync the weekly media set schedule with your active fulls or create multiple GFS media pools based on the primary jobs active full schedule pattern (i.e. one for all jobs with active fulls on Monday; enabled weekly media set to Monday plus daily media set etc.)

soncscy is right, if you dont need GFS - you can simply go with standard media pools and set one for fulls and another for incrementals, Cheers!

[trrobinson]

Thank you for the responses. I added a media pool for incrementals and set my job to use the original keep forever pool for fulls and the new keep for 2 months for incrementals.

I'm running into a different issue now though. We have a few synthetic full jobs so we get weekly fulls. That's fine, I don't mind if they go to tape 4 times a month but it creates a new media set and starts a new tape leaving 9+TB unused on the last tape. I know I can set the media pool to never start a new media set but having a date on the media set helps locate proper tapes if we need to pull from tape a year down the road. The only other option is daily. What I want is monthly.

Any other wisdom here? Thanks!

[Dima P.]

Indeed there is no way to setup the media set creation on the monthly basis, so I've noted as a feature request.

If you have weekly fulls on disk, then you can easily set media set creation to the day of the full backup. In such case you will have full backup at the beginning of each media set which makes recovery easier in terms of finding required tapes. Always continue current media set option is the best in terms of space savings, but you will need to load more tapes during recovery. Cheers!

[trrobinson]

That's what I figured.

Does the GFS handle this better or will it start a new tape set for each day and each weekly etc.

Basically, I want one tape set that I append daily and overwrite after 60 days, don't care what it's media set name is. The fulls need to go a different media pool and append until the month is over then I'll protect the tapes, take them out and assign new media to the pool.

The GFS looks really close on paper but the inability to turn off weekly is kinda sinking it.

[Dima P.]

With GFS, indeed, you can control media set creation option separately. You can either set to create new media set every time (which seems to be a good option for monthly) or append all the data to single media set (which could work perfectly for daily backups as most likely you want to use all the remaining space on tape media).

Just keep in mind that for daily media set we will write what's stored on disk i.e. there is no option to synthesize increment to tape from full on disk. I'd also recommend to play with the schedule to make sure that full backups from disk are picked up by elder media sets, then daily will keep only incrementals linked to the fulls in elder media sets. There is also a minor limitation with daily media set - you must enable weekly backups as well to store the fulls separately. Hope that helps!

[trrobinson]

Seems like a lot is lacking for tape options here which is sad given tape is the only real air gap solution (at scale).

Just keep in mind that for daily media set we will write what's stored on disk i.e. there is no option to synthesize increment to tape from full on disk.

What does this mean? At weekly, if no VBK exists, it doesn't do anything or does it just put the latest vib or does it try to put the full vbk and vib chains on the weekly set?

I'd also recommend to play with the schedule to make sure that full backups from disk are picked up by elder media sets, then daily will keep only incrementals linked to the fulls in elder media sets.

I also don't follow this. You recommend I have multiple jobs one that runs after fulls and tries to slam those into a specific media set and then one that runs daily and slams the incrementals elsewhere? This is what I am trying to do but can't because it starts new sets every day for the full wasting basically 10+TB.