Backup Rotation Question

[ekisner]

Having reviewed the user guide, I believe I've set up jobs to mirror my previous tape solution. Was hoping that someone could confirm that I've set up what I think I have, and not something that I've misunderstood?

Our ultimate goal:
Monthly - Full backups, last Friday of each month, 1 year retention
Weekly - Full Backups, Fridays other than last Friday of the month, 4 week retention
Daily - Incremental-Only backups, retained until one of the two above run.

To that effect, I have 3 media pools within the library... Incremental, Weekly, and Monthly. Each have a number of tapes that will handle their storage load. Tapes are cycled on their retention... so for example I have 12 sets of monthly tapes, 4 sets of weekly tapes, and 1 set of incremental tapes (which are appended to?). Every week before Friday, the tapes are cycled.

The Backup To Disk jobs (one for each VM) are set up to retain 5 restore points, forward incremental, Synthetic Full on Friday, with active full backups on last Friday of the month. Schedule is week-days, in the evening. Each of these jobs (with a few excluded due to a lack of importance) is added to the tape jobs.

There are 2 tape jobs.

Tape_Weekly:
Using the Weekly pool, with 'Process incremental backup files' enabled and targeting the Incremental pool. Both eject and export options are also unchecked. Scheduled for 'As new backup files appear'.

Tape_Monthly:
Using the Monthly pool, with 'Process incremental backup files' disabled. Both eject and export options are also unchecked. Scheduled for 'Monthly at last Friday'

I am probably going to enable the "Eject Media once the job finishes" option on both jobs, to return the tapes to their respective slots.

I've yet to run these jobs, so I've yet to see the results. Anyone have any thoughts on what might catch me? I mostly just want to make sure that during the week, the only thing getting written to tape is the incrementals, and on Fridays it's a full being written (as if it's doing a full backup to tape each day, that library will fill up quickly). And of course that on the last Friday of the month I'm able to tell it to do monthly instead of weekly?

To me, it means that every night, when the D2D backup finishes, it will trigger the weekly job. The weekly job will then back up the incrementals to the Incremental pool except on full/active full days, which would then target the weekly pool instead (as it would roll up the incrementals?). On the last Friday of the month, it looks like it'll auto-trigger the weekly job, and then schedule-trigger the monthly job?

[zibius]

Hi guys,

New to Veeam, what is the best way to configure daily incremental and weekly full to tape. And have the last full of the month copied to tape.

[Dima P.]

Hello,
Your setup looks very smooth, however, I got one little question: what are you using as a target for backup to tape - backup job or repository?

Also, avoid using compression both at backup to disk job (default level) and backup to tape job. I prefer Veeam B&R compression, but if you need to unload the backup server, you can use hardware compression on tape library instead.

[Dima P.]

Hello Martin,
Please, review the ekisner post above. Thank you.

[veremin]

Currently, the GFS rotation scheme can be achieved with the certain workaround. It looks pretty similar to what you're going to implement.

However, be aware, that the backup to tape jobs don't have any priority over each other. In other words, on the last Friday two backup to tape jobs (weekly/monthly) will be executed and corresponding full backup will be copied twice.

Thanks.

[ekisner]

Thanks for the workaround. I may submit a feature request for job priorities I may not though, it's trivial for me to add one more week's worth of weekly tapes to allow for both jobs to run, and then I've just got one more copy of backups... can never have too many of those!

d.popov, I use each individual D2D job as a target, rather than the repository. Should I be using the repository or will it trigger the same?

[veremin]

Thanks for the workaround. I may submit a feature request for job priorities

As mentioned in the adjacent thread, tape GFS rotation is already in our feature list.

Or will it trigger the same?

If you're using individual job as a source for backup to tape job, the backup to tape job will sit in the idle state and wait until the "source" backup job creates a new restore point.

In case of repository being a source, whatever restore point appears in the given repository, the backup to tape job will be started immediately. In other words, there is no direct connection between particular backup and backup to tape job, because various backup jobs can initiate the start of backup job.

Thanks.

[Luka]

Hi

I hope you can advise me on best setup for tapes backups
Currently we have 2 veeam backup disk jobs set: critical servers and standard servers. Critical servers are set to hold 71 restore points (full backups on Fridays) and incremental on working days, Standard servers are set to 7 restore points. We are happy with all the above

On the top of disk backups we need tape backups as per below:

- Yearly - last Friday of the year kept (indefinitely)
- Monthly - full backup at the last Friday of the month ( overwritten yearly)
- Weekly - full backups at Fridays (overwritten every 4 weeks)
- Daily - incremental backups (Monday to Thursday)

Our original ideal was to set a tape job that looks at the respiratory where disk backups are written to with a setting "write only when a new backup file appears". It worked for several weeks, but once we reached 71 restore points on the disk backup we noticed that tape backups started writing more than just last files to tape i.e 3 weeks old vbk files. Obviously it make our solution not working properly due to lack of space on our tapes. The retention period on media pool for offsite tape was set to 7 days. I was advised by Veeam teem to set tapes jobs using a backup copy job first, but still not sure on how to approach this. Can you advise something, so we could keep our tape backup routine

Regards
Luka

[foggy]

Luka, please review the thread above for the available options on implementing GFS rotation for tapes.

[Luka]

Thanks foggy

I have read it through and some good ideas there, so I have set as above 3 medial pools: daily (retention 1 week), weekly(retention 4 weeks) , monthly(retention 12months)
In my case our critical server backup to disk job has 71 restore points, so how will I set the tape job to write to tape only last vbk files for weekly backups and last incremental backups to daily tape backups

Luka

[veremin]

What backup mode you're using? Forward or Reversed incremental one? Thanks.

[Luka]

I am using Forward incremental. I also created a backup copy job that synchronise the job with 71 restore points and keeps only 7 restore points in the copy job location

[veremin]

Then, you can create 3 different file to tape jobs, pointing them to three different media pools. Also, be aware, to specify file mask (.vbk), so that, only full backups are processed by file to tape jobs. This way, you will be able to achieve GFS retention scheme.

Thanks.

[Luka]

Thanks Vladimir

How will the Backup to tape job know which .vbk file is the latest one? Is this done through PS script? What the content of the script should be and how I execute it
How to deal with forward incremental files? We have incremental backups taken twice day, but daily tapes backups are taken only once a day?

I was also considering the following scenario:

a)initial backup to disk with 71 restore points of our critical servers will stay as it is
b) I will create backup copy job for the "71restore points" job to disk and set retention period 14 restore points (that should keep 1 vbk file and 13 vib files at one time) - is that correct?
c) I will set 3 media pools as stated originally (daily, weekly,monthly)
d) I will create 3 tape jobs for daily, weekly and monthly backups setting a source as "backup copy job with 14restore points" respiratory. I will set the tape backup jobs with run the job setting "as new backup file appear"
That way I should have a full backups at the weekends and incremental backups every working day

Would that be a viable option
Sorry I am not very advanced in Veeam, so apologies for any "silly" questions

Luka

[veremin]

First of all, it's not possible to select backup copy job as a source for backup to tape job.

So, if you want to archive files produced by backup copy job, the only option you have is to use file to tape jobs.

With backup copy job in place, you can choose it as a source for file to tape jobs. Backup copy job stores only one full backup which, then, will be processed by corresponding tape jobs.

Thanks.

[Luka]

Vladimir, I didn't mean to use "backups copy job" as a source for "backup to tape" job I know this is not possible. I meant to use "backup repository" as source for the "backup to tape" job. Can this work or I am still limited to use "file to tape" job

[veremin]

Even if you chose the repository backup copy job is writing to, the files produced by this job will be still skipped from processing. So, file to tape job seems to be the best option, if you're after archiving files produced by backup copy job.

Thanks.

[Luka]

Thanks, so coming back to my previous question. How will the "file to tape job" know what is the latest vbk file? And how to deal with the incremental backups only to tape on working days?

[veremin]

How will the "file to tape job" know what is the latest vbk file?

As I have said, with backup copy job, there is only one .vbk file in place. Though, with specified retention (7 days), it wouldn't be the latest, but rather the weekly old one.

And how to deal with the incremental backups only to tape on working days?

You can set the ".vib" mask to "daily" file to tape job, and ".vbk" for weekly and monthly ones. This way, incremental backup files will be copied each day, while, full backups each week/month. However, you should definitely test it with small job and see whether you're able to apply increments copied on daily basis to the full backup copied during the weekend.

Thanks.

[Luka]

Hi Vladimir

I made this work the way as you suggested and it looks like it does do the job. So this is the way it works now

1)We have backup disk job with 54 restore points of all our critical server + another backup to disk job of our standard server with 7 restore points (full backups on Friday and forward incremental on Monday to Thursday)
2)There is a copy backup job to disk that is using 2jobs (critical and standard servers) to write with 7 restore points
3) I have created 2 media polls (daily and weekly) Daily with 7days retention period and weekly with 4 weeks retention period
4) I created 2 file to tape jobs, one called daily is looking at the "copy backup job" repository and writing files with *.vib mask- pointed to daily media poll, second file to tape job called weekly is also looking at the "copy backup job" repository and writing files with *.vbk mask - pointed to weekly media poll. I decided to give up monthlt tape job as I can't see the point (other than adding extra tape protection). I can use weekly tape job for my monthly tape backups

It all works fine for the moment, the only problem I noticed is that when I set "copy backup job" on Tuesday it created full backup, then Wednesday incremental and Thursday incremental - I would like to have full backup *.vbk created on Saturdays, not Tuesdays How can I achieve that?

Regards
Luka

[veremin]

It all works fine for the moment, the only problem I noticed is that when I set "copy backup job" on Tuesday it created full backup, then Wednesday incremental and Thursday incremental - I would like to have full backup *.vbk created on Saturdays, not Tuesdays How can I achieve that?

Not sure whether I completely follow you on that. You mean that backup copy job created its first full restore point on Tuesday, right?

Thanks.

[Luka]

yes this is exactly what I meant, even do the source for the copy backup has full restore point on Fridays

[foggy]

Backup copy job is always incremental and synthetically creates restore points in remote location from the changed blocks extracted from the source storage. The full backup file is always the oldest file in chain.

[Luka]

Thanks Alexander I understand now, so the weakness of my design is that the full restore point *.vbk will be always a week old in my scenario (7 restore points)

[veremin]

Yes, that was exactly what I meant.

Though, with specified retention (7 days), it wouldn't be the latest, but rather the weekly old one.

Thanks.

[Luka]

If on backup copy job settings I tick "keep full backups for archived purposes" and select Weekly backups 1 at Friday 22:00. Will this sort my problem? I should have a full vbk file for last Friday. The only unknown is will I also have "regular" vbk file that is related to retention policy (always 7 days old)?

http://helpcenter.veeam.com/backup/70/v ... arget.html

[foggy]

You will end up with several VBK files this way - the specified number of GFS restore points plus the regular one.

[Luka]

Ok, If I take off the *.vbk mask of the weekly tape job that means that it will copy 7 day old vbk file + 6 vib files, but I will have Weekly tape backup up to date taken over the weekend, which is what I want

The problem I can see now is related to incremental backups that I take on daily basis. First of all with having daily file to tape job filtered for *.vib I will always copy 6 vib files (I would accept it as incremental backups are very small) Obvoiusly to restore incremental backup form tape I will need a full vbk which is the start of the chain, as vbk will change every day - will I be able to restore i.e Thursday incremental backup from tape having full vbk and all vib as per Sunday a few days before

[foggy]

Ok, If I take off the *.vbk mask of the weekly tape job that means that it will copy 7 day old vbk file + 6 vib files, but I will have Weekly tape backup up to date taken over the weekend, which is what I want

Not exactly. I should have mentioned this earlier, GFS restore points are not created but rather existing full is simply copied and marked as weekly GFS on reaching the day specified in the GFS schedule. So they still will be dated by the previous Saturday in your case.

The problem I can see now is related to incremental backups that I take on daily basis. First of all with having daily file to tape job filtered for *.vib I will always copy 6 vib files (I would accept it as incremental backups are very small) Obvoiusly to restore incremental backup form tape I will need a full vbk which is the start of the chain, as vbk will change every day - will I be able to restore i.e Thursday incremental backup from tape having full vbk and all vib as per Sunday a few days before

No, you will need the exact VBK from the chain you've copied over to tape.

[veremin]

First of all with having daily file to tape job filtered for *.vib I will always copy 6 vib files.

Not exactly, only two backup files will be copied on everyday basis - one full backup (the oldest), one incremental backup (the newest). The files in between will stay unchanged, and, hence, not copied to tape appliance.

Thanks.