Discussions related to exporting backups to tape and backing up directly to tape.
ekisner
Expert
Posts: 202
Liked: 34 times
Joined: Jul 26, 2012 8:04 pm
Full Name: Erik Kisner
Contact:

Backup Rotation Question

Post by ekisner »

Having reviewed the user guide, I believe I've set up jobs to mirror my previous tape solution. Was hoping that someone could confirm that I've set up what I think I have, and not something that I've misunderstood?

Our ultimate goal:
Monthly - Full backups, last Friday of each month, 1 year retention
Weekly - Full Backups, Fridays other than last Friday of the month, 4 week retention
Daily - Incremental-Only backups, retained until one of the two above run.

To that effect, I have 3 media pools within the library... Incremental, Weekly, and Monthly. Each have a number of tapes that will handle their storage load. Tapes are cycled on their retention... so for example I have 12 sets of monthly tapes, 4 sets of weekly tapes, and 1 set of incremental tapes (which are appended to?). Every week before Friday, the tapes are cycled.

The Backup To Disk jobs (one for each VM) are set up to retain 5 restore points, forward incremental, Synthetic Full on Friday, with active full backups on last Friday of the month. Schedule is week-days, in the evening. Each of these jobs (with a few excluded due to a lack of importance) is added to the tape jobs.

There are 2 tape jobs.

Tape_Weekly:
Using the Weekly pool, with 'Process incremental backup files' enabled and targeting the Incremental pool. Both eject and export options are also unchecked. Scheduled for 'As new backup files appear'.

Tape_Monthly:
Using the Monthly pool, with 'Process incremental backup files' disabled. Both eject and export options are also unchecked. Scheduled for 'Monthly at last Friday'

I am probably going to enable the "Eject Media once the job finishes" option on both jobs, to return the tapes to their respective slots.

I've yet to run these jobs, so I've yet to see the results. Anyone have any thoughts on what might catch me? I mostly just want to make sure that during the week, the only thing getting written to tape is the incrementals, and on Fridays it's a full being written (as if it's doing a full backup to tape each day, that library will fill up quickly). And of course that on the last Friday of the month I'm able to tell it to do monthly instead of weekly?

To me, it means that every night, when the D2D backup finishes, it will trigger the weekly job. The weekly job will then back up the incrementals to the Incremental pool except on full/active full days, which would then target the weekly pool instead (as it would roll up the incrementals?). On the last Friday of the month, it looks like it'll auto-trigger the weekly job, and then schedule-trigger the monthly job?
zibius
Lurker
Posts: 1
Liked: never
Joined: Feb 10, 2014 6:46 pm
Full Name: Martin Pichette
Contact:

[MERGED] Daily inc, weekly full and monthly to tape

Post by zibius »

Hi guys,

New to Veeam, what is the best way to configure daily incremental and weekly full to tape. And have the last full of the month copied to tape.
Dima P.
Product Manager
Posts: 14396
Liked: 1568 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Backup Rotation Question

Post by Dima P. »

Hello,
Your setup looks very smooth, however, I got one little question: what are you using as a target for backup to tape - backup job or repository?

Also, avoid using compression both at backup to disk job (default level) and backup to tape job. I prefer Veeam B&R compression, but if you need to unload the backup server, you can use hardware compression on tape library instead.
Dima P.
Product Manager
Posts: 14396
Liked: 1568 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Backup Rotation Question

Post by Dima P. »

Hello Martin,
Please, review the ekisner post above. Thank you.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

Currently, the GFS rotation scheme can be achieved with the certain workaround. It looks pretty similar to what you're going to implement.

However, be aware, that the backup to tape jobs don't have any priority over each other. In other words, on the last Friday two backup to tape jobs (weekly/monthly) will be executed and corresponding full backup will be copied twice.

Thanks.
ekisner
Expert
Posts: 202
Liked: 34 times
Joined: Jul 26, 2012 8:04 pm
Full Name: Erik Kisner
Contact:

Re: Backup Rotation Question

Post by ekisner »

Thanks for the workaround. I may submit a feature request for job priorities ;) I may not though, it's trivial for me to add one more week's worth of weekly tapes to allow for both jobs to run, and then I've just got one more copy of backups... can never have too many of those!

d.popov, I use each individual D2D job as a target, rather than the repository. Should I be using the repository or will it trigger the same?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

Thanks for the workaround. I may submit a feature request for job priorities ;)
As mentioned in the adjacent thread, tape GFS rotation is already in our feature list.
Or will it trigger the same?
If you're using individual job as a source for backup to tape job, the backup to tape job will sit in the idle state and wait until the "source" backup job creates a new restore point.

In case of repository being a source, whatever restore point appears in the given repository, the backup to tape job will be started immediately. In other words, there is no direct connection between particular backup and backup to tape job, because various backup jobs can initiate the start of backup job.

Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

[MERGED] Backup to tape recommened setup case#00518195

Post by Luka »

Hi

I hope you can advise me on best setup for tapes backups
Currently we have 2 veeam backup disk jobs set: critical servers and standard servers. Critical servers are set to hold 71 restore points (full backups on Fridays) and incremental on working days, Standard servers are set to 7 restore points. We are happy with all the above

On the top of disk backups we need tape backups as per below:

- Yearly - last Friday of the year kept (indefinitely)
- Monthly - full backup at the last Friday of the month ( overwritten yearly)
- Weekly - full backups at Fridays (overwritten every 4 weeks)
- Daily - incremental backups (Monday to Thursday)

Our original ideal was to set a tape job that looks at the respiratory where disk backups are written to with a setting "write only when a new backup file appears". It worked for several weeks, but once we reached 71 restore points on the disk backup we noticed that tape backups started writing more than just last files to tape i.e 3 weeks old vbk files. Obviously it make our solution not working properly due to lack of space on our tapes. The retention period on media pool for offsite tape was set to 7 days. I was advised by Veeam teem to set tapes jobs using a backup copy job first, but still not sure on how to approach this. Can you advise something, so we could keep our tape backup routine

Regards
Luka
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Rotation Question

Post by foggy »

Luka, please review the thread above for the available options on implementing GFS rotation for tapes.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Thanks foggy

I have read it through and some good ideas there, so I have set as above 3 medial pools: daily (retention 1 week), weekly(retention 4 weeks) , monthly(retention 12months)
In my case our critical server backup to disk job has 71 restore points, so how will I set the tape job to write to tape only last vbk files for weekly backups and last incremental backups to daily tape backups

Luka
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

What backup mode you're using? Forward or Reversed incremental one? Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

I am using Forward incremental. I also created a backup copy job that synchronise the job with 71 restore points and keeps only 7 restore points in the copy job location
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

Then, you can create 3 different file to tape jobs, pointing them to three different media pools. Also, be aware, to specify file mask (.vbk), so that, only full backups are processed by file to tape jobs. This way, you will be able to achieve GFS retention scheme.

Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Thanks Vladimir

How will the Backup to tape job know which .vbk file is the latest one? Is this done through PS script? What the content of the script should be and how I execute it
How to deal with forward incremental files? We have incremental backups taken twice day, but daily tapes backups are taken only once a day?

I was also considering the following scenario:

a)initial backup to disk with 71 restore points of our critical servers will stay as it is
b) I will create backup copy job for the "71restore points" job to disk and set retention period 14 restore points (that should keep 1 vbk file and 13 vib files at one time) - is that correct?
c) I will set 3 media pools as stated originally (daily, weekly,monthly)
d) I will create 3 tape jobs for daily, weekly and monthly backups setting a source as "backup copy job with 14restore points" respiratory. I will set the tape backup jobs with run the job setting "as new backup file appear"
That way I should have a full backups at the weekends and incremental backups every working day

Would that be a viable option
Sorry I am not very advanced in Veeam, so apologies for any "silly" questions

Luka
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

First of all, it's not possible to select backup copy job as a source for backup to tape job.

So, if you want to archive files produced by backup copy job, the only option you have is to use file to tape jobs.

With backup copy job in place, you can choose it as a source for file to tape jobs. Backup copy job stores only one full backup which, then, will be processed by corresponding tape jobs.

Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Vladimir, I didn't mean to use "backups copy job" as a source for "backup to tape" job I know this is not possible. I meant to use "backup repository" as source for the "backup to tape" job. Can this work or I am still limited to use "file to tape" job
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

Even if you chose the repository backup copy job is writing to, the files produced by this job will be still skipped from processing. So, file to tape job seems to be the best option, if you're after archiving files produced by backup copy job.

Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Thanks, so coming back to my previous question. How will the "file to tape job" know what is the latest vbk file? And how to deal with the incremental backups only to tape on working days?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

How will the "file to tape job" know what is the latest vbk file?
As I have said, with backup copy job, there is only one .vbk file in place. Though, with specified retention (7 days), it wouldn't be the latest, but rather the weekly old one.
And how to deal with the incremental backups only to tape on working days?
You can set the ".vib" mask to "daily" file to tape job, and ".vbk" for weekly and monthly ones. This way, incremental backup files will be copied each day, while, full backups each week/month. However, you should definitely test it with small job and see whether you're able to apply increments copied on daily basis to the full backup copied during the weekend.

Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Hi Vladimir

I made this work the way as you suggested and it looks like it does do the job. So this is the way it works now

1)We have backup disk job with 54 restore points of all our critical server + another backup to disk job of our standard server with 7 restore points (full backups on Friday and forward incremental on Monday to Thursday)
2)There is a copy backup job to disk that is using 2jobs (critical and standard servers) to write with 7 restore points
3) I have created 2 media polls (daily and weekly) Daily with 7days retention period and weekly with 4 weeks retention period
4) I created 2 file to tape jobs, one called daily is looking at the "copy backup job" repository and writing files with *.vib mask- pointed to daily media poll, second file to tape job called weekly is also looking at the "copy backup job" repository and writing files with *.vbk mask - pointed to weekly media poll. I decided to give up monthlt tape job as I can't see the point (other than adding extra tape protection). I can use weekly tape job for my monthly tape backups

It all works fine for the moment, the only problem I noticed is that when I set "copy backup job" on Tuesday it created full backup, then Wednesday incremental and Thursday incremental - I would like to have full backup *.vbk created on Saturdays, not Tuesdays How can I achieve that?

Regards
Luka
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

It all works fine for the moment, the only problem I noticed is that when I set "copy backup job" on Tuesday it created full backup, then Wednesday incremental and Thursday incremental - I would like to have full backup *.vbk created on Saturdays, not Tuesdays How can I achieve that?
Not sure whether I completely follow you on that. You mean that backup copy job created its first full restore point on Tuesday, right?

Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

yes this is exactly what I meant, even do the source for the copy backup has full restore point on Fridays
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Rotation Question

Post by foggy »

Backup copy job is always incremental and synthetically creates restore points in remote location from the changed blocks extracted from the source storage. The full backup file is always the oldest file in chain.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Thanks Alexander I understand now, so the weakness of my design is that the full restore point *.vbk will be always a week old in my scenario (7 restore points)
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

Yes, that was exactly what I meant.
Though, with specified retention (7 days), it wouldn't be the latest, but rather the weekly old one.
Thanks.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

If on backup copy job settings I tick "keep full backups for archived purposes" and select Weekly backups 1 at Friday 22:00. Will this sort my problem? I should have a full vbk file for last Friday. The only unknown is will I also have "regular" vbk file that is related to retention policy (always 7 days old)?

http://helpcenter.veeam.com/backup/70/v ... arget.html
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Rotation Question

Post by foggy »

You will end up with several VBK files this way - the specified number of GFS restore points plus the regular one.
Luka
Influencer
Posts: 20
Liked: never
Joined: Feb 21, 2014 12:27 pm
Contact:

Re: Backup Rotation Question

Post by Luka »

Ok, If I take off the *.vbk mask of the weekly tape job that means that it will copy 7 day old vbk file + 6 vib files, but I will have Weekly tape backup up to date taken over the weekend, which is what I want

The problem I can see now is related to incremental backups that I take on daily basis. First of all with having daily file to tape job filtered for *.vib I will always copy 6 vib files (I would accept it as incremental backups are very small) Obvoiusly to restore incremental backup form tape I will need a full vbk which is the start of the chain, as vbk will change every day - will I be able to restore i.e Thursday incremental backup from tape having full vbk and all vib as per Sunday a few days before
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Rotation Question

Post by foggy »

Luka wrote:Ok, If I take off the *.vbk mask of the weekly tape job that means that it will copy 7 day old vbk file + 6 vib files, but I will have Weekly tape backup up to date taken over the weekend, which is what I want
Not exactly. I should have mentioned this earlier, GFS restore points are not created but rather existing full is simply copied and marked as weekly GFS on reaching the day specified in the GFS schedule. So they still will be dated by the previous Saturday in your case.
Luka wrote:The problem I can see now is related to incremental backups that I take on daily basis. First of all with having daily file to tape job filtered for *.vib I will always copy 6 vib files (I would accept it as incremental backups are very small) Obvoiusly to restore incremental backup form tape I will need a full vbk which is the start of the chain, as vbk will change every day - will I be able to restore i.e Thursday incremental backup from tape having full vbk and all vib as per Sunday a few days before
No, you will need the exact VBK from the chain you've copied over to tape.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup Rotation Question

Post by veremin »

First of all with having daily file to tape job filtered for *.vib I will always copy 6 vib files.
Not exactly, only two backup files will be copied on everyday basis - one full backup (the oldest), one incremental backup (the newest). The files in between will stay unchanged, and, hence, not copied to tape appliance.

Thanks.
Post Reply

Who is online

Users browsing this forum: No registered users and 17 guests