Best Practice for Son-Father-Grandfather rotation?

[mfrenzel]

Hi all!

I'm trying to get VEEAM V7 up and running with a Son-Father-Grandfather rotation. We've tried different approaches so far (Veeam Backup and SFG rotation with Backup Exec / Veeam Backup and Xcopy Batch) but I think that it is time to get this running completely in VEEAM V7.

Are there any "Best Practices" docs I've overseen so far? Or can someone from VEEAM shed some light on how to get such a scenario up and running?

I've got 4 tapes for daily backups (Monday, Tuesday, Wednesday and Thursday), 4 tapes for each Friday and 12 tape for each month where the last tape is designated as "yearly backup". They are all located in a Tandberg T24 Library with integrated HP Ultrium 5 Drive.

2 Backup Jobs are running. 1 for Critical Servers and 1 for Non-Critical Servers. I backup onto a Synolgy and keep 28 restore points on disk.

Every help / suggestion would be appreciated.

Thanks in advance!



Regards

Markus

[foggy]

Markus, there's no built-in GFS functionality in the current implementation of tape support in v7. However, as a workaround you can try the following:

1. For the current week's daily backups, you can use Backup to Tape jobs with an ordinary backup jobs as a source.
2. Create a backup copy job with a simple retention of 5 restore points for each of your ordinary backup jobs.
3. Create a File to Tape specifying the target repository of this backup copy job as a source and schedule it to run every Friday, exclude VIB files. This will grab the VBK file weekly.
4. Create a File to Tape with the same repository as a source and schedule it to run once a month, exclude VIB files. This will grab the VBK file monthly.
5. Create the yearly job using similar instructions, if needed.

For each step of GFS retention (daily, weekly, monthly..) you can create a separate media pool containing corresponding tapes and specify this pool as a target for the corresponding Files to Tape job.

However, there are two caveats that should be noted regarding this setup:

1. Backups will be offloaded to tapes with a one week delay (grabbing the Monday's VBK on Friday).
2. With Files to Tape jobs you will lose VM tracking mechanism - you won’t know on which tape media particular VMs reside on, what media is required in order to restore certain VM, etc.

[mfrenzel]

Thanks a lot, foggy. I'll give it a try and will post my experience a few days after my tests.

I'm thinking of a complete backup job redesign with backup jobs for every stage of the GFS retention so that there will be jobs to put the backups on my synology NAS and after that on the tape which will be responsible for the GFS retention part.

I hope that the second mentioned caveat won't be that big problem for us.

[veremin]

I hope that the second mentioned caveat won't be that big problem for us.

Probably, you can use the “description” field of given medias as a place where the information regarding tape content is written. Thanks.

[mfrenzel]

Uh... I have to read the manual more carefully...

What about a GFS rotation via Backup Copy Job?

I've read yesterday evening that there is an option inside the "Backup Copy" function which allows me to keep restore points for archival purposes on a weekly, monthly, quarterly and yearly bases.

Would this be the solution for my little GFS problem? I'll create a Backup Copy Job (5 restore points / target: Main-NAS) for my regular Backup Job (28 restore points / target: Main-NAS). The daily backups will be taken by tape jobs which are scheduled for each "normal" day (Mon, Tue, Wed, Thu). In the Backup Copy Job I'll define the function to keep restore points for archivel purposes... I schedule them as needed.

Would that be an "easy" way of getting a GFS rotation up and running?

What filenames will be given to the weekly, monthly and yearly backups?

Thanks in advance!

[foggy]

That would be the way if you do not need GFS on tapes. However, as far as I get you, you need right this (is that correct?). The problem is that the backup copy job's retention synthetically transforms (i.e. changes) all the restore points in the chain, so you will get the whole bunch of files on the tape every day if using tape jobs against them.

[mfrenzel]

I've thought that this could be the right way to get GFS on tapes...?!

It's a bit confusing now for me.

Maybe I should get a little more into detail what I'm trying to do:

I've got a Backup Job for my critical servers which runs every night at 10:30 pm. This Backup Job stores the backup files on our Synology NAS. Retention policy is set to 28 restore points so that I would be able to restore "quick" the content of the last 28 days if needed.

Now I want to save the backup files via GFS scheme on tape to get them of our building on week-days. The regular backup job (mentioned above) should be also available so that I could restore needed files as fast as possible if an error occures.

So I would like to run my scheduled regular backup job just as before and additionally would like to save via a GFS-scheme to tape. In case of emergency there would be 2 restore ways: 1.) via backup on NAS 2.) via backup on tape (f.e. if a collegue has deleted a important file 30 days ago).

If I understand the "Target" options correctly while creating a new Backup Copy Job then there will be safed restore points for my GFS scheme in the backup repository I declare in the Target options dialog. If I use another backup repository than my Main repository then everything should work fine... correct? If the file names would be "logical" (e.g. "week1", "week2", "week...", "month1", "month2", "month...") it should be possible to save only the needed files to tape. In my opinion...

[veremin]

Hi, Markus.The problem is that files produced by GFS rotation mechanism don’t have any meaningful names (that can be later specified as a mask for files to tape job), they’re just named as common backup files (job name, date). So, the scenario you’ve mentioned is unlikely to work.

Information regarding “who is who” is only shown in the properties of backup file, where restore points are marked with corresponding letters (W, M, Q,Y).

Thanks.

[mfrenzel]

Hi all!

I'm still struggeling with implementing a GFS scheme for our "To Disk -> To Tape" plan. Please help a Veeam novice which is going to surrender otherwise...

I've played around with the solution mentioned by foggy. Weekly, Monthly and Yearly Backups were quick defined as "File to tape" jobs... but I cant find a way to define some kind of perfect backup to tape job for the backups planned Monday to Thursday. If I use my regular backup job as a source for the Backup to Tape job then it's going to backup all files (28 restore points) to tape... right?

I try to backup our servers to our first backup location (a Synology NAS) with a retention policy of 28 restore points. The next step should be a Backup to Tape on a daily basis (Monday to Thursday) so that in case of an emergency I'm able to restore critical servers directly from Tape which is taken away from the office every day. Do I have to create separate Backup to Tape Jobs with Full Backups every day for the Backup from Monday to Thursday?

I didn't find any How Tos or Best Practices on the internet or on the Veeam Website.

Any help would be appreciated.



Regards

Markus

[veremin]

Hi, Markus.

So, you’ve defined weekly, quarterly, yearly jobs without any issues, meanwhile, you’re experiencing problems with setting daily one, right? Then, I’m wondering what backup method is being used by common backup job.

Thanks.

[mfrenzel]

Hi Vladimir,

thanks for your quick response. I try to describe my current backup jobs as good as possible so that my problem might be more clear. Please excuse my slightly rusty english... I still have to improve it.

I've defined a main backup job for my critical servers. It contains 15 objekts (with a total of 1.8 TB). The Backup repository is a Synology NAS with 13.4 TB of space. Retention policy is set to 28. I've scheduled this job to start every night at 10:30 PM.

Foggy pointed me in the right direction about weekly, monthly and yearly jobs via File to Tape (he write about excluding VIB files but I've realized that I have to include VBK files as excluding is not available in the File masks dialog. I've followed his advice and created a Backup Copy Job for my main backup job with a retention policy of 5. Then I've followed his tipps and created the file to tape jobs with the backup copy job as source and include only the VBK files.

He mentions then that I can use Backup to Tape Jobs with an ordinary backup job as a source... but after the Backup to Tape jobs starts I see that the job is going to backup 32 objects which are in the backup repository at the moment and this is quite heavy. I dont want to save everything (all 28 restore points) only the last version of my servers.

[foggy]

I've defined a main backup job for my critical servers. It contains 15 objekts (with a total of 1.8 TB). The Backup repository is a Synology NAS with 13.4 TB of space. Retention policy is set to 28. I've scheduled this job to start every night at 10:30 PM.

What is the backup mode for this job (forward incremental or reversed incremental)?

[mfrenzel]

Hi Alexander,

thanks for your reply.

My fault that I've forgotten to mention this: forward incremental with synthetic-fulls on Fridays. No other options set on the Backup tab in the Advanced Settings.

[veremin]

You can wait till the backup copy job reaches the number of allowed restore points (5) and till it starts to move a full backup on daily basis. Then, you can specify backup copy job as a source for “daily” file to tape job; though, don’t forget to set corresponding exclusions (include “vbk” only).

This way, “daily” file to tape job will copy to tape a new .vbk file each day. Though, in this case, there will be 4-day old full backup file copied to tapes daily, not the freshest one.

If you want to copy the freshest full backup, then, you should stick to reversed incremental mode.

Thanks.

[mfrenzel]

Hi Vladimir,

I think I don't understand the fundamentals of Veeam Backup Methods. Maybe I'm thinking to complex...

The User Guide for Veeam Backup & Replication Version 7 mentions on page 33 that the Forward Incremental Backup is the best choice for using tape backups.

If I try to think simple... maybe this is the easiest way of getting my backup stratety up and running?!

1. Create a regular backup job as Forward Incremental Backup with a retention policy of 28 restore points (or a lower number... this has to be tested) with synthetic fulls on Saturdays. Target: Synology-NAS.
2. Create a Backup Copy Job with a simple retention policy of 5 restore points. (as a shadow backup for the weekly, monthly and yearly backups). Target: Synology-NAS or better: Offsite Backup target (but then I have to create another Backup Copy Job on the Synology-NAS so that Tape backups won't be slow due to the location of the files)
3. Create a File to Tape Job with the Backup Copy Job as a source and schedule it to run every Friday, including only the VBK file. (grabs the "weekly" backups). Target: Tape
4. Create a File to Tape Job with the Backup Copy Job as a source and schedule it to run every last Saturday of the month, including only the VBK file. (grabs the "monthly" backups). Target: Tape
5. Create a File to Tape Job with the Backup Copy Job as a source and schedule it to run every last Sunday of the year, including only the VBK file. (grabs the "yearly" backups). Target: Tape
6. Create a File to Tape Job with the regular backup job as a source and schedule it to run Mondays to Thursdays, including only the VIB files. (grabs the "daily" backups). Target: Tape

Is this the right way? What do I have to do if I have to restore files of VMs from tape (e.g. in case of a complete failure of the Synology-NAS)? For my understanding the Friday backups save a full backup to tape. The daily backups will contain the VIB files for the 28 restore points. Is it possible to restore files or VMs then from the daily backup tapes in a desaster scenario?

Thanks in advance!

[foggy]

The fact is that the Friday's VBK and daily VIBs in this case will be from different backup chains (backup chains from different jobs: VBK from the backup copy job, VIBs from the regular backup job) and you will not be able to restore from those VIBs without their corresponding VBK.

[veremin]

The User Guide for Veeam Backup & Replication Version 7 mentions on page 33 that the Forward Incremental Backup is the best choice for using tape backups.

Yep, but your scenario seems to be different. If you want to have latest full restore point copied to tape on daily basis, then, you should switch to reversed incremental. This way, the copied restore point won’t depend on the other restore points, and you will be able to restore full VMs, using just one tape.

Otherwise, you can make something similar, using backup copy job, instead. Though, in this case, the full restore points copied to tape will be not the latest one, but rather a 4-day old one.

Also, I don’t recommend using off-site backup copy job as a source for tape jobs, since, in this case, the backup data will be copied to tapes from remote location, affecting seriously performance rates.

Thanks.

[mfrenzel]

Thank you very much, Alexander and Vladimir,

I appreciate your support! But I think I'm a little bit confused about the whole topic.

If I want to use the VIB files of my ordinary backup job (which are quite small) then I have to backup first the whole VBK to stay in the same backup chain... correct? This seems to be easy... but the synthetic fulls are also in the same folder located.

I think I should give it a try and experiment with the Reversed Incremental.

Aren't there any other VEEAM customers which are trying to backup to disk and to tape?

[foggy]

If I want to use the VIB files of my ordinary backup job (which are quite small) then I have to backup first the whole VBK to stay in the same backup chain... correct? This seems to be easy... but the synthetic fulls are also in the same folder located.

Correct. Each VIB in the chain depends on all previous VIBs and the corresponding VBK file, be it either active or synthetic full.

[mfrenzel]

So... what is the best way (in youre opinion) to get backup-to-disc-to-tape (with GFS scheme on tapes) up and running? Which backup method (reverse or forward incremental) would you prefer / advice?

I'm stuck at this point of my backup strategy plan and try to figure out what would be the best way.

Thanks in advance.

[veremin]

It depends on your desires. If you want to have independent full backup copied to tape each day, then, stick to reversed incremental. Otherwise, if you’re more interested in copying incremental backups, then, forward incremental should be the way to go.

Thanks.

[mfrenzel]

Well I think that my desires are changeable if someone tells me "that way you can save a lot of tapespace and youre backup is absolutely safe".

I try to figure out which way is recommended and the best because my approach seems to be wrong or led to a never restoreable backup to tape in case of emergency.

Thanks.

[foggy]

There's no universal approach, you would always have to deal with peculiarities of this or that backup method. We tried to give you recommendations based on your specific requirements, however if your requirements tend to change from time to time, you should adjust your backup strategy correspondingly.

[mfrenzel]

Thanks Alexander.

I'm going to read the B&R Manual again and try to figure out which might be the best solution approach for my environment. Maybe I'll understand the manual this time better.

Thanks a lot to you and everyone who participated in this thread.

[foggy]

You're welcome. Feel free to ask for further clarification, if required.

[kennocha]

Is there any plans to build in a Son-Father-Grandfather tape approach in a patch going forward in the future? We have it for backup to disk, so I can't imagine it would be out of the question?

[Dima P.]

Is there any plans to build in a Son-Father-Grandfather tape approach

Yes, this feature is in our longer term list.

[Messenjah]

Hello everybody,

@mFrenzel: Maybe we have the same approach. I am seeking for the right GFS-backupconfig with veeam. Which config have you figured out?

Yes, this feature is in our longer term list.

Very good idea. I want to support the other customers who need this d2d2tape feature with GFS

Thanks

[Dima P.]

Hello,
Please, review this scenario for GFS as a workaround:

Daily Cycle:
Create a first "Daily" media pool with 6 days retention period.
Create a "Daily" backup to tape job pointed to the “Daily” media pool. Schedule it to run 6 times a week (Mon,Tue,Wed,Thu,Fri,Sat)

Weekly Cycle:
Create a second "Weekly" media pool with 4 weeks retention period.
Create a "Weekly" backup to tape job pointed to the “Weekly” media pool. Schedule it to run every Sunday of the week.

Monthly Cycle:
Create a third "Monthly" media pool with 12 months retention period.
Create a "Monthly" backup to tape job pointed to the “Monthly” media pool. Schedule it to run monthly at every last Sunday of the month

[b.vanhaastrecht]

Hello,
Daily Cycle:
Create a first "Daily" media pool with 6 days retention period.
Create a "Daily" backup to tape job pointed to the “Daily” media pool. Schedule it to run 6 times a week (Mon,Tue,Wed,Thu,Fri,Sat)

Weekly Cycle:
Create a second "Weekly" media pool with 4 weeks retention period.
Create a "Weekly" backup to tape job pointed to the “Weekly” media pool. Schedule it to run every Sunday of the week.

Monthly Cycle:
Create a third "Monthly" media pool with 12 months retention period.
Create a "Monthly" backup to tape job pointed to the “Monthly” media pool. Schedule it to run monthly at every last Sunday of the month

With this rotation scheme. Does the monthly job supersede the weekly, or will Veeam run both jobs and backup the data twice on the last Sunday of the month?