File to Tape Media Set question

[tombpcnpa]

Hi. I'm having trouble working out how best to configure a file to tape job to maximise our recovery options. I'm moving across from Backup Exec and the use of Media Sets within Media Pools is confusing me.
I have 1.3TB of files to backup and would like to run a full backup on Friday night, then incremental backups on Monday, Tuesday, Wednesday, Thursday.
I currently have 2 Media Pools, one for full backups and the other for incrementals and intend on assigning tapes a tape to each pool for each week the job is run.

The full backup seems straightforward, my question is how to be able to see each individual incremental backup. It seems as if each incremental backup overwrites the previous one, meaning that the changes to files are saved, but only the most recent one. If a user had been working on a file all week and on Friday wanted to go back to Tuesdays version, all I would be able to restore was the previous night's incremental backup (Thursday)

Sorry for the poor explanation, but hopefully someone can explain this.

[Shestakov]

Hi Tom and welcome to the community!
New increments do not overwrite previous ones, they both will be saved.
By the way, are you going to write on the tapes backup files only or something else? Maybe it`s better to use backup-to-tape rather than file-to-tape.
Thanks!

[tombpcnpa]

Thanks for the quick reply.

The files I'm backing up are user files not VM's so I need to use file to tape jobs.
I've created 2 test Media Pools to backup the IT folders files. The "IT Full" Media Pool has the Media Set option of "Do not create, always continue using current media set".
The "IT Incrementals" Media Pool has the Media Set option of "Create new media set for every backup session". My thinking was that each incremental job would then be available

I ran a full backup of the IT folder using a tape from the IT Full Media Set. I then edited a number of files so that the subsequent incremental job would pick them up
I ran the job again, and it took a tape from the "IT Incrementals" Media Set. When the job had completed I used the view files from the tape properties to confirm that only the changed files had been copied.

I then edited some of the files again, then ran the job once more. When the job had completed, only the most recently edited files were shown in the tape properties.
When checking the file restore options for one of the files that I edited twice - and therefore thought there should be 3 copies on the tape, the full backup, first incremental and second incremental, only the full backup on the full media set and the last incremental were shown.

How do I setup the incremental backups so that I can go back to a file that was backed up at a particular point in time, not just the first or last backup? I'm running Bakup & Replication version 9.5.2.823

[Shestakov]

Tom, what are the data retention policy settings on the media pool you use for increments?
Thanks!

[tombpcnpa]

Hi Nikita

The data retention settings for the "IT Incrementals" Media Pool are "Do not protect data (cyclically overwrite tapes as required)
Each IT incremental job is only copying a few MB, and even when I run the main job it only copys 5-10GB so I thought that leaving the data retention settings at their default would be fine.

For your information I'm using a Quantum SuperLoader 3 with an LTO6 drive. Both the drive and the library have been installed using the specific manufactuers drivers. The tapes are barcoded using the originally supplied labels.

[Shestakov]

If you choose "Create new media set for every backup session" option and do not protect media sets, each new job run will use new tape and they will overwrite each other, that`s probably the reason why you cannot find older increments.
I would suggest to create new media set weekly rather than daily.
Thanks!

[tombpcnpa]

Thanks for the solution to the overwritten tapes Nikita. I can confirm that using a single media set per week will show all the backup copys of a file that was modified multiple times during a week.
With this now working, how can I use a single tape for incremental jobs for 4 weeks? The incremenatal jobs will only take up 100GB per week so I'd like to use a single tape for one month.

If the media set is created on say, Monday 1st, how do I make sure that the following week's media sets don't overwite the first one? Should I enable the data retention policy so that data is protected for 4 weeks?
If that worked should I then end up with 4 media sets on the tape, each containg a weeks worth of incremental backups? That's what I'm hoping.

[Shestakov]

Great that it helped.
Here is how it works: retention policy is implemented to the whole media set starting from the moment when new ones is created. So if you create new media set every 4 weeks and set protection period = 4 weeks, you will have up to 8 weeks of increments.
That needs to be synchronized with "Full" media sets retention. How often do you want to create full backups on the source backup job and for how long do you want keep restore points on tapes?
Thanks!

[tombpcnpa]

I can't see an option to createa new Media Set every 4 weeks, In the Automatically Create New Media Set (Daily) I have the option Everyday or On These Days - with a choice of specific days of the week.

Just as a test, I changed the Data Retention Policy for the exisitng tape I've been using for Incremental backups (with a media set created last Monday) and ran the job.
I was hoping that a new Media Set would be created for this job (as the Media Set rule is to create a new Media Set each Monday, but that the media set from last week would not be overwritten since the data retention policy was set.
Unfortunately the job now wants a tape to be added, as I guess it won't use the current tape with the Data Retention Policy of 4 weeks.

On your last point about synchronization of Full and Incremental Media Sets, I'd like to use somthing along the lines of the following.
Full backup every Friday night using a rotating pool of 6 tapes and a periodic archive:
Week 1, Week 2, Week 3, Month 1, Week 1, Week 2, Week 3, Month 2, Week 1, Week 2, Week 3, Month 3, Week 1, Week 2, Week 3, Archive

The Weekly and Monthly tapes would be re-used as required and the Archive tapes never re-used. The Weekly and Monthly Full Backup jobs would give enough restore options, especically if we could have the incremental jobs backing up the changes during the week. We used to run a Full backup job every night using Backup Exec, but as you can imagine, that was a waste of time and tapes as hardly any files changed night on night. It did give us the easiest of restore options though as we only ever needed 1 tape to restore anything.

[veremin]

Unfortunately the job now wants a tape to be added, as I guess it won't use the current tape with the Data Retention Policy of 4 weeks.

This is an expected behaviour - a tape with retention that is yet to reached cannot be overwritten automatically. You can mark the tape as free and re-use after that.

Full backup every Friday night using a rotating pool of 6 tapes and a periodic archive:

A GFS media pool with monthly and quarterly media sets sound like a good fit. Though, it's a bit different from what you've requested - in the suggested scenario archive point will be created every 3 months, not every 4 ones.

Thanks.

[tombpcnpa]

I didn't think you could create file to tape jobs using GFS media sets, only simple media sets.
If it is possible then that could be an option. I'm not wedded to using my suggested setup, that's just how we've configured backups previously in order to give us good restore options. Having an archive using a quarterly media set would work.

Would your proposed GFS media pool still allow me to create Incremental jobs and run them for a set time before then being re-used?

[veremin]

Sorry, I must have missed that you were talking about file to tape jobs. In this case, indeed two or three jobs and same number of media pools will be required: weekly, monthly and archive one. Thanks.

[tombpcnpa]

OK. Will forget about GFS pool. Could I not just have a single Media Pool for the Full backups and another Media Poll for the incrementals?
The Full job will run every Friday night, it's only when I've run through a cycle that I'll start re-using the tapes based on my Weekly 1,2,3 Month1, Weekly 1,2,3, Month 2, Weekly 1,2,3, Month 3 schedule.

The main problem I seem to be facing is getting multiple weeks Incremental backups onto 1 tape. Knowing now that any Media Set with a Date Retention Policy will stop that tape being re-used until the time-out has been reached (based on the last time it was used) How can I get more than 1 weeks (4 days) worth of Incremental backups on 1 tape. I'm now trying out a Media Set with the option set to Do Not Create Always Continue, and a Data Retention Policy set to Never Overwrite Data. I'll update this tread once the job finishes.

[tombpcnpa]

As I expected, setting the Media Set to Do Not Create - Always Continue, and the Data Retention Policy to Never Overwrite Data, the File to Tape job requested a new tape.

I don't seem to be able to get any sort of file backup comparable to our old Backup Exec setup which is very frustrating. Veeam works well with backing up virtual machines to disk, but struggles to offer enough settings and configuration to backup files to tape.

[veremin]

You can select "do not create, always continue using current media set" option, specify whatever retention you want to (say, 4 weeks) and export a tape manually or automatically (PS script) every four weeks.

Exporting tape will automatically initiate creation of a new media set.

[tombpcnpa]

If I select the Media Set to use the Do Not Create, Continue Using Current Media Set with the Retention Policy of 4 weeks I can't use that tape for 4 weeks worth of Incremental backups.
I'd have to wait until the 4 week expiry time had passed before I could use it again.

What I'd like to do is to use the same physical tape for 4 weeks worth of Incremental backups, with the ability to be able to restore any file from those Incremental backups in the 4 week period.
If this is simply not possible then so be it. Should I open a support ticket to try to explain the situation more clearly?

[Shestakov]

Tom, the support case is not required here.
If you don`t want to set 4 week protection period after the increments are written to the tape, you may either set protection period = 1 week or not set it at all so while new increments rewrite the older ones starting from the beginning of the tape, the 3 other weeks of backups will be available for the restore.
Does it fulfill you requirement?

[tombpcnpa]

So if I create the Incremental Media Pool with the Media Set configuration Do Not Use, Continue Using Current Media Set, with the Data Retention Policy set to nothing, I should be able to use the same tape for 4 weeks and be able to restore any of the backups?
You say that new increments will rewrite the older ones starting from the beginning of the tape, but is that only if the tape is full? or will a backup job overwrite the oldest data on the tape first - supposing it doesn't have any retention settings?

Is it just going to be easier for me to link each weekly job (using 1 tape on a Friday night) to the incremental jobs running Mon-Thurs on one other tape?
This way I have to purchase additional tapes (that will hardly be used) for the incremental jobs, but at least I'll be more confident of my ability to restore files for my users

[Shestakov]

So if I create the Incremental Media Pool with the Media Set configuration Do Not Use, Continue Using Current Media Set, with the Data Retention Policy set to nothing, I should be able to use the same tape for 4 weeks and be able to restore any of the backups?

No, in this case you need to use script as Vladimir suggested above, otherwise the tapes will not be rewritten.

Is it just going to be easier for me to link each weekly job (using 1 tape on a Friday night) to the incremental jobs running Mon-Thurs on one other tape?
This way I have to purchase additional tapes (that will hardly be used) for the incremental jobs, but at least I'll be more confident of my ability to restore files for my users

Yes, you may use several jobs as a source of the tape job to use tapes space more effectively.
Thanks!

[tombpcnpa]

I don't know what script you're referring to. Is it one to script SQL commands in Powershell? Either way, I don't think that will work as I don't have the ability to export tapes as I'm using a Quantum SuperLaoder3 which lacks the required I/E slot to be able to export tapes.

I seem to be going round in circles trying to get incremental backups to work. To confirm my understanding of the setup I ran the incremental job again just now - with Do Not Create Media Set and no Data Retention, and as expected it overwrote the incremental jobs from last week.

So your suggestion that

If you don`t want to set 4 week protection period after the increments are written to the tape, you may either set protection period = 1 week or not set it at all so while new increments rewrite the older ones starting from the beginning of the tape, the 3 other weeks of backups will be available for the restore.

doesn't seem to be correct. If I set any Data Retention period - which is a concept I understand, the tape can't be used until that period has expired. Fair enough. But if a tape is used on Monday, Tuesday, Wednesday, Thursday, I can't use the tape again the following Monday.

I don't understand what you mean by

you may use several jobs as a source of the tape job to use tapes space more effectively.

[Shestakov]

You are right, let`s sum up the discussion.

Our suggestion for your requirement is to select "do not create, always continue using current media set" option and reload the tapes every 4 weeks, so 4 weeks of increments will be in a single media set (and tape).

If you are not able to export tapes, the best way is to configure weekly creation of a new media set with 3-4 weeks of protection period.

By you may use several jobs as a source of the tape job to use tapes space more effectively. I meant that if you write all the disk backups to tape using one tape job instead of several, the tape job will use 1 media set. As long as the problem we are solving is to use tapes effectively without unneeded retention, that`s the way to go.
Thanks!

[tombpcnpa]

I give up!

Our suggestion for your requirement is to select "do not create, always continue using current media set" option and reload the tapes every 4 weeks, so 4 weeks of increments will be in a single media set (and tape).

Seems to contradict your previous answer to my question,

So if I create the Incremental Media Pool with the Media Set configuration Do Not Use, Continue Using Current Media Set, with the Data Retention Policy set to nothing, I should be able to use the same tape for 4 weeks and be able to restore any of the backups?

No, in this case you need to use script as Vladimir suggested above, otherwise the tapes will not be rewritten.

I'm guessing that all of these answers revolve around the ability to export tapes or write PS scripts to achieve the same result, somthing that I'm not going to attempt.
For now I'll simply stick to orignal plan of using a sperate tape each week for both the Full and Incremental jobs.

I can't waste any more time waiting and testing various options and need a solid backup schedule in place for my users.
Hopefully future versions of Veeam will contain more file to tape options and more detailed documentation of the media pool/media set roles.

I appreciate that your original goal was to provide backup for virtual machines and that you're presumably communicating in a language other than your native one, so thanks for trying to work with me.
I'll probably be back with other backup questions in the future.

[Shestakov]

Tom, we`ve suggested you several options, some of them contradict each other, you are right.
Your final plan to new create media set(in your case it equals "use new tape") each week is totally sane and was among our suggestions.
We have taken your request into account, thank you for explaining that!