Retention period on VTL (Quikstation 4) not working as expected

[feiaweng]

Hello community,

I use a RDX quikstation 4 drive as a VTL in Veeam.
In the quikstation are 4 drives, they are divided in 2 media pools.
2 drives/tapes are for full backup, and 2 drives/tapes are for incremental backup.
The media pool full backup has a size from 7,6 TB and the media pool incr. backup has a size from 3,8 TB.
One full backup from NAS to quikstation has a size about 1,1 TB.
When I use in the tape job the option "Do not protect data (cyclically overwrite tapes as required)" then the data will be overwritten on every job.
Means I can not use the whole size of the media pool.
If I use the option "protect data for 3 weeks" (1 full backup will be done weekly), but maybe I missunderstood this.
Every backup should be not overwritten for 3 weeks. After 3 weeks the oldest backup should be overwritten but it does not work as expected.
Veeam sends a message please insert a new tape.
Not sure what I am doing wrong here.
I hope you can help me with my issue.

regards
Ralf

[david.domask]

Hi Ralf, welcome to the forums.

Tape Protection is measured against the tape itself, not individual backups on the tape.

So after 3 weeks, the tape will be marked as "Expired" and will be available for re-use (the job will find expired tapes in the media pool, erase them, then proceed to write new data to them).

Likely when you got the "please insert new tape" message, the currently added tape was not usable due to the protection period being enforced (cannot be appended to/overwritten).

Can you describe your tape-out strategy a bit more? Are you rotating the different RDX disks based on day of the week or just whenever full?

[feiaweng]

Hello David,

thanks for the tip about tape protection. Then I can't use this method.
My strategy is as follows:
2 tapes are in the Mediapool Fullbackup
2 tapes are in the media pool Incr_backup
Monday - Friday incremental backup
Saturday full backup
Write to the tapes in the media pool until they are full or no more complete backups fit on them, then overwrite the oldest backup.
The 4 tapes are in the RDX appliance. No tapes are exported or ejected so that nobody has to change tapes.
Each tape remains in the slot in which it is located.

regards
Ralf

[Andreas Neufert]

Just remember that there is a dependancy chain between tapes and between full and incrementals.
If you write like 2 weeks to the first full tape and 2 weeks to the second full tape and mark it protected from overwrite for 3 weeks, then on the 5th weekly full you do not have a tape to write to, as both tapes are still protected.

[feiaweng]

Hello Andreas, can I write a full backup on the first tape for 3 weeks (3 times in totally) and protect it for 3 weeks and then Veeam takes the 2nd tape and writes another 3 full backups on it. Then the first tape should be free to write again since 3 weeks have passed.
If this is not possible, what would be an alternative strategy to write full backups to the tape until it is full and then overwrite the oldest backup.

with kind regards
Ralf

[Andreas Neufert]

I asked the Tape team to comment here as I am not aware of such a situation.

What I know is, that you can select that you write to a specific tape until it is full and then switch to the next one.
But to be able to select this for 2 weeks is I think not possible.

Maybe you could do it with 2 jobs and 2 libraries. First Job will write to first media the first 2 weeks in a month (scheduler setting) and the second job will do it for weeks 3-5 in a month.

Another thing I might do is to write the full + incremental to the same tapes and close the media before the next full.

Anyway I asked the Tape team here to comment.

[david.domask]

> Write to the tapes in the media pool until they are full or no more complete backups fit on them, then overwrite the oldest backup.

I'm afraid this part is not possible. Right now you cannot erase/overwrite a single backup, and I'm not sure this would work given that the backup sizes may differ from run to run.

For example, suppose on the 1st run, the oldest backup is 1.2 TiB in size. We write other backups to the rest of the tape. The 2nd run comes around, and let's assume we could overwrite "just" the oldest backup (1.2 Tib). However, the current size of the backup we need to write is 1.4 TiB -- it wouldn't work because we'd be overwriting data you still want to protect further down on the tape.

With 4 tapes and the configuration you want, you may consider setting the media pool to create a new media set Daily on a given day when you plan to rotate the tapes. Set the tape protection to the same time frame, so that when it's time for a new media set, the tape protection expires. This would allow you to rotate the tapes.

Similarly, you might consider GFS Media Pools and setting specific tapes with the Append backup files to incomplete tapes option enabled. This way you could have a specific tape to capture on a specific time frame and Veeam will continue writing to those tapes during each session, and you can set the tape protection to allow overwrites.

But please note as you're planning, with tape, the retention applies to the tape itself, not individual backups, so you need to plan that once a tape's protection period is expired, the entire tape must be erased/marked as free before next use.

[feiaweng]

Hello all,

sorry for my late response. I was away for a few days. Let me check your answers and try to get it.
I will come back after testing.
Thanks for your help.

with kind regards
Ralf

[feiaweng]

Hello all,

I have now created a media pool that contains all 4 tapes.
Full backup runs on Saturday at 07:00 AM
Incremental backup runs Tue-Sat 5:30 AM
New media set is created on Sunday at 06:00 PM
Retention time is 7 days
But how does it work?
After the first full backup, the tape is protected for 7 days. Next Sunday, when a new media set is created, only 1 day has passed.
I could also create the new media set on Saturday at 02:00 PM to solve this problem.
After the first full backup has been created, the tape is protected for 7 days.
But the incremental backup is written on Tue-Sat. Tuesday would then be written to tape 2, as tape 1 is protected.
Wednesday would be written to tape 3, and Thursday to tape 4.
I don't have enough tapes for this method.

Could this be a possible solution?
I create 4 media pools. And several jobs.
Monthly job - 1st Saturday on pool 1
Monthly job - 2nd Saturday on pool 2
monthly job - 3rd Saturday on pool 3
Incremental job Tue-Sat on pool 4, but these are overwritten with each job.
Before the tape job runs, all VMs and files are written to the NAS, and in the early morning the files are backed up from the NAS to tape.
A retention policy of 14 days is active in the backup job “Backup VMs”.
How can I ensure that the incremental files are not overwritten each time the tape job is run, but that the tape capacity is utilized?
Many thanks for any suggestions.

with kind regards
Ralf

[david.domask]

Hi Ralf,

I think maybe it would be easier to discuss what your ideal tape-out strategy would be, that is, how often do you want to rotate tapes, what should happen when it's time to rotate a tape back in (we should overwrite it or continue where we left off?), things like that.

I recommended GFS before as you could, for example, have a tape for:

Daily
Weekly
Monthly
Yearly

And simply insert the necessary tape for the GFS period on the scheduled day you have in the GFS media pool. With the "Append" option as mentioned previously, Veeam will not close the media sets, and will simply continue to append to the tapes until they no longer have free space.

Alternatively, if you want a scheme where you have a specific tape for each day or each week, multiple media pools and tape jobs can be used like you were thinking. That would be the same as what we have in our Tape FAQ (last Q and A), just instead of swapping daily, you'd swap them on the schedule you desire.

But I'm guessing at your preferences here, so maybe you can tell your preferred tape-out method and we can better advise.

[feiaweng]

Hello David,

we have the following backup hardware:
Primary backup NAS device
Secondary backup (Air-Gap) Quikstation RDX integrated as VTL with 4 drives
Drive 1 - 3.8 TB space
Drive 2 - 1.9 TB space
Drive 3 - 3.8 TB space
Drive 4 - 1.9 TB space


The backup requirements are as follows:
- all tapes are not changed, but remain in the drive so that nobody has to change tapes
- the tapes are not ejected and not exported

The VMs and some files are backed up to a NAS share with a Veeam job
- Saturday evening - full backup
- Mon-Fri evening - incremental backup

Veeam TapeJob "unstructered data"
Backup from NAS share to RDX tape
Sunday morning - full backup
Tue-Sat morning - incremental backup

previous allocation was:
MediaPool Fullbackup - Drive 1 and 2
MediaPool Incr. backup - Drive 3 and 4

About 6 backups fit on the MediaPool Fullbackup.
How can I realize that several backups are written to one MediaPool until there is no more free space?
If this is only possible with GFS, then I will have to do it this way.
Irrespective of this, I have another question:
If I run the Incremental Job on weekdays, and select the option “Do not protect data.....” in the Mediapool Incr. Backup
will the incremental backup be overwritten every time?
E.g. Monday's incremental backup will be overwritten on Tuesday, and Wednesday's incremental backup will overwrite Tuesday's?
Or are the incremental backups of the whole week next to each other?

Many thanks for your support.

regards
Ralf

[david.domask]

1. How can I realize that several backups are written to one MediaPool until there is no more free space?

Use the media set creation method of "Do not create, always continue" so that the tape is written to its capacity. Veeam will notify when a new tape is needed, and/or the email notifications will tell the tapes used for a job and the current tape status (capacity, free space, location, media set).

2. If I run the Incremental Job on weekdays, and select the option “Do not protect data.....” in the Mediapool Incr. Backup will the incremental backup be overwritten every time?

A tape is not eligible to overwrite until its media set closes or it is written to capacity. So within a single job session, if the tape is filled and there is still more to write, that tape will NOT be overwritten automatically until the next job run.

As for your overall request, as noted currently tape protection works against the entire tape, not individual backups on tape. So you will need to determine a schema that best meets your tape retention needs. Since you're using Unstructured Data Backup to tape and only have 4 tapes, I'm not confident that GFS is the best approach as it will not work with Unstructured Data Backup to Tape currently.

You might consider putting the two larger drives into the Full media pool and the two smaller ones into the increment and configure the protection for a week on both and Create New Media Set for the day when the previous tape is full. That way you're effectively rotating two sets of tapes and have protection for the previous week + the current week until it's time to rotate again.

[feiaweng]

Hello David,

I'm not sure if I've understood you completely.
I have now created 2 media pools.
Media-Pool Fullbackup - 2 Tapes a 3,8 TB
Media-Pool Fullbackup - Retention protect data for 1 week
Media-Pool Fullbackup - create new media set every sunday (after backup) this is the biggest time frame I can set

Media-Pool Incr. Backup - 2 tapes a 1,9 TB
Media-Pool Incr. Backup - Retention protect data for 1 week
Media-Pool Incr. Backup - create new media set every sunday (after backup) this is the biggest time frame I can set
If I assume that 3 backups (fullbackup) fit on one tape, shouldn't the media set be closed after 3 weeks (1 fullbackup per week)?
So that it picks up the next tape? Or am I misunderstanding this?
In the current configuration, I see it like this: after a full backup, the media set is closed, and since the tape is protected for a week, it takes the second tape.
The following week, tape 2 is still protected, the protection on tape 1 has expired and it takes tape 1 again.
Should the protection be no more than 7 days, i.e. more like 10 days?

with kind regards
Ralf

[david.domask]

> If I assume that 3 backups (fullbackup) fit on one tape, shouldn't the media set be closed after 3 weeks (1 fullbackup per week)?

The tape will be full at that time, and the media set will be closed on the time that you set (On Sundays in your current scheme). Media Set closing is not the same as the tape being full, but both will require you to have a new/free/expired tape for the job to continue automatically.

However, because the media set is closing weekly and you're doing weekly full backups, after the first week, the media set will be closed, the tape protection period will start, and a new tape will be required.

From your description though, as I get it you just want to fill the tape, move to the next tape, and protect the original tape for 1 week, correct?

So use:

Do not Create Always Continue for the media set creation
Let the job run until the tape is full, Veeam will then move to the next tape, and original tape will be protected for one week.

And you can just repeat this sequence.

[feiaweng]

Hello David,

> From your description though, as I get it you just want to fill the tape, move to the next tape, and protect the original tape for 1 week, correct?
I want to fill the first tape with full backups until no more fit on it then take the next tape from the pool.
Backup here as well until no more backups fit on it and then the first tape can be taken again and overwritten.
Tape protection is not necessary.
should i remove the tape protection again?

regards
Ralf