Tape retention policy and incomplete chains

[pzaloum]

Hi
I am looking at my media pool retention policies and something has annoyed me for a while:

In one case, I am taking daily tape archives of my Saturday-Seeded-Reverse-Incremental backups. So, if you picture the version chain, every Saturday I have a full (usually synthetic, quarterly actives) and every subsequent backup session I have an incremental. Nightly, I archive this backup-chain to a media pool with a 4 week retention. That's all fine. At any given time, my tapes are good from 4 weeks ago.

My annoyance:
As soon as my tape containing the vbk's becomes expired, it is at risk of being overwritten. At that time, any vib files from the days following, until the next full, become incomplete and unusable, even if the tape they reside on has a non-expired state.

Does there exist (if not, COULD there exist) with media pools the same logic behind tape media expiration that exists with the forward incremental retention? That is: The full must be kept as long as dependent incrementals are still valid even if it has gone past its retention value. Once the tape that the last incrementals of a backup chain are stored on expires, then any pending expirations can take effect.

I suppose an easy way around it for my case is to add an extra week to the retention policy, but the scheduling of a full doesn't necessarily coincide with tape schedules so adding a week might create extra history for nothing.

Has this already been discussed and dismissed?

[Samba222]

Hi,

thx for your post Patrick, i thought i am the only one.
Yesterday i discovered exactly the same problem and was really shocked.
I have a Tape Media Pool with 8 week retention for all our standard VMs. This Pool is used by a Tape Job with Full and Incremental Backups. The Tape Job contains multiple Backup-Jobs (Forw. Incr. with different times of Active Fulls).

I took a closer look at the expired tapes and discovered many vbk's inside which are expired but the depending vib's are on non-expired tapes within the 8 weeks retention.
As soon as the expired tape is overwritten i have the problem with: "Full backup file not found" when i try the restore

Like pzaloum, i also miss the same logic as on B2D-Device, there never a vbk expires in a chain.

The answer from support (Case #02181598) is to change from "... always continue using current media set" - but i think it won't solve the problem, because i have also Pools with weekly new media sets and the SAME problem.

Thx for your help,
regards, Ronald

[pzaloum]

Thanks for the feedback

I confirm that using the same media set has no effect on expiration. I have sets that rotate and sets that never rotate, both show the same behavior. If it was intended to be the case that within a media set, a backup chain is protected then it doesn't work as designed in multiple tape media sets. Single tape maybe but that's not applicable.

[Samba222]

Hello,

please, this topic is really important to us - the support couldn't help me, the answer: it would be a feature to synchronize the tape retention and the backup chains manually. Really???, i can't believe this.

Thx for helping!

Regards,
Ronald

[Samba222]

push

Thank you!

[lyapkost]

Hi.

Tape job's behaviour in the described situation is addressed in user guide (see 'Tape Retention and Disk Retention'). In brief: all files erased from the tape but left in repository will be copied to tapes again during the next run of tape job.

[Samba222]

Hi Konstantin,

sry but exactly this is not described in the user guide, i read it many times.
Again the problem, hopefully a little bit easier shown:

Backup_Job 1 - Forward Inkr. with Active Fulls on Monday -> 14 RP -> daily running
Backup_Job 2 - Forward Inkr. with Active Fulls on Tuesday -> 14 RP -> daily running
Backup_Job 3 - Forward Inkr. with Active Fulls on Wednesday -> 14 RP -> daily running .... and so on

-> they all are in the SAME Backup to Tape Job with one Media Pool (...always continue using crurrent media set, Protect Data for 8 Weeks)
-> Full and Incrementals stored in the same Media Pool
-> this Job runs after the daily Backup_Jobs X

So all the vbk's and vib's are stored for 8 weeks on tape, thats fine, but after 8 weeks when the first tapes expiring, they contain vkb's and all the following tapes, that are not expired until now, containing vib's which i dont get recovered because they need the depending vbk's! There is no check or Logic whether the vbk is still needed or not - it will be hard expired on deadline.
I thought it was the same Logic as on B2D where a vbk never expire (or will be deleted) in a CHAIN - but isn't it??

In brief: all files erased from the tape but left in repository will be copied to tapes again during the next run of tape job.

So i have 14 RP (Corresponds to 2 weeks) on B2D and 8 Weeks on Tape - the tape job cannot copy the missing files and also its not the problem...

Thank you,
Regards Ronald

[lyapkost]

There is no check or Logic whether the vbk is still needed or not

That is correct.

Although you need one more circumstance for that to happen: incremental backups should be copied to another tape than full one, otherwise tape reteintion period will be updated after incremental backup is copied. Not very often situation but to be absolutely sure you always have restore points for the whole 8 weeks on tape you need to extend tape retention period for another week as Patrick suggested.

[pzaloum]

That's not quite true. Although I will point out that saving incrementals to a different media pool is actually an option in Veeam, which should highlight this issue even further, all it takes is large enough incrementals or large enough vbk to be more than the size of one tape, after which the expiration is only as late as the last incremental that is on the same tape as the vbk.

[Samba222]

Hmm, if that is the only answer then it's, in my opinion, really badly implemented
...and also the documentation, nowhere attention: backup chains are not checked on expiration!

[IHeartCats]

Just unearthed this behaviour while I was checking the status of restore points on tape. Luckily the restore points weren't needed I was just monitoring as we have just started doing fulls and incs to tape.

I had stupidly assumed that tape jobs would employ the same logic as the the backups to disk in that it wouldn't get rid of a full until all the dependent incrementals were aged out too.

So to ensure 2 weeks retention I really need to set 3 weeks retention on the media pool as a work around?

Thanks

[veremin]

What kind of backup job is selected as a source for backup to tape job? Forward incremental, forever incremental, reversed incremental? Also, what media set creation and overwrite protection settings are specified on media pool level? Thanks.

[IHeartCats]

What kind of backup job is selected as a source for backup to tape job? Forward incremental, forever incremental, reversed incremental? Also, what media set creation and overwrite protection settings are specified on media pool level? Thanks.

Hi,

Source job is a forever forward incremental (We have only just changed 1 week ago to forever forward from forward incremental).

Tape job is continuous and copies the daily incs and with a synthetic full configured once a week.

Uses a media pool that now has 3w retention but was configured for 2 weeks until I came across this issue. We need 2 weeks of usable restore points on tape.
Media set creation rule is: Do not create, always continue using current media set.

Any advice appreciated.
Thanks

[lyapkost]

So to ensure 2 weeks retention I really need to set 3 weeks retention on the media pool as a work around?Thanks

Hi. 3 weeks retention works in your case. If a tape with a full backup is erased in 3 weeks it will cause lost of maximum next 6 daily incrementals since synthetic full to tapes is scheduled weekly. So there will always be restorable points for at least 2 weeks.

[Samba222]

It sounds so amateurish ".... works in your case" ... where is the problem to implement the same logic from disk for tape? I thought this is an enterprise product?!
Everytime my colleges ask me why i set the tape retention to 10 weeks, i have to say: ok, i need only 8 weeks retention but there is no logic for dependencies for tape backup rotations... and they are laughing...

[lyapkost]

Ronald, we have many feature requests but our resources are limited. Your point is quite clear. We will consider implementing such logic in one of our future releases. Thanks.

[IHeartCats]

It sounds so amateurish ".... works in your case" ... where is the problem to implement the same logic from disk for tape? I thought this is an enterprise product?!
Everytime my colleges ask me why i set the tape retention to 10 weeks, i have to say: ok, i need only 8 weeks retention but there is no logic for dependencies for tape backup rotations... and they are laughing...

I don't understand why backup to disk uses the number of restore points as a measure (Meaning we often have to keep more than the calendar amount we need just incase we run 1 or 2 extra backups) when tapes use a calendar based system.

Wouldn't it be best to only use the number of restore points or the calendar system across both (With similar retention logic as the incremental use on disk) by default. Maybe let the user choose which system or to mix modes.

All most people care about is "how many days/weeks/months" can we go back. It takes a bit of explaining to say "we keep 10 restore points, which is 10 days on disk as scheduled but it could be one hour on disk if someone runs the backup 10 times manually by accident"

H