Discussions related to exporting backups to tape and backing up directly to tape.
Post Reply
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Sanity check - Tape retention period

Post by macademy »

We currently have a retention of 30 days on our backup to disk job and 6 days on our tape media pools.

What we are seeing is that the tape job is now backing up the previous nights job, but also backing up old files from last month.

Reading the help guide section on tape retention, it says "This setting must accord with the retention policy specified for the backup chain that you plan to archive to tape. The retention period for archives on tape must be greater or equal to the time interval for which restore points are kept on the backup repository. In the opposite case, with every new job cycle, Veeam Backup & Replication may write the same set of files to tape, instead of adding new ones."

So, should we set a 30 day retention on our tape media pools too? Do I need to do anything else after that?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

Tapes are considered to be a secondary location for the backup data which in contrast to primary repository guarantees long-term retention. Thus, it's recommended to have tape retention greater than primary repository retention. With such approach you will achieve both: follow best practice and avoid undesired behaviour. Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

Thanks, I've set my media pools to 30 days also.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

Yes, it should help you to avoid unnecessary data duplication. If other questions arise, don't hesitate to let us know. Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

Sorry, I'm going to need more advice on this!

Our Backup to Disk retention period is 30 days, and based on the advice in this thread I changed both our "Daily Incrementals" and "Friday Full Backup" media pools to have 30 days retention also.

Recently, the backup to tape has not started and we get a message - Insert tape to continue.

On looking at the logs, it's seems to be reporting that there are no usable tapes in the loader, but there are three tapes from the Daily Incremental pool currently loaded (we cycle the daily incremental tapes every day). They all have enough free capacity to append to (220GB, 677GB and 308GB free), but are overwrite protected. The amount of data to back up from last night is 66GB, shouldn't it just append to one of the tapes in the loader?

I just changed the Daily Incrementals media pool to have a 5 day retention period then ran the job manually and now it's working.

If we only ever wanted to have 1 months data on both disk and tape, what would our retention settings be on the Daily Incrementals pool? If we run with 30 days on disk, 30 days on Friday Full Backup and 5 days on Incrementals, would that cause problems?

Can I also ask....in the media pools we have a number of tapes that are showing 4.8GB free, so these tapes are full. If the overwrite protection period has passed, will these tapes be overwritten as required?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

The amount of data to back up from last night is 66GB, shouldn't it just append to one of the tapes in the loader?
What mediaset creation settings are specified on the given media pool?
If the overwrite protection period has passed, will these tapes be overwritten as required?
Medium falling out of retention should be overwritten during the subsequent runs.

Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

Both Daily Incremental and Friday Full Backup media pools are set to "Do not create, always continue using current media set".
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

What do you mean by saying over write protected? Have you enabled "protect" option for those tapes? Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

Well, our Daily Incremental pool was set to 30 days retention, based on previous advice (it's now 5).

This morning, it was refusing to load any of the Daily Incremental tapes that were in the loader, I'm guessing this was because the overwrite protection period had not expired. However, there was sufficient capacity on all available tapes, so why didn't it just append the data (appending is not overwriting, right?).

Setting the retention to 5 caused the backup to proceed.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

Hmm, I'm interested in why there is free space left on mediums, if you enable "Do not create, always continue using current media set". With those settings, data should be written to a tape, until it reaches its limit. Or you remove a tape from a device on daily basis? Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

We have:

Monday Diff
Tuesday Diff
Wednesday Diff
Thursday Diff
Friday Week 1
Friday Week 2
Friday Week 3
Friday Week 4
Friday Week 5

The tapes are changed daily, the tape(s) used the previous night are removed from the loader and stored in the appropriate labeled box, this morning we removed two tapes (the first tape must not have had enough free capacity and the backup spilled over onto another tape) and put them in the box labeled "Wednesday Diff". I have inserted the "Thursday Diff" tape today for use tonight.

There are currently two tapes in the loader that are in the daily incrementals pool, one has 263GB free and the other has 677GB free, so it should pick either one of those tapes for use tonight, I'm not sure how it decides! Both are not overwrite protected. There is also always at least one free tape in the loader.

This is the exact same cycle that we used when we were on Backup Exec.

EDIT: I've just marked all of the tapes in the loader as free and moved them to the free pool, apart from the one that was just put in this morning. So right now there are 5 x Free media and 1 x Daily Incremental with 263GB free space.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

It's by design. A tape medium that has been taken offline isn't appendable. Unless a tape job has been suspended since the time cassette was moved offline. Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

One more question, last nights incremental used two tapes (5 & 12), those two tapes are now a media set.

Next Wednesday, those two tapes will be inserted back into the loader, it will continue to append to the second tape, yes?

So, on Thursday when it says that it only used tape 12 for the backup, should we remove only tape 12 or should we remove both tapes 5 & 12 (keeping the media set together).

At some point, tape 12 will be full, it will then overwrite tape 5 again?

Sorry if this is all basic stuff!
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Sanity check - Tape retention period

Post by veremin »

Next Wednesday, those two tapes will be inserted back into the loader, it will continue to append to the second tape, yes?
As I've said, if you take those tapes offline, backup server won't be able to append data to them, once they are inserted back. Depending on specified retention, either they will be overwritten or backup server will request a new media. Thanks.
macademy
Enthusiast
Posts: 27
Liked: 3 times
Joined: Nov 10, 2014 4:07 pm
Full Name: Ian Cullen
Contact:

Re: Sanity check - Tape retention period

Post by macademy »

So, on Thursday, if it says that it only used tape 12 for the backup, should we remove only tape 12 or should we remove both tapes 5 & 12 (keeping the media set together)?

Thanks again.
Dima P.
Product Manager
Posts: 14396
Liked: 1568 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Sanity check - Tape retention period

Post by Dima P. »

Ian,
First, I wonder did you apply retention changes to all existing media or only to new tapes (if you don’t remember you could create your existing media pool from scratch and re add marked as free media - it might do you more good). That might be the reason why you have some space left. Additionally, the append to tape happens only with the last tape in the media set (check the sequence number in the media pool view).

As Vladimir said, once the last tape in the media set is offline - a new media set is started anyway, so previously used tape is no longer appendable. If you want to export media on a weekly/daily basis – its might be better to use a create media set daily at option plus tape job export functionality.

And here is a link to our almighty Helpcenter: Creating Custom Media Pools
nunciate
Expert
Posts: 247
Liked: 39 times
Joined: May 21, 2013 9:08 pm
Full Name: Alan Wells
Contact:

Re: Sanity check - Tape retention period

Post by nunciate »

I have to get in on this one because I have the same issue with duplicate data being written to tape. The way Veeam handles tape out for incremental data is a horrible waste of tape space.
In a real world restoration most people would agree that an incremental older than a week is pretty useless on tape unless you really need the ability to do a point in time restore from tape.
A restore from tape in a disaster situation means I would want the last full backup (weekly) and every incremental up to the current or most recent date (daily). In our situation we would only restore incremental data from tape if we needed a full restore up to the last backup that ran. That is why we send our weekly backups offsite for a month and our daily backups for a week. We keep the extra fulls offsite for the just in case scenario but the most important backups are really the most recent full and incremental backups. We keep local disk for a month as well as that is our quick way to get point in time restores.

This is a pretty common setup so I am not sure why you all chose to force folks to store incremental on tape for the same time retention as our disk backups.

I use LTO6 Tapes, have 4 drives and 4 daily media pools. My incremental backups rarely fill the 2.4Tb that the tapes can hold so I am pretty much wasting a tape or 2 sending that many offsite each day.
If I change my retention on tape to 30 days to match my disk retention that means I will be sending 4 tapes offsite 5 days a week. I will not see those tapes back for a full month and they will barely be used. That is over 80 tapes which are basically wasted money. I also send off about 25 tapes for each weekly full from the weekends. Those stay offsite for 1 month as well except the last week of the month they go off for 3 years. As you can see we use up quite a few tapes.

I would love to see a separate retention on tape that isn't tied to my disk backup retention in any way. It just doesn't make sense.

Edit: One other thing. I was about to switch my daily media pools to a monthly retention and set the option to always continue the media set. My thought was I could at least send my tapes offsite for 7 days, then bring them back and have Veeam append data while leaving the previous backups in place. That way over time I could possibly fill up the tape and let data expire off normally. Sounds like that isn't an option either if you can't append a tape after it has been marked as offsite, even if you put it back in the library and it shows online. Again that doesn't make much sense to me.
Post Reply

Who is online

Users browsing this forum: No registered users and 18 guests