New to tape backups and confused how it works.

Discussions specific to tape backups

New to tape backups and confused how it works.

Veeam Logoby morty » Tue Apr 17, 2018 7:01 pm

I'm trying to implement AWS Storage Gateway VTL.

I have it all setup and seemingly working but I don't really understand what the strategy I should use.

My regular backups have a "Daily" backup to disk job that does a backup once daily of 20 VMware Virtual Machines using the forever-forward incremental method using per-vm files and has 30 days of restore points. The size of this data for 7 days of restore points (1 full and 6 increments) is about 4.5TB.

I'm required to have at least 7 business days offsite (7 restore points). I'm not archiving past 7 days if I don't have to so I haven't been exporting tapes.

Right now I have 10 tapes 1TB in size each all in one media pool. So I have a media pool of 10TB (I don't want to use more than 10TB because I have an AWS budget of $250/month). I really don't understand media sets either but I've created the media pool to never create a new media set and to always use the same one. Is this right? Also I set it to protect data for 7 days. I can't seem to set how many restore points to keep for the tape job so does the tape job just forever keep copying data until it fills the entire media pool and then start overwriting old data assuming it was written past the protection time (7 days)?

What tape strategy should I be using (number of tapes, size of tapes, media pools, media sets, protection, etc)?

Thank you in advance.
morty
Influencer
 
Posts: 20
Liked: never
Joined: Thu Apr 05, 2018 8:34 pm
Full Name: Ryan Mortier

Re: New to tape backups and confused how it works.

Veeam Logoby Dima P. » Wed Apr 18, 2018 9:59 am

Hello Ryan,

morty wrote:I really don't understand media sets either but I've created the media pool to never create a new media set and to always use the same one. Is this right?

Yes. This option allows you to either append or not append to previously written tape media (in case it's still in the library). Keep it always continue if you export your emulated media to amazon on a weekly basis.

morty wrote:Also I set it to protect data for 7 days. I can't seem to set how many restore points to keep for the tape job so does the tape job just forever keep copying data until it fills the entire media pool and then start overwriting old data assuming it was written past the protection time (7 days)?

This is a retention for your tape media and not the backup file. Tape job pulls the entire backup chain from your disk storage and keeps it for the set amount of time. If you are exporting tape media to amazon, I'd rather use 'do not overwrite' option. After tapes are exported to amazon most likely you wont export them back for tape retention to apply (better to kill the tape from amazon to avoid the download costs and then remove it from Veeam B&R console)
Dima P.
Veeam Software
 
Posts: 7607
Liked: 548 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: Prague
Full Name: Dmitry Popov

Re: New to tape backups and confused how it works.

Veeam Logoby morty » Wed Apr 18, 2018 12:47 pm

So if I never want to export tapes, I just want to keep using the same tapes and media pool over and over to only have at least 7 days of restores, how would that look? Is that even possible? I feel like maybe tape is the wrong choice now if it's only for archiving. I just simply want 7 restore points off site.
morty
Influencer
 
Posts: 20
Liked: never
Joined: Thu Apr 05, 2018 8:34 pm
Full Name: Ryan Mortier

Re: New to tape backups and confused how it works.

Veeam Logoby morty » Wed Apr 18, 2018 6:41 pm

What I'm trying to achieve here is essentially just having a 10TB volume (media pool) with the latest 7 restore points from the backup. I looked into AWS Storage Gateway Volumes (both cached and stored) but they seem risky to use because if Veeam reads data from the volume (for example to merge an incremental into a full) then I might get charged for requests which are very expensive. Is tape capable of achieving what I'm looking to do while taking advantage of forever forward incrementals and virtual full backups so that I'm not copying my full data-set more than once (the initial run)? We have a 100mbit upload connection so xfering an entire full of ~4.5TB takes several days.
morty
Influencer
 
Posts: 20
Liked: never
Joined: Thu Apr 05, 2018 8:34 pm
Full Name: Ryan Mortier

Re: New to tape backups and confused how it works.

Veeam Logoby L33T » Thu Apr 19, 2018 8:35 am

If your full backup is 4.5TB and your media is 1tb you`d need 35 tapes to run a 7 day rotation, depending if they get compressed it could be a bit less. We use a 28 day rotation with a monthly tape that gets pulled out of the media pool at the end of each month. The retention policy is set for 28 days so in your case 6 days would need to be set, we create a new media set for every backup session (in the job settings).

Ours is setup very simple but we use a LTO7 media library so capacity and performance isn`t an issue. I get what you are saying about using a pool to rotate X amount of days within it but personally an individual full backup per day on to tape is the most fail safe way of ensuring consistency.
L33T
Enthusiast
 
Posts: 45
Liked: 3 times
Joined: Mon Apr 18, 2016 10:22 am
Full Name: V334M

Re: New to tape backups and confused how it works.

Veeam Logoby Dima P. » Thu Apr 19, 2018 8:53 am

morty wrote:What I'm trying to achieve here is essentially just having a 10TB volume (media pool) with the latest 7 restore points from the backup. I looked into AWS Storage Gateway Volumes (both cached and stored) but they seem risky to use because if Veeam reads data from the volume (for example to merge an incremental into a full) then I might get charged for requests which are very expensive.

Yes, secondary copy to Gateway VTL is better as your offsite backup chain won't be modified.

morty wrote:Is tape capable of achieving what I'm looking to do while taking advantage of forever forward incrementals and virtual full backups so that I'm not copying my full data-set more than once (the initial run)? We have a 100mbit upload connection so xfering an entire full of ~4.5TB takes several days.

If you want to keep the last 7 days of backups you have write virtual full backup anyway at the beginning of every new week.

morty wrote:I just want to keep using the same tapes and media pool over and over to only have at least 7 days of restores, how would that look? Is that even possible?

Sure, but in case you want to cyclically overwrite the tape i'd recommend you to use 'Creating a new media set for each backup session' option with 7 days retention (with such setup every tape is going to expire on the planned day). Ideally one job run will use one tape but you might need to create additional tapes if more capacity is required for specific job run. Additional you must schedule virtual full backup to run at the begging of the week. The workflow should look like this:

Week 1: Tape 1 (Virtual Full) < Tape 2 (Increment) < Tape 2 (Increment) < Tape 3 (Increment) < Tape 4 (Increment) < Tape 5 (Increment) < Tape 6 (Increment) < Tape 7 (Increment)
Week 2: Tape 1 (Virtual Full) is expired and overwritten by virtual full (at this point all tapes with incremental from Week 1 are not operational since full backup was erased) < Tape 2 is expired and overwritten by incremental job run < etc
Dima P.
Veeam Software
 
Posts: 7607
Liked: 548 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: Prague
Full Name: Dmitry Popov

Re: New to tape backups and confused how it works.

Veeam Logoby morty » Thu Apr 19, 2018 12:42 pm

L33T wrote:If your full backup is 4.5TB and your media is 1tb you`d need 35 tapes to run a 7 day rotation, depending if they get compressed it could be a bit less. We use a 28 day rotation with a monthly tape that gets pulled out of the media pool at the end of each month. The retention policy is set for 28 days so in your case 6 days would need to be set, we create a new media set for every backup session (in the job settings).

Ours is setup very simple but we use a LTO7 media library so capacity and performance isn`t an issue. I get what you are saying about using a pool to rotate X amount of days within it but personally an individual full backup per day on to tape is the most fail safe way of ensuring consistency.


Keep in mind this is a VTL in the cloud. We definitely can't be copying a full backup to the cloud everyday.

morty wrote:Is tape capable of achieving what I'm looking to do while taking advantage of forever forward incrementals and virtual full backups so that I'm not copying my full data-set more than once (the initial run)? We have a 100mbit upload connection so xfering an entire full of ~4.5TB takes several days.


If I instead chose to keep 30 days offsite then I'd only have to do a virtual full once every 30 days then, is that correct?
morty
Influencer
 
Posts: 20
Liked: never
Joined: Thu Apr 05, 2018 8:34 pm
Full Name: Ryan Mortier

Re: New to tape backups and confused how it works.

Veeam Logoby Dima P. » Thu Apr 19, 2018 1:29 pm

morty wrote:If I instead chose to keep 30 days offsite then I'd only have to do a virtual full once every 30 days then, is that correct?

Yes. Basically, you will get the my schema from the post above extended from 7 days to 30 days.
Dima P.
Veeam Software
 
Posts: 7607
Liked: 548 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: Prague
Full Name: Dmitry Popov

Re: New to tape backups and confused how it works.

Veeam Logoby morty » Thu Apr 19, 2018 2:09 pm

Great, I'll give that a try then, thank you.
morty
Influencer
 
Posts: 20
Liked: never
Joined: Thu Apr 05, 2018 8:34 pm
Full Name: Ryan Mortier


Return to Tape



Who is online

Users browsing this forum: No registered users and 5 guests