v9 GFS Media Pool & Tape Job

Everything about backing up to tape

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby mario.spaenhoff » Thu Feb 25, 2016 3:24 pm 1 person likes this post

I'd like to add myself to the list of feature requesters for Tape Backup.
    Parallel Processing for GFS
    Daily Incrementals
mario.spaenhoff
Novice
 
Posts: 3
Liked: 1 time
Joined: Fri Jun 22, 2012 10:46 am

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby pkelly_sts » Wed Mar 09, 2016 2:16 pm 1 person likes this post

I'm not yet on v9 but hadn't realised the subtle exclusion of incrementals from GFS tape jobs.

They're also important to me, our current solution (Backup Exec) has the following plan:
4 weeks of "Daily" tapes (Mon-Thu)
4 Weeks of Weekly Tapes (Fridays)
12 Months of Monthly tapes (last Friday of month)
7 Years of yearly tapes (last friday of year)

We also currently ship them off via Iron Mountain but I'm hoping to create a plan that sends backups to tapes on the local site, but also copies them to our remote site (we have lots of tape libraries available!). Having archived backups on two sites means that a site disaster (fire etc.) can never destroy /all/ our archived backups, but we can also save thousands of £ per year in Iron Mountain collection costs as they'd no longer be required.
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 506
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby Mory » Thu Mar 24, 2016 8:49 pm

I too would love to see incremental backups part of the GFS media pool!! This way, it's not necessary to have separate jobs for starters, and then if a weekly backup falls on the same day as the monthly backup, or a yearly backup, no need to make multiple copies.

For example in v8, I had to setup something like this:

Saturday (3/26/2016) we will run a full backup to tape, but because its also the last Saturday in the month, we run a monthly backup, so now I have 10 tapes and many many hours of backing up to tape. With GFS rotation, the weekly backup turns into the monthly backup and I'm not backing up the data twice.

However, not being able to have incrementals included in the GFS media pool, means we need to have a separate job to get the incremental data to tape and offsite each day. Seems like the GFS media pool should have the same options as the GFS backup copy job, that would be great for those of us that just can't get rid of tape! :oops:

Hope that makes sense.

Also, as far as scheduling goes, would love to see a couple options... be nice to have be able to schedule the full and incremental backups at different times. But, I think if GFS supported increments, then using linked jobs would satisfy my situation.
Mory
Novice
 
Posts: 8
Liked: 4 times
Joined: Sat Aug 10, 2013 5:39 pm
Full Name: Chris

Re: v9 GFS Tape

Veeam Logoby Mory » Thu Mar 24, 2016 9:38 pm

cmaier wrote:You are right, but I expect to use backup to tape jobs later, too. So it would be interesting to discuss this anyway because the questions stay the same. The GFS media pools make no sense to me if they don't provide a solution for the daily incremental backups. I think there is no scenario where no incrementals are included in the backup chain. And there is no way to run incremental jobs without a full backup.


This doesn't address any the GFS concerns, as I too want to be able to include incremental backups in the GFS rotation. What I've done though in v8 to backup the incremental files, is created two different file to tape backup jobs. One is a "full" backup that I run on Saturday to backup all *.vbk files (I target the folder I store these in), and then another "incremental" job that backs up the *.vib files. I then have another Monthly file to tape job, that does exactly what the weekly full job does, so I need to run my full backup twice the last saturday of the month. Less than ideal.

So, I think there is a way to backup incremental files and not the full if you use the file to tape job and do not run the "full" backup automatically as part of that job. I'm going to try this with the GFS mediapool - but, I'm having a separate issue with creating a GFS mediapool I just opened a support case on. I get an error that the changer is not found. So I can't confirm if this will work, but I think if I have a GFS mediapool, do the weekly/monthly/yearly here, then do a separate tape backup for the incremental files, this might work for me until incremental files are part of the GFS (please!).
Mory
Novice
 
Posts: 8
Liked: 4 times
Joined: Sat Aug 10, 2013 5:39 pm
Full Name: Chris

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby Daveyd » Sun Apr 03, 2016 4:45 pm

Similar issue here with GFS Media pool. My requirements are simple. Monthly Full backups to tape, retained for 12 months and Yearly Full backups to tape retained for 7 years.

I created the GFS Media pool with the above settings and configured my job to run on the 1st Sunday of the month. For testing purposes, i mapped the Tape job to a Veeam backup job of which I currently have 30 restore points that use daily incrementals with weekly synthetic fulls. The Tape job kicked off this morning at Midnight and has been sitting there idle for the past 12 hours with the job stating "No GFS Candidate for Monthly media set was found for today, waiting" Source backup files detected VBK:0, VBK Map: 0, VIB: 0

I have an open case, 01745711
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby nunciate » Wed Apr 06, 2016 3:41 pm

Just going to jump in on this one. I upgraded to v9 U1 today and was quite disappointed to find out that using a GFS media pool will not resolve my issues as it causes a new issue.

I am very surprised to see that you do not support incremental to tape with the GFS media pool. This is a very common scenario. Almost every place I have ever worked kept Incremental backups offsite for 7 days, Weekly for 1 Month and then Monthly for some longer term and yearly for even longer. In my case Monthly for 3 years and Yearly forever.

It makes absolutely no sense to not keep daily incremental backups offsite for at least 7 days. If I send my Full offsite on Saturday and have a catastrophic failure on Tuesday where I lose my production and backup data I will lose 3 days of data when I restore from tape. I understand this is an extremely rare scenario but it is one that we in IT always plan for and for the life of me I can't understand why you all wouldn't implement it here.

Using the current solution of multiple media pools for various retention is a complete pain in the backside.
First off you have to have a media pool for every retention period so 4 in my case.
Then you have multiply that with a media pool for each tape drive because they aren't smart enough to load balance across drives. I have 4 drives. So that means 16 Media Pools to manage in my case.
If you have multiple jobs using the same pool they all have to wait in line for the pool to be unlocked. What that fixed in v9? If I could simply have 1 media pool for each retention and the jobs could just use whatever free drives are available without locking the pool that would reduce management a bit. (EDIT) After doing some reading and playing around I answered my own question here. Parallel Processing for Tape drives is now available but not for GFS Media Pools. https://helpcenter.veeam.com/backup/hyp ... ssing.html (END EDIT)
Then to make things worse the backup jobs only allow you to choose Daily and Weekly pools so every month I have to edit 140+ jobs and change them from weekly to monthly pools. Then after that weekend I move them all back to Weekly pools.
It really is a lot to have to manage when this could be resolved by simply adding daily backups to the GFS rotation.
nunciate
Expert
 
Posts: 139
Liked: 23 times
Joined: Tue May 21, 2013 9:08 pm
Full Name: Alan Wells

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby StanO » Mon Apr 25, 2016 7:30 pm

Please include incremental to tape with the GFS media pool in a (hopefully soon to be released) future version. It's a bit surprising it has been omitted.

Thanks,

Stan
StanO
Enthusiast
 
Posts: 27
Liked: 4 times
Joined: Mon Jan 04, 2010 4:14 pm
Full Name: Stan

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby StanO » Sat May 07, 2016 9:04 pm

Having worked a bit more with the copy to tape implementation in VB&R, I think it would be very beneficial to include an option for GFR copy to tape function that not only includes daily incremental backups to tape, but also allows for all incremental backups since the last full backup to be included on each daily tape (similar to a differential backup). Further, if a tape is available in the library is in the proper media pool and media set for a job that is selecting a tape to write to, select the tape with appendable space rather than selecting an empty tape for the next copy job. I've observed that currently if a tape in the appropriate media pool with appendable space is available in the library it will be used. If, however, that tape was previously not available in the library when a tape was needed that tape is no longer appendable.

What I'm striving for is the ability to rotate tapes out of the library on a daily (weekly, monthly) basis (for off-site data protection) and still be able to use the full space available on each tape. For example:

Four tapes, LTO701, LTO702, LTO703 and LTO704:

Friday (Week 1)
Weekly full on LTO701
Write VBRJob01W01.vbk

Saturday
Daily incremental on LTO702
Write VBRJob01W01D01.vib

Sunday
Daily incremental on LTO702 (appended)
Write VBRJob01W01D01.vib, VBRJob01W01D02.vib

Monday
Remove tape LTO701 from library for Weekly offsite
Re-insert tape from previous week's offsite to library
Remove tape LTO702 from library for daily offsite
Daily incremental on LTO703
Write VBRJob01W01D01.vib, VBRJob01W01D02.vib, VBRJob01W01D03.vib

Tuesday
Remove tape LTO703 from library for daily offsite
Re-insert tape LTO702 to library
Daily incremental on LTO702 (appended)
Write VBRJob01W01D01.vib, VBRJob01W01D02.vib, VBRJob01W01D03.vib, VBRJob01W01D04.vib

Wednesday
Remove tape LTO702 from library for daily offsite
Re-insert tape LTO703 to library
Daily incremental on LTO703 (appended)
Write VBRJob01W01D01.vib, VBRJob01W01D02.vib, VBRJob01W01D03.vib, VBRJob01W01D04.vib, VBRJob01W01D05.vib

Thursday
Remove tape LTO703 from library for daily offsite
Re-insert tape LTO702 to library
Daily incremental on LTO702 (appended)
Write VBRJob01W01D01.vib, VBRJob01W01D02.vib, VBRJob01W01D03.vib, VBRJob01W01D04.vib, VBRJob01W01D05.vib, VBRJob01W01D06.vib

Friday (Week 2)
Remove tape LTO702 from library for daily offsite
Re-insert tape LTO703 to library for next daily
Weekly full on LTO704
Write VBRJob01W02.vbk

Saturday
Daily incremental on LTO703 (appended)
Write VBRJob01W02D01.vib

Sunday
Daily incremental on LTO703 (appended)
Write VBRJob01W02D01.vib, VBRJob01W02D02.vib

Monday
Remove tape LTO704 from library for Weekly offsite
Re-insert tape LTO701 from previous week's offsite to library
Remove tape LTO702 from library for daily offsite
Daily incremental on LTO703 (appended)
Write VBRJob01W02D01.vib, VBRJob01W02D02.vib, VBRJob01W02D03.vib
.
.
.
Friday (Week 3)
Remove tape LTO703 from library for daily offsite
Re-insert tape LTO702 to library for next daily
Weekly full on LTO701 (appended)
Write VBRJob01W03.vbk

Monthly, Quarterly and Yearly backups would build on this same pattern. Granted this approach results in multiple copies of the same incremental backup on each tape, but I believe it represents the most efficient use of tape when tapes are rotated out and back in to a library.

If this type of functionality and efficient use of tapes is available currently, please provide clarity as it eludes me.


Thanks,
Stan
StanO
Enthusiast
 
Posts: 27
Liked: 4 times
Joined: Mon Jan 04, 2010 4:14 pm
Full Name: Stan

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby Dima P. » Mon May 09, 2016 1:41 pm

Hello Stan,
If, however, that tape was previously not available in the library when a tape was needed that tape is no longer appendable.

That is true. If the latest media set’s tape is not present when tape job starts, another tape will be used instead. Previous one will be counted as ‘closed’ to keep the media set sequence.
If this type of functionality and efficient use of tapes is available currently, please provide clarity as it eludes me.

While there is no option to backup incrementals with the GFS media pool, I was thinking that you could use file to tape job instead of the backup to tape job. Select the folder with the desired incrementals and add include mask for .vib and .vbm files. Schedule this job to run after your GFS full backups. No need to create additional media sets, so ‘keep current media set’ should be used.
Dima P.
Veeam Software
 
Posts: 6494
Liked: 452 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby StanO » Mon May 09, 2016 6:15 pm

Dmitry:

Thanks for the suggestion regarding incrementals. Will I not, however, run into the same limitation with a media set being counted as 'closed' if I alternate tapes in the library on a daily basis?

Dima P. wrote:If the latest media set’s tape is not present when tape job starts, another tape will be used instead. Previous one will be counted as ‘closed’ to keep the media set sequence.


I believe this mechanism is the reason for the inefficient use of tapes when they are rotated out and in of the library. Changing this would be a dramatic improvement in my opinion.

Thanks,

Stan
StanO
Enthusiast
 
Posts: 27
Liked: 4 times
Joined: Mon Jan 04, 2010 4:14 pm
Full Name: Stan

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby Dima P. » Mon May 09, 2016 9:35 pm

Stan,

Indeed, the same logic is used for both F2T and B2T job. May I ask why you remove the media that still have some free space from the library?
Dima P.
Veeam Software
 
Posts: 6494
Liked: 452 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby StanO » Mon May 09, 2016 11:37 pm

We remove daily tapes from the library for a day for offsite storage (to be completely honest, dailys go in a pair of concentric fire safes) and return them the following day. Similarly, we remove weekly backup tapes for a week (truly offsite). We would remove monthly backup tapes for a week as well (substituting the monthly for that particular week's offsite weekly), if our current implementation didn't result in both a monthly tape and a Weekly tape being produced on the last Saturday of the month.

Thanks for the opportunity to log improvement requests,

Stan
StanO
Enthusiast
 
Posts: 27
Liked: 4 times
Joined: Mon Jan 04, 2010 4:14 pm
Full Name: Stan

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby kappa85 » Wed Jun 01, 2016 3:06 pm

I apologize, but I haven't understood how to solve the problem of GFS rotation mixing full backups and incremental. As other people, we had a Backup Exec working with
daily incrementals
full weekly (each friday) - 4 week retention
full monthly (last friday of the month) - 4 month retention
full yearly (last friday of the year) - 1 year retention
I can't understand how to replicate in easy way this rotation with Veeam v9. If I use GFS I cannot write incrementals. If I use a normal backup-to-tape job I can't set period retention and I can't say "ok this is the last friday of the month, you have to run a full monthly and not a full weekly". Can someone help me?
kappa85
Influencer
 
Posts: 21
Liked: never
Joined: Wed Oct 29, 2014 12:31 pm

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby pkelly_sts » Wed Jun 01, 2016 3:23 pm 1 person likes this post

Unfortunately you'll have to configure at least two tape jobs because (currently) GFS tape jobs ONLY process full backups, they won't touch incrementals (hopefully pressure is increasing to address this short-coming).

1) What is your source backup method? (e.g. Reverse Incremental, Forward Incremental, Forever Forward Incremental)
2) How many restore points (RPs) is your source job currently configured with?
2) What is your desired daily (incremental) *retention* requirement? (e.g. how long do you need to retain daily incrementals on tape)

One of the important things I've learnt today is that, if your SOURCE job is a Forever Forward incremental job, then your tape retention plan for that source job MUST be configured such that it retains the same number of restore points as the source job - this is a very significant point (limitation, in my opinion) to consider!

Answers to these would potentially then assist in an answer to your GFS requirements (which sound relatively straight-forward so far).
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 506
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: v9 GFS Media Pool & Tape Job

Veeam Logoby kappa85 » Mon Jun 06, 2016 8:03 am

Hello,
1) since I don't have any active full or synthetic full, my source backup method should be forever forward incremental
2) I have several source jobs based on the type of virtual machine, and each one has a different number of RP retained (for example, exchange retains 30 days, sharepoint 15days, other VMs only 7 days...)
3) I'd like to match the previous retention I had with backup exec, I keep my daily incrementals for 4 weeks
kappa85
Influencer
 
Posts: 21
Liked: never
Joined: Wed Oct 29, 2014 12:31 pm

PreviousNext

Return to Tape



Who is online

Users browsing this forum: No registered users and 16 guests