Feature Request (Tape and Forever Incremental)

Everything about backing up to tape

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby newfirewallman » Wed Apr 29, 2015 12:04 pm

Dave338 wrote:I can't have 2 full backups transfered to tape and so the forever incremental modifies the vbk every day (merging oldest incremental), there are two vbk files copied to tape, the oldest point in the chain, and the synthesized one (plus the incremental files)
I've using reverse incremental to avoid this problem and copy to tape only the full backup, that is the newest one, but It stresses my backup target too much=slow.

Regards.


I also had to use reverse incremental to avoid that and also to work around the shortcomings of tape backups. But it also stressed my backup target to much with 5-6 jobs totaling 10-15TB. Forever incremental is what I need (still slow when merging, but better)

So will there be a fix to backup to tape with synthetic and still allow the daily incremental to disk?
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby v.Eremin » Wed Apr 29, 2015 2:27 pm

Kindly, open a ticket with our support team and provide case number here. It's getting hard to understand through forum correspondence what goal you're trying to achieve and how. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby newfirewallman » Wed Apr 29, 2015 2:51 pm

New Case created.
00892770
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Dave338 » Wed Apr 29, 2015 3:29 pm

I opened a case some weeks ago and the final response was to use reverse incremental and wait for a patch.

Now the patch is released but this issue remains present.

Is simple, there's no way to have incrementals and a simple backup to tape of the most recent restore point, or I haven't found it. You get your full backup copied (which is from some days ago), or many full backups, or the full+synthesized, or also the vib's.... for me, used to other backup softwares ir nearly impossible to understand how it works xD

The option to backup only latest chain that appears if you have more than one full on the target seems the salvation, but I'm testing and seems only working on the first run of the job...

Is any difficult to have an option that makes the tape backup copy ONLY the latest full backup, independently of your backup strategy or type??

Regards :)
Dave.
Dave338
Enthusiast
 
Posts: 35
Liked: 5 times
Joined: Tue Jan 27, 2015 12:21 pm
Full Name: David

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Dave338 » Wed Apr 29, 2015 5:51 pm

Mmmm... after thinking about it for a while... I'll try to resume the "problem"

Problem occurs basically when using Forever Forward. One your retention period is reached, the full vbk file gets modified every day (or backup), merging it the older incremental, so veeam thinks is a new vbk and copy it to tape with your synthesized full, so finally 2 fulls on the tape, plus the incrementals because you had to enable the "process incrementals". This last problem is solved in theory with patch 2 (any confirmation?), so now you don't have to enable the "process incrementals" option to enable synthesized. But the initial problem is solved? Or I will continue getting 2 fulls on tape every week because my "old" vbk is modified?

With reverse incremental, it works well until you make an active full, because also the vbk is modified every day, and when you make an active full, you have two new vbk to copy on tape, unless you make the tape backup between your last incremental and the active full, which is a very narrow windows. So again 2 full backups on tape.

The only way seems to have Forward+active fulls on friday and make the tape copy at weekend, after the active full is done, so it will copy only the new full backup because previous one is supposed to have been copied week before and veeam will dismiss it. I think this one should work and probably is the method I'm going to try, because I'm not comfortable with the reverse incremental method without doing any active fulls, or doing it once a year or so... (I can't be making manual tape backups every week/month). The trick is to start the strategy on friday so the first full backup will be copied to tape that weekend, and next week only new active full will be copied to tape. If you make your first full at Tuesday for example, and have another active full at Friday, on Sunday your tape job will copy again two full backups. Only at first run but is annoying everyway...

I expect it to be much more clear now...
Is all this stuff correct??? xDDD

Regards.
David.
Dave338
Enthusiast
 
Posts: 35
Liked: 5 times
Joined: Tue Jan 27, 2015 12:21 pm
Full Name: David

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby newfirewallman » Wed Apr 29, 2015 6:11 pm

I had to use forever incremental with transform. As long as the my backup storage was fast enough it would be fine because the vbk wouldn't change, but once a week. It became a problem when my file server backups were large enough with enough changes that it wouldn't finish transforming in time for the backup to tape to run.

With Update 2 it doesn't seem to pull two VBK or process incremental's, which is very good as the correct data and amount of data is going to tape. The only issue that continues is the fact that the vbk is changed everyday once your at the retention period (when using forever incremental) and this WILL break the tape job when it is running.
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby newfirewallman » Thu Apr 30, 2015 12:33 am

This is my recent update from my ticket:

This is intended behavior. The tape job will always fail if the backup job kicks off. There are a couple things we can do to mitigate this, though.

1. Continue to disable your standard jobs when the tape job is running.
I don't want to miss backups, nor do I want to have to manually monitor my backups with so much intervention. I have to wait for merging to finish and then start a job while disabling other jobs. It isn't consistent and should be expected.
2. Rethink the job schedules, so that, for example, the tape job runs on Saturdays, and the backup job does not.
If you are a 24/7 shop or even just a 7 day a week shop this isn't acceptable. This is the method i'm shooting for at the moment by just missing a backup on sunday night. It would also help and be a slight work around if i could have the daily backup start at different time on different days. For instance if i could start my saturday backup at 1pm instead of 9pm like a normal day? This would give it more time finish and merge, then i could start the tape job.
3. Change the backup method of the backup job to either Reverse Incremental, or Incremental with weekly Active Fulls. If you make this change, the tape job will not have to do a synthetic and thus will finish much faster.
Revere incremental is no good because it has the same issue, the VBK is constantly changing daily.
Incremental with transform work, until the IO become and issue and it takes DAYS to transform and get caught up and then i can run my tape job. In my instance it was taking 4-6 days to transform all my data to get caught up.
Active fulls would work if Veeam would truly grab only the latest VBK, but my experience hasn't been great with that. Might the first backup job, but it has been inconsistent and sometimes grabs both. Having 1 backup job that is 9.5TB takes a lot more space on disk than i would like when i have to keep two full VBK's and all the incrementals, plus the extra tapes it takes when veeam decides to backup both VBK's


Additionally, there were very significant improvement with merges in Update 2, so if you don't have that, yet, I would strongly recommend applying it to shorten the amount of time the synthetics take.

Thanks,
Greg Bean
Veeam Support
newfirewallman
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Tue Jan 20, 2015 12:08 pm
Full Name: Blake Forslund

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby michaelryancook » Thu Apr 30, 2015 5:39 pm

I am in the same boat and believe there is an issue in the way the backup to tape is handled within Veeam. There needs to be a way to have tape jobs only write the most recent .vbk to tape. My expectation was that if I am using reverse incremental with monthly active fulls when my backup to tape job runs it should either copy my most recent .vbk produced by the reverse incremental job or the .vbk produced by the active full based on whichever was more recent. Not write both to tape which requires twice the number tapes. I'm thinking the backup job should have a checkbox saying active fulls only which would let it write just the most recent active full .vbk and/or a "copy every" interval like backup copy jobs do which would dictate how far back it should search for .vbks to write to tape and would only write one per interval. I think these options would address this issue for reverse incrementals but not other methods.

Thoughts?
michaelryancook
Expert
 
Posts: 113
Liked: 14 times
Joined: Tue Nov 26, 2013 6:13 pm
Full Name: Michael Cook

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby justind » Fri May 01, 2015 2:06 am

The Veeam way to address this is to re-create your tape job and choose Latest backup chain or something similar. Just do this before every backup cycle and you're sweet.

There is another thread that has been going on about this forever and that is the solution... or migrate to the forever incremental.
justind
Enthusiast
 
Posts: 29
Liked: 4 times
Joined: Sun Nov 09, 2014 4:20 am
Location: Australia

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby michaelryancook » Fri May 01, 2015 2:55 pm

Hmm, I'm wondering if the choose latest is selectable via the powershell snapin? If so I could write a script that removes the one tape job and creates a new job with that option selected then have it scheduled monthly.
michaelryancook
Expert
 
Posts: 113
Liked: 14 times
Joined: Tue Nov 26, 2013 6:13 pm
Full Name: Michael Cook

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Dave338 » Wed May 06, 2015 6:48 am

I have re-thinked my last post and maybe I have found a solution.

Actually I have an 8 point retention in my backup job and a 2 daily r.p., 1 weekly and 1 monthly on a GFS backup copy job, so If I do an active full, next week I would have two fulls on my folder, which is undesirable.

I'm going to move to 10 daily restore points on my GFS, eliminate the weekly (with 10 dailys it has no sense, because it would be redundant), and maintain my monthly, leaving only two rps in my backup job.
With that configuration, if I do the active full on Monday, on Friday I'll have only one full backup, and so my tape backup will have no problems.

I'll post my findings If I change again my strategy ,hjehjehe

Regards.
Dave338
Enthusiast
 
Posts: 35
Liked: 5 times
Joined: Tue Jan 27, 2015 12:21 pm
Full Name: David

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Dave338 » Wed May 06, 2015 9:41 am

Oh I've just remembered... If I do that, my monthly backup will be syntethised 10 days later than I want, after the retention date has been expired... another effect that I don't understand about the GFS backup copy policy... but I won't have the downside of having two full backups on the backup job chain when tape job kicks on, so I'll continue with this configuration.

David.
Dave338
Enthusiast
 
Posts: 35
Liked: 5 times
Joined: Tue Jan 27, 2015 12:21 pm
Full Name: David

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Greg.Bean-DeFlumer » Thu May 07, 2015 1:55 pm

What Blake is asking for, specifically, is this:

There would be an option that would cause the source backup job to check if the tape job is currently running when the source job kicks off. If the source job detects that the tape job is currently running, the source job would wait to actually start until the tape job is completes, therefore not causing the tape job to fail.
Greg.Bean-DeFlumer
Veeam ProPartner
 
Posts: 15
Liked: 4 times
Joined: Thu Sep 18, 2014 8:26 pm
Full Name: Gregory Bean-DeFlumer

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Dima P. » Thu May 07, 2015 5:56 pm 1 person likes this post

Hello guys,
Mind me joining back to the discussion I’ve previously abandoned, to sum up Update 2 synthesized full backup logic.

Schedule:
-The synthesized full backup schedule equals the day when the full backup from the source chain should be created. It’s not a backup job schedule trigger – it’s just an actual pointer to the daily increment from the repository, which will be taken into processing once the backup job is started.

How increments are selected for synthesized full backup:
- If there is no incremental restore point for the specified day – logic will pick the incremental restore point for synthesized full backup creation from the day before. If there is no incremental restore point again it will keep checking all previous days in the chain unless there are no unused incremental restore points, by unused I mean previously not written to the tape.
- If there are many increments for the set day – only the latest will be taken for synthesized full source.

Incremental backups processing:
- It’s possible to uncheck the incremental backup processing an leave synthesized full backup only.
- All incremental restore points are written to tape except the taken for synthesized full backup creation. All the further incremental restore points on tape are relinked to the last created synthesized full backup.

Limitations and tricks:
-Synthesized full backup will be created only for forever incremental backup jobs and backup copy jobs. Incremental with active/synthetic full and reversed incremental backup as a source will ignore this option and described logic.
-If you set backup to tape job to run once a week and check more than one day in the synthesized full backup schedule – only one full backup will be generated on tape, closest to the day of the backup to tape job run.
-If you want synthesized full backup to be created more frequently (i.e every day) you need to set backup to tape job schedule to run every day.
Dima P.
Veeam Software
 
Posts: 6242
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Feature Request (Tape and Forever Incremental)

Veeam Logoby Dima P. » Thu May 07, 2015 6:13 pm 1 person likes this post

Now let’s get back to your issues and questions:
Dave338 wrote: The option to backup only latest chain that appears if you have more than one full on the target seems the salvation, but I'm testing and seems only working on the first run of the job...

It is true, however, we recognize this functionality to be highly requested – so it’s now in the top priority list.
Dave338 wrote: Problem occurs basically when using Forever Forward. One your retention period is reached, the full vbk file gets modified every day… But the initial problem is solved? Or I will continue getting 2 fulls on tape every week because my "old" vbk is modified

It should be solved in the Update2. If you see any duplicated .vbk it should be counted as bug and reported to support team
newfirewallman wrote:this is intended behavior. The tape job will always fail if the backup job kicks off…

Have to admit it’s totally looks like a bug. I’ve already forwarded all your findings to QA team for investigation. Will follow up on this one once I hear anything.
newfirewallman wrote:Hmm, I'm wondering if the choose latest is selectable via the powershell snapin?

Not possible. Only via hard links workaround provided in this thread
Dave338 wrote:my monthly backup will be syntethised 10 days later than I want, after the retention date has been expired

It will check the synthesized full backup schedule every backup to tape job run, so if the restore point still persists on repository and was not previously written by this tape job – then yes, but, please, keep in mind the general recommendation to have the tape job’s retention longer than the disk job.
Dima P.
Veeam Software
 
Posts: 6242
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

PreviousNext

Return to Tape



Who is online

Users browsing this forum: No registered users and 5 guests