Backup to Tape Performing Unnecessary Extra Full Backup

Everything about backing up to tape

Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby Princes » Tue Mar 03, 2015 3:40 pm 1 person likes this post

Case ID: 00821217

Just to check I'm not going mad and this is not expected behaviour:

I have recently migrated our tape backup system from Backup Exec 2010 to using the Veeam 8 Tape Backup architecture. To that aim, I have created a Backup to Tape job for each of my existing Backup to Disk jobs, which is set to run after each Backup to Disk job completes. Each Backup to Disk job is of the Incremental Type, set to creating a synthetic Full every Friday PM and to Transform previous backup chains into rollbacks. This means that a new .VBK Full Backup file is created for each Backup to Disk job every weekend, and a new .VIB Incremental file is created Mon-Thur.

I have configured the Backup to Tape job to run after each Backup to Disk job, which on a Friday will pick up the modified .VBK and send it off to the Full Backup tape pool as desired. During the week (Mon-Thur), the latest new .VIB is picked up and sent off to the Incremental Backup pool instead. I have enabled the 'Process incremental backup files' option in the job.

Everything works fine until after the Friday full backup to tape has completed. The .VBK is sent off to tape as expected after the synthetic full and transform process has completed as part of the Backup to Disk job. However the next run of the tape job on the Mon PM results in both the latest new .VIB for that day being backed up, as well as yet another copy of the unchanged, unmodified .VBK that was already backed up successfully to tape when the job last run. Curiously, this only seems to happen for some of the tape jobs, all of which are configured the same way. Different jobs are also affected by the issue each week - others back up just the latest .VIB for the Mon as I would expect. I have checked to see whether the .VBK's have been modified in any way and all retain the timestamp from when the last Backup to Disk job completed and the Full Backup to Tape job ran afterwards. The file appears unmodified in any way - the byte count is also the same - since the Backup to Disk then to Tape.

If I allow this additional necessary backup of the .VBK to take place and finish successfully on the Mon, subsequent backups seem to work fine and the issue goes away, each job committing just the latest new daily .VIB to tape as expected. The unexpected behaviour then returns again for different jobs after the next weekend (Fri) full. The issue therefore only seems to affect the first Incremental tape job run after the last Full Backup to Tape. I would rather not have this behaviour since I cannot afford the backup window time and additional tapes for an extra unneeded Full backup to tape every Monday when I only need the new Incremental file committed to tape.

Am I going mad?
Princes
Novice
 
Posts: 7
Liked: 1 time
Joined: Tue Feb 17, 2015 3:43 pm
Full Name: James Brigden

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby vmff » Tue Mar 03, 2015 10:15 pm

I also am leaving from Backup Exec 2010, after a failed migration to Backup Exec 2012, and just barely bought Veeam after the 30 day trial seemed to be working fine. I'm now experiencing the exact same issue, with a variation on the original job being reverse incremental, but I only want the most recent .vbk on the tape, which shockingly is seemingly not configurable?? I asked support and they said, nope, VB&R can't do that. Huh? You can't backup a full backup to tape each day without issue? Real big problem for me too.

Their suggestion is to try and get someone on the forums to write me a script, however, there are issues with that too, but guess that's my only option now that I've paid them for this.
vmff
Influencer
 
Posts: 21
Liked: never
Joined: Tue Mar 03, 2015 6:24 pm

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby Princes » Wed Mar 04, 2015 11:37 am

With the Reverse Incremental backup mode, does the job not just modify the same .VBK full back file each day, injecting the incremental changes since last backup into the file? In which case, the backup to tape job should be identifying the .VBK file as modified each day and trying to back it up?

My issue is slightly different in that I am using Incremental with a weekly synthetic full created on Fri. The rest of the week (Mon-Thur) is just .VIB incremental files. However the backup to tape job seems to be identifying the .VBK as having been modified since its last successful full backup on Fri after the synthetic full job is completed - which it has not according to bytecount and timestamp....
Princes
Novice
 
Posts: 7
Liked: 1 time
Joined: Tue Feb 17, 2015 3:43 pm
Full Name: James Brigden

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby vmff » Wed Mar 04, 2015 6:11 pm

That is how it works, except when an active full is run, then it starts the reverse incremental chain over again from the latest active full, but leaves the previous reverse incremental .vbk as one of the restore points for the retention policy.

VB&R doesn't seem to be identifying whether the full has been modified for me, I've seen people reference that it should, but instead (with incremental backups ignored on the tape copy job) all full backups in the repository are being copied out to tape each time. I'm only running an active full once a month, so I suppose this problem will go away when I reach my 15 day retention period, but that means half the month I'm backing up twice as much data as I'd like to.

This is a real problem for me, because our tapes are virtual and replicated offsite, so that process is killing our DR connection.

Still haven't had luck finding a script that will work for this, wish they would just add a check box to only back up most recent full backup or something...I might have to go back to Backup Exec to do it, but not really wanting to put a BE agent on the Veeam server.
vmff
Influencer
 
Posts: 21
Liked: never
Joined: Tue Mar 03, 2015 6:24 pm

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby Princes » Tue Mar 10, 2015 11:05 am

Issue reoccurred again last night as expected (first attempted Incremental to tape after the last successful Full to tape), with only 3 jobs working correctly this time, the rest once again incorrectly identifying the last synthetic full .VBK as having been modified since the last successful back to tape, and thus adding it to the Incremental backup to tape job. I have once again checked the .VBK file to ensure that it was indeed successfully backed up to tape during the last backup to tape job and that the file itself has not been modified in any way since. It would seem possible that a bug exists in the internal logic that identifies whether the file has been modified, or perhaps the application is not correctly storing the metadata concerning the last successful full to tape and so is not aware that there has been one and tries to backup the same file once again.

Not getting anything from support who have missed their remote session appointment today as well.... Is there anyone else experiencing anything similar to this where backup to disk files are being incorrectly identified as modified by a tape job causing them to be backed up twice (though only seemingly after the last successful full to tape - if this second backup of the full .VBK file is allowed to complete, the issue corrects itself with the next backup)?
Princes
Novice
 
Posts: 7
Liked: 1 time
Joined: Tue Feb 17, 2015 3:43 pm
Full Name: James Brigden

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby Dima P. » Tue Mar 10, 2015 2:12 pm

Hello James,
I am sorry for delay in response. Let me summarize your issue and setup:

-Source job is incremental with periodic synthetic Full backup (plus rollbacks)
-Backup to tape is scheduled to run ‘After the source job’
-This second unexpected .vbk is placed on tape only during the Friday backup to tape job run

Need to know was your VBR patched with patch1? In addition, how your disk and tape backup job retention was configured? Also, what media set creation option is used in target media pool?

Thank you in advance.
Dima P.
Veeam Software
 
Posts: 6261
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby trustblomme » Thu Mar 12, 2015 8:45 am

Hello,

is there any update on this problem? I have a simular problem and Veeam is eating all my expensive LTO tapes.

Case 00814113

The implemented dll fix worked for 1 week and then the problem returned.

The job dedects a change on file level and rewrites every vib to tape:

11/03/2015 21:33:25 :: Modified storage E:\Backup\XXXXX Backup\XXXX Backup2015-03-09T210057.vib on This server was added to backup list -> this entry returns for every VIB file.

Please advise,

Thanks,

Steven
trustblomme
Influencer
 
Posts: 22
Liked: 3 times
Joined: Fri Nov 16, 2012 9:07 am

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby TommyB » Fri Mar 13, 2015 9:15 am

trustblomme wrote:The implemented dll fix worked for 1 week and then the problem returned.

The job dedects a change on file level and rewrites every vib to tape:


Had the same issues, but in my case the DLL fix worked - so there seems to be some variation in whatever causes this...
TommyB
Expert
 
Posts: 115
Liked: 15 times
Joined: Wed Aug 28, 2013 9:46 am
Location: Germany.Europe.Terra.Sol.Milkyway.Localgroup.Virgo
Full Name: Thomas Braun

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby Princes » Fri Mar 13, 2015 2:24 pm

Hi,

This issue has (likely) been rectified. I will know for sure after the first Incremental Tape backup has run on Monday 16th PM.

Veeam Support remoted into our Veeam B&R Server and identified the issue as one that they are aware of. A new "Veeam.Backup.Core.dll" was supplied with a build date of 15/02/2015, replacing the existing v8 Patch 1 build date of 19/12/2014 version. After all Veeam B&R services were restarted, the issue appears resolved when the tape jobs are rerun, with just the latest Incremental .VIB files now being selected for backup and the last, already backed up and unmodified synthetic Full .VBK no longer being flagged as modified and added to the job.

If this desired behaviour remains after the latest new synthetic Fulls have been compiled over the weekend, then I will be 100% sure the issue has been resolved.

I was told that this fix is in the forthcoming Patch 2 for v8. If you need the fix before then I would contact support and request the latest .dll.

Thanks.
Princes
Novice
 
Posts: 7
Liked: 1 time
Joined: Tue Feb 17, 2015 3:43 pm
Full Name: James Brigden

[MERGED] Unnecessary backups to tape of unmodified full back

Veeam Logoby mfo » Tue Mar 17, 2015 2:16 pm

Hello,
Issue: Unmodified full backup files are repeatedly written to tape unnecessarily.

We use Veeam Backup 8.0.0.917 with disk and tape backups.

Our backup scheme is Forward incremental Backup with syntetic full backup and rollback transformation on Friday. So on Saturday, when Full Tape Backup is scheduled, I have only the fresh *.vbk and *.vrb file in my repository.
These files are than written to one a half LTO5 tapes, which gives me just enought free space on the second tape to write the upcoming incremental backup files during the following week.
The first backup per week runs as expected and all *.vbk files are written to tape.

But lately, the *.vbk full backup files are repeatedly written to tape the following days, which consumes a lot of unnecessary tape space and I'll end up with 5 tapes instead of 2-3 for a weekly media set.
I create a fresh media set just before the full tape backup starts on Saturday in order to have one single media set per week with the full backup and incremental files.
From my point of view, the *.vbk files are unmodified, so there should be no reason to write them again.

Creation of the syntetic full backup and rollback transformation is finished 5h prior to the start of the full tape backup, so there should be no interference.
Full tape backup is only scheduled on Saturday, incremental tape backup is enabled. Tape backup job is scheduled from Monday to Saturday.

Any suggestions?
mfo
Influencer
 
Posts: 17
Liked: 3 times
Joined: Thu May 27, 2010 3:11 pm
Full Name: Christoph Weber

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby foggy » Tue Mar 17, 2015 3:26 pm

Christoph, this looks like a known issue, please contact support for the hotfix.
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby trustblomme » Wed Mar 18, 2015 8:14 am 3 people like this post

Hi,

Contact provided me with a new dll that seems to fix the problem for now.

Thanks Veeam Support! :D

Steven
trustblomme
Influencer
 
Posts: 22
Liked: 3 times
Joined: Fri Nov 16, 2012 9:07 am

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby bertdhont » Wed Mar 25, 2015 9:44 am

We see kind of the same problem, in one specific case with forward incremental and active full backups.
Situation:
    Daily backup job to disk, forward incremental, active full once a week
    Tape job to process full and incremental backups of that backup to disk job

I start a first backup, active full.
After that one is finished, I start an incremental backup to disk
When this one is finished, I start the tape job. This one finds a full and an incremental backup to put on tape (2 different media pools).
During the tape offload, I start another incremental backup.
After the full is written to tape, Veeam notified that another .VIB was created, so it was added to the list of files to put on tape.
BUT, the strange thing is, it also notifies that the full (VBK) is modified and the VBK is written once again to tape....

I don' t understand why that full is modified, because we are using a forward incremental backup.

The log of the tape job below:

19/03/2015 10:28:56 :: New storage E:\VBRRepository\testBDHO\testBDHO2015-03-19T095624.vbk on This server was added to backup list
19/03/2015 10:28:56 :: New storage E:\VBRRepository\testBDHO\testBDHO2015-03-19T101058.vib on This server was added to backup list
19/03/2015 10:28:56 :: Queued for processing at 19/03/2015 10:28:56
19/03/2015 10:28:56 :: Required backup infrastructure resources have been assigned
19/03/2015 10:28:57 :: Media pool Media Pool test BDHO locked successfully
19/03/2015 10:28:57 :: Drive 2 (Drive ID: Tape1) locked successfully
19/03/2015 10:28:57 :: Loading tape ID0578L6 - Mei / Nov - B from Slot 18 to Drive 2 (Drive ID: Tape1)
19/03/2015 10:30:37 :: Current tape is ID0578L6 - Mei / Nov - B
19/03/2015 10:31:07 :: Starting new media set Media set 19/03/2015 10:31
19/03/2015 10:31:09 :: New tape backup session started, encryption type is off
19/03/2015 10:31:34 :: Processing full backups started at 03/19/2015 10:28:55
19/03/2015 10:36:48 :: Backup completed successfully
19/03/2015 10:36:49 :: New storage E:\VBRRepository\testBDHO\testBDHO2015-03-19T103215.vib on This server was added to backup list
19/03/2015 10:36:49 :: Modified storage E:\VBRRepository\testBDHO\testBDHO2015-03-19T095624.vbk on This server was added to backup list
19/03/2015 10:36:49 :: New storage E:\VBRRepository\testBDHO\testBDHO2015-03-19T101058.vib on This server was added to backup list
19/03/2015 10:36:49 :: Changes in source storage detected. Scan for changes
19/03/2015 10:45:19 :: Media pool Media Pool daily bdho locked successfully
19/03/2015 10:45:19 :: Loading tape ID0575L6 - Apr / Okt - A from Slot 15 to Drive 2 (Drive ID: Tape1)
19/03/2015 10:47:01 :: Current tape is ID0575L6 - Apr / Okt - A
19/03/2015 10:47:30 :: Starting new media set Media set 19/03/2015 10:47
19/03/2015 10:47:30 :: New tape backup session started, encryption type is off
19/03/2015 10:47:55 :: Processing incremental backups started at 03/19/2015 10:28:55
19/03/2015 10:48:08 :: Busy: Source 3% > Proxy 3% > Network 5% > Target 99%
19/03/2015 10:48:08 :: Primary bottleneck: Target
19/03/2015 10:48:08 :: Network traffic verification detected no corrupted blocks
19/03/2015 10:48:08 :: Processing finished at 19/03/2015 10:48:08
19/03/2015 10:48:08 :: 0 directories and 4 files backed up
bertdhont
Service Provider
 
Posts: 20
Liked: 3 times
Joined: Fri Nov 08, 2013 2:53 pm
Full Name: Bert D'hont

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby Dima P. » Wed Mar 25, 2015 2:21 pm

Hello Bert,
Steven confirmed above that we have a working hotfix at the support side for a situation like yours. Could you please contact support team and let us know if the existing solution works for you. Thanks in advance.
Dima P.
Veeam Software
 
Posts: 6261
Liked: 440 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Backup to Tape Performing Unnecessary Extra Full Backup

Veeam Logoby krogerss » Tue May 19, 2015 12:58 pm

Can anyone confirm if this fix is included in Update 2? I believe I'm experiencing this very issue and am looking to attain this hotfix. Thanks.
krogerss
Enthusiast
 
Posts: 63
Liked: 1 time
Joined: Mon Aug 18, 2014 6:06 pm

Next

Return to Tape



Who is online

Users browsing this forum: No registered users and 5 guests