Undesired modification of older backup files

[jmcmurry]

Hello everyone,

I have Veeam B&R 5.0.2 running in my environment with a 60TB local volume for Veeam backup files. These are pushed to tape via post job activity scripts and a locally installed IBM TSM client to a central TSM server.

My problem: Older .vib and .vbk files from days at the end of my retention are being modified during Veeam backups. This causes TSM to have to back them multiple times. This is wasting time, tape, etc. In my case, reclaiming Veeam disk space is less important than leaving old files alone so they don't have to be sent to tape over and over again.

The sticky FAQ post on this forum includes a link to a spreadsheet) which states that "Existing backups never touched" is true for incrementals, but this is apparently not the case in my environment.

Anyone have any idea what could be causing this? How can I prevent this behavior? It seems to be related to synthetic fulls, but I'm not sure if I'm missing something or if Veeam is always eager to reclaim space since it's entirely disk based.

My backup jobs are configured with these options:

Backup mode: Incremental with once-per-week synthetic fulls, no transform, no active fulls
Restore points on disk: 28
Deleted VMs retention period: 7 days
Deduplication: Enabled
Compression Level: Optimal
Storage Optimize for: LAN target

Here's a representative directory listing for one of my backup job directories. Notice how older files from mid-June were modified this morning.

Code: Select all

Name                        Modified            GBs
----                        --------            ---
JOB2011-05-27T220057.vbk    2011-07-06 05:11    758
JOB2011-06-16T044156.vbk    2011-07-20 04:56    762
JOB2011-06-16T180113.vib    2011-07-20 04:55     25
JOB2011-06-17T180102.vib    2011-07-20 04:55     16
JOB2011-06-18T180109.vib    2011-07-20 04:55     15
JOB2011-06-19T180058.vib    2011-07-20 04:55     23
JOB2011-06-20T180102.vib    2011-07-20 04:55     49
JOB2011-06-21T180114.vib    2011-07-20 04:55     21
JOB2011-06-23T052031.vbk    2011-06-23 07:33    804
JOB2011-06-23T180058.vib    2011-06-24 05:50     31
JOB2011-06-24T180112.vib    2011-06-25 05:02     24
JOB2011-06-25T180111.vib    2011-06-26 03:56     15
JOB2011-06-26T180107.vib    2011-06-27 04:38     25
JOB2011-06-27T180054.vib    2011-06-28 04:10     15
JOB2011-06-28T180112.vib    2011-06-29 05:11     43
JOB2011-06-30T054047.vbk    2011-06-30 08:05    831
JOB2011-06-30T180105.vib    2011-07-01 05:40     20
JOB2011-07-01T180059.vib    2011-07-02 05:11     21
JOB2011-07-02T180111.vib    2011-07-03 03:45     15
JOB2011-07-03T180117.vib    2011-07-04 03:51     26
JOB2011-07-04T180116.vib    2011-07-05 04:55     16
JOB2011-07-05T180115.vib    2011-07-06 05:11     21
JOB2011-07-07T051840.vbk    2011-07-07 07:36    834
JOB2011-07-07T180100.vib    2011-07-08 05:15     20
JOB2011-07-08T180100.vib    2011-07-09 04:57     21
JOB2011-07-09T180100.vib    2011-07-10 05:00     18
JOB2011-07-10T180105.vib    2011-07-11 03:50     24
JOB2011-07-11T180104.vib    2011-07-12 05:24     18
JOB2011-07-12T180117.vib    2011-07-13 05:58     19
JOB2011-07-14T043713.vbk    2011-07-14 07:00    818
JOB2011-07-14T180114.vib    2011-07-15 05:04     17
JOB2011-07-15T180107.vib    2011-07-16 05:01     14
JOB2011-07-16T180122.vib    2011-07-17 18:01      9
JOB2011-07-17T180102.vib    2011-07-18 04:11     29
JOB2011-07-18T180111.vib    2011-07-19 07:05     18
JOB2011-07-19T180101.vib    2011-07-20 04:55     17

Thanks very much!

James

[Gostev]

Hello James, this is caused by a process which marks backup file blocks belonging to deleted/removed VMs as unused (as per deleted VM retention period). Actual data is not touched, but metadata section is in fact updated, which causes file modification date to change. We are making some changes in v6 to get rid of this, particularly for the reasons you have stated.

As for workaround for you to use with v5, one thing that comes to my mind is PowerShell script that will be started as post-job activity and (on successful backup) set archive bit on all files in the directory except the latest file that was just produced. Earlier, I have posted tape backup script that I suppose can be used as a basis for this (the script determines if the backup job completed successfully, calls EXE with produced backup file name as parameter, and sets archive bit on). Unfortunately, I do not know PowerShell well enough to assist you with this though...

Hoper this helps.

[jmcmurry]

Thanks Gostev. I guess I'll look forward to V6.

FYI, TSM doesn't (by default) respect the archive bit, and I doubt I could (or want to) change that behavior in my environment. There's an option to do an subfile incremental, which would help in this case since a large percentage of the file is unchanged, but it's not recommended for files over 2GB. (My files are way over 2GB.)

My post job activity script is written in PowerShell, and I might extend it to track newly created Veeam files and back up only those, ignoring files that were created weeks ago but modified recently. That's on your assertion that relatively unimportant metadata is the only thing changed in these older files.

Again, thanks for your quick response and your help.

Regards,
James

[EdQ]

We are running 6.0.0.153 and are seeing the same thing.
Look closely and it's evident that the August 15 backup prc0 nightly Windows2012-08-14T180124.vib was retouched on August 16.

Status August 16:

Directory of \\yadayadayada\veeam_backup\prc0 nightly Windows

08/15/2012 06:01 PM <DIR> .
08/07/2012 04:46 PM <DIR> ..
08/15/2012 11:45 PM 879,403 prc0 nightly Windows.vbm
08/08/2012 06:01 PM 592,440,127,488 prc0 nightly Windows2012-08-07T153953.vbk
08/09/2012 06:01 PM 105,747,390,976 prc0 nightly Windows2012-08-08T180148.vib
08/10/2012 06:01 PM 102,395,387,904 prc0 nightly Windows2012-08-09T180119.vib
08/11/2012 06:01 PM 97,216,686,592 prc0 nightly Windows2012-08-10T180136.vib
08/13/2012 06:01 PM 87,048,213,504 prc0 nightly Windows2012-08-12T180125.vib
08/12/2012 08:55 AM 595,150,069,760 prc0 nightly Windows2012-08-12T003729.vbk
08/14/2012 06:01 PM 114,293,135,872 prc0 nightly Windows2012-08-13T180117.vib
08/15/2012 06:01 PM 95,933,567,488 prc0 nightly Windows2012-08-14T180124.vib
08/15/2012 11:44 PM 104,423,426,560 prc0 nightly Windows2012-08-15T180122.vib
10 File(s) 1,894,648,885,547 bytes
2 Dir(s) 1,519,822,094,336 bytes free

Status August 17:

Directory of \\yadayadayada\veeam_backup\prc0 nightly Windows

08/16/2012 06:01 PM <DIR> .
08/07/2012 04:46 PM <DIR> ..
08/16/2012 11:51 PM 983,441 prc0 nightly Windows.vbm
08/08/2012 06:01 PM 592,440,127,488 prc0 nightly Windows2012-08-07T153953.vbk
08/09/2012 06:01 PM 105,747,390,976 prc0 nightly Windows2012-08-08T180148.vib
08/10/2012 06:01 PM 102,395,387,904 prc0 nightly Windows2012-08-09T180119.vib
08/11/2012 06:01 PM 97,216,686,592 prc0 nightly Windows2012-08-10T180136.vib
08/13/2012 06:01 PM 87,048,213,504 prc0 nightly Windows2012-08-12T180125.vib
08/12/2012 08:55 AM 595,150,069,760 prc0 nightly Windows2012-08-12T003729.vbk
08/14/2012 06:01 PM 114,293,135,872 prc0 nightly Windows2012-08-13T180117.vib
08/15/2012 06:01 PM 95,933,567,488 prc0 nightly Windows2012-08-14T180124.vib
08/16/2012 06:01 PM 104,423,426,560 prc0 nightly Windows2012-08-15T180122.vib
08/16/2012 11:50 PM 112,116,351,488 prc0 nightly Windows2012-08-16T180124.vib
11 File(s) 2,006,765,341,073 bytes
2 Dir(s) 1,415,851,220,992 bytes free

[Gostev]

This was addressed in 6.1