Copy To Tape Job - Changes in source backup file detcted

[agrob]

Good Day

I'm using veeam v9 and did setup now some tape jobs. Have several jobs running fine expect for the job which does copy backup files from a copy session to tape.
We have branch offices. backup jobs for those are configured to backup to a local repository. after backup, we use a copy job to replicated the latest backup file to the main datacenter.
Now i configured a TapeJob -> Backup to Tape. Source files are the copy jobs so that the data local stored in our datacenter is written to tape.
Job does start but then it does not process the data. Here is the output of the log

Code: Select all

14.03.2016 15:52:29 :: Building source backup files list started at 14.03.2016 15:52 
14.03.2016 15:52:30 :: No restore point for 13.03.2016 is available for virtual full backup, using previous restore point for 12.03.2016 
14.03.2016 15:52:30 :: New Copy HyperV 2016-02-29T200000_W.vbk restore point will be placed to media set  
14.03.2016 15:52:30 :: Full backup from Copy HyperV 2016-03-12T200000.vib (13.03.2016 00:00:00) restore point will be placed to media set 
14.03.2016 15:52:30 :: Source backup files detected. VBK: 1, VBK map: 1, VIB: 0 
14.03.2016 15:52:31 :: Creating full backup map for M:\Copy HyperV\Copy HyperV 2016-03-12T200000.vib 
14.03.2016 15:53:37 :: Queued for processing at 14.03.2016 15:53:37 
14.03.2016 15:53:37 :: Required backup infrastructure resources have been assigned 
14.03.2016 15:53:39 :: Drive 2 (Server: mediaserver.domain.local, Library: HP MSL G3 Series 1140, Drive ID: Tape1) locked successfully 
14.03.2016 15:53:40 :: Loading tape GXZ396L4 from Slot 44 to Drive 2 (Server: mediaserver.domain.local, Library: HP MSL G3 Series 1140, Drive ID: Tape1) 
14.03.2016 15:54:58 :: Current tape is GXZ396L4 
14.03.2016 15:55:11 :: New tape backup session started, encryption: disabled 
14.03.2016 15:55:26 :: Processing full backups started at 03/14/2016 15:52:29 
14.03.2016 15:55:56 :: Changes in source backup files detected, rescan required:  

after i cancelled the job becaue it always loads a tape, then the same "changes in source..." is displayed again and again...

There is no change in the data and i dont know exactely why this happen. copy jobs are all over so no new data is copied from branch to main datacenter...
any idea why this happens?

Thanks
André

[FredMez]

I have the exact same issue. it also started with v9. I have a ticket open with level 2 support for the second time. First time, i was told to recreate the backup copy, and now i am being told something else is accessing the file. Then to add to the mystery, the job is working today. Since this job worked for over a year without issue, then i started having the issue twice since upgrading to v9, and the fact this is a dedicated veeam server; i have a hard time believing this is not a v9 issue.

[agrob]

Hi Fred
Thanks for the Information. Can you please update here if you have a solution for this Problem?
Thanks

[Dima P.]

Hi Andre,

Can you clarify how your tape job's schedule is configured? Thanks

[agrob]

Hi Dima

Here we go:
Daily at this time 14:00
on these days: Monday

I only copy full backups to tape, no incremental. So under the Full Backup Tab i have selected "Full Backup Schedule" -> "Weekly on selected Days: Sunday"
But i have also changed that to saturday or so... did not help

[Matts N]

Hello all,
I have seen the same thing in my lab. In my case I think it is due to changed and failed backups, only seen this once. Will have to wait a week to see what happens next time.

Source backup job: forward incremental with full on Saturday
Tape copy job: GFS, with weekly on Sundays (only full)

Added a new VM to source job on Saturday, job failed (something about locked files on repository, no client backed up at all)
Tape copy not triggered, since no changes to source
Fixed error in source job and rerun
Tried to force tape copy to run, ended up with the described scenario

[agrob]

interessting Point Matts.

I have 3 different copy Jobs selected as source in the tape Job, all have those Problem. There is also a normaly backup Job in the selection list and this Job can be written to tape. I still think it is a "bug" or something like this.
Any News about that from Veeam side?
Thanks!

[agrob]

Update: Have changed the Tape Job Options. Under Options -> Advanced -> Advanced -> Select "Process latest Fullbackup chain only" This seems to work in my case. And it is also what i Need, i just want to have the latest full on Disk.

[Matts N]

agrob, are you on update 1? I am still on v9 GA and I can't find that setting anywhere.

[veremin]

I think you're referring to different backup to tape jobs. Ordinary backup to job (agrob), GFS backup to tape job (Matts N). Thus, different job settings. Thanks.

[Matts N]

Correct Vladimir. In my case it is a GFS rotation.

[veremin]

It explains why the said option isn't there, as GFS job always archives full backup from the point closest to the specified date.

Anyway, it stands to reason to see whether the issue re-occurs and open a ticket in case it does.

Thanks.

[reinki]

Hi,

No I've the same issue with v 9.0.0.1491, always "Changes in source backup files detected, rescan required:" using GFS Media Pool. I'll open a case.

Brgds
Dennis

[Matts N]

Did anyone find the reason for this behaviour? Would be interesting to know, to be able to work around it if it happens again.

[pkelly_sts]

I think I'm starting to have a similar issue myself. I started a topic at tape-f29/what-can-cause-incr-tape-to-st ... 35434.html and the issue has recurred today but looking closer I can see that it's claiming that all of the source jobs (4 endpoint backups & one backup job) have been "modified" and yet just yesterday it completed another backup of a large chain conatining 2 x 2.4Tb .vbks (in the SAME job).

I've logged my ticket as 01808088

[einhirn]

Me too. *g*

I'm guessing it detects the newly created "VSB"-Files (Virtual Full Backup Map) as change in source files or something. I'm using "per VM backup file" and have my whole repository in that copy to tape job. Every time the backup cycle is cancelled it unloads the tapes. When scheduled "as new files appear", the job fails and is started again. When scheduled as "run after Job X" it keeps on running, but because of the retries with unloading 6 tape drives for each retry it is awfully slow - in the current job it only managed to get about 600Gbytes to 6 LTO4 Tapes in 8 hours... Source files are on a 12-Disk 3TB-NL-SAS RAID-10, so there should be enough throughput available. Tape drives are connected via a single 4Gbit FC link, but I'm far from saturating it, so I didn't think about adding more links yet.

I've logged a case as 01815809 - although I'm waiting to be told how to upload a 6Gbyte Log bundle...

[einhirn]

Just a follow up, my case was successfully closed - we activated incremental backups for VMware templates - that seems to do the trick at least as a workaround.