Comprehensive data protection for all workloads
Post Reply
paoloatuniurb
Novice
Posts: 6
Liked: never
Joined: Mar 15, 2014 4:43 pm
Full Name: Paolo Cecchini
Contact:

Backup copy job failing with *metadata are corrupted* error

Post by paoloatuniurb »

Hello.

Support case #02133769

I have a backup job failing on a few (4 over 111) vm.

20/04/2017 09:39:17 :: Copying restore point 19/04/2017 20:33:35 from backup repository Repository 16.24
20/04/2017 09:39:27 :: Error: All instances of the storage metadata are corrupted.
Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://frontend::CDataTransferCommandSet::RestoreText_{8656b8b9-1119-4026-807a-58c069ce2f7f}]. CHMOD mask: [666].
Agent failed to process method {DataTransfer.RestoreText}.


The *metadata are corrupted* error message has been, of course, already discussed but, figure out,
each of the 4 vm can be successfully restored from the same backup point that is claimed to be corrupted.

I disabled ad enabled again the backup copy job but cannot clear the error. Of course I started a new chain too ... nope.

Any one experienced this behaviour?

TyL on advance. Paolo.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by foggy »

paoloatuniurb wrote:Of course I started a new chain too ... nope.
Do you mean the error persists after starting the backup chain anew?
paoloatuniurb
Novice
Posts: 6
Liked: never
Joined: Mar 15, 2014 4:43 pm
Full Name: Paolo Cecchini
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by paoloatuniurb »

I suppose I'd to tell you all the long story.

I have a couple backup job, name them as an example A,B,C,D,E ... they're forward incremental with weekly active backup.

Then I have some offsite backup copy jobs. They collect *from infrastructure* vm so vm belongs to backup job A,B,C ...

I do have 4 failing vm's belonging to backup jobs A & B. If I restore them from A,B it succeed. I forced an active full on A,B, usually this clear the backup copy error, but this time nope. Of course I restarted the backup copy, too.

So I suppose YES, I mean the error persist after starting the backup chain anew ... or I mean, at least, the usual work-around, force a full, don't work this time.

:)

Cu. Paolo.
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by Gostev »

paoloatuniurb wrote:20/04/2017 09:39:27 :: Error: All instances of the storage metadata are corrupted.
This error indicates massive storage-level corruption. You should create the new Backup Copy job and point it to a different storage device.
paoloatuniurb
Novice
Posts: 6
Liked: never
Joined: Mar 15, 2014 4:43 pm
Full Name: Paolo Cecchini
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by paoloatuniurb »

Yes. I feel like you, and this is the worse side of the problem. The device is (better said, was) a Fujitsu CS800 that has been in production as main backup storage for years and never failed. But now I scaled it down to second line and formatted with Windows server 2016 core and dedup feature.

So the hardware is darn good. The software is windows. It's failing? May I trust in w2016 and deduplication?

Ty. PaoloC
paoloatuniurb
Novice
Posts: 6
Liked: never
Joined: Mar 15, 2014 4:43 pm
Full Name: Paolo Cecchini
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by paoloatuniurb »

paoloatuniurb wrote: But now I scaled it down to second line and formatted with Windows server 2016 core and dedup feature.
Well, I'm planning to restart from scratch and go for refs .. being a backup copy .. looks better.

Paolo
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by foggy »

paoloatuniurb wrote:So I suppose YES, I mean the error persist after starting the backup chain anew ... or I mean, at least, the usual work-around, force a full, don't work this time.
Btw, to start a backup chain from scratch, you need to trigger manual active full on it, active full on original backup job doesn't affect the backup copy job chain.
paoloatuniurb
Novice
Posts: 6
Liked: never
Joined: Mar 15, 2014 4:43 pm
Full Name: Paolo Cecchini
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by paoloatuniurb »

I meanwhile started the *architecture* from scratch :wink:

Delete job, delete repository, proxy, server, format with refs (was ntfs).
Add server et cetera, create new backup copy job.

CU. PaoloC
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by Gostev »

paoloatuniurb wrote:May I trust in w2016 and deduplication?
Not at this time. There is currently a known issue data corruption issue with Windows Server 2016 dedupe... we've recently seen the same corruption in our own lab and notified Microsoft about one. There seems to be a number of customers having this issue at this time.
mrbbs
Influencer
Posts: 11
Liked: never
Joined: Sep 06, 2016 11:00 am
Full Name: mr bbs
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by mrbbs »

Any updates on this issue?
A couple of weeks ago I read in your weekly newsletter this wasn't resolved by Microsoft yet.
Is it known if the patches from May have resolved the issue?

Kind regards,
Kevin
mrbbs
Influencer
Posts: 11
Liked: never
Joined: Sep 06, 2016 11:00 am
Full Name: mr bbs
Contact:

Re: Backup copy job failing with *metadata are corrupted* er

Post by mrbbs »

Another thread with this same issue.

veeam-backup-replication-f2/disk-errors ... 77-15.html
Post Reply

Who is online

Users browsing this forum: No registered users and 165 guests