Declared and actual CRC are different for all bank snapshots

[DaFresh]

Hello,

One of my archive job failed this week end with this error for all VMs. The destination repository is a Linux (via SSH), the file system as been scrubed 1 week ago, actual error counters show no errors (thanks to ZFS), so I truly think there is no corruption problem on/with the storage itself.

Case # 00581153

Could you help to deeper investigate this issue ? Any pointers would be greatly appreciated.

Cheers

Cédric

[foggy]

Cedric, I cannot recall other occurrences of this error. Deep investigation requires logs review, which should be performed by our support engineers within the case you've opened, so please work with them directly on this. Seems that either source or target backup metadata is corrupt (am I understanding right that this is a backup copy job?).

[DaFresh]

Hello Alexander,

I post my issue here in the hope someone already as to deal with this issue, of course I work in // with the support, log are being sending. And no this is not a backup copy job but a simple backup job. I already look for converting it to a backup copy job (which will save CPU and time processing, plus having very nice options for long time retention!), but the thing is it's not possible to keep my actual backup history, I will then need to do a full again which will be a big space wast , but if you have some informations about that it would be great !

Will try to keep information up to date.

[veremin]

Since this is a backup job, can you tell me at what stage did it fail? At the beginning or closer to the end? Thanks.

[DaFresh]

Hello Vladimir,

The whole job fail immediately in about 6sec, so it seems the error is detected very early by, I suppose, checking the metadata of the job/repository.

[veremin]

Was the snapshot even taken or not? Thanks.

[DaFresh]

Indeed nice catch, I had look maybe a bit quickly, here his what happens :

- job has been renamed during the week before the backup windows (just FYI in case it could play something ...)
- then the first try succeed to create/remove snapshot on VMs, but fail at the end of every VMs.
- the next retry (3 exactly) failed immediately without trying the snapshot sequences.

[veremin]

Probably, initial run that had failed closer to the job's end led to metadata corruption or something similar, and the following runs could not proceed any further. As mentioned, no one has raised similar issues before, so, we can't be 100% sure. Though, the support team will be able to shed a light on the root cause of this behaviour, once required logs are analyzed. Thanks.

[DaFresh]

Indeed, strange. Thing is the storage has snapshot, not a big history but I could try to do a rollback, but I would agree I prefer an other way to solve this in order to understand what happens.

Cheers