Backup copy corrupted

[redhorse]

Hello,

I got this message after a backup copy job (with health check) although the status information of this server is “success”:

Warning: Disk Server-flat.vmdk of VM Server is corrupted, possible reason: Storage I/O issue. Corrupted data is located in the following backup files: Server.vm-113762016-02-26T000000.vbk

In the backup properties of this backup copy job I can see that the status of this restore point is “incomplete” and all previous restore points are “corrupted” and the new one are “OK”. I get a warning message when I try to restore from a corrupted restore point, but a test restore and an instant VM recovery work well.

What does that mean?

Greetings

[PTide]

Hi,

What does that mean?

Some blocks belonging to "Server" vm's disk in .vbk file got corrupted. The most recent increment (the one that is reported as "OK) that has been produced by backup copy job contains that block in a valid form which guarantees that you'll be able to restore from the most recent increment.

I get a warning message when I try to restore from a corrupted restore point, but a test restore and an instant VM recovery work well.

Not all blocks are used during Instant Recovery so there is no certainty that the VM will not fail during full recovery from corrupted restore points.

Thank you.

[redhorse]

Okay, so I can rely on the newer restore points which were created after the "OK" one.

I had never an issue like this and I guess it wouldn't be noticed if I wouldn't enable the health check. Why is this not detected during the regular backup? I am afraid to get a serious problem when I need a backup before the monthly heath check detects corrupted data.

[PTide]

Why is this not detected during the regular backup?

Health check option for backup jobs has been added in v9. If you have already enabled that function in your regular backup job then your backups are ok. I'd also recommend you to perform some test on your backup copy destination storage.

Thank you.

[redhorse]

Health check option for backup jobs has been added in v9. If you have already enabled that function in your regular backup job then your backups are ok. I'd also recommend you to perform some test on your backup copy destination storage.

Yes, I perform a monthly health check for the primary and for the back copy job. When the corruption occurs after the health check it would be possible to have a corrupt backup for weeks and data restore is probably not possible? Should I do the health check weekly or even daily?

[PTide]

When the corruption occurs after the health check it would be possible to have a corrupt backup for weeks and data restore is probably not possible?

Correct, data restore will not be possible for the corrupted restore point.

Should I do the health check weekly or even daily?

The schedule of health checks depends on your environment and backup schedule. For example if you run backup copy job after every backup session then you probably don't need to run primary backup health check very often because if the source backup gets corrupted before backup copy job session then backup copy job will fail.

Thank you.

[redhorse]

Hello PTide,

The schedule of health checks depends on your environment and backup schedule. For example if you run backup copy job after every backup session then you probably don't need to run primary backup health check very often because if the source backup gets corrupted before backup copy job session then backup copy job will fail.

Yes, I run a backup copy job after every backup session. So as I now understand during the backup copy job a health check of the source backup is integrated? In this case I would set a monthly health check of the primary backup and a weekly of the backup copy. But there is still a gap where a data corruption is not detected unless I do a daily health check.

[PTide]

So as I now understand during the backup copy job a health check of the source backup is integrated?

Not exactly. The reason why backup copy job will fail lays inside the Backup Copy Job logic and in some. For example in case if you have compression disabled in your source job it may happen so that backup copy job will transfer corrupted block without knowing it.

In this case I would set a monthly health check of the primary backup and a weekly of the backup copy. But there is still a gap where a data corruption is not detected unless I do a daily health check.

Unfortunately there is no silver bullet, however it won't hurt if you run health checks at any opportunity, just keep im mind that health check might be very time consuming depending on the length of the backup chain.