V12: Template restore points corrupted after chain upgrade

[mkretzer]

Hello,

for the first time in over 10 years i got a corruption message - on the source backup, only for templates, only after chain upgrade:
Unable to create synthetic full backup: restore point is corrupted

I find it EXTREMLY unlikely with all we have seen that this is a real corruption - is this a known issue?

Markus

[mkretzer]

Case 06258440

[pirx]

Same here, Case 06217042. But ther will be a new case created for this issue.

Are you using storage snapshots?

[mkretzer]

Nope - no storage snapshots.
How does it happen for you? Also with templates? Also at synthetic time?

[pirx]

Only templates in one job. Not sure if it is only in syn fulls, I've removed them for now from the job. Support suggest to try an active full next.

Edit: just checked. It happend in first incremental backup after chain upgrade. The whole job with all VMs failed. Issue was 3 templates. If I exclude them, all is good. Active full for templates is working. AFAIK during the chain upgrade the templates were not properly upgraded. But upgrade job did not show any errors.

[mkretzer]

Indeed it sounds like this. Currently trying with a second job.

[mkretzer]

I just got an update that @pirx and my problem is not the same (strange...).
Also that i should create a new backup set (for a Veeam bug?!)...

I hope someone finds this issue, for a real corruption it is weirdly specific (and it seems for now happening on the first chain that is upgraded).

[mkretzer]

Veeam Backup Validator verifies that the original Backup files are in fact *not* corrupt:

Statistic:
Backup files count: 3
Incomplete backup files count: 0
Failed backup files count: 0
Files count: 6

[mkretzer]

Update from Veeam:
"Our R&D confirmed the issue with synthetic full operation for templates with non-empty incremental points. Unfortunately for now there is no fix. This issue is planned to be fixed in the future versions.

Until the issues is fixed the possible workaround are:
1) Creating active full backup for a single failed template task. Requires a manual intervention but should be fast and should not take too much space. Also should be rare as issue happens only when template was promoted to VM and back.
2) You can disable skipping incremental points for templates This can be configured in VM exclusions in job properties. This way templates will be processed fully as VMs and the issue should not happen. The downside is increased number of operations during each backup session and slightly higher backup size."