Update to Veaem 12.3 / AHV_7.0.0.241 error

[Amarokada]

I've applied the updates to our Veeam VB&R server this morning, upgrading from 12.2 to 12.3 and upgrading the AHV plugin which contains the new AHV appliance.

All of the upgrade seemed to go fine but I didn't click the "upgrade remote components" during the upgrade, instead I always opt to do those on the first run of the VB&R console.

The console prompted to upgrade the remote components as expected, but when it got to the last one which was the AHV appliance it failed. I was watching the old appliance get deleted and re-created on Prism and saw the workers also get refreshed. However the final result was an appliance that didn't quite work properly. On logging into the appliance portal page it would put up an Ajax Error dialog which I could click away but would return if I went back to the dashboard.

In the VB&R server, the Veeam proxy (AHV) was showing as unavailable, but would allow me to run through the properties until the end (showing how it was checking backup chains etc), but even after that the proxy was showing as unavailable. I rebooted both the VB&R and Appliance servers but am stuck in the same place. If I try to "rescan" the proxy it fails.

What I want to know, is should I just delete this appliance/proxy and get VB&R to re-create and then restore from configuration backups which have been running fine for last 10 days?

Regards
Rob

ps. sorry about the typo in the subject, can't seem to edit that.

[Amarokada]

If I do attempt to delete the broken appliance/proxy it gives the option of:

*Permanently delete the proxy VM from the cluster and all of its backup jobs (irreversable)

If I then push out a new appliance/proxy and restore the configuration from backup, I assume that will bring everything back including the backup jobs and their knowledge of the backup chains. Can someone confirm this?

[ronnmartin61]

Yes it will bring everything back. The prior chains may show as "orphaned" for a bit in the VBR console but will sync as part of normal processing

[Amarokada]

Just giving an update to this.

We did the manual removal and re-creation of the proxy appliance and it ended up failing in the same place, right at the end:

>Failed to synchronize jobs: Infrastructure host not found (InstanceId=3f2f1475-fad9-425e-91b4-e3d9a63f1d27)

We thought we were royally screwed at this point because we expected backups would not run, however the appliance did run the backups that it knew about from the configuration restore and as those backups run the VB&R server gradually received the information. So VMs moved from orphaned to disk section as they were backed up, and backup chains were maintained. However in this state with the sync between VB&R and proxy broken we're hardly in an ideal position. We raised a ticket and got the expected reply that we were on an unsupported AOS version 6.8.0.5. This weekend should see us upgrading to 7.0.0.0 or 7.0.0.5 to finally get on a supported version and hopefully either a resync of the proxy will work or a re-creation/restore of it will.

In the meantime we're limping along with 1500 VM backups working by a miracle.

[ronnmartin61]

Please share support case number so we can track

[Amarokada]

Case # 07575488

Will update after the weekend once we're on 7.0.0.5

[Amarokada]

Hi

So after the AOS upgrade I went through the process of deleting and recreating the proxy appliance and doing a config restore. The exact same thing happened. What seems to be happening is the recreated appliance is fine at first, but once you restore the config from a previous backup the UUID that Veeam refers to is overwritten with the old UUID (which was correct when the config backup was taken). So when it gets to the last step of syncing the jobs it fails specifying the old InstanceID of the app in the error.

I hope we can get some quick resolution now that we are on a supported AOS version. Ticket has been updated.

[TomQB]

Hi,
I think I have the same problem. Do you still have the problem or what was the solution from the support?
Thank you!

[ronnmartin61]

@TomQB our tech support was able to resolve the issue via a small script to correct a database entry. Definitely best to open a support case if you believe you're seeing the same issue.