v12 Config Backup - "Error Failed to get GCP appliances"

[dcolpitts]

I upgraded multiple sites this morning from v11 to v12 using VeeamDataPlatformAdvanced_v12_20230209.zip (12.0.0.1420, which is GA), and at pretty much every single site, the configuration backup is failing with "Error Failed to get GCP appliances". There is a GCP Plugin installed (installed on today's date), that is version 12.0.4.907. We are not using any cloud services with these sites either.

Anyone have any ideas before I start opening 20 support tickets?

dcc

[Dima P.]

Hi Dean,

Checking with the team. Meanwhile can you please raise a ticket (20 is way too much) as we might need some debug logs from configuration backup job? Thank you!

[dcolpitts]

So I did mine on Saturday, and it appears fine. So this morning I upgraded 20 different customers and that happened at all of them. After I posted, I was chatting with a peer, and he said his did the same thing and he just uninstalled the plugin, and the issue went away. So that's what I did - I uninstalled the plugin and the configuration backup now runs at all of them. So I probably won't bother with a ticket now.

dcc

[Dima P.]

Thanks for the update! QA folks have mentioned that possibly the plug-in service was not started after the upgrade and that was the reason why the configuration backup did not work. Did you upgrade the B&R via UI or was in unattended installation via scripts?

[dcolpitts]

UI, and I rebooted each host after the upgrade finished (regardless if it asked for it it not), before going into the UI and checking everything looked good.

dcc

[Dima P.]

Understood, thank you Dean. We will investigate this issue internally!

[dcolpitts]

I assume I have log files here on the hosts if you want them... Although I don't really have a lot of time today, as I'm also dealing with a backup copy failure on all my systems between StoreOnce Catalysts stores that I'm trying to figure out (a nice thing to wake up to in the morning)...

[Dima P.]

Cannot collect your logs without the support case unfortunately, but if you are going to open a support case regarding the StoreOnce issue please upload the full B&R log bundle and share the case ID with us here - I'll pass a word to QA team.

Regarding the SO, I am aware of one particular issue: if gateway selection was set to automatic and B&R server is added to managed servers gateway selection may stuck at the job start. Try to manually select the gateway and see if that helps.

[dcolpitts]

Ok. We'll see what happens in a bit. The Catalyst copy jobs are failing with: [Error] Error: An item with the same key has already been added.

[galbitz]

I just wanted to chime in and say I have seen the same issue with a handful of sites. I upgraded 6 sites, none of them installed new versions of the azure/aws/gcp plugins. We use AWS but not ec2, just object storage to wasabi which I believe is not related to that plugin. I uninstalled those plugins on all 6 sites and the jobs work fine and we don't receive the plugin error anymore during a config backup. What I do find odd though is none of those sites attempted to install a v12 version of those plugins. The site either still had v11 plugins after the upgrade or some sites didnt have them at all. I am not sure what criteria exists to determine if the plugin gets installed. I also had 1 site whos veeam distribution service stayed at v11 after the upgrade. I manually ran the v12 msi for that service after the fact and it is now upgraded. It never exhibited any problem though I just happened to be checking add/remove programs for any v11 components and saw that. When I compared it to other sites I confirmed they were infact upgraded.

Long way of saying I've noticed some odd behavior with the upgrade not updating all components fairly consistently across all deployments I work with.

[Dima P.]

Ok. We'll see what happens in a bit. The Catalyst copy jobs are failing with: [Error] Error: An item with the same key has already been added.

Thanks, passed to the team but they were wondering if you can possibly get us the case and logs.

I upgraded 6 sites, none of them installed new versions of the azure/aws/gcp plugins.

Latest build of Veeam B&R does not have the plug-ins for Azure or AWS, but GCP is already there.

Long way of saying I've noticed some odd behavior with the upgrade not updating all components fairly consistently across all deployments I work with.

Thank you for the feedback! If possible please raise and share support case with us - we would love to troubleshoot every single upgrade issue but in order to do that we need debug logs to identify the root cause and make sure such behavior never bothers you again!

[dcolpitts]

Here's the case number (and I attached logs) for the Catalyst copy failure... 05889837

It's happening on everyone of my customer locations with Catalyst copy enabled.

I also attached the Job.Backup_Configuration_Job.Backup.log to that ticket in case it helps you Dima (it'll probably confuse the tech for 05889837 though ).

dcc

[spiritie]

We were told by support to simply remove the plugins if we were not using them, that resolved our config backup warnings. These plugins were not bundled with either the RTM ISO
(Not sure if they are bundled with GA ISO now?)

[Dima P.]

Here's the case number (and I attached logs) for the Catalyst copy failure... 05889837

Passed to QA team (both Catalyst and GCP). Thank you for your help!

We were told by support to simply remove the plugins if we were not using them, that resolved our config backup warnings. These plugins were not bundled with either the RTM ISO
(Not sure if they are bundled with GA ISO now?)

Thank you for the update! GCP was released together with B&R GA (so it did not make the RTM but is included in GA). Do you have a case I can share with QA folks?