No PVE Hypervisor workloads after inplace upgrade

[Dynamic]

Hi all,

Is it working as designed that Proxmox is no longer directly integrated after an inplace-upgrade from 12.3.2 to version 13.0.1?

My Proxmox workloads can no longer be activated, and I'm also unable to add Proxmox Hosts within the VBR Console.
The VeeamPVESvc service is present and running and the plugin for Proxmox is installed in version 13.3.0.237.

It's my Lab/NFR environment - so no big deal for me - just curious...
Anyone else can confirm this?

The upgrade to 13.0.1 was successful without any issues.
The Plugin was upgraded within the VBR upgrade procedure, from PVE 12.1.3.217 -> 13.3.0.237

A reboot was done after the upgrade.

Case #07892920

[Gostev]

It's not expected, thanks for opening a support case.

[Regnor]

Hi Markus,
In addition to the support case take a look at this article. And also try to restart the PVE service after all other services are started.
https://www.veeam.com/kb4687

[anpa]

I have the same issue after upgrade and rebooting the server, it seems to be certificate related:

Code: Select all

2025-11-20 10:29:33.9942 00010 [10192] ERROR | [DbProvider client]:  ==> Response "Put" "https://localhost:6172/api/proxies/58d815db-dce4-4a9a-b995-e5bef708b70a", "code: 400", duration: "73 msec", headers: "[]", cookies: "[]", body: "{"errorCode":"UnknownError","title":"Failed to generate child certificate from Veeam backup server certificate.","message":"Failed to generate child certificate from Veeam backup server certificate.","statusCode":400}"
2025-11-20 10:29:33.9942 00010 [10192] ERROR | [PlatformService]: Failure in the PlatformService API: An exception occurred while processing the API request. HttpErrorCode: BadRequest, HttpRequestError: . Message: Veeam.Vbf.Rest.Common.ClientGeneration.ApiException: Failed to generate child certificate from Veeam backup server certificate.

I noticed that my Veeam server certificate was originally from 2019, so I generated a new certificate (Options->Security->Install.. - I choose to generate a new one) and that cleared the error messages and I was able to run backups and add PVE servers again. I guess the old cert lacked permissions/features to be used as a CA/signing cert?

[Dynamic]

Hi Markus,
In addition to the support case take a look at this article. And also try to restart the PVE service after all other services are started.
https://www.veeam.com/kb4687

Thanks Max, I just checked your mentioned KB. The DWORD key was already present with the correct port.

I checked the Certificate Topic from @anpa (my whole installation was only ~4 months old) and created a new cert. The behaviour remains.

Code: Select all

2025-11-20 11:26:55.4263 00014 [17016] INFO  | [CertificateStoreBase]: Successfully found the certificate: "Service certificate: CN=hu-vbr-a-20, O=Veeam Software, OU=Veeam Software. EffectiveDate: 24.06.2025 19:38:15. ExpirationDate: 22.06.2035 19:38:15. Thumbprint:71E52729C9E8372AE7AB073A69F3492D640ECF45"
2025-11-20 11:26:55.4263 00014 [17016] INFO  | [VeeamPlatformAuthenticator]: VeeamAuth executable path: 'C:\Program Files\Veeam\Backup and Replication\Backup\VeeamAuth.exe'.
2025-11-20 11:26:55.4263 00014 [17016] INFO  | [VeeamPlatformAuthenticator]: Starting VeeamAuth: "C:\Program Files\Veeam\Backup and Replication\Backup\VeeamAuth.exe" /purpose:DATA /platformId:01208ee8-47fe-4ea8-8727-5115874da1ad
2025-11-20 11:26:56.1202 00016 [17016] ERROR | [VeeamPlatformAuthenticator]: Failed to authenticate using the VeeamAuth client: System.Exception: Failed to execute VeeamAuth: 'Veeam.Backup.Identity.Client.TokenAuthenticationException: Unable to perform authorization: The client does not have access permissions for the resource.

It seems that the new cert (from today) is not recognized by the PVE plugin, logs still shows the cert from 06/2025.
So I switched back to the original certificate.

Update:
I just removed the PVE Plugin and installed it again - this didn't change anything at all.

Additional info: this VBR system is also part of a VRO environment / VRO agent (now on V13) is running on this system.

[newmrac]

Hi.
Have the same issue, after in-place upgrade from the latest B&R 12 to 13.01, all plugins stopped working (PVE, AWS, EntraId, etc). PVE backups are impossible to start. Services of plugins are running, but still no luck.
In VM section only Hyper-V and VMWare are available.
Error notifications from server:

Code: Select all

20/11/2025 10:01:19 Warning Failed to connect to AWS plug-in: Failed to login to platform service (ID: [9a8806c7-85c8-4ecd-8eea-f6fde055e802]): Failed to execute VeeamAuth: 'Veeam.Backup.Identity.Client.TokenAuthenticationException: Unable to perform authorization: The client does not have access permissions for the resource. 
at Veeam.Backup.Identity.Client.HttpIdentityClient.ParseJsonBasedOnStatusCodeAsync[TOk](HttpResponseMessage response, CancellationToken cancellationToken) 
at Veeam.Backup.Identity.Client.HttpIdentityClient.PostTokenAsync(TokenRequest request, CancellationToken cancellationToken) 
at Veeam.Backup.Identity.Client.IdentityClient.RequestTokenByGssAsync(TokenRequestData tokenRequestData, CancellationToken cancellationToken) 
at Veeam.Backup.Identity.Client.SimpleTokenRefresher.RequestTokensAsync(IDateTimeProvider dateTimeProvider, Func`2 refreshFunc, CancellationToken cancellationToken) 
at Veeam.Backup.Identity.Client.SimpleTokenRefresher.CreateAsync(IDateTimeProvider dateTimeProvider, Func`2 refreshFunc, CancellationToken cancellationToken) 
at Veeam.Backup.Identity.Client.AuthorizationSessionFactory.CreateSimpleAsync(Func`2 refreshFunc, CancellationToken cancellationToken) 
at Veeam.Common.Rpc.SAuthorizationSession.CreateByLocalAuthentication(IIdentityClient identityClient, TokenRequestData requestData, Boolean useRefreshToken) 
at Veeam.Common.Rpc.SIdentitySessionProvider.<>c__DisplayClass6_0.<CreateAuthorizedByLocalAuthentication>b__0() 
at Veeam.Common.Rpc.CAuthorizedIdentitySessionProvider.GetIdentitySession() 
at Veeam.Common.Rpc.CGrpcChannelProvider.<>c__DisplayClass10_0.<ForUrl>b__0(IServiceProvider sp) 
at Veeam.Common.Rpc.CGrpcChannelProvider.Initialize(IServiceProvider serviceProvider) 
at Veeam.Common.Rpc.CVbrGrpcClientContextBuilder.Build() 
at VeeamAuth.CProgram.CreateGrpcClientContext(IIdentitySessionProvider identitySessionProvider) 
at VeeamAuth.CProgram.ConnectToVbrServiceWitAuthentication(IIdentityClient identityClient) 
at VeeamAuth.CProgram.AuthenticateAndRegisterPlatformServiceAsync(CVeeamAuthCmdParams parameters) 
at VeeamAuth.CProgram.Main(String[] args) 
' 
20/11/2025 10:01:19 Warning Failed to connect to Microsoft Azure plug-in: Failed to login to platform service (ID: [89622f35-9e48-424b-9d52-a74dc03b65c2]): No valid response from VeeamAuth utility 
20/11/2025 10:01:34 Warning Failed to connect to Google Cloud plug-in: Failed to login to platform service (ID: [30d473aa-1164-4842-959a-c80a776f8cb6]): One or more errors occurred. (The operation has timed out.)

I decided to reinstall Veeam BR 13 completely: saved config, Veeam uninstalled, dropped Postgres database.
After reinstall and import of configuration all worked fine. All plugins were available.
I started backups of PVE VMs. And in the middle of backup interface of the console hanged completely.
I closed console, restarted it and the issue returned: all plugins are absent. Not possible to start\stop PVE tasks. AWS\Entra other plugins stopped working too.
Backups of PVE hanged.

[Gostev]

Folks, please don't dump random unrelated Proxmox issues into this thread (deleted some).

This issue discussed here is likely due to an outdated backup server certificate that does not allow generating child certificates, we have also seen it in some internal labs now.

Backup server fails to renew those old certificates automatically because cryptographic hashes they use are too old. So the issue will not impact relatively fresh VBR installations, but only those first deployed many years ago.

I'm hearing replacing outdated certificate is not a very easy process, we will create a Support KB about this. @tpayton let's prioritize this please.

[tsightler]

Code: Select all

2025-11-20 11:26:55.4263 00014 [17016] INFO  | [CertificateStoreBase]: Successfully found the certificate: "Service certificate: CN=hu-vbr-a-20, O=Veeam Software, OU=Veeam Software. EffectiveDate: 24.06.2025 19:38:15. ExpirationDate: 22.06.2035 19:38:15. Thumbprint:71E52729C9E8372AE7AB073A69F3492D640ECF45"
2025-11-20 11:26:55.4263 00014 [17016] INFO  | [VeeamPlatformAuthenticator]: VeeamAuth executable path: 'C:\Program Files\Veeam\Backup and Replication\Backup\VeeamAuth.exe'.
2025-11-20 11:26:55.4263 00014 [17016] INFO  | [VeeamPlatformAuthenticator]: Starting VeeamAuth: "C:\Program Files\Veeam\Backup and Replication\Backup\VeeamAuth.exe" /purpose:DATA /platformId:01208ee8-47fe-4ea8-8727-5115874da1ad
2025-11-20 11:26:56.1202 00016 [17016] ERROR | [VeeamPlatformAuthenticator]: Failed to authenticate using the VeeamAuth client: System.Exception: Failed to execute VeeamAuth: 'Veeam.Backup.Identity.Client.TokenAuthenticationException: Unable to perform authorization: The client does not have access permissions for the resource.

It seems that the new cert (from today) is not recognized by the PVE plugin, logs still shows the cert from 06/2025.
So I switched back to the original certificate.

Your error is a little bit different, the error with older certificates is "Failed to generate child certificate from Veeam backup server certificate" while your is "Unable to perform authorization". It's a similar issue, but not quite the same, after you generated the new certificate did you restart all Veeam services to make sure they all received new certs issues from the new master VBR cert?

[Dynamic]

I restarted the whole system, didn't change anything. But after that, I reverted back to the original cert (and reboot).
As mentioned, my whole system is about 4/5 month old (started with 12.3.2 @ 06/2025) so this shouldn't be a problem with an outdated certificate.
Providing additional logs for support is no problem.