Veeam agent unable to check revocation for cert.

[controlfreak]

I have a veeam server agent managed by a veeam backup and replication server. Our network is air-gapped. The server has a certificate signed by our CA. The agent has the CA cert in the root certs store, and the veeam server cert in the intermediate CA store. Both the CA cert and the server cert contain our internal CRL distribution point. When I try to start a backup from the server, it failed with message:

Managed session 93ac2d06-cb03-4206-a7e8-9740ec783e7d has failed unexpectedly


Agent logs contain:

Code: Select all

[07.09.2018 09:01:05] <14> Info     Connecting to remote VBR, ips: '192.168.1.111', port: '10005', sessionName: 'ManagedModeSession'.
[07.09.2018 09:01:05] <14> Info     Certificate and certificate id found.
[07.09.2018 09:01:05] <14> Info             [RemoteBackupService] Requesting a new remote session: user identity - local system
[07.09.2018 09:01:05] <04> Info     Cannot validate remote certificate chain. The revocation function was unable to check revocation for the certificate.
[07.09.2018 09:01:05] <04> Info      (System.Exception)
[07.09.2018 09:01:05] <04> Info        at Veeam.Backup.Core.CForeignConnectionPool.CheckRemoteCertificateCallback(Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
[07.09.2018 09:01:05] <14> Error    Cannot connect to a backup server BackupServer. Please contact your Veeam Backup & Replication administrator.
[07.09.2018 09:01:05] <14> Error    Exception has been thrown by the target of an invocation. (System.Reflection.TargetInvocationException)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignInvokerClient.InvokeInternal(CForeignInvokerParams invokerParams)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignInvokerClient.InvokeWithRetry(CForeignInvokerParams invokerParams)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignInvokerClient.Invoke(CForeignInvokerParams invokerParams)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignBackupService.Connect(IPAddress[] ipAddresses, UInt16 port, NetworkCredential credential, String sessionName, CForeignServiceSessionType sessionType, String hostName, Boolean keepLeaseAlive, Boolean ignoreKeepAliveFailures)
[07.09.2018 09:01:05] <14> Error    Unable to establish authenticated client-server connection. (System.Exception)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignConnectionPool.Wait(WaitHandle waitObj, CConnectionState state, Int32 timeout, String timeoutError)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignConnectionPool.CreateConnection()
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Common.CPool`2.Acquire()
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignConnectionPool.GetConnection()
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignInvokerClient.<>c__DisplayClass2.<InvokeInternal>b__0()
[07.09.2018 09:01:05] <14> Error    Cannot validate remote certificate chain. The revocation function was unable to check revocation for the certificate.
[07.09.2018 09:01:05] <14> Error     (System.Exception)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignConnectionPool.CheckRemoteCertificateCallback(Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SecureChannel.VerifyRemoteCertificate(RemoteCertValidationCallback remoteCertValidationCallback, ProtocolToken& alertToken)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.CompleteHandshake(ProtocolToken& alertToken)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
[07.09.2018 09:01:05] <14> Error       at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
[07.09.2018 09:01:05] <14> Error       at Veeam.Backup.Core.CForeignConnectionPool.AuthenticateAsClient4Ssl(TcpClient client)
[07.09.2018 09:01:05] <14> Info         Job event 'vaw_ready2work' was disposed. Session: '0797e25c-d294-4380-b021-119745b04aae'.
[07.09.2018 09:01:05] <14> Info         Job event 'vaw_busy' was disposed. Session: '0797e25c-d294-4380-b021-119745b04aae'.
[07.09.2018 09:01:05] <14> Info         Job event 'vaw_started' was disposed. Session: '0797e25c-d294-4380-b021-119745b04aae'.
[07.09.2018 09:01:05] <14> Error    Job [workstation 1 Backup Test - workstation1.domain.local] is in invalid state. SessionId: [0797e25c-d294-4380-b021-119745b04aae]

The agent certificate chain is OK on the client when viewed in MMC > certificates.

The following KB states that there is a known issue with signed certs.
https://my.veeam.com/#/kb2651

More information
Note: Veeam Agent for Microsoft Windows version 2.1 has a known issue with CRL check if a signed certificate is installed on the VBR server. Please contact technical support in order to obtain a fix.


Has anyone had luck with using signed certs to manage agents on an air-gapped network?

Thanks,

Control Freak

[Dima P.]

Hi controlfreak.

Can you please elaborate why you want to use personal certificate when you network is air-gapped (and there is no way to verify the certificate with trusted authority)? Do you have a case ID to share? Thank you in advance!

[controlfreak]

I was attempting to use a certificate signed by our internal CA. We host an internal CRL, so the agent can check a certificates validity.

In this case, upgrading to version 2.2 of the veeam agent for windows fixed the issue. There is a known issue with signed certificates and CRL checks in agent version 2.1.

Thanks,

Control

[Dima P.]

Thank you for sharing for the details and confirming that upgrade resolved your problem. Cheers!

[controlfreak]

Follow-up, I ended up regenerating my server certificates using this guidance as well. Specifically, we needed to add the additional key usage requirements.

https://helpcenter.veeam.com/archive/ba ... l?ver=95u4

Thanks,

-Control