Veeam Failed to prepare guest for hot backup (Exchange 2010)

[albertwt]

I'm getting the following error today from Veeam Backup & Replication 7.0.0.833 email alert:

Processing 'PRODHT-CAS01' Error: Cannot start service. Machine: [192.168.15.222]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The system cannot find the file specified. Code: 2
Processing 'PRODHT-CAS02' Error: Cannot start service. Machine: [192.168.15.222]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The system cannot find the file specified. Code: 2

and here's the detailed error:

Failed to prepare guest for hot backup.
Error: Cannot start service. Machine: [192.168.15.222]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The system cannot find the file specified. Code: 2
Error: Cannot start service. Machine: [192.168.15.222]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The system cannot find the file specified. Code: 2

why is that all in a sudden the Veeam backup failed ?

[Shestakov]

Hello Albert,
Please provide your support case number, as required by forum`s rules when posting a question about errors.
Thanks.

[albertwt]

Hello Albert,
Please provide your support case number, as required by forum`s rules when posting a question about errors.
Thanks.

Here it is: Case # 00703425

[Shestakov]

Thanks Albert,
Looks like the problem is caused by Veeam VSS integration service which does not work. Logs review with our technical support team is going to help troubleshoot this.

[albertwt]

Thanks Albert,
Looks like the problem is caused by Veeam VSS integration service which does not work. Logs review with our technical support team is going to help troubleshoot this.

Thanks for the quick reply. So is there anything that I need to do to ensure that the backup can be resumed normally ?

[Shestakov]

The best I can advice now is to keep investigating the issue with Veeam technical support team.
Thanks!

[albertwt]

The best I can advice now is to keep investigating the issue with Veeam technical support team.
Thanks!

The error still persist after leaving it running for another day.

Error: Cannot start service. Machine: [NLB Cluster IP]. Service: [VeeamVssSupport].
Cannot start service.
Win32 error:The system cannot find the file specified.
Code: 2

Why it is happening all in a sudden ?

[Shestakov]

Hello Albert,
I`ve reviewed your issue investigation process with the support team and see that you still experience problems.
Please keep working with the technical support team. Also note, that if you are not satisfied with the provided level of support you can ask them to escalate your ticket to the upper Tier. Thanks.

[albertwt]

Yes it is still not resolved.
I’ve just spoke with the Veeam support team and they recommend to add the following registry entry to be inserted into VBR server:

Registry:
HKEY_LOCAL_MACHINE\SOFTWARE\VeeaM\Veeam Backup and Replication
DWORD: InverseVssProtocolOrder
Value = 1

Followed by the Veeam service restart tomorrow.

[albertwt]

The registry change trick somehow didn't work as expected, it gives different type of error:


During the “InverseVssProtocolOrder” registry enabled:

Processing 'PRODHT-CAS01' Error: Cannot upload guest agent's files to the administrative share [C:\Windows]. Cannot create folder [C:\Windows\VeeamVssSupport] in guest. VIX Error: You do not have access rights to this file Code: 13
Processing 'PRODHT-CAS02' Error: Cannot upload guest agent's files to the administrative share [C:\Windows]. Cannot create folder [C:\Windows\VeeamVssSupport] in guest. VIX Error: You do not have access rights to this file Code: 13

During the “InverseVssProtocolOrder” registry disabled / default behaviour:

Processing 'PRODHT-CAS01' Error: Cannot start service. Machine: [192.168.15.222]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The system cannot find the file specified. Code: 2
Processing 'PRODHT-CAS02' Error: Cannot start service. Machine: [192.168.15.222]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The system cannot find the file specified. Code: 2

Our Veeam BR software is on patch 3 as can be seen from the notification email Veeam Backup & Replication 7.0.0.833

There is VBR 7 Update 4 but it doesn't say specifically about this error that I described above.

any other suggestion would be greatly appreciated.

[albertwt]

here's some more details:

VMware Snapshot Provider
Startup mode: manual
status not running

Volume Shadow Copy
Startup mode: manual
status not running

and the result from vssadmin list writers command:

Code: Select all

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Writer name: 'Task Scheduler Writer'
   Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
   Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
   State: [1] Stable
   Last error: No error

Writer name: 'VSS Metadata Store Writer'
   Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
   Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
   State: [1] Stable
   Last error: No error

Writer name: 'Performance Counters Writer'
   Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
   Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
   State: [1] Stable
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {de9b5a83-e88e-49f4-a997-5034af7065f0}
   State: [1] Stable
   Last error: No error

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {30f5f786-901f-481d-b045-592dab640a07}
   State: [1] Stable
   Last error: No error

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {e26eb620-1d82-48c2-8679-c3341fbb83f7}
   State: [1] Stable
   Last error: No error

Writer name: 'IIS Metabase Writer'
   Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
   Writer Instance Id: {7de5be3e-7591-47e0-a432-366f1bb535ba}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {59455792-d7b9-4340-9467-b6656ce11685}
   State: [1] Stable
   Last error: No error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {6b649bba-bd57-4c11-a70d-1cd756a470c2}
   State: [1] Stable
   Last error: No error

Writer name: 'IIS Config Writer'
   Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
   Writer Instance Id: {10b3387c-6fa8-4cf7-b81f-6e9c973694c6}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {5c915e2a-55f7-4a5a-ae66-7a71eaedc6fb}
   State: [1] Stable
   Last error: No error

[icetrain]

Can anyone explain why we have to force Veeam to inverse the order of how it will try to fetch the data? Shouldn't Veeam try to use one protocol and then switch to the other one as a fallback, instead of stopping and failing?