Failed to index guest fiel system - case # 00152341

[dfowler]

Hello,

I am using Veeam B&R v6.1.0.205 (64 Bit) and for the most part have had no issues. However, I have recently added a Blackberry Enterprise Server Express (BESX) server running on Windows Server 2008 to my backup jobs. The backup on just this one VM fails 99% of the time with the following message:

Code: Select all

Failed to index guest file system. Details: Cannot start service. Machine: [IP]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The service did not respond to the start or control request in a timely fashion. Code: 1053
Failed to prepare guest for hot backup. Error: Cannot start service. Machine: [IP]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The service did not respond to the start or control request in a timely fashion. Code: 1053
Error: Cannot start service. Machine: [IP]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The service did not respond to the start or control request in a timely fashion. Code: 1053

I created a new backup job with this as the only VM, pointed it to a new proxy and repository but no change in result. I have tried scheduling the job to run at different times of the day to determine if it is activity related (company BB server so it is very active). As per other posts on the forum I have checked that UAC is disabled and I am using a domain admin account (same account on all backup jobs). The VSS writers on the VM are all fine and I have rebooted the Veeam B&R server.

If I manually launch the backup job it will occasionally run fine! Before I upgrade to 6.5, as suggested by support, I am curious if anyone else has experienced this?

Thanks in advance for any responses.

Dave

[veremin]

As for me, it looks like Veeam VSS integration did not work, which is not good, since you want it to work for consistent backups.
Please continue working with our technical support team directly to troubleshoot this.
Thanks!

[Vitaliy S.]

On a side note, I would also recommend to take a look at Windows Event log on the BESX server for related events which occurred at the same time you took the backup.

[dfowler]

Hi Vitaliy, I upgraded Veeam B&R to 6.5.0.109 (64 Bit) but unfortunately get the same results i.e.:

Code: Select all

Failed to index guest file system. Details: Cannot start service. Machine: [192.168.0.63]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The service did not respond to the start or control request in a timely fashion. Code: 1053
Failed to prepare guest for hot backup. Error: Cannot start service. Machine: [192.168.0.63]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The service did not respond to the start or control request in a timely fashion. Code: 1053
Error: Cannot start service. Machine: [192.168.0.63]. Service: [VeeamVssSupport]. Cannot start service. Win32 error:The service did not respond to the start or control request in a timely fashion. Code: 1053

The corresponding entries in the System Event log on the BESX server are (in order):
1.) The Volume Shadow Copy service entered the running state.
2.) A timeout was reached (30000 milliseconds) while waiting for the VeeamVssSupport service to connect.
3.) The VeeamVssSupport service failed to start due to the following error: The service did not respond to the start or control request in a timely fashion.

On a side note, I had a successfully backup of the BESX server using Backup Exec v12.5 over the weekend, this was done to rule out VSS services on the VM being the issue plus I needed to get a backup.

Thanks

Dave

[dfowler]

The corresponding entries in the Application Event Logs are (in order):

Code: Select all

1.) Faulting application VeeamGuestAgent.exe, version 6.5.0.109, time stamp 0x50900742, faulting module VeeamGuestAgent.exe, version 6.5.0.109, time stamp 0x50900742, exception code 0xc0000005, fault offset 0x0000000000011d3a, process id 0x32ec, application start time 0x01cdbb76234639d3.
2.) Faulting application VeeamGuestAgent.exe, version 6.5.0.109, time stamp 0x50900742, faulting module VeeamGuestAgent.exe, version 6.5.0.109, time stamp 0x50900742, exception code 0xc0000005, fault offset 0x0000000000011d3a, process id 0x2c48, application start time 0x01cdbb762567c83a.
3.) Faulting application VeeamGuestAgent.exe, version 6.5.0.109, time stamp 0x50900742, faulting module VeeamGuestAgent.exe, version 6.5.0.109, time stamp 0x50900742, exception code 0xc0000005, fault offset 0x0000000000011d3a, process id 0x376c, application start time 0x01cdbb76278733b3.

[Vitaliy S.]

Thanks for more details, let's wait and see what our support team says based on these log entries.

[dfowler]

To update the status; Veeam B&R was patched to the latest 6.5 build with the same result i.e. timeout. The Windows 2008 BESx VM was rebooted and the backup job has been running for the last 3-days. The root cause is unknown but it does appear that the standard troubleshooting step of rebooting the windows server corrected the issue. If the root cause is discovered I will post the fix here but for now a reboot seems to be the answer (for BESx servers anyway). Unfortunately it is not always easy to get a maintenance window on a 7x24 production server to do a reboot as can be seen from how long the case has been open.

Dave