SureBackup and VMware Tools

[RubinCompServ]

I've started playing with SureBackup in my lab, and come across a puzzling issue. When I perform SB against my Domain Controllers, the job sits at "Powering On" until it reaches the timeout (even as high as 35 minutes) and then fails, claiming that the VM didn't start. The problem is, the VM did start, and I've verified it by actually accessing it via RDP and browsing out to the Internet. I've opened ticket #04027069 and the engineer suggested that the issue was with VMware Tools not running...except they are. From within the VM, I verified that the VMTools service was running, and from vCenter I verified that VMware actually shows the Tools as running (and current). The engineer suggested that I run 'sc start vmtools' from within the VM, which I did (after manually stopping the service first), and that made the SureBackup job proceed as expected. He also suggested that I change the service to "Delayed Start", which I did yesterday (to get it in last night's backup), and that did not resolve the issue. The suggestion to create a script that manually stops and then starts VMTools at boot is obviously out of the question, as I have 500+ servers that may need it, most of which I don't have access to.

Has anyone else experienced this issue and, if so, what you did to resolve it? This seems more like a Veeam bug than an issue with VMware Tools.

Thanks!

[soncscy]

Hi David,

Sounds suspect to be honest -- my experience playing with SureBackup is that it's just about networking, and the cool thing about SureBackup logs is that they're usually pretty clear cut (just from my poking around).

What did support say about what they found in the logs? Can they see where Veeam failed to make the connection? From what I remember, you need a direct route from Veeam to the Virtual Lab, but that's about it -- after that, the Virtual Lab handles the rest of the translation (probably wrong word, but foggy brain right now), so if your Veeam Server is getting to the Virtual Lab, you can rule that out.

If it really is VMware tools related, what version of the VMware tools are you on? We had a number of issues with one of the 10.x tools on Server 2019, and I know there were some with Server 2016, but can't remember off the top of my head. ))

Regardless, I'd push for a more clear cut explanation. I do agree with the assessment it's VMware Tools related, but it'd be nice to know exactly what Veeam isn't able to find.

[RubinCompServ]

Oh, believe me, I'm still pushing. I know the networking is good because I was able to RDP to the restored machine using the masked IP address. The engineer seems to be focused onthis KB article, but the symptoms aren't the same. I just ran another test and verified that the machine was completely up - to the extent that AD was running (it's a DC) and I could RDP to it - even before VMware Tools started. When Tool started, the SB job still didn't pick up on it. I manually stopped and started them (with 10 seconds after the initial start) and the job proceeded. Something is screwy here, and I don't think it's VMware Tools. This is a Windows 2012 R2 server running on ESXi6.7u2 (vm version with Tools version 10338 (which vCenter says is Current).

[RubinCompServ]

I've closed out my ticket with Veeam. This does appear to have been an issue with VMware Tools. While we don't know the root cause of the issue, we were able to resolve it by uninstalling/reinstalling VMware Tools on the servers. Once I did that, the SureBackup jobs ran as expected. It's possible that it was an issue with an older version of the Tools that stuck around over the multiple upgrades.