Unable to add/rescan Win'2016 domain controller in VBR11

[yorkman21]

Case #05190697 was opened with Veeam Support. Unable to rescan/backup a Win'2016 domain controller that's running on VMWare Workstation 16. I also have Exchange running on it too but no issue installing the Veeam Agent there via VBR and I can telnet to port 6160 on the exchange server. Was able to make a backup of it as well.

-with the DC the issue appears to be with the default Veeam Installer on port 6160. When the windows service is installed by VBR I can only telnet to it from the DC itself but not remotely from any machine on my lan. All servers are on the same subnet. The Windows firewall on the DC and VBR11a is disabled completely and no other firewall software or hardware is installed. Netstat confirms port 6160 is listening on 0.0.0.0. VBR is running on Win'2019 with latest updates installed and windows firewall disabled.

Even if I uninstall the Veeam Installer and bind port 6160 to an IIS site for testing purposes I can still telnet to it from the DC itself only.

For the life of me I can't figure out why I can't telnet to this port remotely. I've tried adding the DC server in VBR manually and setting some different port like 4160 to the Veeam Installer service and then I'm able to add the DC to VBR but when I do a backup it performs a rescan and it always fails with:

"Unable to install backup agent: cannot connect to 192.168.2.253 Error: [192.168.2.253] Failed to check whether remote Installer service is available."

So maybe there's a bug in VBR where if you add a windows machine manually then during rescan it fails to use the configured port 4160 and still tries to use 6160?

I can access the admin shares on the DC remotely from VBR on 192.168.2.156. The account used is domain Administrator and it has no issues installing the Veeam Installer service, until it tries to install the Veeam Agent/rescan.

There are no vlans involved and the DC has only the Veeam required software and its dependencies installed. All windows updates installed.

I tried installing the Veeam Agent on the DC and had no issue but I still can't make backups, same error. There is absolutely no firewall anywhere. The DC did have McAfee installed at some point but that was uninstalled cleanly and to make sure the McAfee Cleanup tool was also used. DC has no issues whatsoever except for somehow not allowing this specific port only, 6160.

When I stop the Veeam Installer service on the DC which is configured on port 6160 then I'm no longer able to telnet to it even from the DC so I know nothing else is conflicting with this port. I have already wasted 3 of days on this as I can't figure out what could be blocking this port. PortQry, when I use it from a remote pc says it's Filtered but I don't see how that's possible because there's just no firewall anywhere:

TCP port 6160 (unknown service): FILTERED
portqry.exe -n 192.168.2.253 -e 6160 -p TCP exits with return code 0x00000002.

Any ideas on what else I can do here? I'm beat.

[Andreas Neufert]

If I understand it correctly, you have 2 VMs running on a VMware workstation that you want to backup with Agents.
One VM works as you have envisioned (Exchange) but the Domain Controller does not allow access from the outside.

I would check in the VMware Workstation network card settings from the VM if you have exactly the same network type selected. Maybe one of the network cards are selected as bridge/NAT while the other had selected another type?

Then I would start Pinging between the VMs. Can they reach each other. You can try as well with your telnet method to see if the services are reachable between them.
If they can not ping/telnet6160 each other, then I would look into the VM itself. If the communication is working, you can as well request support from us to check logs what we see there when we try to connect.

Sometime Domain controller have a wired security setting.

[yorkman21]

Sorry but you misunderstood. All vm's can communicate with each other and that includes the Domain Controller. All I'm trying to do is make a backup of the domain controller but the VBR vm (or any other vm on the same network) cannot telnet to port 6160 on the DC even though it's listening on 0.0.0.0 and I can telnet to it from the DC itself.

So far Veeam Support wasn't able to help (and no pun intended I don't think they would be able to figure out how to fix it). I did find a workaround however. I had to take it to the extreme and create a new domain controller (Win'2019), transfer all the roles and ntp from the current Win2016 DC and just shut down the old DC for now. Went to add the new DC in VBR and had no issues with port 6160 (all vm's could finally telnet to it). So something in the old DC is obviously still blocking port 6160 and likely other ports even though there is no firewall on it at all (Win firewall totally disabled). The only thing I can think of is the McAfee antivirus (with firewall), although uninstalled cleanly, did something with the native windows firewall during its installation and messed it up somehow. I just didn't notice since all other ports had no communication issues and the DC was working well including its vpn.

All I have to do is get the vpn policy installed and configured on the new DC and I should be good. Once all is well I'll completely demote the old DC.

[soncscy]

I'm glad you found a workaround, as indeed seems weird. However, just a comment:

> TCP port 6160 (unknown service): FILTERED
portqry.exe -n 192.168.2.253 -e 6160 -p TCP exits with return code 0x00000002.

> So maybe there's a bug in VBR where if you add a windows machine manually then during rescan it fails to use the configured port 4160 and still tries to use 6160?

No, Portqry not reaching it pretty much exculpates any backup software, __something__ is in your route stopping this. My implicit guess is that it was either Windows' network profiles refusing connections or there really was some firewall somehow in the route.

I'm glad you don't have to dive further, but in the future if you see that stuff like portqry can't get through, it's time to break out wireshark. Reproduce the issue and then watch traffic in the dump with the source/target IPs in your cosmetic filters. I get it's annoying, but for my clients, my team and I just go straight into it if we can't see an immediate firewall configuration issue somewhere. Sometimes we get a clearcut answer, sometimes we just an idea which server is the dorky one and remove it/replace it, but it tends to save a lot of that frustration.

Since it's workstation, I cannot help but second Andreas' idea also that it has to do with Workstation's own networking just not passing the connection through or not playing nice with Windows and forcing it to a public profile as mentioned before.

[yorkman21]

Thanks soncscy. But I don't think wireshark will tell me the source of the problem other than something's blocking the traffic on the DC. And this I already know.
Also I definitely had the domain policy connected, and again, the firewall was off and for all policies so there should be no reason for a port to be blocked. Anyway, I'll be sure to give wireshark a shot next time (was planning on doing that actually but I always found it hard to understand what it's telling me with all its codes).