VBO365 Azure Proxy Server deployment fails with Timeout error

[randerson999]

Veeam Support Case #05029663

I'm having trouble rolling out a new proxy server for VBO365 in Azure. Our architecture is completely in Azure .. we have a VBO Server (which is also running the proxy service) and I'm trying to roll out another proxy server. Every time I try to deploy to the new VM, I get a timeout error; if there Windows firewall is default, the timeout error is 1:03s. If I add specific rules on the backup server (tcp/9191) and the remote proxy vm (tcp/9393) or turn off the firewalls altogether, the timeout is 20s.

I've seen 3 instances of this error: 1) Workgroups were not supported in past versions - we're on v5 so this doesn't apply 2) there is a timeout happening due to a large environment - we're only 2 VMs in Azure, so the default 60s WCF timeout value should be fine (I've tested this and made the WCF timeout 600s in the config.xml file, but that didn't help) or 3) there is an actual connectivity or communication issue due to Windows Firewall settings or due to Azure ACLs. The really odd part is that the Veeam Proxy service appears on the remote VM and starts correctly .. but then abruptly stops and is removed when the deployer fails .. so I ostensibly know the communication has happened, at least enough to roll out the new proxy service.

The Azure ACLs are fine - all intra-vnet traffic is permitted, so that should not be an issue. The only other potential issue I can find is with the Windows firewalls, but as I stated, I've tested with the firewalls completely OFF and got the same error. I've also tried re-creating the remote Proxy VM but see all of the same behavior (ie. errors).

(From the GUI)
9/20/2021 9:53:28 PM :: Configure proxy. Error: The open operation did not complete within the allotted timeout of 00:00:00. The time allotted to this operation may have been a portion of a longer timeout. :: 0:01:03

(From the LOGS)
9/20/2021 10:04:17 PM 63 (7228) Error: The open operation did not complete within the allotted timeout of 00:00:00. The time allotted to this operation may have been a portion of a longer timeout.
.
.
9/20/2021 10:04:17 PM 63 (7228) Error: Open timed out after 00:00:00 while establishing a transport session to net.tcp://xxxxxxxxxx:9193/ArchiverProxy. The time allotted to this operation may have been a portion of a longer timeout.
9/20/2021 10:04:17 PM 63 (7228) Type: System.TimeoutException

What am I missing?!

Rgds

[HannesK]

Hello,
I would guess a network issue... for a definite answer, I recommend working with support as they can check the logs.

The full port list can be found here

Best regards,
Hannes

[randerson999]

Thanks for the feedback - I'm waiting on a resolution from Support. I've tried all different firewall rule configurations and have just turned the Windows firewalls off, but it always results in a timeout error at 20s or 1:03s. The logs all point to a timeout as well, but how can that be if there is no firewall?? This is where I'm stuck - it doesn't seem like it should be this complicated, unless I'm just missing something.

[randerson999]

For anyone that may be struggling with the same situation, this did turn out to be a network/blocked port issue.

It was indeed an issue with a single port (TCP/445) on the Backup Proxy VM. For whatever reason, disabling the firewall completely did not allow that traffic .. oddly.

In addition, the Veeam port table doesn't mention TCP/445 in the Backup Proxy server section .. which is why I missed it at first. It is only listed in the Veeam Backup server section at the top (https://helpcenter.veeam.com/docs/vbo36 ... tml?ver=50). The port is only used during the deployment, I believe.

[HannesK]

sounds strange... disabling all firewalls must allow traffic. I also tested a proxy installation in a workgroup and it works fine with port 445 blocked on the VBO server.

If it works for you, that's fine. If you like to know the root cause, I suggest sending all logs to support and double check.