Why does a vmware backup proxy operating in virtual appliance/hotadd mode comm with ESXi hosts?

[HYF_JE]

Relevant support case 06170910.

I'm confused by this behavior in Veeam. Assume a backup job is being ran against VMs in a vCenter environment. One VBR server, one vmware backup proxy (preferring the hotadd transport mode), and a VCSA with ESXi hosts.

My understanding is basically the following when a VM is being backed up:

1. Veeam B&R server communicates with the vCenter and handles all snapshot creation/deletion and reconfiguration of disks on the backup proxy VM.
2. The backup proxy connects to TCP port 902 on the ESXi hosts to download (at the very least) VMX files from the ESXi host directly.
3. The backup proxy reads the disks that have been attached to it and makes the backup files (including compression/dedup/encryption) before sending onward to the gateway servers/WAN accelerators/repos.

My problem is understanding point #2. In my environment/case, the backup proxy can connect to the vcenter server via port 443. Why does Veeam need to talk to the ESXi hosts? If it only needs to download the vmx files (very small), why doesn't Veeam get these files through vcenter? Presumably that would be better because as a backup admin, I only need to get my firewall admins to punch one socket through the firewall as opposed to multiple holes for every ESXi host. I can also re-IP ESXi VMK adapter as needed without needing to update firewall exceptions.

As a human operator, I can easily connect to vcenter and download vmx files through the vcenter web UI so I don't think there is a specific limitation here as support suggested.

I'm having difficulty understanding the product limitation or design decision here. Hoping someone can explain it better than support. The documentation is weak on this nuance.

[HannesK]

Hello,
the product always worked like that and port 902 was always documented as requirement. An explanation including the exception for VMware Cloud on AWS is given here.

Best regards,
Hannes

[Andreas Neufert]

Yes, it is possible to work through the vCenter only (https.folders) and not use TCP902 to the ESXi host when Hotadd is used.
This processing has hard limitations on the VMware side. We only use it therefore when we are forced to => VMware Cloud on AWS.

Some of the limitations are around supported characters, throughput and the main point is that the error handling is very limited. As well we can do less things through it directly interacting with the guest files. So that things like Re-IP are not possible.

There is a reg key where you can enforce the usage, but it is not tested outside of VMware Cloud on AWS, so I would not use it.

In the end VMware even on VMware Cloud on AWS will open up TCP902 and we will move to our regular processing there as it is way more stable at scale and has way less support cases which is a huge factor for our 350.000+ vSphere backup customers.

Maybe can you share a bit your concerns related to this port usage. In the end you can harm through both protocols the environment in the same way.

[Andreas Neufert]

Regarding the firewall settings.
You need TCP 443 to the ESXi hosts anyway as this is a requirement by the VMware VDDK kit that all backup vendors use (even at HotAdd).
So I understand the point of the extra port, but as shared above you can harm the ESXi host in the same way through the vcenter https.folder processing.

Regarding IP changes.
Usually you create objects in the firewall (hostname of the server) and then add the IP addresses to it. Then you make rules based on the objects (this ensures clean human readable rules in large environments). So to change the ESXi host IP, you just need to update the firewall object and not the rules.