Error: Cannot get service content. Soap Fault

[Zew]

I know this problem as been beatin' to death but I'm a bit stumped here.

I dealt with this issue in the path and the error was a bit more pronounced... but this time it was different. For my previous time tackling this see my blog post here: https://zewwy.ca/index.php/2020/09/15/v ... oap-fault/.

In this case, I first noticed my firewall was blocking port 2500 from Veeam to the Proxy and the remote site. So I created a rule to really ensure the Veeam server was able to communicate openly with the proxy server. This then brought on the error "Error: Cannot get service content. Soap fault. No DataDetail: 'get host by name failed in tcp_connect()', endpoint: 'https://vcenter.domain.local:443/sdk'"

Now it's not clear here from the error is actually from the proxy in question, but I knew this from my previous experience, and I went to the proxy server and validated my old fix was still in place. I then checked the first firewall and it was allowing the connection, and then the secondary firewall which was sure enough blocking it. Now I can't remember what possibly changed on our end, but I pretty much know what I can do to fix it, create/extend my rules to allow the connection from the Proxy server to the vCenter server on port 443. But now I have to ask the question... why? like why is this needed? which seems to want to create an NFC stream? All other posts on this seem to post to https://www.veeam.com/kb1198, but that just mentions DNS (Which I fixed in the past), but no mention of the proxy needing to contact vcenter on port 443.

I even read this just to be safe https://helpcenter.veeam.com/docs/backu ... ml?ver=120. But I couldn't locate any network drawings showing how a remote proxy needs to contact vCenter. Just wanna hear others responses before I go opening yet another hole in my firewalls.....

[Zew]

Huh... didn't expect complete dead silence... sigh

[PetrM]

Hello,

Please keep in mind that access from a proxy to vCenter is a requirement that is clearly stated on the page that describes ports and connections (connection to web service).

Furthermore, VMware recommends communicating with vCenter for VDDK usage in order to ensure VM location transparency and minimize the number of connections to save infrastructure resources. In addition, some low-level operations are performed at the VDDK level, which we refer to as API methods.

NFC stream is required to download the VM configuration files (f.g. .vmx, .nvram), but I don't think that the error you're getting is related to NFC.

Although I can't recall more reasons at the moment, here are a few additional examples:
1. In HotAdd mode, we require access to mount disks to a proxy
2. If the infrastructure includes vCenter, we utilize the QueryChangedDiskAreas method to request change tracking information from vCenter
3. When restoring to a thin or thick lazy zeroed disk, VDDK makes specific requests to vCenter to allocate space for writing blocks

Thanks!

[Zew]

Hey PetrM,

Thanks so much for pointing it out in the official reference. Now that you mention it, that makes sense, but I would have figured that the primary Veeam server could initiate all those operations (As it already has access to vCenter as that's how all jobs currently work in this case it's a replication job of a VM but all source and destination targets are at the proxy server/location), and the proxy would do nothing more than handle the I/O traffic since it has access to all the local host resources. So in my understanding there would be no need for the proxy to make the API calls to vCenter as the main Veeam server does that and again the proxy only handles to read/writing of the VM's files (VMX,VMDK,etc)

[PetrM]

Hello,

I agree that the answer does not lie on the surface, and it completely makes sense to assume that a proxy does not need to access vCenter since it solely manages data traffic. However, to handle this traffic, a proxy requires the ability to open virtual disks and interact with a VM managed by vCenter. Therefore, it has to speak with vCenter.

Thanks!

[Zew]

Thanks for clearing that up for me. Always love the Veeam forums and the product.
I adjusted my rule as required, and everything works again.

[aaduncan]

Good day all,

I have been experiencing the similar issue for the past two days:

Processing <VM NAME> Error: Cannot get service content.
Soap fault. HTTP ErrorDetail: '403 - Forbidden | Access to this page is forbidden.
', endpoint: ''
SOAP connection is not available. Connection ID: [<HOST NAME>].
Failed to create NFC download stream. NFC path: [nfc://conn:<HOST NAME>,nfchost:ha-host,stg:63f7bc3f-5522f8e1-e8a8-e43d1a603367@<VM NAME>/<VM NAME>.vmx].
Agent failed to process method {Transfer.FileToText}.

After removing and re-attaching the Standalone Host and vCenter Server to Veeam several times successfully, I follow Veeam KB article https://www.veeam.com/kb1198 with the troubleshooting steps all passing.

I finally resolved the issue when I notice the Host Server NTP service was not running, and the time was 5mins out-of-sync; Starting the NTP service and syncing fixed the issue.

Hope this can helps someone.