V6 Replication job failing on specific ESX Host

[SunkistDavid]

I initially opened this case with Veeam under case: 5167894 but suports end conclusion is the issue is a VMWare issue. VMWare so far has been unable to diagnose issue and potentially feels its a Veeam issue based on a timing issue they have seen in previous Veeam products. Actually, VMWare took time to read through the Veeam logs extensively whereas Veeam support did not.

Product: V6 V. 158 64Bit

Issue: When performing a replication job on VM located on a specific host VM fails with the following error:

Processing configuration Error: Client error: File does not exist or locked. VMFS path: [[Datastore Name] VMName/VMName.nvram]. Please, try to download specified file using connection to the ESX server where the VM registered. Failed to create NFC download stream. NFC path: [nfc://conn:VirtualCenter Name,nfchost:host-125,stg:datastore-30@VMName/VMName.nvram].

I first moved the VM to a different Datastore same Host and received same error. When I moved VM to different ESX host and kept same Datastore job completed. I then moved the VM back to original host and the job failed.

I can browse the datastore and manually upload the NVRAM file.

I have attempted to replicate other VM's on this host but they all fail with same error. I have 6 other ESX hosts that are working properly for replication jobs.

The 2nd Host I moved the VM is the same configuration and version as the 1st (original) ESX host where job is failing.

I can perform a succesful backup job of the VM when its on the original host just not replication job.

Per VMWare we rebooted the problem host but this did not resolve problem.

One last thing: Replication jobs from V5 pointing to host are successful

Any thoughts or insight would be helpful.

[SunkistDavid]

Forgot to add as part of debugging by VMWare we did NSLookups to verify DNS was working properly and it is.

[SunkistDavid]

Mistake: The Host I'm trying to create replication job on that is failing is an ESXi host whereas my other systems are ESX. Am I not able to create replication jobs for ESXi hosts?

[Gostev]

Host type makes absolutely no difference for v6, you can use either.

[Gostev]

And regarding the issue - if you are able to demonstrate to our support that v5 works fine and v6 does not on the same VM, that should give them enough ammo to escalate the issue into R&D right away. Thanks.

[SunkistDavid]

Thanks I will run a job on V5 so I have logs

[SunkistDavid]

Job in V5 starts successfully and starts replicating. I cancelled job so not to run during production time. I will contact support with this information.

I also verified using same permission account on both V5 and V6 for the VCenter host, the local and remote ESX(i) hosts. I will post update as soon as support has an answer for me.

[Ben Milligan]

Hi David - I've taken ownership of your case and have requested those updated logs. NFC call to download these vm config files virtually is identical from v5 to v6, so I am very interested in seeing the comparison.

Thanks!

[SunkistDavid]

An update to this case after Ben took over. In the end it was found that the remote proxy was the proxy performing the NFC Call to download the VM Config file. In our case we did not have a route between our DR site and the specific ESX host on our DR firewall and thus the job was failing. We added the route and all started working correctly. Interesting in that the remote proxy is performing the call versus the local proxy. I would say possible a design issue that can be addressed but Veeam knows this stuff better then me.

[Gostev]

Yes, this can be classified as a bug.

[Shogan]

Hi all,

I found this thread and have been experiencing a similiar issue with v6 on a POC for using Veeam B&R 6 to do a series of worldwide data centre migrations. Has this bug been patched or is it still an issue awaiting a fix?

[Ben Milligan]

This should be addressed in v6.1.

[Shogan]

Hi Ben,

Thanks for the quick reply

Cheers,
Sean

[Andreas Neufert]

Client error: File does not exist or locked. VMFS path: [[cx3_1] ****.vmx].
Please, try to download specified file using connection to the ESX server where the VM registered.

Had the same issue at a customer yesterday with and ESX 5 host. The root cause was a wrong Subnet Mask at VMKernel Interface.

[Ben Milligan]

When NFC fails on *.vmx, it can also be ports, permissions, or DNS. Most cases it is the latter. Our source agent logs show this connection clearly, and where it is failing.

[Andreas Neufert]

I thought I share this with the Subnet Mask because it is a uncommon one.
You can add the ESXi as Managed Server, you can download Files over the "Files" area, but Job fails.

[Ben Milligan]

Agree, Andy, that is far from common.