Veeam Ver 6 & vCenter 4.1 Job Failure Issues

VMware specific discussions

Veeam Ver 6 & vCenter 4.1 Job Failure Issues

Veeam Logoby paps40 » Fri Mar 16, 2012 11:37 pm

I'm hoping that someone can give me a hand with this very frustrating error that I have been receiving for the past 2 days which has caused all of our backups to fail repeatedly. The exact error message is the following:

Error: Client error: File does not exist or locked. VMFS path: [[DataStoreName] Server1/Server1.vmx]. Please, try to download specified file using connection to the ESX server where the VM registered. Failed to create NFC download stream. NFC path: [nfc://conn:123.456.789.50,nfchost:host-65,stg:datastore-3433@Server1/Server1.vmx].

I have an open support request with veeam and vmware on this issue. Both vendors are pointing the finger at each other which is very frustrating.

The error seemed to be caused by our switch dying which in turn caused vcenter to lose communication to it's Service Console connection, ESX Hosts and all the vm's. Our entire company went down until we setup a new switch later that day. Everything magically started coming backup up again and working except that our backups failed that night and every night since then. We are running Veeam Version 6 Patch 3, vcenter 4.1, and ESX 4.1 update 2.

Veeam Tech Support is saying that there is an NFC communication issue that vmware should assist in resolving but vmware is saying that veeam is using their API incorrectly. Here is vmware's official responce "Unfortunately, VMware will not be addressing this issue until the next major release at this point, as from our perspective, the API in question is not actually reacting poorly, simply being used incorrectly. The API in question is called the CopyDatastoreFile_Task API, and is designed for use on files that are not locked by our VMFS Distributed Locking system. There are alternative APIs that are available for copying/accessing locked files appropriately (by farming the task to the lock-owning ESX host, or by using one of the alternative lock-slots, depending on what type of lock is in place). "

Here are the things that I have tried before putting tickets in with both veeam and vmware.

1. Rebooted vcenter and veeam server
2. Rebooted esx servers to try and clear this lock.
3. Deleted and re-setup jobs on the veeam server.
4. Verified communication from veeam server to vcenter and all esx hosts.
5. Powered off vcenter server and veeam server.
6. vmotioned vm's to different esx hosts and also different datastores.
7. Restart mgmt agent and vcenter agent on all esx servers.

The jobs are still failing with that same issue.

The weird thing is that If I setup my ESX hosts by IP address individually in veeam I can back them up my vm's but just not through vcenter. I can also not download the .vmx file or any other file through vcenter. I can only download it directly through the esx host. This issue goes away after I reboot the host but comes back once veeam tries to run a backup job and fails.

I'm open to any suggestions that any one has.

Thanks in advance.
paps40
Member
 
Posts: 20
Liked: 10 times
Joined: Mon Dec 12, 2011 4:10 pm
Full Name: Peter Pappas

Re: Veeam Ver 6 & vCenter 4.1 Job Failure Issues

Veeam Logoby paps40 » Fri Mar 23, 2012 4:44 am

This issue has now been fixed. A big thanks to Cody from Veeam Tech Support for figuring this out.

The fix was to uninstall and reinstall the VPXA agent. I guess there was some corruption in the vCenter database that occurred when we lost the Service Console connections. A reboot by itself did not correct the behavior as the problem is that vCenter didn't have the correct NFC path in it's DB.

Here is a link to the article on the vmware support site
http://kb.vmware.com/selfservice/micros ... Id=1003714

Here are the steps that I followed to get this working again.

1. Run the following command via Putty into ESX Host once all vm's have been v-motioned off to another host.
Code: Select all
service mgmt-vmware stop && service vmware-vpxa stop && service vmware-vmkauthd stop && service xinetd restart && rpm -qa | grep -i vpxa | awk '{print $1}' | xargs rpm -ef $1 && userdel vpxuser && rpm -qa | grep -i aam | awk '{print $1}' | xargs rpm -ef $1 && service mgmt-vmware start && service vmware-vmkauthd start

2. Login to vCenter < Choose ESX Host < Right Click < Connect - This initiates a re-install of the agent and prompts you to re-authenticate to the host.

3. Reboot the Host.

Hopefully this post will help someone else who might be having the same issue.
paps40
Member
 
Posts: 20
Liked: 10 times
Joined: Mon Dec 12, 2011 4:10 pm
Full Name: Peter Pappas

Re: Veeam Ver 6 & vCenter 4.1 Job Failure Issues

Veeam Logoby foggy » Fri Mar 23, 2012 10:23 am

Peter, thanks for coming back to update the thread with the resolution!
foggy
Veeam Software
 
Posts: 6582
Liked: 366 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] Error: Client error: File does not exist or

Veeam Logoby Marc.H » Thu Jan 24, 2013 7:26 am

Hi,

I am getting this error from time to time: Error: Client error: File does not exist or locked. VMFS path:....

When the Backup of this VM is retried everything finishes without a problem. The error randoms over different VM's, ESXi Server and Datastores. So It's not a problem as the retry always succeeds, but it's a bit annoying. Anything I can do to troubleshoot this?

Regards
Marc
Marc.H
Enthusiast
 
Posts: 31
Liked: 3 times
Joined: Wed May 23, 2012 7:51 am
Full Name: Marc Holbe

Re: Veeam Ver 6 & vCenter 4.1 Job Failure Issues

Veeam Logoby Vitaliy S. » Thu Jan 24, 2013 7:37 am

If the issue you're experiencing is different from the issue described in this thread, then please contact our support team directly for further troubleshooting. Thanks!
Vitaliy S.
Product Manager
 
Posts: 12015
Liked: 439 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

[MERGED] : Failed to create NFC download stream.

Veeam Logoby AndrewTech » Thu Jan 31, 2013 2:48 pm

Hi,

I have the following problem on a host with esx 4.1 (502767) :

"Error: Client error: File does not exist or locked. VMFS path: [[a5_fc_r5_5d_v1] Exchange2007/Exchange2007.vmx]. Please, try to download specified file using connection to the ESX server where the VM registered. Failed to create NFC download stream. NFC path: [nfc://conn:192.168.255.200,nfchost:host-12140,stg:datastore-2158@Exchange2007/Exchange2007.vmx]. "

The esx OS is fresh installed, the problem occurs on all VMs on the host. The workaround that i found is to migrate the VM on another host (identical), then i can do the backup. If i migrate the VM back on the host the error persists.


What can i do?
AndrewTech
Lurker
 
Posts: 1
Liked: never
Joined: Thu Jan 31, 2013 2:34 pm
Full Name: Andrei Popescu

[MERGED] Error: Client error: File does not exist or locked.

Veeam Logoby kte » Tue Sep 17, 2013 4:08 pm

Error: Client error: File does not exist or locked. VMFS path: [[VMFS_LUN07] xxxx/xxxx.vmx].
Please, try to download specified file using connection to the ESX server where the VM registered.
Failed to create NFC download stream. NFC path: [nfc://conn:vcenter,nfchost:host-462,stg:datastore-155@xxx/xxx.vmx].

this corrupted my vmx and the VM lost his disks, and everything was very slow.

case 00446537
kte
Expert
 
Posts: 140
Liked: 4 times
Joined: Tue Jul 02, 2013 7:48 pm
Full Name: Koen Teugels

Re: Veeam Ver 6 & vCenter 4.1 Job Failure Issues

Veeam Logoby Andreas Neufert » Fri Nov 22, 2013 8:39 am

Had the same issue at a customer yesterday with and ESX 5 host. The root cause was a wrong Subnet Mask at VMKernel Interface.
Andreas Neufert
Veeam Software
 
Posts: 1027
Liked: 127 times
Joined: Wed May 04, 2011 8:36 am
Full Name: Andreas Neufert @AndyandtheVMs Veeam SA


Return to VMware vSphere



Who is online

Users browsing this forum: acatic, Brent_King and 6 guests