NFC Errors after vMotion and other issues

[sheehanje]

I am a new VEEAM user. We are migrating virtual machines from an EMC SAN to an Equallogic array (2 members, 48 SAS drives per machine)

I have a physical host for VEEAM, with 4 NICS - 1 on the management network - 3 on the iSCSI network. I am using the Microsoft iSCSI initiator for direct SAN backups. The server is fairly beefy - A Dell Poweredge R510 - 8GB RAM, 2 E5520 processors @ 2.27GHZ (Quad Core - 16 logical processors with Hyperthreading).

For the backup repository we are using a 5TB Exagrid for testing (this will go up to 2 5TB Exagrids).

During my migrations, I have been converting all thick provisioned VM's to thin. I have been upgrading the VM hardware level to version 8 and installing the latest VMWare tools. I have the latest Dell Equallogic MPIO software installed - and it is showing connections. I can see my volumes in Disk Management.

The problems started after I added another ESXi 5 host. After installation, VMWare initiated some DRS recommendations to balance the load. All the machines that got put in the new host are experiencing the following message:

4/13/2012 10:23:52 AM :: Error: Client error: NFC server [esxhost] is busy.
Failed to retrieve next FILE_PUT message. File path: [[EQLVolume] VM/VM.vmx]. File pointer: [0]. File size: [3216].

I also notice that sometimes the method is using [san;nbd] and other times it is using just [nbd]. The virtual machines are a mix of Windows and Linux guests.

I've done a little troubleshooting -- I'm getting NFC errors when I download .vmx files from vSphere client connected to our vSphere server - but not when I connect directly to the esx hosts. While VMWare shows the resolution for downloading these files, it really doesn't help my issue with VEEAM.

Also, my backup rates are wildly flucuating from anywhere between 21MB/s (should be much faster) to 180MB/s ... I am getting a bottleneck at source 98%

My questions are: How do I eliminate the NFC errors -- vmotion is a big part of our operations and sometimes change nightly due to server loads. How to I verify my SAN settings are correct, and how do I know iSCSI is using the correct NIC's to connect to the SAN? Do these NICS need to be bridged?

This is the first time I've connected a Microsoft Server (Windows 2008R2 with all the latest patches) to my SAN, so I'm not fully confident I am doing it right, even after following various guides here on VEEAM and the Equallogic site. Would I be better off scrapping this setup and going with the Virtual Appliance mode? I was hoping to offload the backups so we can do some during the day.

Thanks in advanced for any comments/advice you can give -

John Sheehan

[Vitaliy S.]

I've done a little troubleshooting -- I'm getting NFC errors when I download .vmx files from vSphere client connected to our vSphere server - but not when I connect directly to the esx hosts. While VMWare shows the resolution for downloading these files, it really doesn't help my issue with VEEAM.

Does VMware's resolution fix this issue while trying to download VMX through a vCenter Server? If you do not have this error with direct hosts connection, then you can reconfigure Veeam backup server to work with standalone hosts (though it is not recommended approach) and this issue will also go away.

Also, my backup rates are wildly flucuating from anywhere between 21MB/s (should be much faster) to 180MB/s ... I am getting a bottleneck at source 98%

Do you see these numbers for incremental or full job run? Is it the same VM in question?

My questions are: How do I eliminate the NFC errors -- vmotion is a big part of our operations and sometimes change nightly due to server loads. How to I verify my SAN settings are correct, and how do I know iSCSI is using the correct NIC's to connect to the SAN? Do these NICS need to be bridged?

I would recommend to open a support case with VMware, since you've stated that you started to receive these errors when you added one more host to your vCenter Server.

This is the first time I've connected a Microsoft Server (Windows 2008R2 with all the latest patches) to my SAN, so I'm not fully confident I am doing it right, even after following various guides here on VEEAM and the Equallogic site. Would I be better off scrapping this setup and going with the Virtual Appliance mode? I was hoping to offload the backups so we can do some during the day.

I wouldn't recommend scrapping off the existing installation, as it did work for you previously (before adding new host), however to reduce a backup window and offload your existing proxy, you can definitely deploy an additional proxy server to the VM and use this proxy in Virtual Appliance mode to get maximum of jobs performance.

Hope this helps!

[JoshF]

Has anyone else had any success in making these NFC errors disappear. Notice that a scan of google seems to show a lot of us suddenly seemed to start getting these in April