Can't back up with Veeam from new SAN

[Cotswold]

Hi,

Got a bit of a problem here - we've been using Veeam Backup & Replication for a year or so, no problems. We installed a new SAN last week and created some VMFS LUNs on it, and have been able to use Storage vMotion to move a test VM to it.

I have set up a new job in Veeam to back up the test server that I've moved to the new SAN but Veeam cannot back it up. The error is:

Backing up object "[CX4_LUN_02] TLCTEST/TLCTEST.vmdk"
Unable to establish direct connection to the shared storage (SAN).
Please ensure that:
- HBA is properly installed in the Veeam Backup server computer, or software iSCSI initiator is configured correctly.
- SAN volume can be seen by operating system in the Windows Disk Management snap-in on the Veeam Backup server.
- Read access is allowed for the Veeam Backup server computer on the corresponding LUN (refer to your SAN documentation).

Direct SAN connection is not available, failing over to network mode...


The LUNs are (I believe) correctly zoned - we are backing up from an EMC Clariion CX4-120c, and there is a storage group created with the ESX hosts and the backup proxy HBA in it.

I can run SanSurfer on the Veeam backup proxy server and it shows that it can see the new LUNs.

However, when I run vcbsandbg.exe, I get the message Lun does not contain any VMFS/LVM signatures for all the LUNs on the new SAN. If I check Windows Disk Management, it shows the new LUNs presented as Unknown, Unreadable, Unallocated.

I can browse the datastores within the Veeam Backup front end, and can see the new LUNs and the VMDK files within it, but I understand that this part of the Veeam tool uses an IP connection with a username/password, rather than a SAN connection, so this may be a red herring.

Can anyone help? Why can Veeam not see VMFS stores on my new SAN? There are definitely VMFS stores there, as we've created the volumes and successfully moved VMs to the LUNs.

[Gostev]

Hi, if vcbsandbg.exe cannot see VMFS signatures, then the problem is not with Veeam. I recommend opening a support case with your SAN vendor on this, because AFAIK some SAN storage devices have special security/access settings that must be set correctly before the data from mounted LUN can actually be accessed. As soon as vcbsandbg.exe can see signatures, Veeam will start working immediately as well. Thanks.

[Cotswold]

Hi Gostev,

Thanks for the update - I have read some other forums and some posts on here and decided to download and install EMC PowerPath 5.5, as the new SAN is an EMC Clariion. I rebooted after installing the software (which automatically installs some drivers) and it's looking much better now - I can see the disks in Disk Management, and when I run the vcbsandbg.exe again now, instead of the message saying "Lun does not contain any VMFS/LVM signatures " it now shows:

Lun contains VMFS/LVM signatures:
VMFS LVM signature

I will run another test backup job tonight and will report back tomorrow, but at the moment it is looking like the installation of EMC PowerPath did the trick.

[Gostev]

Cool, thanks for the update. Sure, Veeam jobs will work just fine now.

[Cotswold]

Hmm, something's still not right - the overnight backup last night still said:

Backing up object "[CX4_LUN_00] TLCTEST2/TLCTEST2.vmdk"
Unable to establish direct connection to the shared storage (SAN).

So....I can see all the disks in Disk Management, I have run vcbsandbg.exe and it reports that it can see VMFS partitions, yet Veeam 5 thinks it can't see the SAN directly. Very odd - I edited the backup job, removed the TLCTEST2 server, browsed for the server using the Datastore button and found it okay (Veeam could see it on the CX4_LUN_00 LUN), but I still get an error.

I will move another server to the new SAN and will try to back it up from scratch in Veeam, using a new backup job, and will report back.

[Gostev]

Sounds like this particular LUN is not connected to a backup server, but some other LUNs are? Veeam does not really interact with SAN directly, we get the data from vStorage API for Data Protection. In this case, the API reports it cannot establish connection to SAN LUN hosting this file in direct access mode.

[Cotswold]

Something's still not right - I've used Storage vMotion to move a test server to another new LUN on the SAN, and now when I create a Veeam job to back that server up and run it I get:


Backing up object "[CX4_LUN_25] TEST/TEST.vmdk"
Unable to establish direct connection to the shared storage (SAN).
Please ensure that:
- HBA is properly installed in the Veeam Backup server computer, or software iSCSI initiator is configured correctly.
- SAN volume can be seen by operating system in the Windows Disk Management snap-in on the Veeam Backup server.
- Read access is allowed for the Veeam Backup server computer on the corresponding LUN (refer to your SAN documentation).

The strange thing now is that the message on the log no longer says "Direct SAN connection is not available, failing over to network mode...". Before I installed PowerPath, each job failed with the above completion message AND said "failing over to network mode". After I installed PowerPath, the job completion message (above) no longer says it's failing over to network mode, even though the "failover to network mode if primary backup mode fails" is checked.

[Gostev]

Try uninstalling PowerPath completely and rebooting, as PowerPath may be causing the issues (I've seen that before). Actually CX4 is very common SAN that many of our customers are using, and nobody ever reported similar issues - so I would be looking at your setup more closely. Feel free to open a support case for assistance and continue working with them. While this topic brakes a number of forum rules, I will keep it open in case other CX4 users can chime in and assist with SAN access configuration. Thanks.

[Cotswold]

My apologies if this is breaking forum rules - I'll go and read the rules to make sure I don't transgress in future. I appreciate you keeping the fault open - I hope to add a resolution in case anyone has the same problem in future.

I have had some success like this: let's say the VM on the new SAN is attached to vSphere host VSPHERE1. If I go into Veeam and add that vSphere host server to the server list, then create a backup job and add the host through the direct link to the VSPHERE1 server, which is now visible as an object when I create the job, then everything works fine - the jobs backs up successfully from the new SAN.

If I create another new job with exactly the same settings, but add the VM to back up by browsing through the vCenter object and picking it from there, it fails, saying it can't get direct access to the SAN. However, the user that I used to add the vCenter server as a server in Veeam DOES have full administrator rights to the datastores in vCenter.

So, the first part of the fix to my problem was to install EMC PowerPath on the Veeam proxy server - this allowed the server to see the VMFS volumes on the new storage. The final part of the fix seems to be related to permissions, but I'm not able to find a solution yet. I will update this thread when I find the answer, for people who may read this thread in future.

[Cotswold]

Hi,

Update on this call, which was logged as support ID 585399:

This is now working correctly. It appears that the configuration we are using, with an EMC Clariion CX4-120c SAN with two fibre switches, requires EMC PowerPath to be installed on the backup server.

Initially all the disks on the new SAN could not be read correctly on the backup server (they were all showing as unreadable/unallocated, and the vcbsandbg.exe utility showed that VMFS volumes could not be read on these disks) so I installed EMC PowerPath v5.5, the latest available, and the disks appeared correctly in Disk Management and vcbsandbg.exe. However, the Veeam jobs still failed and reported that they couldn't find a direct path to the SAN.

The final fix was to uninstall PowerPath v5.5, reboot, then install PowerPath 5.1, and reboot again. This time not only did the backup server report once again that it could see all the disks, but the Veeam backup jobs worked fine.

The thing that triggered this was that I found a page on the VMware knowledge base that showed a compatibility matrix between PowerPath versions and VCB versions, and it showed that the latest version of VCB doesn't support PowerPath later than 5.3. I had a hunch that maybe Veeam 5 was affected by the same problem, and sure enough it seems to be the case. The compatibility matrix page is here:
http://kb.vmware.com/selfservice/micros ... Id=1007479

I have updated this thread for future reference, even though I understand that support type calls should no longer be logged in this forum.

[Gostev]

Thanks for the update! Technical issues are fine to post about as long as you mention your support case ID.