SAN-mode backups failing

[stephaneb]

Hello,

I'm running Veeam backup & Replication v4.1 on Windows Server 2008 Std 32bit SP1.
I followed the instructions to configure SAN mode (http://www.veeam.com/forums/viewtopic.php?f=2&t=2259), and I can see the VMFS datastore disks in the WIndows disk management MMC.

However, when I try to run my abckup jobs in SAN mode, they fail with the following error:

Backing up file "[HYP_HP_DS001] test 03/test 03-flat.vmdk"
Unable to establish direct connection to the shared storage (SAN).
Please ensure that:
- HBA is properly installed in the Veeam Backup server computer, or software iSCSI initiator is configured correctly.
- SAN volume can be seen by operating system in the Windows Disk Management snap-in on the Veeam Backup server.
- Read access is allowed for the Veeam Backup server computer on the corresponding LUN (refer to your SAN documentation).

The job log file contains the following entry:

[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 trivia 'App'] Attempting to open LVID:4b6af210-b0c60d9b-0103-00215ec44a70/4b6af20f-82010e49-adf2-00215ec44a70/1.\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 error 'App'] No path to device LVID:4b6af210-b0c60d9b-0103-00215ec44a70/4b6af20f-82010e49-adf2-00215ec44a70/1 found.\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 error 'App'] Failed to open new LUN LVID:4b6af210-b0c60d9b-0103-00215ec44a70/4b6af20f-82010e49-adf2-00215ec44a70/1.\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: \n
[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 verbose 'App'] Cannot access LUN LVID:4b6af210-b0c60d9b-0103-00215ec44a70/4b6af20f-82010e49-adf2-00215ec44a70/1, cannot open disk snapshot-94[HYP_HP_DS002] test 09/win2008 Datacenter 64bit test02.vmdk. Giving up.\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 error 'BlockList'] \n
[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 info 'BlockList'] Closing connection 129.39.141.233:username\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: [2010-02-09 16:27:17.391 03624 trivia 'SOAP'] Sending soap request to [TCP:xxxx:443]: logout\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: SAN: Cannot access a SAN/iSCSI LUN backing this virtual disk. (Hint: If you are using vcbMounter you can use the option "-m nbd" to switch to network based disk access if this is what you want.) If you were attempting file-level access, stop the vmount Service by typing "net stop vmount2" on a command prompt to force vmount to re-scan for SAN LUNs and re-try the command.\n
[09.02.2010 16:27:17] <13> Info (Client) Service output: DISKLIB-LINK : "san://snapshot-94[HYP_HP_DS002] test 09/win2008 Datacenter 64bit test02.vmdk@xxxx?username/XXX" : failed to open (You do not have access rights to this file). \n

I have opened a support case (#520187), but I was wondering if anyone ever experienced similar issues.
The storage backend is an IBM SVC cluster connected to IBM DS8000 disk bays.
Note that I am not using VCB.

I am wondering how Veeam maps the VMFS datastore to the disks Windows sees... What reference is used?

Regards,

S.Bourdeaud

[Gostev]

Stephane, our product does not do the actual VMFS datastore to disk mapping. vStorage API is what does the low-level storage interactions and actual block level I/O, so these operations are completely abstracted from us.

Such errors usually indicate issues with multipathing software. Try updating it to the latest available version, also removing multipathing software completely and rebooting Veeam Backup often helps (if the issue is connected with multipathing software). It could as well be issues with HBA, its drivers, or specific storage (DS8000). Frankly, you are the first customer I encounter with this type of SAN (at least on forums, may be our support seen customers with this storage before.).

Out of curiosity, did you try to deploy and run Veeam Backup in VCB mode to see if it can access storage normally? If this would fail with the same error, you simply could copy vcbmounter.exe line from our log file and open support case with your SAN vendor as well. This was the course of actions other customers were taking when experiencing similar issues.

Also, I just remembered one case when the issue was caused by failing HBA, and it all started working after HBA was replaced... but this is most unlikely cause probably...

[stephaneb]

Hi,

I actually solved my problem which was that the account used to start the "Veeam Backup and FastSCP" service was not a member of the local "Administrators" user group.
As this is running on a Windows Server 2008 server, I can only assume that a non-privileged account was not able to access the disks to check the LVID.

It seems indeed that the vStorage API uses this "LVID" which seems to be part of the disk information when it is formatted in VMFS to recognize which volume contains the virtual machine files. Once the LVID has been returned by the API, Veeam accesses each disk visible to Windows to search for the LVID (which it finds only if the volume is a VMFS volume).

Anyway, it is now working.
For people out there trying to make it work on IBM SAN Volume Controller (SVC), here is my setup:

On the Veeam server:

• Windows Server 2008 Std 32bit
  
  HBA: Qlogic 2562
  Qlogic EFI Driver version: 02.12
  Firmware version: 04.04.07
  BIOS version: 02.04
  FCode version: 02.05
  
  Qlogic Storport driver v9.1.7.45
  
  IBM SDDDSM Version 2.4.2.1-2
  Microsoft MPIO Version 6.0.6001.18000

SVC is running v5.1.0.697

Regards,

S.Bourdeaud

[Gostev]

Thanks for the update!