Recently several server backups within a few of our jobs have been failing with the following error:
Backing up file "[DataStore] Server/Server-flat.vmdk" Unable to establish direct connection to the shared storage (SAN). Please ensure that: - HBA is properly installed in the Veeam Backup server computer, or software iSCSI initiator is configured correctly. - SAN volume can be seen by operating system in the Windows Disk Management snap-in on the Veeam Backup server. - Read access is allowed for the Veeam Backup server computer on the corresponding LUN (refer to your SAN documentation). BackupDisk failed Client error: Failed to open VDDK disk [[DataStore] Server/Server.vmdk] ( is read-only mode - [true] ) Failed to open VMDK. Logon attempt with parameters [VC/ESX: [ConsoleServer.company.com];Port: 443;Login: [vmadmin];VMX Spec: [moref=vm-11976];Snapshot mor: [snapshot-16076];Transports: [san]] failed because of the following errors: Server error: End of file
We have 5 jobs with mutliple servers in each job. Each job is configured to use the VMware vStorage API in SAN mode Only. Our backup server is connected to the the datastores in read-only mode (connectivity to the datastores is definitely not the issue). Each job spans multiple datastores. We usualy see this error, for example, let's say job 1 is backing up server1 connected to datastore-1. Job 2 kicks off and it tries to backup server2 also connected to datastore-1 while job 1 is still backing up server1. Server2 in job 2 fails. However, on the retry of job 2 server2 is backed up successfully.
So with that long winded explanation my question is can two seperate jobs in Veeam access the same datastore at the same time? Or is this a function of the vStorage API? Is there a work around?
Any help is much appreciated!
-
- Novice
- Posts: 5
- Liked: never
- Joined: Dec 22, 2009 1:31 pm
- Full Name: Jeremy Jon Hilsen
- Contact:
-
- Chief Product Officer
- Posts: 31695
- Liked: 7207 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Multiple jobs connecting to same Datastore
Hello Jeremy, generally we do not advise on running parallel jobs hitting the same LUN, because it causes a number of problems with VMFS like SCSI reservation timeouts (actually VMware has technical white paper about this, I will try to look it up for you).
Usually, this does not cause actual errors (just some operations like snapshot management start to take ridiculous time), but I guess depending on specific storage/HBA make and model this can lead to issues like that.
To help our customers better align with VMware recommendations around parallel datastore access, we are planning to give ability to create datastore-based jobs in v5 (in other words, stuff jobs with datastore containers). This has been common request from our customers, because it would allow to increase reliability of backups and simplify job management; so I am trying really hard to make sure this makes into v5 release.
Anyhow, if you'd like us to investigate this further to ensure the issue is not caused by something else (unreliable network, overloaded vCenter), then we will need full logs to be submitted to support@veeam.com.
Thank you.
Usually, this does not cause actual errors (just some operations like snapshot management start to take ridiculous time), but I guess depending on specific storage/HBA make and model this can lead to issues like that.
To help our customers better align with VMware recommendations around parallel datastore access, we are planning to give ability to create datastore-based jobs in v5 (in other words, stuff jobs with datastore containers). This has been common request from our customers, because it would allow to increase reliability of backups and simplify job management; so I am trying really hard to make sure this makes into v5 release.
Anyhow, if you'd like us to investigate this further to ensure the issue is not caused by something else (unreliable network, overloaded vCenter), then we will need full logs to be submitted to support@veeam.com.
Thank you.
-
- Novice
- Posts: 5
- Liked: never
- Joined: Dec 22, 2009 1:31 pm
- Full Name: Jeremy Jon Hilsen
- Contact:
Re: Multiple jobs connecting to same Datastore
Mr. Gostev,
Thank your for the quick reply. The white paper would be helpful if you have time to find it. The enhancement of datastore-based jobs would be very helpful. Matter of fact to get around this issue I was planning on doing that manually. I opened a case with support and I am submitting the logs as requested.
Once again your help is very much appreciated!
Best regards,
Jeremy
Thank your for the quick reply. The white paper would be helpful if you have time to find it. The enhancement of datastore-based jobs would be very helpful. Matter of fact to get around this issue I was planning on doing that manually. I opened a case with support and I am submitting the logs as requested.
Once again your help is very much appreciated!
Best regards,
Jeremy
-
- Chief Product Officer
- Posts: 31695
- Liked: 7207 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Multiple jobs connecting to same Datastore
Jeremy, I have finally tracked the white paper down for you
http://www.vmware.com/files/pdf/scalabl ... rmance.pdf
See page 6 for SCSI reservation issues described, and some actual numbers showing the performance effects of SCSI reservations.
Also, just as an update, datastore-based jobs have made it into v5.
Hope this helps!
http://www.vmware.com/files/pdf/scalabl ... rmance.pdf
See page 6 for SCSI reservation issues described, and some actual numbers showing the performance effects of SCSI reservations.
Also, just as an update, datastore-based jobs have made it into v5.
Hope this helps!
Who is online
Users browsing this forum: Google [Bot], Semrush [Bot] and 86 guests