Host-based backup of VMware vSphere VMs.
Post Reply
ferrus
Veeam ProPartner
Posts: 299
Liked: 43 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

VM Restore using nbd, instead of direct SAN

Post by ferrus »

Hi

We have a new Veeam 8 installation, with multiple physical proxies. Each proxy has several TBs of local storage, and each is FC attached to the SAN source repositories.

We've just tried a full VM restore, and while it worked successfully - it chose a different proxy to the one where the files are located, and dropped down to network mode.
The messages in the restore log are:
03/12/2015 15:00:06 No available proxies are running on ESX(i) management interface subnet. Using proxies from a different subnet, performance may be impacted.
03/12/2015 15:00:11 Using source proxy 'incorrectproxyserver' [nbd]
We haven't - yet, set up a vmkernel NIC on the Veeam subnet, for Instant Restore/SureBackup etc. But I didn't think this was a requirement for traditional restores?
I've had a quick search and I thought that thick disks were the only requirement for Direct SAN restores.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by Shestakov »

Hi ferrus,
Yes the thick disks is one of the requirements, besides the proxy must have write access to LUNs where VM disks are located.
What about backup, when you perform it, is the proxy used? What version of vSphere are you at?
Thanks!
ferrus
Veeam ProPartner
Posts: 299
Liked: 43 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by ferrus »

Direct SAN is used during the backups, and we're on ESX 5.1
Might be a good point about the RW access. I read that Veeam automatically prevented write access to the LUNs to stop accidental damage, but our storage admin might have presented them RO as well.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by Shestakov »

Yeah, it worth checking if the proxy has the write access.
Also note that for ESX 5.1 restore of IDE disks in the Direct SAN Access mode is not supported.
I would also recommend reading VMware article about transport modes.
Thanks
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by skrause »

ferrus wrote:Direct SAN is used during the backups, and we're on ESX 5.1
Might be a good point about the RW access. I read that Veeam automatically prevented write access to the LUNs to stop accidental damage, but our storage admin might have presented them RO as well.
Veeam doesn't automatically make the LUNs read only, it does stop them from being auto-mounted and potentially being formatted accidentally though.

You have to do some work in DISKPART to get a LUN to be read only (thus why we never really went to direct SAN for our setup, the running of commands in DISKPART to keep my production storage from being wiped made me nervous).
Steve Krause
Veeam Certified Architect
ferrus
Veeam ProPartner
Posts: 299
Liked: 43 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by ferrus » 3 people like this post

Just a quick update, I've managed to workaround the issue.
The disks were presented RW, the VMs satisfied all the requirements - the issue appears to be entirely down to the incorrect proxy selection.

In the Restore VM wizard, if the correct proxy is manually selected rather than left to Auto-detect - Direct SAN mode is used.
Thanks for all the replies.
btridon
Service Provider
Posts: 9
Liked: 1 time
Joined: Oct 26, 2015 10:39 am
Full Name: Benjamin T.
Location: FRANCE
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by btridon » 1 person likes this post

Hello ferrus,

i've encountered the same issue few months ago.
I want just to add some tricks that I found for this same case.

So, we used the Direct-SAN mode to backup and we wanted to use it for the recovery part.
We all know now that mounted LUN on Microsoft Server are in RO but we didn't want to put any LUN online (because of some people in our company who are very curious)

So this is the trick (and it's work) :

In a Server 2008 OS

Run CMD in admin mode and type :
1.diskpart
2.list disk (check the windows disk ID of the RO LUN)
3.select disk n (n is the windows disk id)
4.attrib disk (check the read-only state of the disk)
This will show the attributes disk. If the disk is set to Read Only we will be unable to write data to it. Run the commands below to remove the read only attributes and allow us to write to the disk
5.ATTRIBUTES DISK CLEAR READONLY
6.attrib disk (check again the new read-only state of the disk, see the difference)

With this tricks, LUN are always offline in the disk management window and VMFS datastores are always in the same state.
The only difference, and the great one that it's now possible to write on it (D-SAN restore is possible now!)

For suspicious person, you could try it on a test's VMFS Lun.

With Server 2012 and higher
Open the Powershell console in admin mode and type :
2.Get-disk (list disk)
3.Set-disk "Windows disk id" -IsReadOnly $False (the same that before with CMD : ATTRIBUTES DISK CLEAR READONLY)

It does the same that before.

I hope it will be interesting for someone.

Have a good day ! :D
Buh95
Influencer
Posts: 21
Liked: 4 times
Joined: Nov 17, 2015 1:31 pm
Full Name: H.Schema.
Contact:

[MERGED]: DirectSAN thinking...

Post by Buh95 »

Hi all,

Just want to share a subject with you, about DirectSAN configuration, and precisely about the way to protect VMFS Datastores that are presented (zoned and mapped) to a physical Veeam Server.

Two months ago, we’ve put in production a new Veeam server (V8u3) :
- Physical Dell Server with his own SAS local storage as repository
- Only one Veeam Proxy : the Veeam server itself
- The Veeam Server is “FC attached” to our Production SAN, to be able to backup VM using DirectSAN access.

No real problems at the end to put the infrastructure in place (thanks Veeam for simplicity…).
And no regret, DirectSAN access performance is here…
… Until we decided to test Full VM restoration:

As you may know, when you enable DirectSAN access on your physical Veeam proxy server, default behavior for full restauration is to use DirectSAN access too : When the restauration process begins, proxy is setup for DirectSAN, Datastore are connected, so let’go…

But by default, Your Veeam installer have set the Windows Server SAN policy to “Offline Shared”, so all your connected datastore are, by default and by design, not initialized, and so in Read Only mode.

At this point, any attempt to restore a VM via DirectSAN will failed with a pretty “VDDK Error 16000” (I.e. unable to write to your datastore).
So what… if we want to take advantage of DirectSAN for restauration, we have to put our datastores in Write access mode… but HOW ? What is the best way to do that ? I can say, after looking for a while, that any instruction about this is not clearly mentioned by Veeam documentation (and support confess me this after some rich discussions).

As far as I know, you have those options:

- You decide to give up DirectSAN access for restore operation. But even if you configure your proxy to failover to network (NBD) mode, this failover does not occur if the datastore is write protected (perhaps any new features about this in V9…). To use network mode, the only way is to install another Veeam proxy server and force this proxy to act as end point for sending data to your vCenter, not a really “sexy” solution…

- Initialize each disk/datastore from the Veeam Server point of view (Right-Click -> Initialize, from your Veeam server Disk Management Console). In theory, this not as risk (I’ll let you read the excellent article about this http://rickardnobel.se/vmfs-exposed-to- ... an-policy/ ). Until, one of your colleague decides to work in the disk management console for any reason, and make THE mistake that he shouldn’t have done (delete volume on datastore for example). VMFS header corrupted and so on… Several dozen of VMs lost, Not really reassuring…

- The option that we choose : by default, we were all agree that letting datastores in read only mode was the best solution to protect our production VMware infra. So we’ve scripted, with Powershell, a way to simply put the concerned datastore in write mode using the SET-DISK command let. Example : you want to restore a full VM in the datastore that is viewed by you Veeam server as disk number 6, you pass this command before restoring :

* Set-Disk 6 –isreadonly $false
After restore operation, you rollback your datastore in read only mode :
* Set-Disk 6 –isreadonly $true

I don’t know if there is another solution… and I’m curious having your experience of this DirectSAN feature in production.

I think that Veeam should manage this datastore access layer automatically in the job (really simple in powershell). It would be the easiest and the most secured way of using DirectSAN access. But today, it’s not the case and I hope V9 will be the one…
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by PTide »

Hi,

Thank you for your feedback!
(and support confess me this after some rich discussions)
Could you please post your case ID?

Thank you.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: [MERGED]: DirectSAN thinking...

Post by foggy »

Buh95 wrote:- Initialize each disk/datastore from the Veeam Server point of view (Right-Click -> Initialize, from your Veeam server Disk Management Console). In theory, this not as risk (I’ll let you read the excellent article about this http://rickardnobel.se/vmfs-exposed-to- ... an-policy/ ).
I guess you meant bring the LUN online, but not initialize it (at least, this is what the article is talking about).
ferrus
Veeam ProPartner
Posts: 299
Liked: 43 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by ferrus » 1 person likes this post

Or you could just leave it presented RW, safe in the knowledge that when the rogue sysadmin wipes the disk - you can be back online in no time, thanks to your safe & reliable Veeam backups!! :wink: :lol:

I want a Veeam pen for that ...
ferrus
Veeam ProPartner
Posts: 299
Liked: 43 times
Joined: Dec 03, 2015 3:41 pm
Location: UK
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by ferrus »

Seriously though, are we unusual in having our disks presented RW?
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by PTide »

are we unusual in having our disks presented RW?
Placing disks in RW is OK but only for restore purposes - one should not keep them RW all the time, especially if some rouge-admin is around. Here is the example that some people definetily put disks in RW mode.
Buh95
Influencer
Posts: 21
Liked: 4 times
Joined: Nov 17, 2015 1:31 pm
Full Name: H.Schema.
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by Buh95 »

Case ID : 1081850

Foggy : yes, you're right, it's "bring online" and not "initialize".
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by foggy »

Veeam B&R should switch the LUN to write mode during restore, so I recommend contacting support if this does not occur and you have to do this manually.
Buh95
Influencer
Posts: 21
Liked: 4 times
Joined: Nov 17, 2015 1:31 pm
Full Name: H.Schema.
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by Buh95 »

Oh... Foggy, are you sure ? Because after long discussion with support about this, nobody seemed aware of this fonctionnality.
And, unless I'm mistaken, there's no reference about this in documentation. Right ?

But I confirm that in my environment, Luns do not turn on write mode before restoration.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by foggy »

Yes, Veeam B&R issues a command to switch readonly attribute for the target LUN to false prior actual restore.
Buh95
Influencer
Posts: 21
Liked: 4 times
Joined: Nov 17, 2015 1:31 pm
Full Name: H.Schema.
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by Buh95 »

Well, I'll contact the support again to clarify this. Thanks for your help.
wgrixti
Influencer
Posts: 11
Liked: never
Joined: Oct 30, 2015 1:02 am
Full Name: Wayne Grixti
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by wgrixti »

Did you hear anything further on this? I just had the same thing happen to me where my proxy is Direct SAN connected with volumes assigned the Read Only attribute.
When doing the restore it defaulted to NBD instead of SAN.

I am curious to know if what foggy said is true? I am running v9 Update 1.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by veremin »

Did you hear anything further on this?
This question appears to have been discussed outside of the provided case, so we cannot check it unfortunately.
I am curious to know if what foggy said is true? I am running v9 Update 1.
It is supposed to work the way Alexander's described it, if that doesn't happen, kindly, reach our support team directly.

Thanks.
kroerig
Novice
Posts: 8
Liked: never
Joined: Nov 11, 2014 10:45 am
Full Name: Klaus Rörig
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by kroerig »

I do have the same problem. Veeam B&R 9.5 is not using SAN-Mode for restore.
Case ID: 02000544
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by veremin »

Are you restoring thin disks by any chance?
apronk
Enthusiast
Posts: 76
Liked: 9 times
Joined: Mar 23, 2015 2:47 pm
Full Name: Arend Pronk
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by apronk »

I have the same problem as well. Veeam B&R 9.5 is not using SAN-Mode for restore.
Case ID: 02360276

Used Diskpart to clear the read-only flags, but it doesn't help, restore still happens using NBD.
All my VM's are thick provisioned and backup does work over SAN.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by foggy »

Hi Arend, have you reviewed the possible limitations of direct SAN restore? If nothing applies, please continue looking into this with your support engineer.
apronk
Enthusiast
Posts: 76
Liked: 9 times
Joined: Mar 23, 2015 2:47 pm
Full Name: Arend Pronk
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by apronk »

Hi Alexander,

Unfortunately those limitations do not apply.
It has worked in the past, so it has to do either with Veeam changes or Windows Server 2016 changes (on witch VBR resides).
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by foggy »

I believe our technical staff will be able to track those changes down. Keep us informed.
apronk
Enthusiast
Posts: 76
Liked: 9 times
Joined: Mar 23, 2015 2:47 pm
Full Name: Arend Pronk
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by apronk »

So far I have to say they are not very useful, as you can probably see from the Case ID.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by foggy »

I can see that investigation is still going on, but you can ask for the case escalation, if you feel deeper analysis required.
apronk
Enthusiast
Posts: 76
Liked: 9 times
Joined: Mar 23, 2015 2:47 pm
Full Name: Arend Pronk
Contact:

Re: VM Restore using nbd, instead of direct SAN

Post by apronk » 2 people like this post

Hi Alexander, the case took a good turn and Veeam Support was able to steer me in the right direction.
The problem was in our case duplicated disks, this isn't a problem for backup purposes but for restore purposes it apparently is.
I'm told this would also be solved in Update 3, but I chose to go with MPIO as presented as a solution by Veeam Support.
Configuring MPIO so that the disks are not listed twice anymore solved my restore problem, now VM's are properly restored using SAN transport mode again.

P.S. I never gave MPIO any thought since we switched from iSCSI to FC a couple of years ago...
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 73 guests