Host-based backup of Microsoft Hyper-V VMs.
Post Reply
ArturSchneider
Lurker
Posts: 1
Liked: never
Joined: Oct 30, 2014 1:06 pm
Full Name: Artur Schneider
Contact:

HP Lefthand VSS Provider Import Snapshot Error

Post by ArturSchneider »

Case [ID#00667058]

Hello,

i have a big Problem with my Hyper V 2012 R2 and Lefthand P4500 Environment .

My Backup Server is a 2012 R2 physical Server, Hyper-V Role is activated, HP Application Aware Snapshot (Hardware VSS Provider) installed (on Hyper V Hosts too) and he is a Offhost Backup Proxy.
The Hyper V Cluster contains two Hyper V Hosts with Windows Server 2012 R2. I also connected the CSV Luns over iSCSI to the Veeam Server (Offhost Proxy) but the Backup Jobs finish with:
"Unable to create snapshot LeftHand Networks VSS Provider on Hyper-V Offhost Backup Proxy (mode: Crash consistent). Details: Failed to import snapshot: Import snapshot task has failed. Vss error: '0x80042320' "

On the Storage CMC Console i see that one of the Hyper V Server creates a Snapshot but it disappears when the Job Finishs with error.

"
Hope someone can help me.

Best Regards

Artur

EDITED: job log has been removed by a moderator as per forum rules.
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

No One with same Issue ???
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by veremin »

Hi Artur, it seems that your case is still open and under investigation. So, please keep working with support team on addressing the said problem. Thanks.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

Can you create a hardware VSS snapshot on the Hyper-V host itself and not on the off-host proxy server?
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

Hello Vitaly,

yes already tested it on the Hype-Host itself and it worked fine.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

Artur, As far as I remember you have to configure access list to the Hyper-V server (acting as an off-host proxy) in HP storage management console. Basically, both production host and off-host proxy should have identical hardware VSS configuration, in this case there should not be any problems with snapshot creation.
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

Vitaly, what you mean with Access list to the Hyper-V Server ?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

In the HP management console did you give read-only access to the LUNs to your off-host proxy server? Did you configure off-host proxy in the same way as production host?
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

Yes i did, actually it looks like its a Problem with the HP Lefthand Hardware Provider ... Veeam Supporter will have some testings tomorrow with his colleagues from TK2... than we decide if its suggestive to take HP colleagues in this call...

i will continue ...
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

I would recommend checking VSS providers status via command line in that server. Additionally, while you're waiting take a look at Windows Event log, might be helpful.
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

vss Provider Status Looks alright :/ ... ty will check the Event log
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

So here is a Update to my Call..

Supporter made same Tests as before with same result... manually making HP VSS Provider Snapshots from Hyper-V works fine... with Veeam B&R its just not working ...

Call will be escelated to TK 2 ... hope this Guy will find the Issue...

Just found an old Article where someone had the same Issue with HP SAN ... but without Results ...

http://forums.veeam.com/microsoft-hyper ... 14879.html

Dont know if this is an Veeam or an HP Issue but i have to say with Backup Exec the HP VSS Provider worked fine ... Maybe there is another mechanism but at this Call State (after 1 month now) iam very disappointed with the Veeam Support... because there is no Information where the Problem is located ...
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

JaYaKaAzZ wrote: manually making HP VSS Provider Snapshots from Hyper-V works fine... with Veeam B&R its just not working ...
Manual snapshot creating should work just fine, it seems like there is an issue with snapshot transfer, let's see what our T2 engineer will find out after reviewing the debug information.
JaYaKaAzZ wrote:Details: Failed to import snapshot: Import snapshot task has failed. Vss error: '0x80042320'
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ » 2 people like this post

So here is the final Update to my Call...

At first i want to thank Denis Bozhkov and Nicolay Sergeev the T2 engineers who found the solution for our Problem! Really great Job !

So here is the Problem:
In the CMC of the HP SAN the HyperV Servers where all in a Cluster Group which was assigned to the LUN´s. We made a new single entry for the Proxy Server and assigned the LUN´s too and thats the major issue. U have to add the Proxy Server to the Cluster Group and than the Direct SAN Backup will work fine with the Lefthand Hardware Provider. I know sounds very strange but its the solution !

Hope this gonna help someone with same Issue

Best Regards
Artur Schneider
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by frankive »

If you add the off-host proxy to the same LUN, will you be able to set only read-access to the LUNs for the off-host proxy?
JaYaKaAzZ
Influencer
Posts: 24
Liked: 4 times
Joined: Feb 03, 2014 8:01 am
Full Name: Artur Schneider
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by JaYaKaAzZ »

Didnt test that Frank... but its a good question
lowlander
Service Provider
Posts: 450
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by lowlander »

JaYaKaAzZ wrote:So here is the final Update to my Call...

U have to add the Proxy Server to the Cluster Group and than the Direct SAN Backup will work fine with the Lefthand Hardware Provider. I know sounds very strange but its the solution !
Thanks for this solution. It indeed works. We added the off host proxy to the cluster group and the backup is working again.

However, all CSV volumes are presented read/write to the off host proxy.

We followed the following advises:
- san policy=OfflineAll
- automount disable
- automount scrub

This is working perfectly and settings remain after a reboot. However when we start a backup that is using the off host proxy, the automount setting is being enabled again. When rebooting the off host proxy this is quite undiserable regarding the CSV volumes being in read write mode.

I opened a support case 00935459

Anyone experienced similar behaviour ? Offhost proxy is Windows 2008 R2
lowlander
Service Provider
Posts: 450
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by lowlander »

I monitored the backup processes and apparently the process VeeamHvIntegrationSvc.exe initiates registry set command on the key NoAutoMount=0. As a workaround we wrote a post backup script to set this value to 1 after a backup. Also created a computer policy startup / shutdown to disable automount.

We wait now on support regarding the official statement regarding this issue. Above is a workaround for now.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

I have discussed this behavior with our R&D team and automount is, indeed, set to enabled mode on the Hyper-V off-host proxy server. This is required by backup job VSS process, however if san policy is configured as OfflineAll, then CSV should not be mounted with a drive letter. Let's see what our support engineer can find out after looking through the logs.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

@lowlander, do you see the same behavior on any other off-host proxy server?
lowlander
Service Provider
Posts: 450
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by lowlander »

Hi,

we only have one off-host proxy. We looked into the SAN policy in combination with the iSCSI volumes. When enabling SAN policy "offline all", the iSCSI volumes are booted in an online state. When enabling SAN policy "offline shared" iSCSI volumes are listed in disk manager as offline (as expected). I will try to simulate this behaviour in our lab, when time is available :)

* SAN policy we configure in diskpart
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

Hmm... that's weird, cause offline all should also keep new volumes from initialization. I bet you will not see this behavior on another Windows box.
lowlander
Service Provider
Posts: 450
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by lowlander »

Quick update. When Veeam is backing up a Hyper-V VM the snapshot volume is mounted with a drive letter. When backup finishes, the disk is removed from diskmgmt.msc. So basically even when SAN policy is set to "offline shared" mode the disk is configured as online disk (with a drive letter). Disk is listed as RAW volume.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by Vitaliy S. »

Under normal conditions drive letter assignment should not be happening, so I wonder whether these volumes have been initialized previously or not, cause SAN policy should definitely help here.
lowlander
Service Provider
Posts: 450
Liked: 30 times
Joined: Dec 28, 2014 11:48 am
Location: The Netherlands
Contact:

Re: HP Lefthand VSS Provider Import Snapshot Error

Post by lowlander » 1 person likes this post

Hi,

I captured this in a mp4 and updated it with case 00949962
Post Reply

Who is online

Users browsing this forum: No registered users and 29 guests