VSS related error with Equallogic and Windows Server 2012 R2

Hyper-V specific discussions

VSS related error with Equallogic and Windows Server 2012 R2

Veeam Logoby StephanF » Mon May 30, 2016 2:27 pm 1 person likes this post

Hi there,

we have a disgusting error with Dell Equallogic Hardware VSS and WIndows Server 2012 R2 for with we are searching help since more than 6 months. We had support cases with Veeam and Dell but nobody seems to find the root cause.
Maybe some here as a comparable setup or any idea what we can do. Seems to be my last hope...

Over the months tried to remove as much complexity as possible out of our test scenario. I try to summarize the whole case as short as possible.

Problem:
I am taking a backup of the VMs on my Hyper-V Host. The Equallogic Hardware VSS provider is used. The snapshot is created and the backup is taken successfully.
After the backup the snapshot is released by Veeam. During the snapshot release several errors are logged in the eventlog of the Hyper-V host, starting with many:
MSiSCSI 116 Removal of device MPIO\DISK&VEN_EQLOGIC&PROD_100E-00&REV_8.1_\1&7F6AC24&0&363031394342373134343430343637454332373442353044313432303641 on iSCSI session ffffe00139a9d020-4000013700000008 was vetoed by STORAGE\Volume\{0605d165-264b-11e6-80c0-5cf9ddf26177}
And some more errors afterwards.

According to Dell this happens because the snapshot is removed and the iSCSI session are released while something is still accessing the volume. This leads to the error above, followed by some unsuccessful reconnection attempts etc.
Veeam itself does not log an error. After some minutes the backup job finishes fine.

What makes it really interesting is:
- This only happens on Windows Server 2012 R2. The completely same setup on the same hardware with Windows Server 2012 as the Hyper-V Hosts OS does not produce any errors.
- This only happens when Veeam is creating and releasing the snapshots. Using Microsofts Vssadmin tool for creating and releasing snapshots does not produce any errors.
- Of course, this only happens with the Equallogic Hardware VSS provider.
- The issue does not depend on standalone or clustered Hyper-V Hosts, application aware processing on or off, VMs powered on or off, etc.

I compared the behaviour and logs between Windows Server 2012 and 2012 R2. The only difference that happens is when releasing the VSS snapshot. In Windows 2012 the additional disk in Windows Disk Management (= mounted snapshot) disappears, the iSCSI sessions are closed an the snapshot is deleted. All fine. In 2012 R2 the iSCSI sessions are closed an the snapshot is deleted but the disk stays. Then the errors start. Unsuccessful reconnection attemps etc.

Nobody could find the root cause of this until now. This could also be a disgusting timing issue that behaves a little bit different in Windows Server 2012 R2 or something.

I can not believe we are the only one in the world with this setup and hardware combination. Maybe somebody has seen such errors before or has the same hardware and can tell me if it is working or not.

Environment:
1x Hyper-V Host
- Dell PowerEdge R720
- OS: Windows Server 2012 R2 english
- clean install, only latest patches and drivers
- Dell Equallogic Host Integration Tools 4.8.0

2x Equallogic SAN (PS4110)
- latest Firmware


Veeam case# 01812617


Any input is highly appreciated.

Thanks
Stephan
StephanF
Enthusiast
 
Posts: 30
Liked: 6 times
Joined: Thu Mar 26, 2015 1:15 pm

Re: VSS related error with Equallogic and Windows Server 201

Veeam Logoby PTide » Mon May 30, 2016 3:43 pm

Hi,

What's the name of the vetoing service? Should be listed right after "Vetoing device"

Thanks
PTide
Veeam Software
 
Posts: 3019
Liked: 246 times
Joined: Tue May 19, 2015 1:46 pm

Re: VSS related error with Equallogic and Windows Server 201

Veeam Logoby StephanF » Tue May 31, 2016 7:59 am

Hi,

no vetoing service. That's the whole error message that I posted.
I tried tu use the handle tool and process explorer from Microsoft to see what is still open. But no luck here.

Dell Support said it could be that the volume (mounted snapshot) itself is not completely closed/dismounted/whatever before the VSS snaphot is removed.
StephanF
Enthusiast
 
Posts: 30
Liked: 6 times
Joined: Thu Mar 26, 2015 1:15 pm

Re: VSS related error with Equallogic and Windows Server 201

Veeam Logoby Vitaliy S. » Fri Jun 03, 2016 12:34 pm

Did you try to run different version of Dell Equallogic Host Integration Tools? Just guessing that there could be something wrong with the VSS provider version.
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: VSS related error with Equallogic and Windows Server 201

Veeam Logoby brett.hopkins » Thu Jun 23, 2016 12:47 am

I am also having this issue. Case # 01834223
Environment:
7x Hyper-V Host In Failover Cluster
- Multiple Physical host types
- OS: Windows Server 2012 R2
- Dell Equallogic Host Integration Tools 4.8.0

4x Equallogic SAN (PS4000, PS4100, PS6100 X2)
- Firmware 7.1.8
brett.hopkins
Novice
 
Posts: 3
Liked: never
Joined: Fri Oct 31, 2014 4:43 pm
Full Name: Brett Hopkins

Re: VSS related error with Equallogic and Windows Server 201

Veeam Logoby moderniste » Tue Aug 02, 2016 8:31 am

Same issue here...

3 Hyper-V 2012 r2 (french language) node R630
Dell Equallogic Host Integration Tools 4.8.0
Equallogic PS4210XS Firmware 7.1.12

HIT 4.9 are realeased, anyone tested it?
moderniste
Influencer
 
Posts: 11
Liked: 4 times
Joined: Fri Jan 17, 2014 10:25 am
Full Name: jeremy

Re: VSS related error with Equallogic and Windows Server 201

Veeam Logoby Vitaliy S. » Tue Aug 02, 2016 2:42 pm

Cannot find any solution in both cases mentioned in this thread. One of them was closed due to no response from the customer.
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov


Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 4 guests