Host-based backup of Microsoft Hyper-V VMs.
Post Reply
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

v7 and Dell MD3000i errors

Post by aeccles »

I just installed v7 into a setup of Server 2012 and an MD3000i

I'm getting the following three errors throughout the night. I've gotten the messages at the exact time jobs start AS WELL as other times when jobs are not running (perhaps some background checks veeam does?)
Error #1
Event Description: Faulting application name: SMPROV~1.EXE, version: 10.84.26.2, time stamp: 0x50405699 Faulting module name: ntdll.dll, version: 6.2.9200.16420, time stamp: 0x505ab405 Exception code: 0xc0000374 Fault offset: 0x00000000000ea485 Faulting process id: 0x1128 Faulting application start time: 0x01ceaafb22b60747 Faulting application path: C:\PROGRA~2\Dell\MDSTOR~1\VDS_VSS\provider\SMPROV~1.EXE
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll Report Id: 45d2fbba-16ef-11e3-93f6-001517cc709f

Error #2
Event Description: An attempt was made to issue a command to a password protected storage system and the correct password is not known.

Error #3
Event Description: The Modular Disk Storage Hardware Provider service terminated unexpectedly. It has done this 10 time(s).

There is also a post on a dell forum about the exact same messages although he is using different VM backup software.
http://en.community.dell.com/support-fo ... ?pi21859=2


Anyone have any insite on this? I'm traveling today and I'll open a ticket with Veeam on Monday when I am back in the office.

Thanks
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

Hello,

Can you please tell me if you select your hyper-v server in the backup console and click the "Rescan" button do you see the same errors in the event log of your Dell MD3000i storage?

Thank you!
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

A little more information on this...

I just noticed this... I have 3 Hyper-V hosts and they are all listed under Managed Servers/Microsoft Hyper-V within Veeam. However, I've only backed up VMs on two of the hosts. Those two hosts are the ones generating the event logs. The third host hasn't reported anything.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

I'm not an MD3000i expert, but based on the event description it seems like the connection to your storage might be protected with some kind of a password, so when the backup job tries to retrieve VMs data from the storage and does not provide any passwords, it generates this event in the event log. Anyway, try a quick test with a "Rescan" button and let me know the results. Thanks!
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

Vitaliy S. wrote:Hello,

Can you please tell me if you select your hyper-v server in the backup console and click the "Rescan" button do you see the same errors in the event log of your Dell MD3000i storage?

Thank you!
Hi Vitaliy, I just ran the rescan and YES - all three errors occurred.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

Ok, now everything is clear, but it doesn't explain why you still have successful backup jobs. BTW, you DO have successful backup jobs with these errors, right? ;)

To resolve this issue you should look at the configuration of your storage connection. Every 24 hours Veeam performs automatic object/topology discovery action to check whether anything changed in your environment or not. The same procedure is triggered when you run the backup job, this is required for VM backup operation. Now reading back through the error message above, I would recommend contacting Dell support to verify where this password protection option is configured. Once you disable this option or configure some sort of trust relationship between Dell storage and Veeam backup proxy/backup server, this error should go away.

Hope this helps!
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

I do not have my SAN setup to share the LUNS with the Veeam server, so I'm using network backup. Do you think this lack of access could be causing it? If so, is there a way I can tell Veeam to default to network backup instead of direct access?


And yes, all of my backups are running and are error free as far as Veeam is reporting

Thanks
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

I believe by saying network backup, you refer to on-host backup mode? Are you using software or hardware VSS provider? What other errors do you see in the event log?
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

I haven't gotten a clear response from Dell, but it looks like they are suggesting that this may be happening because I did not purchase the premium support features for this SAN. One of those features is for Snapshotting virtual disks.

Does that make sense to you? I could see how it could come up when the jobs run, but I don't know why it would happen when Veeam performs it's daily topology discovery.

As for your last questions. Yes, I was referring to on-host backup mode. ( I think I confused some Hyper-V/VMware terms)
And sorry if this is a stupid question, but how do I know if I am using a software or hardware VSS? Perhaps the feature I was talking about above would have been it?

Thank you
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

aeccles wrote:I haven't gotten a clear response from Dell, but it looks like they are suggesting that this may be happening because I did not purchase the premium support features for this SAN. One of those features is for Snapshotting virtual disks.

Does that make sense to you?
Not really, as during scan job there is no virtual disk snapshotting operation.
aeccles wrote:As for your last questions. Yes, I was referring to on-host backup mode. ( I think I confused some Hyper-V/VMware terms)
And sorry if this is a stupid question, but how do I know if I am using a software or hardware VSS? Perhaps the feature I was talking about above would have been it?
Can you run this command on your backup server/on-host proxy - "vssadmin list providers"? What is the output?
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

Here's the results of the list providers:


C:\Windows\system32>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2012 Microsoft Corp.

Provider name: 'SmVssProvider'
Provider type: Hardware
Provider Id: {24e4e8a3-69cb-4370-8b7b-c5276cd49765}
Version: 10.84.02

Provider name: 'Microsoft File Share Shadow Copy provider'
Provider type: Fileshare
Provider Id: {89300202-3cec-4981-9171-19f59559e0f2}
Version: 1.0.0.1

Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7


C:\Windows\system32>
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

Looks like you're using hardware VSS provider - "Provider name: 'SmVssProvider'" to backup your VMs. Try updating it to the most recent version, if it doesn't help, then I believe you can ignore this message (assuming all your backups are valid and backup jobs are running fine).

Also I wonder if there is a way to disable password protection of your volumes since this is a "premium support feature for this SAN" which you are not using anyway.
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles » 1 person likes this post

OK, to hopefully wrap this up and help out others who may run into this...

It looks like the solution to this is to uninstall the "SmVssProvider" which, when you run "vssadmin list providers," is listed as:
Provider name: 'SmVssProvider'
Provider type: Hardware
Provider Id: {24e4e8a3-69cb-4370-8b7b-c5276cd49765}
Version: 10.84.02

That service initiates the premium add-on Snapshot service that you can buy for the MD line through Dell. If you didn't buy it, you don't need this provider. (as far as I understand it at least)

To uninstall this provider, navigate to the VDS-VSS uninstall folder in the MD Storage folder. By Default it is at:
C:\Program Files (x86)\Dell\MD Storage Software\VDS_VSS\Uninstall Modular Disk Storage Hardware Provider (Be careful here and double check your path. There is more than one uninstaller with a similar path and name.)
Once in that folder, launch the "Uninstall Modular Disk Storage Hardware Provider.exe"
Note that the uninstaller is very rudimentary - there's no GUI or confirmation that it completed. The folder and service will just disappear.

This will remove the "SmVssProvider" and the related Windows Service:
"Modular Disk Storage Hardware Provider" (SMPROVIDER1.EXE - Same as SmProv~1.exe).


In my case, the "Modular Disk Storage Hardware Provider" service was set to "Automatic" start, although it was not running, so I was unable to uninstall the provider without stopping I/O or causing any disruption to my VMs. There was no reboot request either. YMMV though so proceed with caution.

THANKS VERY MUCH to Vitaliy for leading me to the solution!
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

Aeccles, can you please clarify one more thing before we wrap this up? :)

I want you to look through your backup job sessions and verify what VSS prodiver was used previously (search for this line - "Creating snapshot....")? Was it a "SmVssProvider"? Also what provider is used now after you have uninstalled SmVssProvider?
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

Vitaliy S. wrote:Aeccles, can you please clarify one more thing before we wrap this up? :)

I want you to look through your backup job sessions and verify what VSS prodiver was used previously (search for this line - "Creating snapshot....")? Was it a "SmVssProvider"? Also what provider is used now after you have uninstalled SmVssProvider?

In both cases it was "Microsoft Software Shadow Copy provider 1.0 (mode: Veeam application-aware processing.)
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v7 and Dell MD3000i errors

Post by Vitaliy S. »

Ok, then you can keep using your current configuration, I've just thought that you were using hardware provider previously.
timofcourse
Enthusiast
Posts: 36
Liked: 1 time
Joined: Nov 28, 2011 5:18 pm
Full Name: Tim Graffam
Contact:

Re: v7 and Dell MD3000i errors

Post by timofcourse »

I realize this thread is a bit old, but I am having a similar issue as mentioned in the o.p. Unfortunately when trying to uninstall using this referenced file, nothing seems to happen. The folder and services stay untouched and the SmVssProvider are not removed.

Any one happen to have any ideas how to force the removal of this?

Thanks!
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: v7 and Dell MD3000i errors

Post by veremin »

Hi, Tim, since the said hardware provider is shipped by Dell, the Dell support or online community might be a better place to ask this sort of questions. Thanks.
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: v7 and Dell MD3000i errors

Post by aeccles »

timofcourse wrote:I realize this thread is a bit old, but I am having a similar issue as mentioned in the o.p. Unfortunately when trying to uninstall using this referenced file, nothing seems to happen. The folder and services stay untouched and the SmVssProvider are not removed.

Any one happen to have any ideas how to force the removal of this?

Thanks!
Tim,
Is the "Modular Disk Storage Hardware Provider" service started on your host system? If so you may need to restart for the provider to be removed.
timofcourse
Enthusiast
Posts: 36
Liked: 1 time
Joined: Nov 28, 2011 5:18 pm
Full Name: Tim Graffam
Contact:

Re: v7 and Dell MD3000i errors

Post by timofcourse »

Bah... I figured it out. Run as administrator! Not sure why I never thought of that until now. That removes it as previously described.

Thanks!
JamesNT
Service Provider
Posts: 69
Liked: 17 times
Joined: Feb 15, 2016 2:28 pm
Full Name: James Summerlin
Contact:

Re: v7 and Dell MD3000i errors

Post by JamesNT »

I have acquired a Dell MD3200 direct attach storage and am getting the same error. Does removing this provider solve the problem? Our backups using Veeam 9.0 are running fine but it is annoying to see the error in the event logs.

JamesNT
timofcourse
Enthusiast
Posts: 36
Liked: 1 time
Joined: Nov 28, 2011 5:18 pm
Full Name: Tim Graffam
Contact:

Re: v7 and Dell MD3000i errors

Post by timofcourse » 1 person likes this post

Uninstalling did resolve the issue for us.
Post Reply

Who is online

Users browsing this forum: No registered users and 27 guests