VEEAM Hyper-V CSV Backup - Blue Screen

Hyper-V specific discussions

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby Gostev » Fri Feb 24, 2012 1:27 pm

Hi, actually the issue was addressed in Patch #3 that was released couple of weeks ago, please see a sticky topic in the main forum.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby smurteira » Fri Feb 24, 2012 3:18 pm

I'm not so sure about that. I installed patch 3 and one of my hosts bluescreened yesterday
smurteira
Enthusiast
 
Posts: 26
Liked: never
Joined: Wed Feb 22, 2012 1:45 pm
Full Name: Sergio Murteira

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby Gostev » Fri Feb 24, 2012 4:14 pm

Please open a support case then. Reading through my response above, since this issue is not on the Veeam side, but in the Hyper-V design, patch only introduces some workarounds which dramatically reduce the chance of issue appearing, but does not really "fix" anything. There is, however, a setting that you can change on Hyper-V side that will actually prevent the issue (by disabling bugcheck response to a certain event) - our support will be able to share it with you. But first, it would also be good for us to make sure that the patch was deployed correctly, because frankly, in the past few week we have closed few dozens of similar support cases by deploying patch 3 - and you seem to be the only one for whom the patch did not help... please let me know your support case ID as well. Thanks!
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby collinp » Thu Mar 01, 2012 7:35 pm

I am having the same issues as you describe with the same event id's.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby foggy » Fri Mar 02, 2012 10:22 am

Colin, do you have Patch 3 installed? If yes, it would be great if you could also let our support know about that. Thanks!
foggy
Veeam Software
 
Posts: 14765
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby collinp » Wed Mar 07, 2012 2:20 pm

Yes, I have Veeam patch 3 installed as well as the 2 hyper-v hotfixes mentioned earlier in this thread. Case # is 5177545. In the case I uploaded the event logs from both Hyper-v hosts. I get this in the eventlog: The computer has rebooted from a bugcheck. The bugcheck was: 0x0000009e (0xfffffa8062b01b30, 0x00000000000004b0, 0x0000000000000000, 0x0000000000000000). A dump was saved in: C:\Windows\MEMORY.DMP. Report Id: 030712-28002-01.

Hopefully Veeam support will respond to my case. I am still on the trial edition. I would purchase the license if I could just keep my hyper-v hosts online!
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby Gostev » Wed Mar 07, 2012 4:23 pm

For what it's worth, if nothing else helps, there is a really simple way to disable bug check response to the Hyper-V event that is causing this.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby collinp » Wed Mar 07, 2012 4:33 pm

Are you referring to setting this setting to 1? http://blogs.technet.com/b/askcore/arch ... 0009e.aspx

If so, I am worried that setting this to 1 will have worse consequences then a simple reboot of the Hyper-v host.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby hopways » Thu Mar 08, 2012 3:47 pm

having the same issue, licensed 14 cpu sockets for a client
do not set cluster setting HangRecovery=1 as it freezes the cluster CSV volume of machine being backed up goes into deadlock and you have to find the host that owned that
particulat csv volume and then restart its cluster service the remaining vm's on other hosts will then respond and the vm's on the failed deadlocked node will restart on remaining hosts

i have now gone through a process of updating all possible firmware and driver updates and have set repository to 1 concurrent job and selected single hyper v hosts and not the cluster

will update this when i get results back
veeam support just say install patch 3 - did that !! still doesnt work
hopways
Novice
 
Posts: 6
Liked: 4 times
Joined: Wed Mar 07, 2012 3:36 pm
Full Name: john waugh

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby Gostev » Thu Mar 08, 2012 11:59 pm

collinp wrote:Are you referring to setting this setting to 1? http://blogs.technet.com/b/askcore/arch ... 0009e.aspx
If so, I am worried that setting this to 1 will have worse consequences then a simple reboot of the Hyper-v host.

You can just increase the timeout instead, that also helps. Again, the root cause of this issue is CSV ownership transfer (which is quite standard cluster operation) taking long time with some storage devices, causing this bugcheck to go off. While backup of VMs from CSV requires that CSV ownership is transferred to the host running the backed up VM.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2350 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby smurteira » Fri Mar 09, 2012 2:15 pm

I'm glad to see I'm not the only one having this problem. I also get the 0x0000009e bugcheck. My Luns are on a NetApp storage by the way. Wondering what storage everyone else is using?
smurteira
Enthusiast
 
Posts: 26
Liked: never
Joined: Wed Feb 22, 2012 1:45 pm
Full Name: Sergio Murteira

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby collinp » Fri Mar 09, 2012 3:31 pm

I am using a fiber attached EMC CX4. Hopefully there is a patch 4 that will address this? The funny thing is that I never saw these problems when backing up our test Hyper-v cluster with 5 vm's. But once I started trying to back up our production Hyper-v cluster with 25 VM's (and got in big trouble for taking the environment down night after night), we started seeing these issues.
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby collinp » Fri Mar 09, 2012 4:13 pm

I went back and looked at the logs and noticed that the Hyper-v host blue screens usually happen when trying to backup a particular vm. It sometimes occurs on other vm's but most of the time it is this particular vm. There isn't anything unusual about this vm. There are larger vm's in the cluster on slower storage than this one. Are any of you seeing the same thing? Here is the error message that I see on the particular vm when the Hyper-v BSOD occurs.
Code: Select all
2/29/2012 3:26:25 AM :: Unable to create snapshot (Microsoft Software Shadow Copy provider 1.0). Details: This operation returned because the timeout period expired
Failed to prepare cluster shared volume for backup. File name: [C:\ClusterStorage\Volume1\VirtualMachine1\Virtual Machines\37E31A18-C436-4AF1-8641-59899BFFBADD.xml]
--tr:Failed to obtain shadow path for backup item 'C:\ClusterStorage\Volume1\VirtualMachine1\Virtual Machines\37E31A18-C436-4AF1-8641-59899BFFBADD.xml'.
--tr:Failed to perform pre-backup tasks.

2/29/2012 3:59:32 AM :: Unable to allocate processing resources. Error: Unable to allocate resources for taskId: '1a06a110-b78f-4901-9ed6-fc2d21bddcc2'

2/29/2012 3:59:32 AM :: Processing finished with errors at 2/29/2012 3:59:32 AM
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby collinp » Fri Mar 09, 2012 4:17 pm

The only thing that is different about this VM is that in the VMM console its CPU average is 19% whereas the other vm's show a CPU average between 0% and 4%
collinp
Expert
 
Posts: 115
Liked: 10 times
Joined: Tue Feb 14, 2012 8:56 pm
Full Name: Collin P

Re: VEEAM Hyper-V CSV Backup - Blue Screen

Veeam Logoby smurteira » Fri Mar 09, 2012 4:35 pm

I'll have to pay attention to see if this happens on the same one all the time too. When you added your cluster, did you do it by adding the vmm server or did you just add the cluster to veeam?
smurteira
Enthusiast
 
Posts: 26
Liked: never
Joined: Wed Feb 22, 2012 1:45 pm
Full Name: Sergio Murteira

PreviousNext

Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 5 guests