Host-based backup of VMware vSphere VMs.
Post Reply
carlos.hernandez
Lurker
Posts: 2
Liked: never
Joined: Mar 11, 2013 6:26 pm
Full Name: carlos hernandez
Contact:

EFI boot issue

Post by carlos.hernandez »

Hello,
Sorry if this post is a duplicate, searched and couldn't find something similar. Has anyone seen the issues outlined here? http://communities.vmware.com/message/2170214
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: EFI boot issue

Post by Vitaliy S. »

Hello Carlos,

EFI boot support was added as a part of version 6.1 patch 1 update, since then we haven't heard any negative feedback on this.

Thank you!
carlos.hernandez
Lurker
Posts: 2
Liked: never
Joined: Mar 11, 2013 6:26 pm
Full Name: carlos hernandez
Contact:

Re: EFI boot issue

Post by carlos.hernandez »

Hi,
I just started getting this issue today with my 2012 domain controller. Deleting the nvram file seems to be the only fix. I'm running Veeam 6.5 with latest patch.. do you think this issue is unrelated to Veeam then?

Thank you
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: EFI boot issue

Post by Vitaliy S. »

If you can reproduce this issue without Veeam, then this should be investigated by VMware support team. Anyway, I would appreciate if you could report about your findings to our technical support team first.
lejim
Novice
Posts: 3
Liked: never
Joined: Mar 11, 2013 9:53 pm
Full Name: Jimmi
Contact:

Re: EFI boot issue

Post by lejim »

Hi,

Interesting, I also get the same issue with me domain controller only.
Like I said in the vmware communities post, I ran a bunch of test and was unable to reproduce the bug, using only vmware snapshots from vcenter and lot of backup and replication job from veeam on a fresh 2012 vm and on a 2012 promoted to dc.
On my first DC impacted I removed the corrupted nvram file, it was a week ago, as for now the issue doesn't appear again.
On friday my second DC was impacted. Same path for resolution, still working.
I opened a case this week end at veeam, they just awnser that it should be vmware related and I should open a case at vmware. Vmware want me to open a case at veeam :)
Oh I provided a corrupted nvram to veeam in my case.

Ok. At this moment everything is working but issue could came back.

But one strange thing is that it seems only DC get their nvram corrupted. So maybe we should open a case at microsoft ^^

The bad thing is that restore job based on veeam lab won't work anymore since the vm can't boot.
VMwareDarius
Lurker
Posts: 1
Liked: never
Joined: Apr 01, 2013 4:50 pm
Contact:

Re: EFI boot issue

Post by VMwareDarius »

Hi Vitaliy,

I'm one of the developers of VMware's virtual EFI firmware, and I'm investigating this issue from our side. We not yet succeeded in replicating this issue here at VMware, although I'm yet to gain access to a Veeam setup, and we have observed what so far seems to be a strong correlation between reported Veeam usage and this particular failure mode.

The NVRAM files we're seeing from affected customers have had the EFI Boot Options (the Boot#### variables) replicated massively until there is no more EFI NVRAM space left. The final failure occurs when there is insufficient NVRAM to write the bloated BootOrder variable (~400 entries), which leaves the VM without a BootOrder and renders the VM unbootable until the .nvram file is removed.

Possibly relevant information: Our virtual EFI firmware sometimes uses one of the "reserved" bits in the Attributes field of the EFI_LOAD_OPTION descriptor (to solve a problem that we could find no other way of solving). We're seeing that any Boot Option that our firmware automatically creates with that reserved bit set is subsequently replicated many times, with the copies having that reserved bit cleared. Boot Options that do not have the reserved bit set do not get replicated.

I emphasize that we haven't yet determined how the entry is replicated, so it still might not be anything to do with Veeam, or it might be that Veeam is using a Windows API or tool which indirectly triggers this behavior. I'd be interested to learn more about how Veeam interacts with the EFI Variable Store and/or the Windows Boot Configuration Database so that we can try to get to the bottom of this.

Thanks!
--
Darius
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: EFI boot issue

Post by Gostev »

Hi Darius, we are using ImportStore and ExportStore functions to interact with BCD. Thanks!
http://msdn.microsoft.com/en-us/library ... s.85).aspx
http://msdn.microsoft.com/en-us/library ... s.85).aspx
dellock6
VeeaMVP
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: EFI boot issue

Post by dellock6 » 2 people like this post

I would only add to the discussion that is soooo nice to see two companies working together to solve customers' issues instead of the usual "pass the buck" behaviour.

Really appreciated, even if I'm not one of those beeing affected by the problem.

Thanks guys!

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: EFI boot issue

Post by Gostev » 1 person likes this post

We have confirmed the issue without Veeam present using just BCD import/export functions, but also in our testing we have found an easy way to prevent the issue from happening (cleaning BCD before importing), so we are going to address this from our side in Patch 3 which we were planning to release in the coming weeks anyway.
regi
Enthusiast
Posts: 53
Liked: 22 times
Joined: May 10, 2010 7:21 am
Full Name: Regi Pradeeswaran
Contact:

Re: EFI boot issue

Post by regi » 3 people like this post

Thanks for the patch I have installed and done 10+ application aware backups on my 6 X Win 2012 VM AD's & all seems good :D but I like to test in few weeks' time. Hope your patch fixed this issue which is only happening in Win 2012 AD VM's.

:mrgreen:
regi
Enthusiast
Posts: 53
Liked: 22 times
Joined: May 10, 2010 7:21 am
Full Name: Regi Pradeeswaran
Contact:

Re: EFI boot issue

Post by regi » 1 person likes this post

Happy to say all my Win 2012 AD VMs are working fine, looks like patch 3 worked. Like to monitor another 2 weeks. Thanks Veeam & VMWare – Keep up the good work :D :mrgreen: :) :lol:
Post Reply

Who is online

Users browsing this forum: No registered users and 109 guests