Host-based backup of Microsoft Hyper-V VMs.
Post Reply
rskinner
Influencer
Posts: 19
Liked: 10 times
Joined: Feb 11, 2014 3:05 pm
Full Name: Richard Skinner
Location: London, UK
Contact:

Issue with Windows Server 2012 R2 Update (April 2014 rollup)

Post by rskinner » 3 people like this post

Having installed the recently released Windows Server 2012 R2 Update, AKA Windows 8.1 Update 1 KB 2919355, my Veeam Hyper-V backups failed with errors as shown below.

Code: Select all

03/04/2014 20:58:28 :: Processing 'SERVERNAME' Error: Client error: The system cannot find the file specified.
Failed to open file [\\?\Volume{36db85be-bb6a-11e3-80ce-000af72f0333}\SERVERNAMEWINDOWS SERVER 2012 R2 STANDARD_DISK_1_117A5A0B-5976-4766-B256-3707A2D8CFA7-AutoRecovery.avhdx] in readonly mode.
The engineer (Anatoly) was excellent and was able to prove that the error wasn't related to Veeam by running Windows backup commands using PowerShell. Following the uninstall of KB 2919355, the only one of the six hot fixes comprising the Windows Server 2012 R2 Update listed in Add Remove Programs, I was able to run backups as before. If you're thinking of installing this update, ensure you test it well.

For reference the configuration is:

4x Hyper-V 2012 R2 hosts
Dell EqualLogic SAN 6.0.7
HIT Kit 4.7 EPA
Veeam Backup & Replication 7.0.0.839
m.novelli
Veeam ProPartner
Posts: 504
Liked: 84 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by m.novelli »

Any info if this bug raise only on Hyper-V hosts or also on backup/proxy servers? I've migrated many Veeam Backup servers to Windows 2012 R2 in the last months and they have Windows Updated active

Thanks!
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev » 1 person likes this post

Hi Marco, you are fine. This bug is specific to Hyper-V virtual machine backup from Hyper-V 2012 R2 (once the update is installed, Hyper-V starts to return a different path to one of the VM files). Thanks!
m.novelli
Veeam ProPartner
Posts: 504
Liked: 84 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by m.novelli »

:)
Susanb
Lurker
Posts: 2
Liked: 1 time
Joined: Apr 04, 2014 9:41 pm
Full Name: SusanBradley
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Susanb »

Are you guys working with Microsoft to resolve this? This update has to be installed otherwise these servers won't get future security updates starting in May. We have 30 days to deploy that patch.

If you don't have contacts with Microsoft, let me know. There's a time sensitivity to this update.
m.novelli
Veeam ProPartner
Posts: 504
Liked: 84 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by m.novelli »

Don't worry, Veeam has strict partnership with Microsoft and is super fast in hotfix release (Veeam, not Microsoft) :)

BTW we should speak about a workaround, not an hotfix, because the bug has been introduced by Microsoft and not by Veeam
JRH
Novice
Posts: 5
Liked: never
Joined: Jan 03, 2014 9:05 pm
Full Name: James
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by JRH »

Hi,

Just to add that we have the same issue on our 2012 R2 cluster (with the new updates) and currently have a support ticket open with Veeam.

Has anyone had any update or ETA from Microsoft?
rskinner
Influencer
Posts: 19
Liked: 10 times
Joined: Feb 11, 2014 3:05 pm
Full Name: Richard Skinner
Location: London, UK
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by rskinner »

I haven't opened a call with MS. The problem is that I've had to put the cluster into production, so I can no longer carry out testing etc.

JRH - what storage are you using? I'm just wondering if it's related to the Dell EqualLogic VSS provider.
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev » 4 people like this post

All, our devs worked on Saturday to produce the hot fix. We will now need a couple of days for full regressive testing, and thus are hoping to make it available by the Patch Tuesday next week (which is when this will hit Windows Update). Thanks!
JRH
Novice
Posts: 5
Liked: never
Joined: Jan 03, 2014 9:05 pm
Full Name: James
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by JRH »

Hi,

It's not an EqualLogic issue as I'm running a Nimble SAN with a software VSS - I've had to uninstall the windows update, but now have working backups again, hopefully Veeam release the fix soon and I will reinstall the windows update.
Susanb
Lurker
Posts: 2
Liked: 1 time
Joined: Apr 04, 2014 9:41 pm
Full Name: SusanBradley
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Susanb » 1 person likes this post

Kudos to the Veeam folks for being very responsive to this issue.
Shralok
Enthusiast
Posts: 51
Liked: 5 times
Joined: Aug 29, 2012 5:36 pm
Full Name: Shralok
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Shralok »

This thread was a great find. I will be holding off on this KB 2919355 until there is a resolution for Veeam. Thanks to all, especially devs for working on Saturday :mrgreen:
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev »

QC has signed off the hot fix for distribution, so you should be able to obtain it through support at this time, and I am sure they will create a KB article tomorrow. Thanks!
kasacompanies
Lurker
Posts: 1
Liked: never
Joined: Apr 09, 2014 5:21 pm
Full Name: Adam Staggenborg
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by kasacompanies »

Can I get a link to the KB article? I can't seem to find it. Thanks
Vitaliy S.
VP, Product Management
Posts: 27025
Liked: 2709 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Vitaliy S. »

There is no KB yet, I believe our support team is still working on it. If you want to get the hotfix, then contact our technical team directly for now.
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev »

Just a head up, our QC has found another issue with this update. This second issue is completely unrelated to the previous one, and has to deal with Windows 2008 and earlier VMs.

Quick summary of all known issues so far:
1. Backing up Windows 2008R2 and later VMs on Hyper-V 2012 R2 hosts fails after update. Hotfix for this issue is already available from Veeam support.
2. Restoring Windows 2008 and earlier backups made on updated Hyper-V 2012 R2 hosts fails. No solution at the moment (my contact at Microsoft has just confirmed the issue present).

I will keep providing updates on the situation in this topic. You can subscribe to notifications by clicking the link at the bottom left of this page.
Mrdoff
Novice
Posts: 5
Liked: never
Joined: Mar 16, 2014 6:12 pm
Full Name: Ola Haraldsson
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Mrdoff »

Any news yet? I started a support chat and asked about the KB but they only gave me this link...
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev »

KB was taken offline to add a note about issue #2, but now it is back online > http://www.veeam.com/kb1863
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev »

Hot fix for the 2nd issue has been implemented, and put into automated test lab for overnight stress testing. If all goes well, we will start pilot deployments tomorrow morning.
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev »

Download in the KB article above has been updated to include fixes for both issues.
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by aeccles »

The KB is missing the file path on step 5. It's: C:\Program Files\Veeam\Backup and Replication\Backup

Also, the KB doesn't state what type of backup is required in Step 7. Is an incremental sufficient or does it need to be a full?
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by Gostev » 1 person likes this post

Incremental run is sufficient. I have asked support to update the article. Thank you!
aeccles
Enthusiast
Posts: 82
Liked: 6 times
Joined: May 01, 2012 3:00 pm
Contact:

Re: Issue with Windows Server 2012 R2 Update KB 2919355

Post by aeccles »

Great thanks, Gostev
KelKnowles
Service Provider
Posts: 30
Liked: 2 times
Joined: Sep 15, 2012 8:01 pm
Full Name: Kelly Michael Knowles
Contact:

Re: Issue with Windows Server 2012 R2 Update (April 2014 rol

Post by KelKnowles »

Have applied the hotfix. Works like a charm.
Kelly Knowles
Principal Systems Architect at PNJ Technology Partners
Veeam Certified Architect and Veeam Certified Engineer - Advanced: Design & Optimization
frankive
Service Provider
Posts: 1091
Liked: 132 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Issue with Windows Server 2012 R2 Update (April 2014 rol

Post by frankive »

For me the path is C:\Program Files\Veeam\Backup and Replication\
and not C:\Program Files\Veeam\Backup and Replication\backup
why is that?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Issue with Windows Server 2012 R2 Update (April 2014 rol

Post by foggy »

Frank, what path are you referring to? The fixed DLL files should be placed in the C:\Program Files\Veeam\Backup and Replication\Backup folder, there are no DLL files in the parent directory.
sebastien
Lurker
Posts: 2
Liked: never
Joined: Jun 27, 2014 9:19 am
Full Name: Sebastien
Contact:

Re: Issue with Windows Server 2012 R2 Update (April 2014 rol

Post by sebastien »

Hi,
I have the same issue on a server but can't download the hotfix. The ftp link provided in the KB (http://www.veeam.com/kb1863) doesn't work for me...
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Issue with Windows Server 2012 R2 Update (April 2014 rol

Post by foggy »

Sebastien, this hotfix was included into Veeam B&R patch 4, please consider upgrading.
sebastien
Lurker
Posts: 2
Liked: never
Joined: Jun 27, 2014 9:19 am
Full Name: Sebastien
Contact:

Re: Issue with Windows Server 2012 R2 Update (April 2014 rol

Post by sebastien »

Thanks a lot, it worked perfectly !!
Post Reply

Who is online

Users browsing this forum: No registered users and 14 guests