Host-based backup of Microsoft Hyper-V VMs.
Post Reply
lenart.plausteiner
Influencer
Posts: 22
Liked: 5 times
Joined: Apr 13, 2015 9:23 pm
Full Name: Lenart Plausteiner
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by lenart.plausteiner »

@Salm

Yes, all those W2k8 R2 VMs are up to date, I'm looking for some kind of offline bundle for IntegrationTools in HyperV just to try re-installing those drivers. This is real pain.


And just for an update, with domain controller W2k8 R2 I have used this workaround:

Stop the "COM+ Event System" service (be careful, it has a few dependencies, such as DHCP Server, so make sure to restart it after).
Restart the "Volume Shadow Copy" service
Start the "COM+ Event System" again (and all it's dependencies).

And the result is that with this workaround AD is not recognized as application integration and therefor you can not restore any Active Directory item (User, Group, GPO...).
Salm
Enthusiast
Posts: 29
Liked: 3 times
Joined: Jan 15, 2017 7:39 pm
Location: France
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Salm » 1 person likes this post

lenart.plausteiner wrote:
And the result is that with this workaround AD is not recognized as application integration and therefor you can not restore any Active Directory item (User, Group, GPO...).
Really ? I didn't even think to check (bad I know, but to be fair I'm only keeping this 2012 R2 DC to check for the bug correction).

Edit : Just checked, can confirm that the 2012 R2 DC doesn't appear in the pre-selected VMs list by Veeam in the AD restoration wizard.

However, AD objects restore works when you go get the VM manually (at least I was able to restore a test user, with a mailbox).
Simon_
Novice
Posts: 3
Liked: never
Joined: Mar 14, 2017 4:22 pm
Full Name: Simon
Location: UK
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Simon_ »

@Mike

Thanks for your reply. I see KB2919355 (April 2014 update rollup for Server 2012 R2) is installed on both of my 2012 R2 servers. But KB3037623 does not appear as an available update. We have installed optional update KB3063109 (Hyper-V integration components update for Windows virtual machines that are running on Windows 10 or Windows Server 2016-based hosts) on both 2012 R2 servers, does this replace KB3037623?

I rebooted the Hyper-V 2016 host server yesterday afternoon (~14:00 GMT 15th March) and could then successfully run Veeam backups of all four guest servers. I have been repeatedly running backups of the four VMs yesterday, last night and this morning, all working perfectly up until an hour ago (~11:00 GMT 16th March).

When the backup fails with error "Failed to create VM recovery snapshot" I can see that Checkpoint files (avhdx) are created in the Hyper-V Virtual Hard Disks directory for each of the four virtual machines and are not removed. When the backup works we see that checkpoint files are created whilst the Veeam backup runs, they are then removed when the job completes.
Simon_
Novice
Posts: 3
Liked: never
Joined: Mar 14, 2017 4:22 pm
Full Name: Simon
Location: UK
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Simon_ »

To add to my previous post. Support case now logged with Veeam (02101451).
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by mkaec »

lenart.plausteiner wrote:Yes, all those W2k8 R2 VMs are up to date...
I wonder if a W2K8 R2 system needs to have the convenience update rollup installed to be considered up to date. This was a "kind of" SP2 and a very late attempt at resolving the issue caused by stopping production of SPs without replacing the system with something that would get bug fixes out of the hotfix stage. Micorosft did do that with Windows 8/Windows Server 2012, but Windows 7/Server 2008 R2 were left in a bad state for a long time. I think it's a good idea to install it, but I had to back it out of the remote desktop servers because there was an issue with roaming profiles. I'm not suggesting it will help with this issue, though if it's not installed it might be worth trying.
lenart.plausteiner
Influencer
Posts: 22
Liked: 5 times
Joined: Apr 13, 2015 9:23 pm
Full Name: Lenart Plausteiner
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by lenart.plausteiner »

I've solved my issue with installing KB3063109. This KB wasn't offered trough Windows Update I had to do this install manualy. Thanks guys, this was very helpfull.

Now I just have to deploy this KB to our other VMs in new HyperV cluster.
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

Good to know Lenart.

But please note that installing this KB won't fix the issue when the VM is a DC (and not running W2K16)
AlexLeadingEdge
Veteran
Posts: 456
Liked: 57 times
Joined: Dec 14, 2015 9:42 pm
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by AlexLeadingEdge »

So for the TLDR crowd, since I don't have time to read through seven pages of comments, the March 2017 cumulative update still doesn't allow the successful backup of a DC using Veeam B&R 9.5 SP1 on Hyper-V?
Salm
Enthusiast
Posts: 29
Liked: 3 times
Joined: Jan 15, 2017 7:39 pm
Location: France
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Salm »

AlexLeadingEdge wrote:So for the TLDR crowd, since I don't have time to read through seven pages of comments, the March 2017 cumulative update still doesn't allow the successful backup of a DC using Veeam B&R 9.5 SP1 on Hyper-V?
Nope, it doesn't.
AlexLeadingEdge
Veteran
Posts: 456
Liked: 57 times
Joined: Dec 14, 2015 9:42 pm
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by AlexLeadingEdge »

Salm wrote:Nope, it doesn't.
Thanks. I still need a resolution for this issue, the exchange logs won't truncate until a backup completes successfully, and since it isn't we're seeing a huge blow-out in log sizes, to the point that we are running a local backup just to get them to truncate.

We are running this every morning, it might help someone else out:

Code: Select all

(Open Command prompt)

Diskshadow
Add volume F: (drive letter for exchange server location)
Begin Backup
Create
End Backup

(At this step you should notice the events in the application log indicating that the backup was indeed successful and logs will now be deleted)
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

AlexLeadingEdge,

Does that mean your DC and Exchange are one VM?

And thanks for the script and sharing it with the community!
AlexLeadingEdge
Veteran
Posts: 456
Liked: 57 times
Joined: Dec 14, 2015 9:42 pm
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by AlexLeadingEdge »

Yes, SBS servers are DC + Exchange Server all in one :)
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler » 1 person likes this post

Ah, SBS, yes, probably even more than DC and Exchange ;-)
frankive
Service Provider
Posts: 1091
Liked: 132 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by frankive »

Ah, gotta miss those SBS 2008/2011 servers...
jed-hyper
Enthusiast
Posts: 39
Liked: 4 times
Joined: Feb 26, 2014 4:42 am
Full Name: Jed Parkes
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by jed-hyper »

I also have this issue.
Running Windows Server 2016 Hyper-V Host with Windows 2012 R2 guest which is a DC
Veeam error: Failed to create VM recovery checkpoint (mode: Veeam application-aware processing) Details: Job failed (''). Error code: '32768'.
Can someone please post here when Microsoft have a fix for this issue ?
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

Hi Jed-hyper,

We certainly will post but unfortunately... No fix yet :-(

What I have done in my environment for now (I know, it is just a lab :-)) is deployed a core 2016 server (minimum config/ resources) and made it a DC. That is being backed up but the 2012 R2 is the one doing the heavy lifting (that one got the resources, roles, DHCP and so on). But at least I have a workable backup of my DC database so I can do item recovery. And in worst case scenario, I can even recover the DC but then I will be without the 2012 R2 (but that is disaster scenario...)
jed-hyper
Enthusiast
Posts: 39
Liked: 4 times
Joined: Feb 26, 2014 4:42 am
Full Name: Jed Parkes
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by jed-hyper »

My config is a live config and i don't have licenses to cover another Windows Server installation. I've just had to disable VSS on the DC until this issue is fixed
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

Can I ask how many VM's are running on your hosts and what license you are running? (Standard/ Datacenter).

The reason I ask is that there is sometimes confusion and I have noticed (and helped) a few people already a few times in the last years when they actually had the licenses covered without knowing it.
jed-hyper
Enthusiast
Posts: 39
Liked: 4 times
Joined: Feb 26, 2014 4:42 am
Full Name: Jed Parkes
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by jed-hyper »

Host is 2016 Standard with 2x VMs
My understanding is that is the max
Salm
Enthusiast
Posts: 29
Liked: 3 times
Joined: Jan 15, 2017 7:39 pm
Location: France
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Salm »

A standard license does not limit you to 2 VMs.

You can have any number of VMs as long as you have individual licenses for those VMs.

2 of those licenses are provided by the standard license you got for your host.
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

Salm is correct

With that license you are allowed to run 2 VMs per host and the Windows OS license is included. The other VMs need to be licensed separately. Depending on the amount of VMs it is sometimes better to license your hosts with the datacenter edition. I can't remember exactly but I thought it was 8 VMs which is the turnaround point when datacenter becomes "cheaper" than licensing your VMs separately

Mike
nmdange
Veteran
Posts: 527
Liked: 142 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by nmdange »

Here's a good overview: http://www.aidanfinn.com/?p=13090

Although with 2016 moving to per-core licensing it might be slightly different.
jed-hyper
Enthusiast
Posts: 39
Liked: 4 times
Joined: Feb 26, 2014 4:42 am
Full Name: Jed Parkes
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by jed-hyper »

That's correct i'm only licensed for 2 VM's so i can't install another one legally without purchasing another license which is not an viable option for me to get around this issue.
So i'm left with waiting for Microsoft to fix this issue
There's no progress on getting this Microsoft issue fixed i'm assuming?
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

Unfortunately I am not aware of any progress on this matter :-(
frankive
Service Provider
Posts: 1091
Liked: 132 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by frankive »

Cant believe Microsoft does not support backing up their own software 8 mintes after GA
-=MACROS=-
Influencer
Posts: 16
Liked: 2 times
Joined: Jun 27, 2016 12:18 pm
Full Name: Rodion Fedin
Location: Germany, Nuremberg
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by -=MACROS=- »

twinstate_ainsley wrote:I find myself in the same position:

- Hyper-V host running Windows Server 2016
- VM running Windows Server 2012 R2 and Microsoft Exchange 2013 CU15
- Microsoft patch KB3037623 (referenced in prior posts) was already installed via Windows updates
- AAP still fails on backups

I was considering uninstall that patch and re-installing using the guidelines from the post by SALM (microsoft-hyper-v-f25/9-5-hyper-v-known ... ml#p230347 , see below). What do folks think?
=======================================
My own Exchange runs on 2012 R2 and Exchange 2016 RTM and CU3.

I have confirmed that backups with AAP work for Exchange 2013 and 2016 (latest CU) on Windows Server 2012 R2, provided KB3037623 is installed on the guest OS (this KB updates integration services).

If the KB has not yet been installed, the vss will fail with the following error
Failed to create VM recovery checkpoint (mode: Veeam application-aware processing) Details: Job failed (''). Error code: '32768'. Failed to create VM recovery snapshot, VM ID '064dac1c-9652-4378-a69c-fa3429ca32a9'.
Retrying snapshot creation attempt (Failed to create production checkpoint.)


The KB is unable to be properly installed if Exchange is already installed (OS will hang on reboot)

The following service must be disabled prior to KB installation
AD Topology.
These services should be set to manual :
Microsoft Exchange Diagnostics
Microsoft Exchange Transport
Microsoft Filtering Management Service
Microsoft Exchange Search Host Controller

The services should be set to Automatic after successful KB installation.

I do not have any information on 2008 R2 Guest OS or previous versions of Exchange.
=======================================
Yes, I can also confirm (accepted for Server 2008 R2, 2012 and 2012 R2)

Regards

Rodi
fireflymaster
Lurker
Posts: 1
Liked: never
Joined: Apr 26, 2017 4:54 pm
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by fireflymaster »

# 02123686

I need a sollution on server 2008R2. On windows 2008R2 is installed exchange 2013, and App Aware not work.
Have you some advice step by step, how to try?

Thank you all, for help.

BR

T.
Mike Resseler
Product Manager
Posts: 8042
Liked: 1262 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by Mike Resseler »

Hi Fireflymaster

Please see above in this thread for the resolution when you are running a 2008R2 VM with Exchange on a 2016 host server.
Is this the problem you have?
mvalpreda
Enthusiast
Posts: 74
Liked: 3 times
Joined: May 06, 2015 10:57 pm
Full Name: Mark Valpreda
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by mvalpreda »

For those that installed KB3063109 or KB3037623....which is newer and how did you do it? It is just a CAB file and everything I have found to install it on 2008 R2 does not work.
mvalpreda
Enthusiast
Posts: 74
Liked: 3 times
Joined: May 06, 2015 10:57 pm
Full Name: Mark Valpreda
Contact:

Re: 9.5 Hyper-V 2016 Known Issues

Post by mvalpreda »

To install the CAB manually

DISM.exe /online /add-package /packagepath:c:\windows6.x-hypervintegrationservices-x64.cab
Post Reply

Who is online

Users browsing this forum: No registered users and 23 guests