9.5 Hyper-V 2016 Known Issues

[lenart.plausteiner]

@Salm

Yes, all those W2k8 R2 VMs are up to date, I'm looking for some kind of offline bundle for IntegrationTools in HyperV just to try re-installing those drivers. This is real pain.


And just for an update, with domain controller W2k8 R2 I have used this workaround:

Stop the "COM+ Event System" service (be careful, it has a few dependencies, such as DHCP Server, so make sure to restart it after).
Restart the "Volume Shadow Copy" service
Start the "COM+ Event System" again (and all it's dependencies).

And the result is that with this workaround AD is not recognized as application integration and therefor you can not restore any Active Directory item (User, Group, GPO...).

[Salm]

And the result is that with this workaround AD is not recognized as application integration and therefor you can not restore any Active Directory item (User, Group, GPO...).

Really ? I didn't even think to check (bad I know, but to be fair I'm only keeping this 2012 R2 DC to check for the bug correction).

Edit : Just checked, can confirm that the 2012 R2 DC doesn't appear in the pre-selected VMs list by Veeam in the AD restoration wizard.

However, AD objects restore works when you go get the VM manually (at least I was able to restore a test user, with a mailbox).

[Simon_]

@Mike

Thanks for your reply. I see KB2919355 (April 2014 update rollup for Server 2012 R2) is installed on both of my 2012 R2 servers. But KB3037623 does not appear as an available update. We have installed optional update KB3063109 (Hyper-V integration components update for Windows virtual machines that are running on Windows 10 or Windows Server 2016-based hosts) on both 2012 R2 servers, does this replace KB3037623?

I rebooted the Hyper-V 2016 host server yesterday afternoon (~14:00 GMT 15th March) and could then successfully run Veeam backups of all four guest servers. I have been repeatedly running backups of the four VMs yesterday, last night and this morning, all working perfectly up until an hour ago (~11:00 GMT 16th March).

When the backup fails with error "Failed to create VM recovery snapshot" I can see that Checkpoint files (avhdx) are created in the Hyper-V Virtual Hard Disks directory for each of the four virtual machines and are not removed. When the backup works we see that checkpoint files are created whilst the Veeam backup runs, they are then removed when the job completes.

[Simon_]

To add to my previous post. Support case now logged with Veeam (02101451).

[mkaec]

Yes, all those W2k8 R2 VMs are up to date...

I wonder if a W2K8 R2 system needs to have the convenience update rollup installed to be considered up to date. This was a "kind of" SP2 and a very late attempt at resolving the issue caused by stopping production of SPs without replacing the system with something that would get bug fixes out of the hotfix stage. Micorosft did do that with Windows 8/Windows Server 2012, but Windows 7/Server 2008 R2 were left in a bad state for a long time. I think it's a good idea to install it, but I had to back it out of the remote desktop servers because there was an issue with roaming profiles. I'm not suggesting it will help with this issue, though if it's not installed it might be worth trying.

[lenart.plausteiner]

I've solved my issue with installing KB3063109. This KB wasn't offered trough Windows Update I had to do this install manualy. Thanks guys, this was very helpfull.

Now I just have to deploy this KB to our other VMs in new HyperV cluster.

[Mike Resseler]

Good to know Lenart.

But please note that installing this KB won't fix the issue when the VM is a DC (and not running W2K16)

[AlexLeadingEdge]

So for the TLDR crowd, since I don't have time to read through seven pages of comments, the March 2017 cumulative update still doesn't allow the successful backup of a DC using Veeam B&R 9.5 SP1 on Hyper-V?

[Salm]

So for the TLDR crowd, since I don't have time to read through seven pages of comments, the March 2017 cumulative update still doesn't allow the successful backup of a DC using Veeam B&R 9.5 SP1 on Hyper-V?

Nope, it doesn't.

[AlexLeadingEdge]

Nope, it doesn't.

Thanks. I still need a resolution for this issue, the exchange logs won't truncate until a backup completes successfully, and since it isn't we're seeing a huge blow-out in log sizes, to the point that we are running a local backup just to get them to truncate.

We are running this every morning, it might help someone else out:

Code: Select all

(Open Command prompt)

Diskshadow
Add volume F: (drive letter for exchange server location)
Begin Backup
Create
End Backup

(At this step you should notice the events in the application log indicating that the backup was indeed successful and logs will now be deleted)

[Mike Resseler]

AlexLeadingEdge,

Does that mean your DC and Exchange are one VM?

And thanks for the script and sharing it with the community!

[AlexLeadingEdge]

Yes, SBS servers are DC + Exchange Server all in one

[Mike Resseler]

Ah, SBS, yes, probably even more than DC and Exchange

[frankive]

Ah, gotta miss those SBS 2008/2011 servers...

[jed-hyper]

I also have this issue.
Running Windows Server 2016 Hyper-V Host with Windows 2012 R2 guest which is a DC
Veeam error: Failed to create VM recovery checkpoint (mode: Veeam application-aware processing) Details: Job failed (''). Error code: '32768'.
Can someone please post here when Microsoft have a fix for this issue ?

[Mike Resseler]

Hi Jed-hyper,

We certainly will post but unfortunately... No fix yet

What I have done in my environment for now (I know, it is just a lab ) is deployed a core 2016 server (minimum config/ resources) and made it a DC. That is being backed up but the 2012 R2 is the one doing the heavy lifting (that one got the resources, roles, DHCP and so on). But at least I have a workable backup of my DC database so I can do item recovery. And in worst case scenario, I can even recover the DC but then I will be without the 2012 R2 (but that is disaster scenario...)

[jed-hyper]

My config is a live config and i don't have licenses to cover another Windows Server installation. I've just had to disable VSS on the DC until this issue is fixed

[Mike Resseler]

Can I ask how many VM's are running on your hosts and what license you are running? (Standard/ Datacenter).

The reason I ask is that there is sometimes confusion and I have noticed (and helped) a few people already a few times in the last years when they actually had the licenses covered without knowing it.

[jed-hyper]

Host is 2016 Standard with 2x VMs
My understanding is that is the max

[Salm]

A standard license does not limit you to 2 VMs.

You can have any number of VMs as long as you have individual licenses for those VMs.

2 of those licenses are provided by the standard license you got for your host.

[Mike Resseler]

Salm is correct

With that license you are allowed to run 2 VMs per host and the Windows OS license is included. The other VMs need to be licensed separately. Depending on the amount of VMs it is sometimes better to license your hosts with the datacenter edition. I can't remember exactly but I thought it was 8 VMs which is the turnaround point when datacenter becomes "cheaper" than licensing your VMs separately

Mike

[nmdange]

Here's a good overview: http://www.aidanfinn.com/?p=13090

Although with 2016 moving to per-core licensing it might be slightly different.

[jed-hyper]

That's correct i'm only licensed for 2 VM's so i can't install another one legally without purchasing another license which is not an viable option for me to get around this issue.
So i'm left with waiting for Microsoft to fix this issue
There's no progress on getting this Microsoft issue fixed i'm assuming?

[Mike Resseler]

Unfortunately I am not aware of any progress on this matter

[frankive]

Cant believe Microsoft does not support backing up their own software 8 mintes after GA

[-=MACROS=-]

I find myself in the same position:

- Hyper-V host running Windows Server 2016
- VM running Windows Server 2012 R2 and Microsoft Exchange 2013 CU15
- Microsoft patch KB3037623 (referenced in prior posts) was already installed via Windows updates
- AAP still fails on backups

I was considering uninstall that patch and re-installing using the guidelines from the post by SALM (microsoft-hyper-v-f25/9-5-hyper-v-known ... ml#p230347 , see below). What do folks think?
=======================================
My own Exchange runs on 2012 R2 and Exchange 2016 RTM and CU3.

I have confirmed that backups with AAP work for Exchange 2013 and 2016 (latest CU) on Windows Server 2012 R2, provided KB3037623 is installed on the guest OS (this KB updates integration services).

If the KB has not yet been installed, the vss will fail with the following error
Failed to create VM recovery checkpoint (mode: Veeam application-aware processing) Details: Job failed (''). Error code: '32768'. Failed to create VM recovery snapshot, VM ID '064dac1c-9652-4378-a69c-fa3429ca32a9'.
Retrying snapshot creation attempt (Failed to create production checkpoint.)


The KB is unable to be properly installed if Exchange is already installed (OS will hang on reboot)

The following service must be disabled prior to KB installation
AD Topology.
These services should be set to manual :
Microsoft Exchange Diagnostics
Microsoft Exchange Transport
Microsoft Filtering Management Service
Microsoft Exchange Search Host Controller

The services should be set to Automatic after successful KB installation.

I do not have any information on 2008 R2 Guest OS or previous versions of Exchange.
=======================================

Yes, I can also confirm (accepted for Server 2008 R2, 2012 and 2012 R2)

Regards

Rodi

[fireflymaster]

# 02123686

I need a sollution on server 2008R2. On windows 2008R2 is installed exchange 2013, and App Aware not work.
Have you some advice step by step, how to try?

Thank you all, for help.

BR

T.

[Mike Resseler]

Hi Fireflymaster

Please see above in this thread for the resolution when you are running a 2008R2 VM with Exchange on a 2016 host server.
Is this the problem you have?

[mvalpreda]

For those that installed KB3063109 or KB3037623....which is newer and how did you do it? It is just a CAB file and everything I have found to install it on 2008 R2 does not work.

[mvalpreda]

To install the CAB manually

DISM.exe /online /add-package /packagepath:c:\windows6.x-hypervintegrationservices-x64.cab