9.5 Hyper-V 2016 Known Issues

[Mike Resseler]

@salm

Wow, that is a special procedure. Can you tell us from where that workaround comes from? I'm having troubles seeing how this can solve the issue MSFT has with older DC's running on 2016

Thanks
Mike

[Salm]

I found it on Reddit : https://www.reddit.com/r/Veeam/comments ... h=3e393548

As to how or why it fixes the issue, I have absolutely no idea.
I'm guessing there is some error on one of these services initialization, and restarting them clears it ?

Saw it, tried it manually and found it worked 100% (on my 2012 R2 lab at least).

[Mike Resseler]

Thanks @Salm

[frankive]

Hmm.
if we shutdown v,. run backup and fire it up again, i guess it will not truncate serverlogs either, so it seems like @Salm solutions is the way to go.ay

I should mention that we have a 2008 R2 terminalserver which has the excact same error so not sure this is related to only domain controller funcationality..?

[frankive]

Actually have a replication going on now with 15 Vms from 2012 to 2016.
Wondering if we should dealy the transistion until Microsoft has solved this..

[frankive]

Soo, to wrap things up... Veeam/Microsoft doesnt work to backup guest os lower thatn 2016 with Veeam 9.5 and Hyper-v 2016?
Is other backupvendors affected by the same bug?

[Mike Resseler]

Frank,

I am not aware of the issue other than with guest OS-es lower than 2016 and which are running Active Directory at this point in time.

[frankive]

we are seeing this with a 2008 R2 Citrix terminalserver also which are running as a guest.

[Mike Resseler]

Is that terminal server up-to-date? It was indeed an issue when 2016 server went GA, but for other applications besides AD Microsoft has updates (those are updates for the guest system, not on the host)

But maybe this is another application which they have missed?

Can you check if you are up-to-date with that server? (PS: This is with Service Pack 1? Otherwise there is no support anymore. 2008 R2 without SP1 is end-of-life since somewhere 2013 if I am not mistaken)

Let me know
Mike

[frankive]

Hmm, let me check if that terminalserver are up to date
I see also the hyper-v host doesnt include guest services by default, is this somehow also related or is the "backup integration service" which should be do the job?

edit: it is updated last time 6.januar 2017. and yes; it is servicepack 1.

[Mike Resseler]

Frank,

In that case, I would like to ask you to create a support case. Probably this means there is another application that is not in a good shape and that means we need to get that information to MSFT (But first check by our support engineers). Probably it would be good afterwards (if that is the case) to log a call to MSFT from your side also.

As usual, please log the support case ID here and the follow-up

Thanks
M/

[frankive]

case id 02074842

[Mike Resseler]

Thanks Frank,

Keep us informed!

[frankive]

the suggestion so far is up upgrade my domain controlles to 2016.. Not too happy with that answer..
And the issue is the same for the customers 2008 R2 terminalserver.

[frankive]

it seems like downgrade to 2012 R2 Hyper-v is the only viable solution.
That means out buying SAN again instead of using storage spaces direct..
Soory Veeam/Microsoft, this time it didn't just work, even after Windows Server 2016 over 6 months in production.

[Mike Resseler]

Frank,

This is indeed not good. Did you also log a support call with MSFT. I'm certainly not saying we are perfect but imho this is something where our hands are tied. And the more people complain / report to MSFT, hopefully the faster we get a solution...

[frankive]

I will certainly open a case with MSFT.
How Critical is the Application aware Processing on a domain Controller?

[Gwisdala]

Hi everyone!

I must back up two virtual server. A server is backed up normally and the other I get following error:

Code: Select all

Failed to finalize guest processing: Microsoft Exchange transaction logs may not have been truncated. Error: Failed to take in-guest VSS snapshot COM error: Code: 0x80042308
Retrying snapshot creation attempt (Failed to process guest: server.domain.local)
Task has been rescheduled
Queued for processing at 20.02.2017 22:01:58
Unable to allocate processing resources. Error: Failed to take in-guest VSS snapshot COM error: Code: 0x80042308

I've read that that should be solved when an update is applied by Microsoft. But that is not the case.

Enough space is also available.

Does anyone have a solution?

Thanks!

[Mike Resseler]

Pretty critical in my book. Since AD is actually a running database, plus some other information that is only included in a system state backup of a machine, you need to get the VSS writers to do their work before the actual backup works. Otherwise you might end up with inconsistency.

There are users that don't mind this because they are running multiple DC's and in case they loose one they either build a new one (and let AD synch do its work) or restore a DC in non-authoritative way but I always prefer to have at least 2, application-aware backups of my DC's (depending how much you have obviously)

My 2 cents

[frankive]

How about shutting down all vms for one hour each night. Do backup and program hyper-v to start them. Best solution for now? Exchange logs will not be truncated though..

[Salm]

Exchange logs will not be truncated though..

Exchange is not affected by this issue (at least not from what I've seen).

[Mike Resseler]

I also have no reports that Exchange is affected by this. As I said, we know about the DC's and now you reported the TS / RDS server.
@Salm, what OS are you running for your virtual exchange on Windows server 2016 Hyper-V?

[Salm]

@Salm, what OS are you running for your virtual exchange on Windows server 2016 Hyper-V?

My own Exchange runs on 2012 R2 and Exchange 2016 RTM and CU3.

I have confirmed that backups with AAP work for Exchange 2013 and 2016 (latest CU) on Windows Server 2012 R2, provided KB3037623 is installed on the guest OS (this KB updates integration services).

If the KB has not yet been installed, the vss will fail with the following error

Failed to create VM recovery checkpoint (mode: Veeam application-aware processing) Details: Job failed (''). Error code: '32768'. Failed to create VM recovery snapshot, VM ID '064dac1c-9652-4378-a69c-fa3429ca32a9'.
Retrying snapshot creation attempt (Failed to create production checkpoint.)

The KB is unable to be properly installed if Exchange is already installed (OS will hang on reboot)

The following service must be disabled prior to KB installation
AD Topology.
These services should be set to manual :
Microsoft Exchange Diagnostics
Microsoft Exchange Transport
Microsoft Filtering Management Service
Microsoft Exchange Search Host Controller

The services should be set to Automatic after successful KB installation.

I do not have any information on 2008 R2 Guest OS or previous versions of Exchange.

[Mike Resseler]

Thanks @salm,

Really appreciate this!

Mike

[frankive]

Just mgirated 12 VMs for a customer to a new 2016 storage spaces direct 2-node cluster.
Same errors on all he Vms!

[Salm]

Update your guests. It will correct the errors, except for Domain Controllers, which aren't fixed yet.

[frankive]

with all windows updates? is there a specific windows update we could deploy manually?

[Salm]

KB3037623

Beware of this KB on Exchange servers, it will break them.

See my post above.

[frankive]

i can see the update for 2012 servers, but not 2012 R2, is this correct?

[Salm]

Update should be present on 2012 R2 as well, but you may not see it if your updates are too far back, as there are some prerequisites.

Read the KB info to know which ones : https://support.microsoft.com/en-us/hel ... view-hosts

(Yes, it says technical preview but it also applies to RTM).