Azure DC error

[BCT-Tech]

Deployed a new Server 2012 R2 VM in Azure, and promoted it to a Domain Controller. Installed VEB 1.5.0.306 via "run as Administrator" and ran initial backup job. Support Case # 01932351

Getting this error:

Code: Select all

Creating VSS snapshot Error: Failed to create snapshot: Backup job failed. Cannot create a shadow copy of the volumes containing writer's data. Cannot prepare the [NTDS] data to a subsequent restore operation. Cannot process NTDS data. Updating BCD failed. Default OS loader not found.
Error: Failed to create snapshot: Backup job failed. Cannot create a shadow copy of the volumes containing writer's data. Cannot prepare the [NTDS] data to a subsequent restore operation. Cannot process NTDS data. Updating BCD failed. Default OS loader not found.

C:\Windows\system32>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.

Writer name: 'Task Scheduler Writer'
   Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
   Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
   State: [1] Stable
   Last error: No error

Writer name: 'VSS Metadata Store Writer'
   Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
   Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
   State: [1] Stable
   Last error: No error

Writer name: 'Performance Counters Writer'
   Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
   Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
   State: [1] Stable
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {b3c9ae85-8a3c-452f-83ba-04a92d74407d}
   State: [1] Stable
   Last error: No error

Writer name: 'SqlServerWriter'
   Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
   Writer Instance Id: {ea57b6a9-ad13-4f99-86e5-8a906538c8ad}
   State: [7] Failed
   Last error: Timed out

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {351f7ad8-2360-41c8-a3b6-98c6ae6bfef5}
   State: [7] Failed
   Last error: Timed out

Writer name: 'FRS Writer'
   Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
   Writer Instance Id: {b75d3280-b727-434c-a726-149aa02306c1}
   State: [7] Failed
   Last error: Timed out

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {baa3ff3e-a504-46d5-9447-bc6f67cf8c60}
   State: [7] Failed
   Last error: Timed out

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {bedc5ef8-d14a-4199-95e0-f0379d5571ae}
   State: [7] Failed
   Last error: Timed out

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {905ab9a7-b52d-46db-ab76-cb2e4679cbe8}
   State: [7] Failed
   Last error: Timed out

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {6c924055-01c2-4ddb-82f3-12e71f64318a}
   State: [7] Failed
   Last error: Timed out

Writer name: 'BITS Writer'
   Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
   Writer Instance Id: {96ba75c6-9b7a-4f3e-b9f8-e1db8e90ce13}
   State: [1] Stable
   Last error: No error

Writer name: 'NTDS'
   Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
   Writer Instance Id: {98abd5db-6d7f-46b2-97c2-c2fecb26d595}
   State: [7] Failed
   Last error: Timed out

[Dima P.]

Hi BCT-Tech,

What was the backup mode and what was the backup destination? Thanks.

[BCT-Tech]

What was the backup mode and what was the backup destination? Thanks.

Entire computer, backing up to our local Veeam Repository. We have 10 other Azure VMs backing up successfully this way.

[Vitaliy S.]

Can you please check out Windows Event log for VSS related errors as they might shed some light on the issue?

[enoch]

Hi everyone,

I have the same issue as BCT-Tech. Currently I've only tried on two 2012r2 DC's from Azure.

Newest VEB version to Veeam Reposistory by VPN.

Before backup vssadmin list writers shows no errors, after backup it shows the NTDS, WMI and DFS Replication service, shadow copy optimization, registry, COM+ REGDB and SQL Server writers as "Timed out"

It would be great to have someone at Veeam to respons to this if possible

I need it working ASAP

[Dima P.]

Hi Peter,

Please open a case and post the ID to this thread. Meanwhile, I'll check the results of the OP investigation.

[enoch]

Thanks for quick reply.

I'll open a case now.

[enoch]

For information support case number is 01984029.

[enoch]

For information, tried with other Azure VM which is not a DC, it works!

So it is something todo with DC on a Azure VM.

[enoch]

Veeam support respons with this:

..
[23.11.2016 09:32:22] Error <1c48> Backup job failed.
[23.11.2016 09:32:22] Error <1c48> Cannot create a shadow copy of the volumes containing writer's data.
[23.11.2016 09:32:22] Error <1c48> Cannot prepare the [NTDS] data to a subsequent restore operation.
[23.11.2016 09:32:22] Error <1c48> Cannot process NTDS data.
[23.11.2016 09:32:22] Error <1c48> Updating BCD failed.
[23.11.2016 09:32:22] Error <1c48> Default OS loader not found.
..

Please try to recreate BCD store on your system It can be done by booting from WinPE image, deleting current BCD entries, rescanning drives and rebuilding BCD store. In brief, commands are the following:

bcdedit /delete {%entry%} , where %entry% are all your current entries in a bcd store.

bootrec /scanOs

bootrec /RebuildBCD

I don't know if it's possible on a Azure VM to do this?

I cannot do it on current servers so does anyone have the possiblility to test on a test Azure VM (DC role needs to be added!)

[Dima P.]

Asked QA and DEVs to take a look at your case.

[enoch]

Hi everyone,

I created a new Azure VM just to check and not to play with production DC

I also fails on this new Azure VM (also configured as a domain controller)

I disabled UAC and tried again, still error.

From Veeam support I got the information to try this before backup from a CMD: bcdedit /default {current}

Then tried backup and it works! - I don't know it it only works with UAC off or if it also works with UAC on (someone else test)

I'll try to reboot the Test VM to be sure it start again, so please be carefull

[enoch]

And the Azure VM work after reboot again.

Don't know if the: "bcdedit /default {current}" is needed each time before backup or after a reboot on the server.

Veeam: maybe you should look into a fix for this "problem"?

[Dima P.]

Peter,

Thanks for confirming that suggested solution worked. We are still investigating this issue

P.S. bcdedit is a one-time operation, so there is no need to repeat it after restart.

[Mike Resseler]

It is a bit weird that for a VM on Azure, being a domain controller, the boot information needs to be changed. In the end, what you do with that command is simply tell the boot loader that the OS that is active is the default OS. This is used in multi-boot environments but as far as I know, an Azure VM is not a multi-boot VM... (Unless there is something behind the scenes...)

Did everone used a windows server 2012 R2 OS image or another one?

[enoch]

I've only tested with Windows 2012r2 machine.

It toke me less then 30 min. to create new test VM with DC role, so it should be possible to test on a Windows 2016 pretty easy also.

[Mike Resseler]

You are always allowed to do that test...

But VEB 1.5 does not support server 2016 so we will need to wait for that with VAW 2 (the new name for VEB). I was also wondering if some people still used older versions such as 2012 or 2008 R2.

Thanks for the info