Error: Failed to enable DC SafeBoot mode

[RobMiller86]

Yeah I don't know. I have 1 2019 DC that with 23.3.3.264 S1 and all of the steps listed in this thread above completed, it still fails. Remove S1, everything works fine. Once installed, no combination so far has worked on this particular DC. Guess we will try one final time with a full uninstall, and reapply the fixes, and if not, I'll have to open an S1 ticket. S1 really needs to make this easier.

[gigarun]

Nothing solve in my case. Removed SentinelOne solve. Add another backup solution is also an alternative.

[david.domask]

Hi Thomas and welcome to the forums.

Sorry to hear about the issues, but to confirm, removing SentinelOne assisted here? The solutions from earlier in the thread did not assist? Removing AV naturally will prevent AV from incorrectly interfering with backup operations, but obviously this is not a long term solution, so it's probably best to check the behavior with Veeam Support to confirm the behavior, then reach out to SentinelOne if you have not already.

Thanks!

[vmikhelson]

@SomewhereinSC

In addition to:
cd "\Program Files\SentinelOne\Sentinel Agent 24.1.4.257" *** Elevated CLI, current agent version ***
sentinelctl config antiTamperingConfig.allowSignedKnownAndVerifiedToSafeBoot true -k "<pass>"
sentinelctl config antiTamperingConfig.allowSignedKnownAndVerifiedToSafeBoot *** To verify ***

1. sentinelctl unload -a -k "<pass>"
2. sentinelctl load -a

It will allow to avoid the unnecessary reboot.

-Vladimir

[ShenRaiden]

We are still dealing with this too. Sometimes it works, and sometimes it does not. S1 has been a real pain with backing up DCs. I'm dealing with 1 DC now that throws this no matter what I do:

Failed to prepare guest for hot backup. Details: VSSControl: -805306334 Backup job failed.
Cannot create a shadow copy of the volumes containing writer's data.
Cannot prepare the [NTDS] data to a subsequent restore operation.
Cannot process NTDS data.
Updating BCD failed.
Cannot execute [SetIntegerElement] method of [\\SERVERNAME\root\wmi:BcdObject.Id="{cd0922c3-4ef8-11ee-9786-8af7d491816a}",StoreFilePath=""].
COM error: Code: 0xd0000022

Will be opening an S1 ticket I guess to see what they say.

In the exact same situation, have you found any solution?
So far, the only thing i could manage to do after trying everything else is turning that failed status into a warning, so it actually performs the backup, yet it's not backing up the NTDS, so it's a half-useful (pretty useless) solution.

[david.domask]

Hi ShenRaiden, welcome to the forums.

Sorry to hear about the challenges with SentinelOne and backups of the DCs -- did the suggestions from vmikhelson in the post above yours help or the config changes proposed on the previous page? What was the result?

[ShenRaiden]

Thank You for welcoming me David, I work for a MSP company, so I have a multitenant situation. Sadly, nothing helped so far.
As of now I've spent 60+ hours on this, I've implemented all the possible, safe, solutions, yet nothing worked.

This is a problem I already faced in the past, and the "sentinelctl config antiTamperingConfig.allowSignedKnownAndVerifiedToSafeBoot true -k "PASSPHRASE"" command had been the solution (it is still working for most of my customers DCs).


Worth mentioning, this tenant has 3 domain controllers; They were working fine with the above command prompted. Then a month ago "DC1" started failing... then after a couple of weeks also "DC2"... and after a week maybe, even "DC3"....... then suddenly, without me doing anything relevant on it, "DC3" started working again a couple of days ago. O.o

So, now DC1 and DC2 give that error everytime I try to perform the job, and DC3 works fine. Mind that, configuration-wise, they're the same (compared their configuration files). O.O!

We're using SentinelOne (agents version is 24.1.5.277, i will request the company managing S1 to update them asap, but i'm not positive about a resolution coming from this), and for this particular tenant I've even added some exceptions for some of the veeam processes and folders, with no good results.

The verification command "sentinelctl config antiTamperingConfig.allowSignedKnownAndVerifiedToSafeBoot" returns "true", yet I would say it is still interferring somehow.

I've expanded the available space for VSS, succesfully performed a shadow copy manually (to verify), but still getting the error on the job.

This is the infamous error, same as RobMiller86:

<Failed to prepare guest for hot backup. Details: VSSControl: -805306334 Backup job failed.
Cannot create a shadow copy of the volumes containing writer's data.
Cannot prepare the [NTDS] data to a subsequent restore operation.
Cannot process NTDS data.
Updating BCD failed.
Cannot execute [SetIntegerElement] method of [\\MyDC\root\wmi:BcdObject.Id="{74b16b4e-7439-11ee-9dc0-dd1cc76b4b19}",StoreFilePath=""].
COM error: Code: 0xd0000022>

Doing the backup causes this, the only error among the VSSWriters:

<Writer name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {12ec3752-22db-4f33-bd78-c835561ef59d}
State: [8] Failed
Last error: Non-retryable error>

Restarting the service fixes it's status, then it brakes again after a backup.

Credentials for the job are fine.
I won't disable SafeBootProtection, i won't exclude vsswriters and their protection as none of this is good, security-wise.

I've almost finished the google pages with results pertaining this issue, so when I saw RobMiller86's comment (which is the only post with the exact same error i've found so far among hundreds) I decided to ask if He/They managed to find a proper solution.

As of now, the only thing I could do to "get it working", meaning going from Failure to Warning, is to check the box "try application processing, but ignore failures" in the application-aware settings for the specific vm.
Yet I can't be satisfied with this, since it's not backing up the NTDS, and we're talking about a domain controller with AD here, so backing up without that becomes pretty useless.

Anyone has a solution?

[RobMiller86]

In the exact same situation, have you found any solution?
So far, the only thing i could manage to do after trying everything else is turning that failed status into a warning, so it actually performs the backup, yet it's not backing up the NTDS, so it's a half-useful (pretty useless) solution.

Following the steps posted above has been working for us for a while now on DCs. Make sure to use updated S1 agents.
https://community.sentinelone.com/s/article/000006996

[itg@dynatronics.com]

I got these instructions from Sentinal One Support. They were excellent!

This was happening on MS Server 2016 where the servers were a Domain Controller

Here is what worked for me!

1. Retrieve the passphrase. The passphrase can be retrieved here (https://community.sentinelone.com/s/article/000005375)

2. On the endpoint on which the backup is failing, open CMD as Admin.

3. cd C:\Program Files\SentinelOne\%AgentVerion%

\\ie cd C:\Program Files\SentinelOne\Sentinel Agent 24.1.6.313

4. Then run this cmd

sentinelctl config antiTamperingConfig.allowSignedKnownAndVerifiedToSafeBoot true -k "Passphrase"

5. To make sure this configuration is True run:

sentinelctl config antiTamperingConfig.allowSignedKnownAndVerifiedToSafeBoot

The output should be True.

6. Reboot the machine.

7. After the machine is fully loaded, run Veeam Backup again.

[adamVAC]

We were backing up a Windows 2019 Server Essentials Physical DC and it was working until we installed Sentinel One which has since been removed but we are still getting errors with backup. I have updated Veeam to 12.3.2

Error: Failed to disable DC SafeBoot mode Cannot get [BcdObject.Id="{9dea862c-5cdd-4e70-acc1-f32b344d4795}",StoreFilePath=""] object. COM error: Code: 0x80041010

Does anyone have some best guidance to get through this?

[david.domask]

Hi adamVAC, welcome to the forums.

I've merged your post with an existing topic on the subject. While I understand you've uninstalled SentinelOne, I would double-check if some of its services are still active on the DC (from Powershell/cmd, use the command fltmc and likely you'll see the SentinelOne filter is still active) Typically it can take several reboots before the filters are uninstalled.

[adamVAC]

David, there is probably a connection but I did want to clarify that my error is failed to DISABLE whereas this topic is failed to ENABLE.

Also, S1 was uninstalled 6 months ago and several reboots have occured and I don't see anything that looks like an S1 filter when running fltmc.

[david.domask]

Got it, in that case, please open a Support Case and allow Veeam Support to review the behavior, though I do suspect it's still leftovers from that best to let Support review the situation.

[pat_ren]

Hi all,

I recently had a similar problem (Veeam case #05377961) and wanted to add my solution here in case Googling brought anyone else this way.

My error was: Error: Failed to disable DC SafeBoot mode Cannot get [BcdObject.Id="{9dea862c-5cdd-4e70-acc1-f32b344d4795}",StoreFilePath=""] object. COM error: Code: 0x80041010

We do not use SentinelOne, however.

The agent log on the server contained the lines below:

Code: Select all

[12.04.2022 19:05:34] <01> Info     Trying to backup system volume to temp file on dc
[12.04.2022 19:05:34] <01> Info     Enabling AD safe boot mode
[12.04.2022 19:05:34]      Info     <10e8>     Enabling AD SafeBoot mode
[12.04.2022 19:05:34]      Info     <10e8>             Connecting to WMI namespace.
[12.04.2022 19:05:34]      Info     <10e8>             Connecting to WMI namespace.. Ok.
[12.04.2022 19:05:34]      Info     <10e8>             Enabling DC SafeBoot mode
[12.04.2022 19:05:34]      Info     <10e8>                 UpdateSafeBootForAllLoaders registry value is not set. Using default value: false
[12.04.2022 19:05:34]      Info     <10e8>             Enabling DC SafeBoot mode. Failed.
[12.04.2022 19:05:34]      Info     <10e8>     Enabling AD SafeBoot mode. Failed.
[12.04.2022 19:05:34] <01> Info     Disabling AD safe boot mode
[12.04.2022 19:05:34]      Info     <10e8>     Disabling AD SafeBoot mode
[12.04.2022 19:05:34]      Info     <10e8>             Connecting to WMI namespace.
[12.04.2022 19:05:34]      Info     <10e8>             Connecting to WMI namespace.. Ok.
[12.04.2022 19:05:34]      Info     <10e8>             Disabling DC SafeBoot mode
[12.04.2022 19:05:34]      Info     <10e8>                     Loading original SafeBoot values from file [C:\ProgramData\Veeam\Endpoint\Backup\bcdorig.xml]
[12.04.2022 19:05:34]      Info     <10e8>                     Loading original SafeBoot values from file [C:\ProgramData\Veeam\Endpoint\Backup\bcdorig.xml]. Failed.
[12.04.2022 19:05:34]      Warning  <10e8>                 Unable to read original SafeBoot values.
[12.04.2022 19:05:34]      Warning  <10e8>                     Cannot load the specified XML file: [C:\ProgramData\Veeam\Endpoint\Backup\bcdorig.xml].
[12.04.2022 19:05:34]      Warning  <10e8>                     COM error: The system cannot locate the resource specified.
 Code: 0x1
[12.04.2022 19:05:34]      Info     <10e8>                 UpdateSafeBootForAllLoaders registry value is not set. Using default value: false
[12.04.2022 19:05:34]      Info     <10e8>             Disabling DC SafeBoot mode. Failed.
[12.04.2022 19:05:34]      Info     <10e8>     Disabling AD SafeBoot mode. Failed.

After various troubleshooting steps, I used a tool called WMI Explorer to compare WMI entries between the problem server (below, right) to another machine. Notice how the "BCD" entries were simply gone from the server when comparing a search side-by-side.



To fix it, I opened an Administrator command prompt and CD'd to c:\windows\system32\wbem. Running the command "mofcomp bcd.mof" fixed the issue. Re-running the search in WMI Explorer found the BCD entries just like it did on the comparison machine. The Veeam job then successfully ran.

I have no idea how the BCD entries disappeared from WMI in the first place, but I hope this helps someone.

thanks for sharing this, i had this issue tonight, never seen it before in years of doing this
this post was literally the only info i could find about it and it fixed the issue for me too

you're a legend, cheers