Comprehensive data protection for all workloads
Post Reply
Sanai95
Novice
Posts: 3
Liked: 2 times
Joined: Aug 17, 2022 2:42 pm
Full Name: Maik von der Fecht
Contact:

SQLVSSWriter Errors on multiple servers

Post by Sanai95 »

Hello,

Case numer #05572428.

We are getting silimar errors to this for multiple customers:

"Processing WFO-DC-01 Error: Failed to call RPC function 'Vss.Unfreeze': Error code: 0x80004005. Failed to invoke func [Unfreeze]: Unbekannter Fehler. Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer's data. A VSS critical writer has failed. Writer name: [SqlServerWriter]. Class ID: [{a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}]. Instance ID: [{7b26d6f7-1b70-4cf7-830d-1776c4dcf2a5}]. Writer's state: [VSS_WS_FAILED_AT_PREPARE_SNAPSHOT]. Error code: [0x800423f4].]."

They are mostly occuring on the Domain Controllers. After restarting the Writers service it sometimes works, but not all the time.
One customer (D) got this error on re SQL Server.

In the eventlog we got the events:
"SQLVDI: Loc=SignalAbort. Desc=Client initiates abort. ErrorCode=(0). Process=6220. Thread=8372. Client. Instance=LOCALDB#SHA17731. VD=Global\{43A9108A-7E77-4B2C-B47E-A73DFDE9CBCB}1_SQLVDIMemoryName_0. "

On other server it's another instance(just for a database for a application that the customers uses).
But also this one:
"Sqllib error: OLEDB Error encountered calling ICommandText::Execute. hr = 0x80040e14. SQLSTATE: 42000, Native Error: 3013
Error state: 1, Severity: 16
Source: Microsoft SQL Server Native Client 11.0
Error message: BACKUP DATABASE is terminating abnormally.
SQLSTATE: 42000, Native Error: 18210
Error state: 1, Severity: 16
Source: Microsoft SQL Server Native Client 11.0
Error message: BackupVirtualDeviceSet::Initialize: CoCreateInstance failure on backup device '{43A9108A-7E77-4B2C-B47E-A73DFDE9CBCB}3'. Operating system error 0x800703fa(Es wurde versucht, einen Registrierungsschlüssel einem unzulässigen Vorgang zu unterziehen, der zum Löschen markiert wurde.)."


We belive those may be linked to the lastest microsoft updates.
Those were installed in the last days on the backued up servers:

KB: 890830 [Windows-KB890830-x64-V5.104.exe] - Windows Malicious Software Removal -> F, J, P, D
KB: 5012170 [Windows10.0-KB5012170-x64.cab] - Security Update for Boot DBX -> F, J and D (P has to install updates, that why the update ist not installed so far)
KB: 5017095 [Windows10.0-KB5017095-x64.cab] - Updates to servicing stack - P and D
KB: 5016622 [Windows10.0-KB5016622-x64.cab] - Security Update for Printers, KDC, LSASS, -> J, P, D

Only F:
KB: 5016623 [Windows10.0-KB5016623-x64.cab] - Security Update for Local Security Authority Server Service -> Only F

F has a WS2019, other customers WS2016.



Anyone else experiencing this errors?
HannesK
Product Manager
Posts: 14487
Liked: 2933 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by HannesK »

Hello,
yes, I saw something yesterday. The partner did not want to open a case, so the post was deleted.

If I remember correctly, then the solution to the problem was restarting the SQL writer service on the SQL machine.

Best regards,
Hannes
Sanai95
Novice
Posts: 3
Liked: 2 times
Joined: Aug 17, 2022 2:42 pm
Full Name: Maik von der Fecht
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by Sanai95 »

Hello,

I restarted the correspondig VSS Service "SQL Server VSS Writer".

For most of those servers there is no SQL Service, since it's "only" a Domain controller without an additional SQL server installed.

We try deinstalling the said windows updates on one system.
On another we try to de- and reinstall the "SQL VSS Writer" (Recommandation from the veeam support).

Best regards,
Maik
LickABrick
Enthusiast
Posts: 62
Liked: 31 times
Joined: Dec 23, 2019 7:26 pm
Full Name: Lick A Brick
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by LickABrick »

I posted about this yesterday. For some customers stopping instead of restarting the writer service fixed the issue.
SnakeSK
Service Provider
Posts: 85
Liked: 18 times
Joined: Feb 09, 2019 5:06 pm
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by SnakeSK » 1 person likes this post

Yep, currently have to stop the VSS SQL service as part of the prefreeze script.

Also guys - do you have Azure AD connect installed on those servers?
LickABrick
Enthusiast
Posts: 62
Liked: 31 times
Joined: Dec 23, 2019 7:26 pm
Full Name: Lick A Brick
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by LickABrick »

I've seen the issue on servers without AzureAD connect (I'm pretty sure), a colleague of mine mentioned the following fixes the issue on server with AAD Connect though: https://www.veeam.com/kb2326
n.koch
Service Provider
Posts: 20
Liked: never
Joined: Jan 08, 2020 2:09 pm
Full Name: NiKo
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by n.koch »

I have the same case. On some servers it was enough to restart the VSS Writer. For others it had to be a server reboot.

What I noticed was that all DomainControllers were running Windows 2019.
mjr.epicfail
Veeam Legend
Posts: 238
Liked: 52 times
Joined: Apr 22, 2022 12:14 pm
Full Name: Danny de Heer
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by mjr.epicfail »

Ive seen an increase as well. But only on domainservers with Azure AD connect.
VMCE / Veeam Legend 1*
Sanai95
Novice
Posts: 3
Liked: 2 times
Joined: Aug 17, 2022 2:42 pm
Full Name: Maik von der Fecht
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by Sanai95 » 2 people like this post

Hallo,

Update to the last steps.
Both deinstalling the last updates and re-registering the VSSWriters work.

Link on how to re-registering the writers:
http://woshub.com/vss-writer-failed-re- ... ws-server/

Best regards
Maik
YouGotServered
Service Provider
Posts: 175
Liked: 52 times
Joined: Mar 11, 2016 7:41 pm
Full Name: Cory Wallace
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by YouGotServered »

All - I can confirm that we have seen this at several clients with Azure AD Connect, now and in the past. The light version of SQL that MS installs with AD Connect is garbage. We have tried various "permanent" solutions across the web with mixed results. If anyone finds one that holds up in several environments for more than a couple of months, I'd love to hear about it.
Steve-nIP
Service Provider
Posts: 123
Liked: 53 times
Joined: Feb 06, 2018 10:08 am
Full Name: Steve
Contact:

Re: SQLVSSWriter Errors on multiple servers

Post by Steve-nIP »

SnakeSK wrote: Aug 19, 2022 10:33 pm Yep, currently have to stop the VSS SQL service as part of the prefreeze script.

Also guys - do you have Azure AD connect installed on those servers?
That was my first thought. Azure AD connect, when it updates, causes this issue. Restarting the server fixes the issue. (If it doesn't, repair localDB and restart).
I've seen the same problem for many years, it's a classic.
Post Reply

Who is online

Users browsing this forum: No registered users and 57 guests