VSS Writer failures when backing up Oracle database server

[PaulOliver]

We have an Oracle 11g database server on Windows 2008 R2 which I'm trying to backup & replicate using Veeam B&R 6.5

Whenever the backup or replication job runs I get VSS writer errors preventing the job from completing. When the job retries a few minutes later it generally runs ok which I suspect means that the VSS snapshot is timing out but has completed by the time the Veeam process retries.

Code: Select all

Unable to release guest. Error: Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer's data. A VSS critical writer has failed. Writer name: [Oracle VSS Writer - LIVE]. Class ID: [{26d02976-b909-43ad-af7e-62a4f625e372}]. Instance ID: [{87d92348-6579-4bfb-ae57-d1bc449bc156}]. Writer's state: [VSS_WS_FAILED_AT_POST_SNAPSHOT]. Error code: [0x800423f3].]
Error: Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer's data. A VSS critical writer has failed. Writer name: [Oracle VSS Writer - LIVE]. Class ID: [{26d02976-b909-43ad-af7e-62a4f625e372}]. Instance ID: [{87d92348-6579-4bfb-ae57-d1bc449bc156}]. Writer's state: [VSS_WS_FAILED_AT_POST_SNAPSHOT]. Error code: [0x800423f3].]

Searching around it appears that people have experienced similar errors when backing up MS Excahnge servers but I've not found any reports of problems with an Oracle database server.

[dellock6]

Hi Paul,
I've found on Microsoft webiste this interesting list of VSS states:
http://msdn.microsoft.com/en-us/library ... s.85).aspx
Your error seems to happen "after" snapshot creation, so this could be the reason why during the retry the VSS invokation succeeds. There are sadly no further explanations on MS website about "why" it happens...

About finding no other cases on Oracle, maybe is because the majority of customers run Oracle on linux, so there is no VSS at all there.

Luca.

[Dima P.]

Hello Paul,

First step is to gather some background info:

- Double check that VSS service is running under backed up server. (restarting VSS service could also help)
- Check the windows even log regarding VSS Writer errors.

Also it would be great to get the following output from the cmd on the backed up server:

Code: Select all

vssadmin list writers
vssadmin list providers

Regarding the Error code: [0x800423f3] its typical Microsoft transiet error - when a transient error occurs, this usually indicates that one or more VSS
writer(s) has returned an error.

[yajith]

Hello Everyone,

I'm also experiencing this issue when trying to backup a similar Windows environment running SAP with Oracle DB. Witnessed the same issue on two systems actually.
Appreciate some advice on fixing this.

@PaulOliver Hope you were able to resolve this issue ?

--
Backup job failed. Cannot create a shadow copy of the volumes containing writer's data.

A VSS critical writer has failed.

Writer name: [Oracle VSS Writer - XDV].
Class ID: [{26d02976-b909-43ad-af7e-62a4f625e372}].
Instance ID: [{e4e9c9c3-b52e-411d-a7c0-4156451474e7}].
Writer's state: [VSS_WS_FAILED_AT_POST_SNAPSHOT]. Error code: [0x800423f3].]
--

[Dima P.]

Hello Yajith,

I would start from re installing Oracle VSS Writer component, the main idea is that Veeam uses VSS writer preinstalled with Oracle. And due the time out (20secons for VSS snapshotting) - you got the error and as a result a failed job.
Please try to uninstall VSS service; reboot Oracle server and install back VSS.

Thank you.

[foggy]

I would also check with the VSS writer vendor (XDV) regarding this issue. Btw, we have another existing thread with Oracle VSS Writer issues, worth reviewing.

[yajith]

@foggy If i'm not mistaken, XDV is the name of the system that is running SAP (with Oracle). I will check further from oracle end if there is any clue that i can find from the database logs.

@d.popov it is a little difficult to uninstall anything from this system. I have restarted VSS services, rebooted the server without any improvement.

I was having this same issue on two systems. Suddenly when i was trying to backup one of them today, it ran successfully on one of the systems. This is without doing any changes at Veeam end. I'm trying to find out if any change was done at the application which may have contributed.

[foggy]

It would be much appreciated if you could get back with the results of your findings as such issues seem to become quite typical. Thanks!

[rurouni]

I got the same problem here on two of my Windows Server with oracle VSS writers.
sometimes the backup works fine, some other it ends with this error:

Code: Select all

09/09/2013 21:02:16 :: Unable to release guest. Details: Unfreeze error: [Backup job failed.
Cannot create a shadow copy of the volumes containing writer's data.
A VSS critical writer has failed. Writer name: [Oracle VSS Writer - PL1]. Class ID: [{26d02976-b909-43ad-af7e-62a4f625e372}]. Instance ID: [{300690b1-4fb3-4339-a05a-153e9a4338a0}]. Writer's state: [VSS_WS_FAILED_AT_POST_SNAPSHOT]. Error code: [0x800423f3].]

vssadmin list writers prints out this:

Code: Select all

Writer name: 'Oracle VSS Writer - PL1'
   Writer Id: {26d02976-b909-43ad-af7e-62a4f625e372}
   Writer Instance Id: {300690b1-4fb3-4339-a05a-153e9a4338a0}
   State: [11] Failed
   Last error: Retryable error

If i restart the oracle vss service, it changes to "no error". But the next backup fails.

event logs comes out with VSS error 8193:

Code: Select all

Description:
Volume Shadow Copy Service error: Unexpected error calling routine XML document is too long.  hr = 0x80070018, The program issued a command but the command length is incorrect. 

Operation:
   Writer Modifying Modifying Backup Document

Context:
   Execution Context: Requestor
   Writer Instance ID: {300690B1-4FB3-4339-A05A-153E9A4338A0}

I couldn't find much information about this error on the internet.
Thanks for the help.

[Vitaliy S.]

Hello,

If i restart the oracle vss service, it changes to "no error". But the next backup fails.

When you restart Oracle VSS service and run regular Windows system backup, do you also see this issue?

Thank you!

[veremin]

As mentioned, it might be also worth informing VSS (PL1) vendor about this issue. Thanks.

[ALBEA]

Hi,

I had the same error.
The problem was on Microsoft/Windows side.
Try to install this KB :
http://support.microsoft.com/kb/2807849/en-us
After that the problem was solved for me.

Regards,

[Piotr]

Sometimes it happens the VSS Writer is in error state - command VSSAdmin List Writes gives "Non-retryable error" for this writer. Am I allowed to restart VSS service associated with this writer without affecting other services?

In my case the writer is "Oracle VSS Writer" and the service is "Oracle VSS Writer Service".

[foggy]

Yes, restarting the service is pretty common troubleshooting step in such cases.

[dwallace]

I have the same symptoms as user rurouni was there a resolution?

Thanks for any information

[Vitaliy S.]

Cannot check the solution, since rurouni didn't post his case ID. Please contact our technical team for assistance with this issue.

[sergeKala]

I solved this issue by setting the ORACLE VSS WRITER SERVICE to LOGON as the Domain Admin.

I think the issue stemmed from the server not being able to write the data to a shared drive location.
Needless to say, VEEAM, and others were not helpful. The KB they mentioned was for testing VSS, and the VSS errors were of no benefit either. Finally solved!

I used the trick here listed above to use the Windows Server Backup first and got errors with Oracle VSS, and I changed the Oracle vss properties not windows ones like the veeam support was telling me to resolve this issue.

[hjot]

Hi sergeKala

I can confirm your solution.

"setting the ORACLE VSS WRITER SERVICE to LOGON as the Domain Admin"