Availability for the Always-On Enterprise
Post Reply
leebtish
Influencer
Posts: 23
Liked: 3 times
Joined: Nov 12, 2012 9:07 am
Full Name: Lee Russell
Contact:

Sophos Central Endpoint AV Causing VSS timeout Issues

Post by leebtish » Feb 26, 2018 5:33 am

Hi,

Just posting this in case anyone else running Sophos Central Endpoint AV experiences the same issue. We were experiencing the following error on multiple servers:
VSSControl: Failed to freeze guest over network, wait timeout

We also found warnings in our app logs showing the following:

A VSS writer has rejected an event with error 0x800423f2, The writer's timeout expired between the Freeze and Thaw events.
. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.

The issue was resolved by disabling the Sophos Health Service. If you run this from an admin command prompt - "fltmc instances" you will see Sophos accessing VSS. Another Veeam customer also has the same problem - post272884.html#p272884

FYI, this thread has a lot of background information - https://community.spiceworks.com/topic/ ... 713?page=1

At the moment Sophos and speaking with Microsoft about the issue. According to Sophos they are accessing VSS correctly and the problem should not be happening. Hopefully between them and MS they can resolve the issue.

Thanks

Mike Resseler
Veeam Software
Posts: 4994
Liked: 529 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by Mike Resseler » Feb 26, 2018 6:11 am

Hi Lee,

Thanks for the heads up! Really appreciated. Please keep us further informed about the outcome when Sophos and MS have figured out what is happening

Cheers
Mike

leebtish
Influencer
Posts: 23
Liked: 3 times
Joined: Nov 12, 2012 9:07 am
Full Name: Lee Russell
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by leebtish » Feb 26, 2018 9:02 am

No problem, will do Mike

mboucher87
Novice
Posts: 3
Liked: never
Joined: Jun 07, 2018 11:51 pm
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by mboucher87 » Jun 07, 2018 11:54 pm

We are also experiencing this issue (case 02779434). Has there been any updates at all?

Mike Resseler
Veeam Software
Posts: 4994
Liked: 529 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by Mike Resseler » Jun 08, 2018 3:13 pm

Hi mboucher87,

First: Welcome to the forums

Unfortunately the OP did not came back to us so we have no outcome on the sophos/ MSFT conversation. However, any change you are using intercept (I believe that is a component within Sophos...)

mboucher87
Novice
Posts: 3
Liked: never
Joined: Jun 07, 2018 11:51 pm
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by mboucher87 » Jul 11, 2018 10:13 pm

Hi Mike, Sorry for the delay I've been away on holiday. Yes we are using Intercept also...

stephen@sophos
Novice
Posts: 3
Liked: 1 time
Joined: Aug 07, 2018 8:49 am
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by stephen@sophos » Aug 07, 2018 8:54 am

Hi all,

I found this thread as a result of a post in our community forum; https://community.sophos.com/products/s ... -installed

I have included updates from our conversations with MSFT in that thread:
  • MSFT has confirmed that the Sophos product is using their APIs correctly
  • We have established that the way in which the Sophos Health Service (part of the Sophos agent) writes to the Windows registry as part of a transaction, has uncovered a behaviour that can, intermittently, result in a backup failure
  • MSFT are expecting to release a fix into their RS5 builds imminently, and back-port the fix to Windows Server 2016
  • Sophos have also create a configuration that will fix this; this is currently being tested and should be released to all customers in the next 4-6 weeks
Regards,
Stephen

Mike Resseler
Veeam Software
Posts: 4994
Liked: 529 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by Mike Resseler » Aug 07, 2018 12:24 pm

Stephen!

First: Thanks for coming to our forums and taking the time to post the results here! Really appreciated
If you have more information around that configuration, please let us know!

Thanks again
Mike

mboucher87
Novice
Posts: 3
Liked: never
Joined: Jun 07, 2018 11:51 pm
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by mboucher87 » Sep 04, 2018 10:55 pm

Looks like the fix from Sophos has arrived: https://community.sophos.com/kb/en-us/132691

Mike Resseler
Veeam Software
Posts: 4994
Liked: 529 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by Mike Resseler » Sep 06, 2018 5:34 am

Hey mboucher87

Thanks for letting us know! It looks like a bit of a workaround but at least it is something! And I am glad that Sophos continuous to work with MSFT to find the root cause!

Cheers
Mike

stephen@sophos
Novice
Posts: 3
Liked: 1 time
Joined: Aug 07, 2018 8:49 am
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by stephen@sophos » Sep 10, 2018 9:34 am

Hi all,

The release of the version that allows for the workaround is due to complete on Thur 13th Sept. Please note, we are discussing the KBA that MSFT is due to publish and finalising the details of that KBA. I hope that they will publish it, with a reference from our own KBA, by the 13th.

Note: We have had to provide a workaround as the responsiveness of our alerting and security systems depends on the transactional frequency of the Health service. Changing this value across the board is not viable for the security story.

Regards,
Stephen

Mike Resseler
Veeam Software
Posts: 4994
Liked: 529 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by Mike Resseler » Sep 12, 2018 9:17 am

Good news Stephen! Please let us know the URL when it is published

stephen@sophos
Novice
Posts: 3
Liked: 1 time
Joined: Aug 07, 2018 8:49 am
Contact:

Re: Sophos Central Endpoint AV Causing VSS timeout Issues

Post by stephen@sophos » Oct 01, 2018 8:08 am 1 person likes this post

Hi all, we have updated our KBA here: https://community.sophos.com/kb/en-us/132691

This includes a link to the MSFT KBA: https://support.microsoft.com/en-us/hel ... eeze-error

Stephen

Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 65 guests