SBS2011/Exchange VSS Issue - 9.5U3

Availability for the Always-On Enterprise

SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Mon Dec 25, 2017 1:40 pm

We have noticed two separate sites where SBS2011 backup jobs are not completing since upgrade to 9.5U3 due to a VSS issue:
Unable to release guest. Error: VSSControl: Failed to freeze guest over network, wait timeout
Error: VSSControl: Failed to freeze guest over network, wait timeout

VSS writer error:
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {f1128c1d-a6fa-4a2b-a8dc-1fa94443beba}
State: [1] Stable
Last error: Retryable error

Next step is a support case, but wanted to see if anyone else is experiencing this? Both of these environments have not had any changes.

For now we need to shut down the guest to get a successful backup.
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby vmniels » Tue Dec 26, 2017 11:23 am

VSS errors can be related to multiple things so best would be to contact support for inspection.
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be
vmniels
Veeam Software
 
Posts: 1706
Liked: 373 times
Joined: Mon Jul 15, 2013 11:09 am
Full Name: Niels Engelen

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby foggy » Tue Dec 26, 2017 12:27 pm

Try to restart the Exchange Writer service or the entire server and see whether this error shows up after that.
foggy
Veeam Software
 
Posts: 15621
Liked: 1164 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Tue Dec 26, 2017 4:50 pm

Restarting Exchange VSS server and server restart does not result in any change.
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Tue Dec 26, 2017 4:55 pm

Case ID 02442756
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby Ibeme85 » Tue Dec 26, 2017 6:29 pm

I already opened a case relating a possible VSS issue.

In my case VSS doesn't seem to fail but in a Sure Backup Job, the SBS2011 doesn't reboot out of AD Restore Mode into normal Mode and so the SureBackup Job fails.

Backups created with U2 Work. Backups created with U3 don't.

Case 02434632
Ibeme85
Lurker
 
Posts: 1
Liked: never
Joined: Sun Dec 03, 2017 7:43 pm
Full Name: Bjoern Ebner

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Tue Dec 26, 2017 9:06 pm

We have demonstrated to Support that the issue points to B&R rather than VSS. diskshadow full completes successfully for Exchange volumes. Exchange Writer in stable/no error state after diskshadow backup.
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby Gostev » Wed Dec 27, 2017 2:51 pm

Just wanted to note that there are different ways to trigger VSS and it is safe to say that diskshadow, not being a backup application, does not instruct VSS Writer to do transaction logs processing for example... so, this would not be a valid comparison from my perspective. But no worries, you're in good hands now and our support engineers will get to the bottom of this.
Gostev
Veeam Software
 
Posts: 21739
Liked: 2462 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Thu Dec 28, 2017 10:25 am

The issue is that B&R thinks VMware snapshots are taking in excess of 20 seconds and failing the job. In actuality, the snapshots are taking 1-2 seconds. This has all been verified with Support. Case has been escalated to level 3, but we won't be able to resume activity for several more hours.

If anyone has experienced this and has some feedback, or any suggestions, we'd really appreciate it.
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Thu Dec 28, 2017 10:48 pm 2 people like this post

For anyone else affected by this, Veeam Support has isolated the issue to something involving vCenter. If you create a new Backup Job and point it to vSphere direct instead of vCenter, there are no issues.

Good workaround, but not excited about starting a new Full that we have to get offsite at 20 Mbps upstream.
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby ic_mcourtney » Mon Jan 01, 2018 4:45 am

gatorheelz wrote:For anyone else affected by this, Veeam Support has isolated the issue to something involving vCenter. If you create a new Backup Job and point it to vSphere direct instead of vCenter, there are no issues.


Did Veeam Support say if this affects all versions of vCenter, or if it's specific to one version?

@Gostev: Will there be a KB article published about this issue soon? I'm hesitant to proceed with upgrading to U3 before I fully understand the version(s) of vCenter affected, and what the impact is to VBR's ability to make backups.
ic_mcourtney
Influencer
 
Posts: 20
Liked: 2 times
Joined: Thu Sep 22, 2016 7:15 am
Full Name: mcourtney

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby gatorheelz » Tue Jan 02, 2018 11:14 am

They have not indicated which versions of vCenter are affected. In my case it's vCenter 5.5.0 (4180647) with internal DB. Very basic config.

Also, I'm not sure that this is limited to SBS2011. To me it seems like any job with Exchange (possibly only 2010) would be affected. The Level 2 engineer I was working with indicated they don't seem to have other reported cases, but don't think this is a unique case because we're seeing it in multiple environments.

Still awaiting an update from Support. I'm sure the timing of this hasn't helped (holidays), but sure hope to hear something today.
gatorheelz
Influencer
 
Posts: 16
Liked: 2 times
Joined: Mon Jun 19, 2017 4:38 pm
Full Name: gatorheelz

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby Mike Resseler » Tue Jan 02, 2018 1:08 pm

Gatorheelz,

Please keep working with the support engineer. I see there are things going on in the support case so I am keeping my fingers crossed that you will have a solution as soon as possible.

@Matty: We can't say at this moment. Sorry about that

Best wishes
Mike
Mike Resseler
Veeam Software
 
Posts: 3672
Liked: 402 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby Shunx239 » Wed Jan 03, 2018 11:45 am

We have the same issue with the SBS2011 / vCenter Server 6.0.0 build-4541948 (SA).
Case #02452552

That all seems to be about the VSS timeout: https://www.veeam.com/kb1680

SBS 2011 is relevant since Persistent Snapshots can not be used on domain controller:
https://helpcenter.veeam.com/backup/80/ ... shots.html

So we are in grace of the Veeam Team to come back with a solution

gatorheelz wrote:Still awaiting an update from Support. I'm sure the timing of this hasn't helped (holidays), but sure hope to hear something today.

You probably underestimate that the holiday decade has just started in Russia :x
Shunx239
Novice
 
Posts: 7
Liked: 1 time
Joined: Sat Feb 11, 2017 9:39 pm
Location: Milan
Full Name: Nikolay

Re: SBS2011/Exchange VSS Issue - 9.5U3

Veeam Logoby Mike Resseler » Wed Jan 03, 2018 12:40 pm

@Nikolay,

It is not because holiday has just started that we are not working ;-). I noticed that there is still quite some action in Gatorheelz case.

EDIT: And PS: We have support engineers around the world ;-)
Mike Resseler
Veeam Software
 
Posts: 3672
Liked: 402 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: No registered users and 1 guest