Availability for the Always-On Enterprise
gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Dec 25, 2017 1:40 pm

We have noticed two separate sites where SBS2011 backup jobs are not completing since upgrade to 9.5U3 due to a VSS issue:
Unable to release guest. Error: VSSControl: Failed to freeze guest over network, wait timeout
Error: VSSControl: Failed to freeze guest over network, wait timeout

VSS writer error:
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {f1128c1d-a6fa-4a2b-a8dc-1fa94443beba}
State: [1] Stable
Last error: Retryable error

Next step is a support case, but wanted to see if anyone else is experiencing this? Both of these environments have not had any changes.

For now we need to shut down the guest to get a successful backup.

vmniels
Veeam Software
Posts: 2054
Liked: 453 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by vmniels » Dec 26, 2017 11:23 am

VSS errors can be related to multiple things so best would be to contact support for inspection.
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be

foggy
Veeam Software
Posts: 17087
Liked: 1396 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by foggy » Dec 26, 2017 12:27 pm

Try to restart the Exchange Writer service or the entire server and see whether this error shows up after that.

gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Dec 26, 2017 4:50 pm

Restarting Exchange VSS server and server restart does not result in any change.

gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Dec 26, 2017 4:55 pm

Case ID 02442756

Ibeme85
Novice
Posts: 3
Liked: never
Joined: Dec 03, 2017 7:43 pm
Full Name: Bjoern Ebner
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by Ibeme85 » Dec 26, 2017 6:29 pm

I already opened a case relating a possible VSS issue.

In my case VSS doesn't seem to fail but in a Sure Backup Job, the SBS2011 doesn't reboot out of AD Restore Mode into normal Mode and so the SureBackup Job fails.

Backups created with U2 Work. Backups created with U3 don't.

Case 02434632

gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Dec 26, 2017 9:06 pm

We have demonstrated to Support that the issue points to B&R rather than VSS. diskshadow full completes successfully for Exchange volumes. Exchange Writer in stable/no error state after diskshadow backup.

Gostev
Veeam Software
Posts: 23199
Liked: 2964 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by Gostev » Dec 27, 2017 2:51 pm

Just wanted to note that there are different ways to trigger VSS and it is safe to say that diskshadow, not being a backup application, does not instruct VSS Writer to do transaction logs processing for example... so, this would not be a valid comparison from my perspective. But no worries, you're in good hands now and our support engineers will get to the bottom of this.

gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Dec 28, 2017 10:25 am

The issue is that B&R thinks VMware snapshots are taking in excess of 20 seconds and failing the job. In actuality, the snapshots are taking 1-2 seconds. This has all been verified with Support. Case has been escalated to level 3, but we won't be able to resume activity for several more hours.

If anyone has experienced this and has some feedback, or any suggestions, we'd really appreciate it.

gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Dec 28, 2017 10:48 pm 2 people like this post

For anyone else affected by this, Veeam Support has isolated the issue to something involving vCenter. If you create a new Backup Job and point it to vSphere direct instead of vCenter, there are no issues.

Good workaround, but not excited about starting a new Full that we have to get offsite at 20 Mbps upstream.

ic_mcourtney
Influencer
Posts: 20
Liked: 2 times
Joined: Sep 22, 2016 7:15 am
Full Name: mcourtney
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by ic_mcourtney » Jan 01, 2018 4:45 am

gatorheelz wrote:For anyone else affected by this, Veeam Support has isolated the issue to something involving vCenter. If you create a new Backup Job and point it to vSphere direct instead of vCenter, there are no issues.
Did Veeam Support say if this affects all versions of vCenter, or if it's specific to one version?

@Gostev: Will there be a KB article published about this issue soon? I'm hesitant to proceed with upgrading to U3 before I fully understand the version(s) of vCenter affected, and what the impact is to VBR's ability to make backups.

gatorheelz
Influencer
Posts: 18
Liked: 3 times
Joined: Jun 19, 2017 4:38 pm
Full Name: gatorheelz
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by gatorheelz » Jan 02, 2018 11:14 am

They have not indicated which versions of vCenter are affected. In my case it's vCenter 5.5.0 (4180647) with internal DB. Very basic config.

Also, I'm not sure that this is limited to SBS2011. To me it seems like any job with Exchange (possibly only 2010) would be affected. The Level 2 engineer I was working with indicated they don't seem to have other reported cases, but don't think this is a unique case because we're seeing it in multiple environments.

Still awaiting an update from Support. I'm sure the timing of this hasn't helped (holidays), but sure hope to hear something today.

Mike Resseler
Veeam Software
Posts: 5000
Liked: 530 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by Mike Resseler » Jan 02, 2018 1:08 pm

Gatorheelz,

Please keep working with the support engineer. I see there are things going on in the support case so I am keeping my fingers crossed that you will have a solution as soon as possible.

@Matty: We can't say at this moment. Sorry about that

Best wishes
Mike

Shunx239
Novice
Posts: 8
Liked: 2 times
Joined: Feb 11, 2017 9:39 pm
Full Name: Nikolay
Location: Milan
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by Shunx239 » Jan 03, 2018 11:45 am

We have the same issue with the SBS2011 / vCenter Server 6.0.0 build-4541948 (SA).
Case #02452552

That all seems to be about the VSS timeout: https://www.veeam.com/kb1680

SBS 2011 is relevant since Persistent Snapshots can not be used on domain controller:
https://helpcenter.veeam.com/backup/80/ ... shots.html

So we are in grace of the Veeam Team to come back with a solution
gatorheelz wrote:Still awaiting an update from Support. I'm sure the timing of this hasn't helped (holidays), but sure hope to hear something today.
You probably underestimate that the holiday decade has just started in Russia :x

Mike Resseler
Veeam Software
Posts: 5000
Liked: 530 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: SBS2011/Exchange VSS Issue - 9.5U3

Post by Mike Resseler » Jan 03, 2018 12:40 pm

@Nikolay,

It is not because holiday has just started that we are not working ;-). I noticed that there is still quite some action in Gatorheelz case.

EDIT: And PS: We have support engineers around the world ;-)

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 37 guests