Wierd Veeam B&R server after Windows Updates / Reboot

Hyper-V specific discussions

Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby enoch » Fri Apr 28, 2017 3:09 pm

Hi all,

Had a pretty solid B&R running Windows 2012r2 DataCenter Edition with current Veeam 9.5 U1 running.

All Windows patches is up-to-date from Windows Update feature in Windows. Last time I updated the server is months ago. In the window we updated the SQL 2014 SP2 to SP2 CU5.

After this on the B&R server it self I have had some "lag" in the interface and 3-4 times today got the message:

"The connection to backup server has been lost. Attempting to reconnect to your session"

"Connection attepmt: 1 of 10"

All services: console, SQL, B&R is on same server!

A lot of jobs I suddenly gets the error (both VMware, Hyper-V backup jobs and replication jobs):
"Full backup file merge failed error: ChannelError: ConnectionReset"

All Backup repositories is on local storage!

Tried restore from some of the jobs failling and they seems to Work fine!

I have a current open case at Veeam: 02140683 but if anyone have any fix/ideas I'll be glad to hear from you!
enoch
Service Provider
 
Posts: 114
Liked: 8 times
Joined: Sat Jun 29, 2013 12:14 pm
Full Name: Peter Enoch

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby Vitaliy S. » Mon May 01, 2017 10:02 am

Hi Peter,

Do you have any SQL Server related error messages in the Windows Event log?

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19571
Liked: 1104 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby enoch » Mon May 01, 2017 10:12 am

Hi,

I can't find any SQL related errors in the System or Application Event Logs.

It's seems a reboot "fix" the problem, 4-6 hours and then problems again.

A stop of the Veeam Backup Service results in not able to start the service - a server reboot is needed! :cry:
enoch
Service Provider
 
Posts: 114
Liked: 8 times
Joined: Sat Jun 29, 2013 12:14 pm
Full Name: Peter Enoch

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby Vitaliy S. » Mon May 01, 2017 10:28 am

OK, then let's see what our support engineer says after reviewing the debug log files.
Vitaliy S.
Veeam Software
 
Posts: 19571
Liked: 1104 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby enoch » Mon May 01, 2017 12:16 pm

Yes, running the case with Veeam support now.

Hope they find the problem and a fix.
enoch
Service Provider
 
Posts: 114
Liked: 8 times
Joined: Sat Jun 29, 2013 12:14 pm
Full Name: Peter Enoch

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby JosvwHLT » Tue May 02, 2017 8:14 pm

Last Friday I updated 2 Veeam B&R 9.0 servers to 9.5 u1. One of the 2 is experiencing the same problems. A server reset is the only option. In the systemlog i see evenitid 4227 - TCP/IP Network Connectivity (EVENT_TCPIP_TCP_TIME_WAIT_COLLISION). NIC disable/enable option is not available because server is almost non-responsive.
JosvwHLT
Influencer
 
Posts: 14
Liked: 1 time
Joined: Thu Dec 24, 2015 4:15 am
Full Name: Jos van Wordragen

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby enoch » Tue May 02, 2017 8:45 pm

Hi all,

I've tried REALLY much right now.

I see there is a issue with the Microsoft patch: kb4015553 for a lot of people - Windows 2012r2 - the patch is from late April 2017.

Sadly I cannot remove the patch from the server without an error :-(

Maybe if you have this installed you could try with uninstall?
enoch
Service Provider
 
Posts: 114
Liked: 8 times
Joined: Sat Jun 29, 2013 12:14 pm
Full Name: Peter Enoch

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby JosvwHLT » Wed May 03, 2017 3:55 am

Don't want to uninstall any patches right now before knowing what is really going on. We are running a tape backup procedure in which we mount a backup image (with VeeamFLR) and retrieve data from that mounted image and dump it to tape. During this procedure the server crashed already twice. I'm running the Veeam B&R server now without running any tape procedure. Want to wait for that outcome first before i change any thing on that server. We are also experiencing a failed service start of the "VeeamCatalogsvc" service, this service won't start anymore after we experienced the issues with the "unresponsive server" but don't know if it is related. In the application log i see the following "Service Cannot be started", part of the message is the following "Attribute "LastSyncTimeUTC" was not found under the tree".
JosvwHLT
Influencer
 
Posts: 14
Liked: 1 time
Joined: Thu Dec 24, 2015 4:15 am
Full Name: Jos van Wordragen

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby JosvwHLT » Fri May 05, 2017 4:35 am

Applied TCP/IP settings as described in URL vmware-vsphere-f24/veeam-flr-t32119-15.html This seems to have fixed my issue, server is running for a couple of days now without any issues. Server froze when running a tape procedure which copied a lot of small files to tape. That is running fine now too.
JosvwHLT
Influencer
 
Posts: 14
Liked: 1 time
Joined: Thu Dec 24, 2015 4:15 am
Full Name: Jos van Wordragen

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby enoch » Sat May 06, 2017 1:29 pm 1 person likes this post

Hi all,

Afters days suppport with Microsoft who didn't have any luck removing the KB4015553 which could not be removed in Windows, I succeed removing the patch.

After the patch was removed my system have now run 24 hours without issues.

Before I could only have the system running about 8 hours before I was needed to do a reboot.
enoch
Service Provider
 
Posts: 114
Liked: 8 times
Joined: Sat Jun 29, 2013 12:14 pm
Full Name: Peter Enoch

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby kstrp » Wed May 10, 2017 2:09 pm

enoch wrote:Hi all,

Afters days suppport with Microsoft who didn't have any luck removing the KB4015553 which could not be removed in Windows, I succeed removing the patch.

After the patch was removed my system have now run 24 hours without issues.

Before I could only have the system running about 8 hours before I was needed to do a reboot.


Mind sharing how you removed it? Couldn't get that to happen on our own and microsoft couldn't figure out when we spoke to them briefly.
kstrp
Lurker
 
Posts: 2
Liked: never
Joined: Wed May 10, 2017 1:59 pm
Full Name: RP

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby enoch » Wed May 10, 2017 3:09 pm

I would like to, but not 100% sure what did it :D

Please try this:
Dism /Online /Cleanup-Image /CheckHealth - check everything is OK

Dism /Online /Cleanup-Image /ScanHealth - check everything is OK

If these are not OK, then: DISM.exe /Online /Cleanup-image /Restorehealth and: sfc /scannow

Download the KB4015553 manually from Microsoft: http://www.catalog.update.microsoft.com ... =KB4015553

Then extract thethe KB4015553.msu package.

Then do a DISM.exe /Online /Add-Package /PackagePath:"path to KB4015553.cab file"

Then do a dism /remove-package with this update. Find the package ID with: Dism /Online /Get-Packages

After this I went in the Add/Remove program an the uninstall worked!
enoch
Service Provider
 
Posts: 114
Liked: 8 times
Joined: Sat Jun 29, 2013 12:14 pm
Full Name: Peter Enoch

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby kstrp » Wed May 10, 2017 3:21 pm

Hmm - I've tried similar steps already - but I'll give it another go covering what you listed.

Thanks again.
kstrp
Lurker
 
Posts: 2
Liked: never
Joined: Wed May 10, 2017 1:59 pm
Full Name: RP

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby LittlePiGPiG » Fri Jun 02, 2017 3:47 pm

I experienced exactly / very much similar symptoms of everyone above said and sadly the Veeam support had not yet come up with any conclusions.

Then, I tried removing latest "updates" (Except those M$ "security patches") and I get a normal V&R server back.

Very lucky in my case, hope it will be useful to you all as well.
LittlePiGPiG
Lurker
 
Posts: 1
Liked: never
Joined: Fri Jun 02, 2017 3:43 pm

Re: Wierd Veeam B&R server after Windows Updates / Reboot

Veeam Logoby Mike Resseler » Mon Jun 05, 2017 8:39 am

Can you tell us what patches you removed? It might help other community members also and I don't want everyone to install all patches ;-)

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 3171
Liked: 362 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Next

Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 8 guests