Comprehensive data protection for all workloads
Post Reply
wa15
Expert
Posts: 280
Liked: 21 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Veeam B&R Console Freezing

Post by wa15 »

Just updated to V10 from 9.5U4b and we have noticed that the B&R console keeps on freezing and we have to end the Backup Shell process in task manager to restart it. This is happening on multiple systems (Windows 10 1909 and Server 2016 Standard 1607). I have been able to see a pattern on when B&R freezes, but usually it's after some time of inactivity. Sometimes after 15 minutes, sometimes after 2 hours.

Not sure if related, but when opening the console, it sits at the splash screen for 3-5 minutes before starting.

Has anyone else experienced similar symptoms? Unfortunately, this is making our backup environment almost useless as we are still trying to deploy AHV proxies etc.

Case #04095157

Dima P.
Product Manager
Posts: 11623
Liked: 1007 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Veeam B&R Console Freezing

Post by Dima P. » 1 person likes this post

Hello wa15,

Thank you for your post, I've merged it to existing discussion. Please keep working with our support team in order to get the mentioned fix when it's out. Cheers!

wa15
Expert
Posts: 280
Liked: 21 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Re: Veeam B&R Console Freezing

Post by wa15 »

Thanks Dima. Any ETA on when it'll be out? This is making our environment almost unusable. We are backing up about 150 VMs with this environment. Veeam sits on a physical 14gen Dell server on flash storage (SQL Express install) so having Veeam freeze every 20 minutes, it's very difficult to use.

Also, unlike the title of this post, the console completely freezes for 10-15 minutes (sometimes longer) so it's not just slow, it just freezes.

wa15
Expert
Posts: 280
Liked: 21 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Re: Veeam B&R Console Freezing

Post by wa15 »

Spending more time with this, the issue I am running into is more severe (and different) than what this thread is reporting. In my case, the console simply just freezes for up to an hour, sometimes 2. Eventually starts responding again for a few minutes, then dies again.

Resources are good (24 physical cores, 64GB of memory, ~150VM environment). Rebooted multiple times. Quite frustrating.

Gostev
SVP, Product Management
Posts: 26498
Liked: 4150 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam B&R Console Freezing

Post by Gostev »

Sorry for a bad merge. It's a different issue (looks like bug 204167), also known by now and support may have a hotfix already. Thanks!

wa15
Expert
Posts: 280
Liked: 21 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Re: Veeam B&R Console Freezing

Post by wa15 »

@Gostev, is there anything we can do to speed up access to the hotfix? We are dead in the water right now as Veeam is next to unusable. Been following up for support but still waiting to hear back as of 3/31 at 5PM PST. Considering rolling back to v9.5

Also, not sure how common this bug is or what the cause is, but if it's somewhat common, perhaps it should be added to the top issues tracker thread? If I would have known the pain, I would have held off on updating :)

Gostev
SVP, Product Management
Posts: 26498
Liked: 4150 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam B&R Console Freezing

Post by Gostev » 1 person likes this post

No, it's not common, which is why it is not listed in the top issue tracker. If this is the bug I'm thinking about, then it only triggers due to overloaded or misbehaving SQL Server where the configuration database is hosted. So, it's more of a corner case. But at this point it's just guess based on your description, your issue could be different than the bug I'm thinking about.

Gostev
SVP, Product Management
Posts: 26498
Liked: 4150 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam B&R Console Freezing

Post by Gostev » 1 person likes this post

Checked our support system and indeed there's not much activity on your case it seems, so I asked support management to escalate it into R&D.

Gostev
SVP, Product Management
Posts: 26498
Liked: 4150 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam B&R Console Freezing

Post by Gostev »

@wa15 so I was checking on this support case, and looks like you asked to close it because the issue resolved itself? Do you have any guesses as to what may have caused it, was it some temporary problem with the environment?

wa15
Expert
Posts: 280
Liked: 21 times
Joined: Jan 02, 2014 4:45 pm
Contact:

Re: Veeam B&R Console Freezing

Post by wa15 »

@Gostev - Yes, the issue resolved itself around 72 hours later. Interestingly enough, CPU/Memory/Disk utilization during the console freezes were low so I can't imagine Veeam doing any data intensive tasks. Also, during the 72 hours, nothing changed that I can think of, besides usual jobs kicking off here and there as jobs were still completing successfully from what I could see, but the console itself was NOT responsive.

This environment does have AHV proxies (7 of them). I performed the V10 update in another environment with very similar physical hardware (Dell R740xd), and that completed successfully but that one did not have AHV proxies so maybe there is a bug in environments where AHV is involved? Just a rough guess.

mrholm
Expert
Posts: 104
Liked: 9 times
Joined: Apr 20, 2018 8:12 am
Full Name: Mats Holm
Contact:

Re: Veeam B&R Console Freezing

Post by mrholm »

Hi
We also see this behaviou ron a newly upgraded site, 10.0.0.4461 SP2 on Windows 2016 servers,
The GUI freezes and in Resource Manager we see it as "not responding". During this time Veeam uses the temdb files a lot for both read and write although the only jobs running are SQL t-log jobs nothing else.
CLosing tdown the GUI from Task Manager and opening up again seems to work as a workaround...

wishr
Veeam Software
Posts: 1745
Liked: 195 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Veeam B&R Console Freezing

Post by wishr »

Hi Mats,

Have you already raised a support case? If so, please share your case ID with us.

Thanks

mrholm
Expert
Posts: 104
Liked: 9 times
Joined: Apr 20, 2018 8:12 am
Full Name: Mats Holm
Contact:

Re: Veeam B&R Console Freezing

Post by mrholm »

04234067 is the support ID, now support want me to follow https://www.veeam.com/kb1995 but I still say this is after upgrade to v10 and we will have a webex session on Tuesday.

mrholm
Expert
Posts: 104
Liked: 9 times
Joined: Apr 20, 2018 8:12 am
Full Name: Mats Holm
Contact:

Re: Veeam B&R Console Freezing

Post by mrholm »

Hi
had a webex session with support and still refering to the above KB and also decrease the number of sessions to keep in the database, today we show 500 sessions and keep sessions for 53 weeks (defaultvalues). The KB says to decrease the number of sessions and also to drop old sessions directly in the database. To me this is just a workaround, we have had the same configuration during the 2½ years we have run veeam in the whole enviroment and have never seen this on previous versions on this server. If there's a problem keeping sessions in the database for 53 weeks in an environmnet of 75 backup jobs and similair amount of copy jobs then there should be a info about that written in some best practise or so.

wishr
Veeam Software
Posts: 1745
Liked: 195 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Veeam B&R Console Freezing

Post by wishr »

Hi Mats,

We are keeping an eye on the case and will assist where needed.

With each B&R release, the dataset is obviously growing due to lots of new objects and parameters being added with each release and that forces us to decrease the data retention in order to guarantee the appropriate level of performance. Speaking about the direct solution, we are planning to enhance the approach to storing sessions data in the next product versions, but it's obviously not a "quick fix".

Thanks

Post Reply

Who is online

Users browsing this forum: No registered users and 29 guests