SQL usage goes to 100% immediately after upgrading to 11a version

[omegagx]

after upgrading Veeam ONE yesterday to version 11a, the SQL server was stuck at 100% of CPU with the following query causing the issue with the Veeam ONE database:

delete #IDs
where ID not in
( select ID from #IDS_Current )

I had to stop the Veeam ONE service to be able to free up SQL server CPU. Starting it again and starting the collection caused the issue to occur again about 8 minutes after the collection started. Looks like it's a bug.

[Vitaliy S.]

Hi Mike,

Please open the support case and let our technical team review the issue.

Thanks!

[omegagx]

I already did open the case #05078863 yesterday but so far 2 support engineers couldn't help and even though it's severity 2, my Veeam ONE is currently down.

[Vitaliy S.]

Feel free to ask for escalation, our RnD team should be able to help once we have this ticket in our queue.

[omegagx]

Just asked for the escalation - hopefully it's in your queue now.

[wishr]

Hi Mike,

As far as I see the case has been escalated and the support team is waiting for additional logs from you.

Thanks

[omegagx]

logs uploaded

[MFranzen]

Hi,

we got the same issue, but we dont use Veeam One that much.
Stopping and disabling the Service "Veeam One Reporting Service" immediately stopped
the hight CPU usage.

It's not a permanent or perfect solution but it solves the bigger problem for now.

[Vitaliy S.]

Hi MFranzen,

Thanks for your feedback, but if you want to resolve it, please open a support case as additional details might be required for review.

Thanks!

[omegagx]

We did open case 05078863 but haven't had any progress at all. I've been asking for updates for a week already.

[Vitaliy S.]

I see that your case has been escalated to a higher-tier engineer. Let's see what he can find out.

[omegagx]

so far they weren't able to find out anything and it's been 3 weeks.

[wishr]

Hi Mike,

As far as I see from the case notes our tech. support engineers are analyzing the data you provided. Stay tuned for updates.

Thanks

[omegagx]

How long do you think it will take to get a fix? Any idea why this wasn't caught in QA?

[wishr]

Hi Mike,

The fix is ready and currently being verified by our QA team. The engineer assigned to the case will let you know once it's ready to be distributed. We are also planning to include this issue in the Top Issues Tracker forum post once get all the necessary information.

I cannot say yet why the issue was not discovered during our regular QC cycle as the data about it is still being formalized, but we'll make sure to get to the bottom of that.

Thanks

[stfconsulting]

I am going through this right now. Case 05233717 When you stop the reporting service it calms down immediately. I provided the most expensive queries to support and all the logs.

[wishr]

Hi Stfconsulting,

It sounds like issue #4 from the aforementioned list of known issues. You can get a hotfix for it through our support team by referencing the specified issue ID.

Upd: I've pinged our support management and made sure the engineers are proposing the correct fix.

Thanks

[dali@iae.nl]

Same high CPU issue for us when Veeam ONE Reporting Service is running, Case #05771747

[jorgedlcruz]

Hello Da Li,
I can see that our Support Engineers are actively sending you a few scripts, and things to try. Please keep working with them wherever possible.

I have added myself to the ticket so any update, I will be informed.

Thanks a lot!

[dali@iae.nl]

Today had a Webex with Veeam on Case #05771747 to analyze Veeam ONE database server 100% CPU.
Got the Fix_330950_4004cac594.zip and added a registry ParallelWidgetDataDbRequestCount. After that CPU Usage went from 100% to below 10%. Thanks Veeam Support!