[V13] Known Issue - High CPU Usage while monitoring VBR v12.x

[jorgedlcruz]

Summary
We have identified a known issue that can lead to sustained high CPU usage when environments upgrade to Veeam ONE v13 and enable the new Veeam Analytics Service while the monitored backup servers run Veeam Backup & Replication v12.x.

Who is affected

• Veeam ONE v13 with Analytics Service enabled
• One or more Veeam Backup & Replication servers on v12.x

Observed symptom

• Abnormal, and constant elevated CPU on the VBR server
• Abnormal, and constant elevated Windows Defender CPU usage

Immediate recommendation

• Upgrade to the latest Veeam ONE v13 Patch, download information and steps here
• Add the standard Veeam antivirus exclusions on all Veeam Backup & Replication servers, with special attention to the Analytics Service path. Follow our official guidance in KB1999: Antivirus exclusions for Veeam components

Notes
Keep us updated in case you experience any issue with the patch upgrade.

[jorgedlcruz]

Update: We have released the first Veeam ONE v13 patch that fixes this specific CPU abnormal consumption, plus a few other ones, the recommendation to all Customers is to apply the Patch as soon as possible.

[chris.childerhose]

The patch fixes this issue as well as others. Tested and works great

[jorgedlcruz]

Awesome feedback, thanks Chris.

I am glad everything is know stable, if anything changes, please share it with us.

Thank you

[chris.childerhose]

Not a problem, Jorge. I will continue my testing and feedback.

[ndaviscmm]

Installed the patch today as well and so far it has made a significant improvement to performance! Will continue to monitor and update if it changes.

[pablo.gutierrez]

We installed the patch and the CPU usage of the VBR 12.3 returned to its normal state. Will continue to monitor.

[jorgedlcruz]

Awesome Pablo, thanks a lot. Keep us posted about that, or any other issue you might encounter.

Thank you

[JRRW]

For what it's worth, we're seeing massive latency on our VBR server even with the patch installed.

When I stop Veeam One (running on a different server) the latency goes down to sub 1ms.

Our VBR Management server is running on an R650 connected with 32gb FC to a PureStorage X50R4 - 16 cores and 64GB Memory - and is the only VM running on the R650. Xeon Gold 6326.

We have the veeambackupreporting DB still on C, but the Veeam Backup db temp and logs have all moved to dedicated VHDX (but again, they're all backed by a SAN that can do 150k iops without breaking a sweat)

We're on community edition so I'm not opening a case as it's not 'supported' - but I wanted to offer the data.

I'll have to disconnect VBR from Veeam One for the time being though.

[jorgedlcruz]

Hello Ryan,
We have not observed disk latency on any of our scalability labs, or had being reported before either with new Veeam Analytics Service. Have you excluded the Veeam paths on the VBR Server from your antivirus? https://www.veeam.com/kb1999

Also, the screenshot doesn't show which process is consuming that disk latency, during initial data collection (maybe that it is your case), some minutes up to a few hours we would need some resources to perform that initial intake, but never seen disk itself.

It is difficult to troubleshoot in the forums, and without a support ticket it is even more difficult.

Please leave the service up for a few hours, also explore that it is the Veeam Analytics Service consuming the disk, and exclude the paths from Antivirus.

Thank you!