Console wont Load and backups fail after V12 update (N-able N-central)

[Egor Yakovlev]

Current situation looks like this:
- N-Able agent issues periodic polls on VBR Service(refreshing status), triggering a locally-executed PowerShell script
- some of commands used in that script are no longer supported by v12, and return errors back
- There seems to be a problem cleaning up unfinished sessions caused by that script (still under investigation)

It is best to turn off N-Able agent monitoring on VBR Server for now (to stop polling), and wait for both fixes (N-Able to update the script and stop spamming our service with unsupported commands, and Veeam to properly cleanup stale sessions).

/Thanks!

p.s. spot-on diagnosis by Darrin above!
p.p.s. Veeam One presence has no role here

[Ci2Group]

We are having this exact issue and can confirm no Veeam One installed on server. (#05960758)
I have removed the monitoring of veeam jobs from the server but the issue still persists. Do I need to disable all Nable monitoring ?

[LickABrick]

Disable the 'Automation manager' service and reboot veeam services or the whole server.

[redboom]

hello!

we also have problems with V12 and are also using nable (latest version)

we are getting timeout errors. sometimes the jobs work and then they fail for some reason...

i will try the regkey. will update tomorrow!

but just to clarify, what are you guys disabled exactly (the automation services?)

update:
I only disabled nable agent and backups all work now. (after reboot...)

[YouGotServered]

All,
Can confirm that removing the N-Central Veeam Service Template from our backup servers in N-Central, then one last reboot of the VBR server fixed the issue. This also removes all of the Veeam job monitors that it applies to the server, and those are the ones running the automation policies that are incompatible. When you remove the service template, you may want to disable the rule that applies it to Veeam servers (remove the scoping filter or deny it to your N-Central client sites), or it will just apply again.

When you do that, you don't need to disable or uninstall any other N-Central components, you're targeting the exact part that causes the issue

[StevenMeier]

I have this issue. I have no N-able agents and no Veeam One.

I upgraded everything from v11 to v12 and all worked for a few days and then it just got stuck ...(hung in the console) A reboot seemed to fix this ...but after 4 days same issue....this time I was just removing a Veeam Agent job in console we no longer needed. it seemed it was going to take some time so left it......it never completed and reboot required.

I have not logged a job yet but this seems to be just a veeam v 12 backup and recovery issue.

[rennerstefan]

Hi Steven,
may I ask you to open a ticket with veeam support? That would help to check details and looks into the issue.
Feel free to drop the ticket ID here as well.
Gruss

[StevenMeier]

I have logged a case and escalated it as responses have been very slow.

Case #06020076

[nroyepic]

Anyone find a fix for this? I have an open ticket with ncentral and they are brutally slow, again the only fix temporarily is to disable the monitoring jobs

[Egor Yakovlev]

Hi Nick.

Fix for that issue is included in latest cumulative patch.
Please note that it does not change the fact N-Able officially does not support VBR v12 and keeps hammering our service with unsupported commands

/Cheers!

[YouGotServered]

Just a polite reminder to those still asking for a fix, I posted my confirmed solution up above If it isn't working for you, let me know and I'm happy to help. As a Veeam and N-Central user, I've got a bit of insight into both areas.

[StevenMeier]

So I am now unto my 3 or 4 week of this issue.

Case #06020076

Support seem to have no idea and are not making much atempt to solve.
Since I have no Ncentral Agent (N-able) on the box they appear lost)
I supplied a full sql Backup for them....provided job logs.
I stated I was concerned to see very many veeam.sateliite process in taskmanager as well as backup manager process also in task manager.....spawning over 3 days.....then I have to restart the server after 3 days and manually retry and resync jobs and the server is good for 3 days.
I have applied sql timeouts in registry as requested because the jobs were timing out was the error (903 secs).
After many emails with grasping at straws the veeam support said please apply veeam 12 latest patch...which I did...... then we get a new issue. this error message

5/05/2023 11:42:47 AM Warning Failed to enumerate NAS backup metadata files Details: Agent: Failed to process method {MessageChannel.EstablishConnectionByRoles}: Unsupported NAS protocol version. Message version: 12.0.0.151 (201326743), Required version: 12.0.0.150 (201326742)
Now we have Refs volumes on our landing zone backups but we also have hpe 5250 storeOnce devices which do not use ReFS but get the same error.

Luckily I took a snap shot as I will roll back the patch.....
Veeam this is a mess.....why am I having no decent effort put into solving my issue ?
This is a failure on Veeam support and I suggest QA .
We never had issues on Veeam 11......wish I could go back to that now.

[theruck]

Similar issue with V12
using external share as a backup repository which uses USB drive which gets rotated. The V12 always hangs after few days, the console never starts while the backup run until the disk is changed again. I run the sync-vbrbackuprepository every day but it stucks too
nothing works until I restart veeam backup service and also the CDP coordinator service - which both I have to kill.

The V12 officially sucks as it breaks itself on the very simplest installation with 2 backup clients on HyperV.

Code: Select all

[14.06.2023 10:18:50.463] <149> Info         [SatelliteProcessClient]Satellite process is not running for current user: DOMAIN\administrator
[14.06.2023 10:18:50.473] <149> Error        Failed to start satellite process (System.Exception)
[14.06.2023 10:18:50.473] <149> Error           at Veeam.Backup.Interaction.Management.CSatelliteClient.Connect()
[14.06.2023 10:18:50.473] <149> Error           at Veeam.Backup.Interaction.Management.CSatelliteClient..ctor(CBackupServiceSessionStartupInfo startupInfo, Guid clientSessionUid)
[14.06.2023 10:18:50.473] <149> Error           at Veeam.Backup.Interaction.Management.CSatelliteGateway.<>c__DisplayClass17_0.<TryRegisterRoute>b__1()
[14.06.2023 10:18:50.473] <149> Error           at System.Lazy`1.CreateValue()
[14.06.2023 10:18:50.473] <149> Error           at System.Lazy`1.LazyInitValue()
[14.06.2023 10:18:50.473] <149> Error           at Veeam.Backup.Interaction.Management.CSatelliteGateway.<>c__DisplayClass20_0.<StartAsync>b__0()

[LickABrick]

@theruck Install the latest V12 patch, as mentioned above this 'fixes' the issue. We have had no issues after we installed the latest patch (with N-Able Veeam monitoring enabled)

[theruck]

that patch is included in the iso i have downloaded V12 on and is included as my build number is one number higher than the mentioned patch.

[Gostev]

I'm locking this thread because it is quickly becoming a parking spot for random issues totally unrelated to the OP's issue (caused by N-able) which has been resolved. Not to mention unprofessional comments and multiple forums rules violations. If you choose to continue participating in this community, kindly try your best to avoid both with your future contributions.