Stalling Jobs

[t7MevELx0]

@mjr.epicfail

Case #07583409


I just fixed the indexing issue on the two repos (you'll see that issue in that ticket too...). The copy jobs just sit there, transferring very little, or absolutely nothing.

Things were working great on 8.05. These copy jobs haven't worked since upgrading to 8.1 in January.

[mjr.epicfail]

Hi t7MevELx0,

I dont have access to the support ticket, im not a Veeam Employee.
I asked because the Veeam members on the forum can look it up and maybe shed some light on the issue, or maybe that R&D is working (actively) on this issue.

[t7MevELx0]

Yeah, I usually share the ticket number here for that reason. I'm still seeing jobs stall on version 8.1.1. It happens randomly, and I have to use a quirky workaround to get things running again. Just clearing the NATS JetStream isn't enough. I typically have to place each proxy in the pool into maintenance mode and then bring them back out. In some cases, I need to remove the proxies from the pool and re-add them to get things unstuck.

[pat_ren]

Have been continuing to have this issue intermittently, even on 8.1.1.159 and on my smallest servers and smallest jobs that usually take 1-2 mins max to run. Have received a fix from support yesterday that they say should address the issue.
My case is 07708238, as it's intermittent can't confirm if it's resolved anything yet, but for anyone waiting on a fix there may be some progress now.

[pat_ren]

Fix from support did not help so this is still an ongoing issue, every couple of days a job is getting stuck and running indefinitely until manual intervention occurs.
Case is still open

[MarkBoothmaa]

It looks like our issue (the original post on the thread) has been fixed by an hotfix. We are currently monitoring progress but results have been positive for the last week.

[pat_ren]

Got some more progress on this recently, confirmed it is still an issue and a known issue being worked on by R&D.
From what I have been able to determine, the issue is related to NATS streams getting stuck with an object they don't know where to put and the job just stalls.

If you see something like this in the log, especially with repo id 00000000-0000-0000-0000-000000000000 then it means the nats is confused and your job is stuck.

[10.06.2025 05:10:41.165] 466 (1) [Info] Adding object to job report (session ID: xxx-xxx-xxx-xxx)...
[10.06.2025 05:10:41.165] 466 (1) [Info] Object was successfully unlocked: xxx-xxx-xxx-xxx (repository: 00000000-0000-0000-0000-000000000000, session: xxx-xxx-xxx-xxx)

No fix yet but it does seem like the issue is understood by R&D but they don't know how to fix it yet.

My case is still open.

[tkutil]

I believe we have the same issue, but only when it comes to our mail job. We have the users broken into 4 groups all in the same job and this seems to work, but if we try to switch from using groups to using the organization the job never seems to finish. I have even tried a combination of user groups and the organization, but that also does not work. For those having the problem are you also trying to backup the organization?