Feature Request: Keep "Resource not ready" in job log

[nmdange]

When running a backup job for a large number of VMs, many of those VMs will spend a lot of time in a "Resource not ready" state. This could be any number of resources, like the backup repository, the proxy, etc. However, once the VM starts processing, the job details only show the message "Required backup infrastructure resources have been assigned".

This makes it hard to see after the fact what resources were causing the delay. It would be nice if you could see in the UI how long each VM spent waiting for each specific resource. This would help us know what piece of the backup infrastructure is limiting backup parallelism when managing a large environment. So we would know if adding a new backup proxy makes sense, for example. As it is, right now I can only see this information by watching a backup job live while it's running, which is a bit of a problem when the jobs always run in the middle of the night

[wishr]

Hi again Nmdange,

Thank you for the request. It's noted.

Btw, have you managed to identify the root cause?

Thanks

[wishr]

I would like to double check the following: is there anything else after "Resource not ready" phrase in your case, because usually there is an indication which resource is not ready.

Also, could you please let me know if you, by any chance, use Veeam MP or Veeam ONE? The second part of your post sounds like a good potential feature for some of our monitoring tools

BR,
Fedor

[Shestakov]

Hello,
what if you could see statistics of proxies busyness? E.g. proxy A had all its resources (parallel tasks) used by job X during particular period of time. So you can understand which proxies are overused and/or which jobs is better to re-schedule.
By the way, do you use Veeam ONE?

[nmdange]

Since most of my environment is Hyper-V, I see a lot of "Resource not ready: Snapshot", but I have also seen "Resource not ready: Backup Proxy" and "Resource not ready: Backup Repository". I know what the causes are and how to fix them, it's just I can't see what the type of resource is unless I watch the backups live.

I don't have Veeam ONE licensing. I could definitely see Veeam ONE having detailed reports on parallel tasks, but at a minimum, just being able to look at a job in the console after the fact and seeing which resource type was waiting would be enough for me.

[Vitaliy S.]

Yes, this log entry would be a nice to have feature, I agree. Also, I would recommend taking a look at Veeam ONE 9.5 U4 (once available) new features that might also be able to solve your existing issue. Thanks!

[hyvokar]

Hi,

When job action log states, that "resource not ready" (usually backup repository in my case), could you please add WHICH resource is not ready. I mean I have several backup repositories.

[foggy]

Hi, typically it should state the particular resource name, but seems there are situations when it is not the case. We will look into it, thanks.

[hyvokar]

Thanks for the reply foggy.

Personally I don't recall ever seeing the resource name, but could be my memory.

[crackocain]

+1

[nmdange]

I made a similar request recently veeam-backup-replication-f2/feature-req ... 55693.html

[Zew]

Running Veeam 11. Same issues still exists. Job @ 99% stating "Resource not ready: Backup repository".
Same boat as others, doesn't specify which (although I know which one cause there is only one for these jobs).

Which is super weird cause the jobs process (data transferred) but then cause other jobs of the same nature all started at the same time (source of problem was a long list of "old style" BCJ were paused and then restarted after some physical maintenance work was completed). It's funny cause they will transfer the data and then wait at this state instead of completing:

E.G.

Job 1 - Processing 21% (no resource issues)
Job 2 - Processing 0% (waiting for resources)
Job 4 - Processing 99% (Resource not ready: Backup repository) (waiting to merge)
Job 5 - Processing 99% (Resource not ready: Backup repository) (waiting to merge)
Job 6 - Processing 99% (Resource not ready: Backup repository) (waiting to merge)
Job 7 - Processing 0% (waiting for resources)
Job 8 - Processing 99% (Merging oldest incremental into full backup file) (finally after some of the other processes completed)

What's super weird about this, is you figured the job as a whole gets processed as a "batch" and gets resources to complete all required tasks. Instead what appears is each jobs tasks are queued, so they have to wait to completed the final task to complete their tasks.

Not suggesting a change in design, but interesting to note based on the symptoms.

[foggy]

That basically means your repository doesn't have the required amount of slots to perform all these tasks in parallel. If you feel the repository is speced for all this load - then it's worth opening a case for a closer look. Remember though that each backup chain merge process takes a separate slot and if this repository is per-VM chains enabled, then it should have enough slots to process all VMs in the job since each of them has its own chain and requires a slot.

[Zew]

I totally get the slot "tasks" requirements, and understanding jobs have multiple tasks completing for resources the constraints are pretty obvious actually.

Just noted the repo name was not listed in the state of the job. That's all, the rest makes sense once I printed out the design myself on paper.

[foggy]

Ok, misread your post then. Thanks for the feedback, anyway.

[Zew]

No worries, thanks for following up Foggy, always good to hear from you.

Just wondering if the root cause of when the resource is not directly named was determined as you mentioned in previous posts it was to be looked at.

Hi, typically it should state the particular resource name, but seems there are situations when it is not the case. We will look into it, thanks.

if not, no worries again I knew which one it was based on the deployment. Just a curious George here.