Comprehensive data protection for all workloads
Post Reply
SteelContainer
Service Provider
Posts: 146
Liked: 21 times
Joined: May 21, 2014 8:47 am
Location: New Zealand
Contact:

Waiting for backup infrastructure - More Detail?

Post by SteelContainer » 2 people like this post

Hi There,

I'm just wondering if there are any plans to add a feature to the VBR console which can provide more detail on this when jobs are waiting? We have slowly been increasing the number of VMs backed up in our environment (up to 330 at the moment) and I have found more and more jobs becoming queued. This is fine because they do complete on time but I need more detail in the console.

I would like and easy way to figure out
1. Which jobs are actually transferring data and which jobs are Pending - We have 100+ Jobs and clicking on each one to check which VMs are transferring is a pain. The job status does not help as this will often have the overall transfer rate if one VM in the job has completed backup and the rest are pending the status will still have a transfer rate even though no data is currently being transferred in that job.

2. When Jobs are waiting for infrastructure resources, what resource are they waiting for? Source, target? Why? Is it IO restriction or the concurrent limit reached?

Adding extra columns to the console with an actually transfer rate and a meaningful status could solve these.

Thanks,
Cullan
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Waiting for backup infrastructure - More Detail?

Post by dellock6 »

Hi,
even if all the jobs are queued and in status "Running" I think a quick way to check which one are actually using available processing slots is to see the progress percentage, all the queued ones should be at 0%. By the way our parallel processing works, once a job is effectively started (thus assigned processing resources) it should complete before other jobs are moved from queue. But there are some limits in the infrastructure that could make other jobs enter the processing queue before other can progress, for example the maximum number of snapshots per datastore: once one has hit this limit (4) a job that is protecting VMs in another datastore may be started even if another job is already running. This is done to optimize the use of proxies and try to always keep them completely busy.

With 300+ VMs, I would check:
- how many proxies you have and how many CPU they have (count the number of processing slots)
- repositories and their limit on concurrent backup jobs
- limits reached on the (is it VMware?) infrastructure on concurrent snapshots: 4 per datastore, 8 per vCenter, 5 per single ESXi. All are tunable if vCenter is not overloaded in cpu and memory. But better first check the situation with default parameters.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
SteelContainer
Service Provider
Posts: 146
Liked: 21 times
Joined: May 21, 2014 8:47 am
Location: New Zealand
Contact:

Re: Waiting for backup infrastructure - More Detail?

Post by SteelContainer » 2 people like this post

Thanks for the reply,

We are using Hyper-V and on-Host proxies. This has worked well but as we have grown I think it may be time to go for Off-host. All off our jobs are scheduled to run at 6pm each day, perhaps staggering could also help.
We definitely have jobs where for example 2/10 VMs process then another job will start making it hard to tell if that first job is actually transferring data as the status will say XX% completed at XXMB/s. I guess it would just be nice to know what jobs are actually doing the work.
I have been talking to support regarding the waiting for resources and there are svc logs that I can check but again it would be nice to see this somewhere in the console even if it was just 'Waiting for resources (Source snapshot limit reached)' etc.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Waiting for backup infrastructure - More Detail?

Post by foggy » 1 person likes this post

Cullan, thanks for your feedback, I admit some improvements in reporting can indeed be handy.
Post Reply

Who is online

Users browsing this forum: No registered users and 170 guests