Discussions specific to the VMware vSphere hypervisor
Post Reply
Jim Mc
Influencer
Posts: 18
Liked: 2 times
Joined: Aug 13, 2014 8:49 am
Full Name: James McGuire
Contact:

Proxy task utilisation

Post by Jim Mc » Jan 30, 2015 1:02 pm

Hi there,

* If have a proxy with 4 vCPUs and configured to allow 4 tasks. If I have 'multiple TCP/IP connections per job' set to '2' does this mean I am essentially using '2' of the 'task' slots on the proxy? I.e. does this limit my total number of jobs via that proxy which can run concurrently to '2'?

* What actually happens when I enable multiple TCP/IP connections per job? How does Veeam use the individual streams to transmit data? E.g. is it one disk per connection?

* If I have a proxy with 4 vCPU and 4 tasks configured. And I only have two jobs running on that proxy, are only 2 vCPU being used or are the other vCPU being utilised? Would this be a waste of resources and I should maximise the concurrent number of jobs running on that proxy at any given moment, or would the jobs utilise any processing resources available?

Many thanks,

Jim

veremin
Product Manager
Posts: 16908
Liked: 1440 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Proxy task utilisation

Post by veremin » Jan 30, 2015 1:29 pm

Number of upload streams option isn't related to number of simultaneous tasks a proxy server can run.

Jim Mc
Influencer
Posts: 18
Liked: 2 times
Joined: Aug 13, 2014 8:49 am
Full Name: James McGuire
Contact:

Re: Proxy task utilisation

Post by Jim Mc » Jan 30, 2015 1:37 pm

Hi Eremin, thanks for that information, I note you've linked an article but it does not have any answers or additional information to the questions posed?

Dave338
Enthusiast
Posts: 36
Liked: 5 times
Joined: Jan 27, 2015 12:21 pm
Full Name: David
Contact:

Re: Proxy task utilisation

Post by Dave338 » Jan 30, 2015 8:00 pm

I only have used v8, and un my rig with only one concurrent job sometimes i ser high CPU usage on my 4 Cores. With 2 concurrents i have CPU a lot of time over 80-90 or full throtle (speeds around 200mb/s) so i think is not 1 job=1 core like people say un previos versions of vbr.

So review your CPU utilization while running two jobs, if the load is high you should se little to no benefit from going from 2 to 4 simultaneous jobs. It depends con your compression settings. I use optimal

Regards

foggy
Veeam Software
Posts: 18291
Liked: 1568 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Proxy task utilisation

Post by foggy » Jan 30, 2015 9:25 pm

James , provided the first two questions in your post are irrelevant, the link provided by Vladimir answers the next three of them. And the following link answers the rest of the questions. Feel free to ask for additional clarification, if something is not clear after reviewing those resources.

Jim Mc
Influencer
Posts: 18
Liked: 2 times
Joined: Aug 13, 2014 8:49 am
Full Name: James McGuire
Contact:

Re: Proxy task utilisation

Post by Jim Mc » Jan 31, 2015 1:24 pm

Hi Alexander,

Thanks for that link, actually it raises yet more questions:

"Task limits specified for backup proxies and backup repositories influence the job performance. For example, you add a VM with 4 disks to a job and target this job at the backup proxy that can process maximum 2 tasks concurrently. In this case, Veeam Backup & Replication will create 4 tasks (1 task per each VM disk) and start processing 2 tasks in parallel. The other 2 tasks will be pending."

What this essentially means is that I have no control over the scheduling of Jobs! And they will start only once the first Job is complete, as the first Job will consume all of the available task slots on the proxy! Unless of course the number of disks to be processed concurrently within a Job (and therefore the number of tasks generated by that job) can be limited. Which goes back to my original question on what exactly do the individual TCP/IP streams do? Perhaps e.g. one disk per stream? And sorry but I did not find any additional information in Vladamir's link which addressed any of my questions.

In essence what I'm getting at here is; How do I control how many task slots are consumed on a proxy by an individual Job so I can manage how many Jobs I have running concurrently.

This is important to me, because if I know exactly when a job will start, then I can start to predict when it will end, and this will allow me to schedule other tasks against the VM.

foggy
Veeam Software
Posts: 18291
Liked: 1568 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Proxy task utilisation

Post by foggy » Jan 31, 2015 10:19 pm

Jim Mc wrote:What this essentially means is that I have no control over the scheduling of Jobs! And they will start only once the first Job is complete, as the first Job will consume all of the available task slots on the proxy!
This also essentially means that each job is completed as fast as possible, since the job's tasks have priority in getting resources over the jobs that have started later. Isn't that what you're basically after?
Jim Mc wrote:In essence what I'm getting at here is; How do I control how many task slots are consumed on a proxy by an individual Job so I can manage how many Jobs I have running concurrently.
You can disable parallel processing, this will provide the number of concurrently running jobs on the proxy equals the max number of allowed concurrent tasks on it.
Jim Mc wrote:This is important to me, because if I know exactly when a job will start, then I can start to predict when it will end, and this will allow me to schedule other tasks against the VM.
What kind of tasks are you referring to? If those are performed by Veeam B&R, then you should not worry about overlapping, Veeam B&R handles this automatically. Basically, if you have defined the backup window, you shouldn't care about how jobs are processed within it, unless they run out of the window. All other tasks can be performed after the window is over.

dellock6
Veeam Software
Posts: 5734
Liked: 1626 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Proxy task utilisation

Post by dellock6 » Feb 02, 2015 2:37 pm

As a side note, the rule in parallel processing of 1task=1core has always been a suggested practice, not a mandatory one. You can push 2 tasks per core for example, and except for the warning sign in the proxy configuration, this will be the behaviour of the proxy. On some high-end CPUs I've seen customers doing this and, even if the result is not to have double the speed, it can increase even more the processing speed of a proxy.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Post Reply

Who is online

Users browsing this forum: No registered users and 34 guests