More efficient use of backup proxies

VMware specific discussions

More efficient use of backup proxies

Veeam Logoby hoFFy » Wed Sep 16, 2015 6:22 am

Hello,

maybe the feature is already there, but I couldn't find it or it may be a new feature:

We have at one of our customers 2 new hosts and one old host.
All VMs run on the two new hosts with shared storage, but there is a LTO-drive connected to the old host and it has a some disks in it which aren't used any longer.... so I'm using the old host as a backup proxy (6 cores), as a remote target and as a remote tape server.
On one of the new hosts I have a VM with 2 vCPUs running the VBR server acting as a normal VBR-server with all roles (backup proxy, etc.).

Now the question: Veeam allocates the VMs in the task to the available backup proxies in a static way. Let's say we have 10 VMs, 6 of them are allocated to the "VMware Backup Proxy" -> the VBR-server itself and 4 are allocated to the Backup Proxy on the old host (wich uses NBD, because it has no access to the shared storage).... what if the VMs assigned to the VMware Backup Proxy are not as fast finished as the four ones assigned to the other backup proxy? Are they being switched to the other, idle, backup proxy, or do they stay in queue even if there are free ressources?

Another thing: At this customer we installed VBR on the fileserver VM (very bad, I know), and the configuration database is saved in the SQL-Server VM (also bad...), this leads to messages like "VM is a baxkup server - no parallel task" (1:20 hours), very long wait times (this VM was waiting for about 5 hours with the message "Required backup infrastructure resources have been assigned) and "Waiting for completion of other tasks (VM is SQL server...)" (44 minutes).
Is there a way to change this behaviour, like backup up these VMs with the other backup proxy?
Or do I really have to install VBR on another (best practise would be a standalone VM..) VM, also hosting the config database itself?
VMCE 7 / 8 / 9, VCP-DC 5 / 5.5 / 6, MCITP:SA
Blog: machinewithoutbrain.de
hoFFy
Veeam ProPartner
 
Posts: 161
Liked: 25 times
Joined: Fri Apr 27, 2012 1:10 pm
Location: Germany / Lohne
Full Name: Sebastian Hoffmann

Re: More efficient use of backup proxies

Veeam Logoby foggy » Wed Sep 16, 2015 2:46 pm

hoFFy wrote:Now the question: Veeam allocates the VMs in the task to the available backup proxies in a static way. Let's say we have 10 VMs, 6 of them are allocated to the "VMware Backup Proxy" -> the VBR-server itself and 4 are allocated to the Backup Proxy on the old host (wich uses NBD, because it has no access to the shared storage)....

Sebastian, what are the concurrent tasks limits on the proxy servers?

hoFFy wrote: what if the VMs assigned to the VMware Backup Proxy are not as fast finished as the four ones assigned to the other backup proxy? Are they being switched to the other, idle, backup proxy, or do they stay in queue even if there are free ressources?

Task can be assigned to a proxy server only if there is a free slot available there and processing of the task starts immediately. So situation you're describing is not possible (task waiting in a queue after being assigned). Once being assigned, tasks never switch proxy servers.

hoFFy wrote:Another thing: At this customer we installed VBR on the fileserver VM (very bad, I know), and the configuration database is saved in the SQL-Server VM (also bad...), this leads to messages like "VM is a baxkup server - no parallel task" (1:20 hours), very long wait times (this VM was waiting for about 5 hours with the message "Required backup infrastructure resources have been assigned) and "Waiting for completion of other tasks (VM is SQL server...)" (44 minutes).
Is there a way to change this behaviour, like backup up these VMs with the other backup proxy?
Or do I really have to install VBR on another (best practise would be a standalone VM..) VM, also hosting the config database itself?

Tasks related to Veeam B&R server itself or the server holding its database are placed at the bottom of the queue and wait for the resources assignment until all other job tasks are completed. So moving Veeam B&R to another VM looks right in this situation.
foggy
Veeam Software
 
Posts: 14765
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: More efficient use of backup proxies

Veeam Logoby hoFFy » Fri Oct 02, 2015 2:16 pm

foggy wrote:Task can be assigned to a proxy server only if there is a free slot available there and processing of the task starts immediately. So situation you're describing is not possible (task waiting in a queue after being assigned). Once being assigned, tasks never switch proxy servers.

That's exactly what I meant:

20 VMs are assigned to Proxy1 and 20 VMs are assigned to Proxy2.

The 20 VMs from proxy1 are earlier finished than the VMs assigned to proxy2. Then proxy1 becomes idle and proxy2 has some VMs waiting for the backup proxy assigned to them to become available... the optimization in my opinion would be if these VMs were reassigned to the idle proxy1 instead of waiting for free ressources on proxy2.
VMCE 7 / 8 / 9, VCP-DC 5 / 5.5 / 6, MCITP:SA
Blog: machinewithoutbrain.de
hoFFy
Veeam ProPartner
 
Posts: 161
Liked: 25 times
Joined: Fri Apr 27, 2012 1:10 pm
Location: Germany / Lohne
Full Name: Sebastian Hoffmann

Re: More efficient use of backup proxies

Veeam Logoby foggy » Fri Oct 02, 2015 3:16 pm

hoFFy wrote:Then proxy1 becomes idle and proxy2 has some VMs waiting for the backup proxy assigned to them to become available...

Again, this is possible only in case the task cannot be processed in parallel with other tasks (i.e. is related to Veeam B&R server itself or the server holding its database) and waits unassigned for other tasks in the job to be completed. So it simply cannot be started earlier, whatever proxy it will be assigned to.
foggy
Veeam Software
 
Posts: 14765
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: More efficient use of backup proxies

Veeam Logoby tsightler » Fri Oct 02, 2015 3:33 pm 5 people like this post

hoFFy wrote:That's exactly what I meant:

20 VMs are assigned to Proxy1 and 20 VMs are assigned to Proxy2.

The 20 VMs from proxy1 are earlier finished than the VMs assigned to proxy2. Then proxy1 becomes idle and proxy2 has some VMs waiting for the backup proxy assigned to them to become available... the optimization in my opinion would be if these VMs were reassigned to the idle proxy1 instead of waiting for free ressources on proxy2.

I think what foggy is trying to say is that what you describe is not how Veeam proxy assignment works. We don't start a job and immediately assign 20VMs to one proxy and 20 VMs to the other.

To use use your example, if a job had 40 VMs to backup and 2 proxies with 4 tasks each (total of 8 parallel task), then, when the job started, Veeam would find all 40 VMs, determine which VMs are eligible to be assign to which proxies, and then assign the first 8 VMs (technically first 8 VM disks) to the available proxy slots. The other 32 VMs would be in "waiting for resources" mode, waiting for task slots to complete. As soon as a proxy finishes a task and that slot becomes available, Veeam evaluates the remaining VMs that are in "waiting for resources" mode and determines if the new proxy slot can service any of them, if so, it assigns that task to the slot and moves on. It's a totally dynamic assignment. If the first proxy gets assigned a large VM with 4 disk at the very start of the job, and that VM takes many hours, the second proxy will happily process all 39 of the remaining VM tasks.

However, there are a lot of factors that go into the proxy selection algorithm that may come into play. For example, in your environment, you have noted that you have two proxies, it appears one is virtual and the other is physical, but can only use NBD mode. NBD mode is considered a "non-preferred" mode, in that, if a VM has a proxy that can back it up via hotadd, Veeam will wait for that proxy to become available rather than assign it to the NBD proxy. There are good reasons for this in general, although they might not apply to your specific case, but this is most likely why VMs don't appear to be assigned "equally" in your case.

Ideally if you could attach your physical proxy directly to the storage to enable direct SAN mode then you would likely see very even assignment of proxy tasks. Alternatively you could force NBD mode on the virtual proxy as well, at which point Veeam would consider them equally "non-preferred" but since there aren't any other options would still spread the tasks.

I hope this explanation makes some sense. If you can share a few more specifics about your setup we can probably explain exactly what Veeam is doing with proxy assignment and why.
tsightler
Veeam Software
 
Posts: 4775
Liked: 1740 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: More efficient use of backup proxies

Veeam Logoby hoFFy » Mon Oct 05, 2015 10:30 am

Thank you Tom,

great explanation!
VMCE 7 / 8 / 9, VCP-DC 5 / 5.5 / 6, MCITP:SA
Blog: machinewithoutbrain.de
hoFFy
Veeam ProPartner
 
Posts: 161
Liked: 25 times
Joined: Fri Apr 27, 2012 1:10 pm
Location: Germany / Lohne
Full Name: Sebastian Hoffmann


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 15 guests