Host-based backup of VMware vSphere VMs.
Post Reply
hoFFy
Service Provider
Posts: 183
Liked: 40 times
Joined: Apr 27, 2012 1:10 pm
Full Name: Sebastian Hoffmann
Location: Germany / Lohne
Contact:

More efficient use of backup proxies

Post by hoFFy »

Hello,

maybe the feature is already there, but I couldn't find it or it may be a new feature:

We have at one of our customers 2 new hosts and one old host.
All VMs run on the two new hosts with shared storage, but there is a LTO-drive connected to the old host and it has a some disks in it which aren't used any longer.... so I'm using the old host as a backup proxy (6 cores), as a remote target and as a remote tape server.
On one of the new hosts I have a VM with 2 vCPUs running the VBR server acting as a normal VBR-server with all roles (backup proxy, etc.).

Now the question: Veeam allocates the VMs in the task to the available backup proxies in a static way. Let's say we have 10 VMs, 6 of them are allocated to the "VMware Backup Proxy" -> the VBR-server itself and 4 are allocated to the Backup Proxy on the old host (wich uses NBD, because it has no access to the shared storage).... what if the VMs assigned to the VMware Backup Proxy are not as fast finished as the four ones assigned to the other backup proxy? Are they being switched to the other, idle, backup proxy, or do they stay in queue even if there are free ressources?

Another thing: At this customer we installed VBR on the fileserver VM (very bad, I know), and the configuration database is saved in the SQL-Server VM (also bad...), this leads to messages like "VM is a baxkup server - no parallel task" (1:20 hours), very long wait times (this VM was waiting for about 5 hours with the message "Required backup infrastructure resources have been assigned) and "Waiting for completion of other tasks (VM is SQL server...)" (44 minutes).
Is there a way to change this behaviour, like backup up these VMs with the other backup proxy?
Or do I really have to install VBR on another (best practise would be a standalone VM..) VM, also hosting the config database itself?
VMCE 7 / 8 / 9, VCP-DC 5 / 5.5 / 6, MCITP:SA
Blog: machinewithoutbrain.de
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: More efficient use of backup proxies

Post by foggy »

hoFFy wrote:Now the question: Veeam allocates the VMs in the task to the available backup proxies in a static way. Let's say we have 10 VMs, 6 of them are allocated to the "VMware Backup Proxy" -> the VBR-server itself and 4 are allocated to the Backup Proxy on the old host (wich uses NBD, because it has no access to the shared storage)....
Sebastian, what are the concurrent tasks limits on the proxy servers?
hoFFy wrote: what if the VMs assigned to the VMware Backup Proxy are not as fast finished as the four ones assigned to the other backup proxy? Are they being switched to the other, idle, backup proxy, or do they stay in queue even if there are free ressources?
Task can be assigned to a proxy server only if there is a free slot available there and processing of the task starts immediately. So situation you're describing is not possible (task waiting in a queue after being assigned). Once being assigned, tasks never switch proxy servers.
hoFFy wrote:Another thing: At this customer we installed VBR on the fileserver VM (very bad, I know), and the configuration database is saved in the SQL-Server VM (also bad...), this leads to messages like "VM is a baxkup server - no parallel task" (1:20 hours), very long wait times (this VM was waiting for about 5 hours with the message "Required backup infrastructure resources have been assigned) and "Waiting for completion of other tasks (VM is SQL server...)" (44 minutes).
Is there a way to change this behaviour, like backup up these VMs with the other backup proxy?
Or do I really have to install VBR on another (best practise would be a standalone VM..) VM, also hosting the config database itself?
Tasks related to Veeam B&R server itself or the server holding its database are placed at the bottom of the queue and wait for the resources assignment until all other job tasks are completed. So moving Veeam B&R to another VM looks right in this situation.
hoFFy
Service Provider
Posts: 183
Liked: 40 times
Joined: Apr 27, 2012 1:10 pm
Full Name: Sebastian Hoffmann
Location: Germany / Lohne
Contact:

Re: More efficient use of backup proxies

Post by hoFFy »

foggy wrote:Task can be assigned to a proxy server only if there is a free slot available there and processing of the task starts immediately. So situation you're describing is not possible (task waiting in a queue after being assigned). Once being assigned, tasks never switch proxy servers.
That's exactly what I meant:

20 VMs are assigned to Proxy1 and 20 VMs are assigned to Proxy2.

The 20 VMs from proxy1 are earlier finished than the VMs assigned to proxy2. Then proxy1 becomes idle and proxy2 has some VMs waiting for the backup proxy assigned to them to become available... the optimization in my opinion would be if these VMs were reassigned to the idle proxy1 instead of waiting for free ressources on proxy2.
VMCE 7 / 8 / 9, VCP-DC 5 / 5.5 / 6, MCITP:SA
Blog: machinewithoutbrain.de
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: More efficient use of backup proxies

Post by foggy »

hoFFy wrote:Then proxy1 becomes idle and proxy2 has some VMs waiting for the backup proxy assigned to them to become available...
Again, this is possible only in case the task cannot be processed in parallel with other tasks (i.e. is related to Veeam B&R server itself or the server holding its database) and waits unassigned for other tasks in the job to be completed. So it simply cannot be started earlier, whatever proxy it will be assigned to.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: More efficient use of backup proxies

Post by tsightler » 5 people like this post

hoFFy wrote:That's exactly what I meant:

20 VMs are assigned to Proxy1 and 20 VMs are assigned to Proxy2.

The 20 VMs from proxy1 are earlier finished than the VMs assigned to proxy2. Then proxy1 becomes idle and proxy2 has some VMs waiting for the backup proxy assigned to them to become available... the optimization in my opinion would be if these VMs were reassigned to the idle proxy1 instead of waiting for free ressources on proxy2.
I think what foggy is trying to say is that what you describe is not how Veeam proxy assignment works. We don't start a job and immediately assign 20VMs to one proxy and 20 VMs to the other.

To use use your example, if a job had 40 VMs to backup and 2 proxies with 4 tasks each (total of 8 parallel task), then, when the job started, Veeam would find all 40 VMs, determine which VMs are eligible to be assign to which proxies, and then assign the first 8 VMs (technically first 8 VM disks) to the available proxy slots. The other 32 VMs would be in "waiting for resources" mode, waiting for task slots to complete. As soon as a proxy finishes a task and that slot becomes available, Veeam evaluates the remaining VMs that are in "waiting for resources" mode and determines if the new proxy slot can service any of them, if so, it assigns that task to the slot and moves on. It's a totally dynamic assignment. If the first proxy gets assigned a large VM with 4 disk at the very start of the job, and that VM takes many hours, the second proxy will happily process all 39 of the remaining VM tasks.

However, there are a lot of factors that go into the proxy selection algorithm that may come into play. For example, in your environment, you have noted that you have two proxies, it appears one is virtual and the other is physical, but can only use NBD mode. NBD mode is considered a "non-preferred" mode, in that, if a VM has a proxy that can back it up via hotadd, Veeam will wait for that proxy to become available rather than assign it to the NBD proxy. There are good reasons for this in general, although they might not apply to your specific case, but this is most likely why VMs don't appear to be assigned "equally" in your case.

Ideally if you could attach your physical proxy directly to the storage to enable direct SAN mode then you would likely see very even assignment of proxy tasks. Alternatively you could force NBD mode on the virtual proxy as well, at which point Veeam would consider them equally "non-preferred" but since there aren't any other options would still spread the tasks.

I hope this explanation makes some sense. If you can share a few more specifics about your setup we can probably explain exactly what Veeam is doing with proxy assignment and why.
hoFFy
Service Provider
Posts: 183
Liked: 40 times
Joined: Apr 27, 2012 1:10 pm
Full Name: Sebastian Hoffmann
Location: Germany / Lohne
Contact:

Re: More efficient use of backup proxies

Post by hoFFy »

Thank you Tom,

great explanation!
VMCE 7 / 8 / 9, VCP-DC 5 / 5.5 / 6, MCITP:SA
Blog: machinewithoutbrain.de
Post Reply

Who is online

Users browsing this forum: No registered users and 84 guests