Alternative for backup of VMs with Ress. Pools (Tags maybe?)

[Lasr]

Hi,

at the moment we have some performance issues on our ESX-Farm (ESX 5.5 U2) because we have around 200 VMs running (550 VMs at all, but many are powered off). The performance problems are IMO caused by the setup of our ressource pools.

We use this pools in our cluster:

noBackup: 180 VMs (40 powered on)
Dev1: 70 VMs (20 powered on)
Dev2: 70 VMs (40 powered on)
Dev3: 70 VMs (30 powered on)
Dev4: 70 VMs (20 powered on)
Production: 50 VMs (50 powered on)

The ressource pools are primarily used to manage our backup. We created one job per pool, to split the VMs to multiple backup jobs. This is a good way for the backup, but screws our performance because of the usage of too much pools. I had the idea to use tags instead of those pools for our backup, but worry a bit, if this would work like with the pools too. Did anyone use tags already for the backup or has a better idea how we could manage our backup? I could put all the Dev-VMs in one pool for sure, but we had some other problems with such high amount of VMs in one job already. I'm a bit clueless at the moment, so I would really appreciate a tip, how we could put our backup to the next level


Thanks in advance

Lasr

[emachabert]

Why not simply using folders in the incentory ? And mapping the jobs to the folders.

Using ressource pool to classify VMs is just the worse thing you can do with VMWare. If you don't explicitly need ressource sharing management then do not use ressource pools. Even an unconfigured ressource pool will penalize the whole performance.
I often destroy ressource pools at customer location, and without adding any hardware I obtain 20 to 30% performance improvment by removing the resources sharing constraints inherent to the concept of pools.

[cupplesey]

Resource pools are for alocating resource to a particular type of server group or tasks, e.g. a SQL cluster to prevent it ballooning out and hogging lots of resources from the rest of the farm but ensuring it has the resouces it needs from other taks taking reources. They need to be provisioned correctly and can affect overall performance as emachbert has mentioned.

It sounds lke you have configured your infrastructure around your backups to the detrement of you production performance which interm could mean even slower backups. IMO you should use Veeam to pool the servers into back tasks by type rather than the other way around. I would make your production servers P1 interms of performance and backup time, the rest can suffer if need be as your core business is being impacted and ultimately losing time and money.

[veremin]

I'm along the same lines with Eric here. VMware recommends against using resource pools as VM grouping means, because their main idea is proper resource allocation.

If I were you, I'd probably stick to folders, indeed; seems the easiest option here.

Thanks.

[Lasr]

First I completly agree with you, that ressource pools are the wrong solution just to handle backups. They just exist from times as we haven't had veeam and still used VMWare Backup appliances.

I don't think that folders will solve our problem globally. For sure, its a better grouping than with pools. But we fighted hard here, to kick out ~200 Dev-VMs from our backup. If we would switch to folders, we have to backup them again or we have to make an exclusion list, that will not be very flexible.
I've discussed that with my mate right now and we will give tags a try. Our idea is to make similar groups of VMs, like in the ressource Pools just with Tags. We could reduce our pools to 3: noBackup, Dev, Production. To be honest, I've no idea what is the best solution here: minimal amount of ressource pools or completly no ressource pools. Because we have less production and many Dev-VMs we can't risk that if somebody makes anything stupid, he will screw our production VMs.


Thanks for your ideas ^^

[foggy]

Other customers do use tags to add VMs to backup jobs, you can search this forum for some of them mentioning it here, so worth giving a try.

[Lasr]

Thanks for the hint, I'll check this.

[dellock6]

Hi Lasr,
sorry for the shameless plug, but if you have the chance to come to VMworld this year, either US or EMEA, I will have a session exactly on vSphere + Veeam + Tags, maybe you can find some useful info.

For your scenario, I see two different problems:
- grouping VMs: I think both folders and vSphere tags are ok, tags are more flexible as one VM can have multiple tags, while it can belong to only one folder at a time. Also, by changing tags, a VM can be immediately moved to a different backup job that groups a different tag
- performances: Veeam has different performance control systems, regardless the used grouping method. Parallel processing uses different proxies to process VMs cuncurrently, and you can choose how many proxies you want to use. Then, you can leverage Backup IO control to limit the impact on storage latency created by Veeam jobs. There are many topics here on the forums regarding performance control.

[Lasr]

Hi Lasr,
sorry for the shameless plug, but if you have the chance to come to VMworld this year, either US or EMEA, I will have a session exactly on vSphere + Veeam + Tags, maybe you can find some useful info.

For sure I would like to join this event, but I think I would have to pay it by myself, because my boss is a noob and would never pay this for me.

- grouping VMs: I think both folders and vSphere tags are ok, tags are more flexible as one VM can have multiple tags, while it can belong to only one folder at a time. Also, by changing tags, a VM can be immediately moved to a different backup job that groups a different tag

We setup the tags on wednesday already and changed our jobs to work with it. Everything seems to be fine, we just have to write a short powershell script which will assign the tag, with the lowest amount of VMs assigned, to a VM without tag. Then we should be fine and don't have to move the VMs between the tags.

- performances: Veeam has different performance control systems, regardless the used grouping method. Parallel processing uses different proxies to process VMs cuncurrently, and you can choose how many proxies you want to use. Then, you can leverage Backup IO control to limit the impact on storage latency created by Veeam jobs. There are many topics here on the forums regarding performance control.

So what does this mean for our situation? Are tags worse in performance thans pools?


Wish you a nice and sunny friday

[dellock6]

The only thing to be careful when using tags is exactly to check if there are VMs with no assigned tags, and thus not covered by any backup job, but seems you already managed this. There are different ways to fix this, from automatic assignment of default tags, or you can use VBR Enterprise Manager or Veeam ONE to find out VMs that are not protected; or again, workflows in vRealize that find VMs with no tags.

For VMs that do not need to be protected, you can create a "no backup" tag, so they do not create a warning for a missing tag, but are not going to be saved.

Finally, no, there's no difference for performances from resource pools compared to tags, both are going to group multiple VMs, so just check how many VMs are going to have the same tag and thus be protected by the same job.