Running multiple vm backups inside 1 job

[abiltcliffe]

Hello,
I am currently using vRanger and I am evaluating Veeam as a replacement, but although Veeam seems to be quicker on single vm's when you batch run them with more than 1 vm in the job it processes them sequentially. This makes it very slow overall.....
What vRanger offered was to set how many concurrent vms could process simultaneously and how many per Host. So we ran 8 vms simultaneously with no more than 2 per host (we have 4) Our backups would then take around 2-3 hours.

If there is a way of setting this it would help as at the moment I have had to group vms into folders, and so have 5 running at the same time, but the maintenance and job creation is a bit of a pain.....

Many Thanks.

[Gostev]

Hello Anthony, we don't allow this functionality currently. Parallel processing is not recommended because it causes issues in larger environments, especially around concurrent snapshot operations on the same LUN (causing snapshot/file operations timeouts, SCSI reservation timeouts etc. - all leading to backup failures). This is explained in more details in the following VMware document (see page 6). So you may get a little faster speed at cost of much worse backup reliability.

Yes, as you mentioned you can indeed setup multiple jobs and run them in parallel (best is to setup your jobs in a way to ensure that they are not accessing the same LUN simultaneously). In most cases, 2 jobs is enough to fully saturate your Veeam Backup console network and target backup storage I/O capacity. And if you are using VCB SAN on FC4, due to the speed this backup mode provides target backup storage disk speed becomes your bottleneck even with a single job, so running parallel jobs does not typically bring you any speed improvements here. Hope this helps! Thank you.

[abiltcliffe]

Thanks for the info - I do have one question...

As one file is created that can contain lots of VM's as opposed to 1 file per VM - What happens if vm's move from one container to another?

Eg - I have a backup set for 1 of our 4 hosts and it contains 4 vm's
If we vMotion a vm from that host to another what will happen with the next backup? ie what wll happen to the files and their sizes on disk as one file in theory has one less vm data in in and the other has one more vm data in it? Does it all get cleaned up?

Thanks.

[Gostev]

Anthony, I am sorry for spotting your question so late. If VM falls out of scope of the job, it will not be removed from backup file automatically (what if it is gone because it got deleted unintentionally - and you would want to restore it)? If you use VMotion, you would want to avoid creating jobs on host level though. Instead, consider creating them on cluster level or above, or using VM folders (Templates and Folders view), or by individually picking VMs you need. Thank you.

[toreagan]

Hi Gostev, I have a question regarding this statement "Parallel processing is not recommended because it causes issues in larger environments, especially around concurrent snapshot operations on the same LUN (causing snapshot/file operations timeouts, SCSI reservation timeouts etc. - all leading to backup failures). "

We currently have 3 ESX Hosts - each using direct storage - no SAN. We have VEEAM on a seperate physical server and using direct storage to store the backups. In this scenario, are there still issues with running multiple jobs in parallel? We typically have 2 or 3 jobs running at once to increase the efficiency of the jobs and have not had any issue yet (that I am aware of). Are we at risk for inconsistent backups or are there any other issues we should be concerned with by running parallel jobs?

Thanks,
~Trent

[Gostev]

Hello Trent, there are no issues with this scenario, since each ESX hosts uses its own storage. So, you can setup 3 jobs (1 per ESX server) and run them in parallel without any risk. The problems only start to appear when you are running parallel backup of VMs located on the same shared storage. This is what my above post referred to.