parallel vDisk processing not using all possible tasks

[pirx]

Hi,

I've enabled parallel processing and I see that a lot of VMs are processed in parallel. What I'm missing is that disks are backed up in parallel. Or I'm only seeing VMs where it does not happen. Currently there is a copy job running that has to copy ~50TB of backups. 99% is done and only one VM is missing, this VM is 5,4 TB big and has 10 vDISKs. The backup for this VM is now running for 17 hours and it's copying one disk after the other, so this job takes ages. There are plenty of resources free. SOBR extends are set to 20 parallel tasks. I'm not sure why this job does not use more streams for the disks? Are the resources only checked and assigned at beginning of a job? There are a couple of copy jobs that have the same problem, they are at 98-99% for a very long time because the remaining VM/disks are using only one task for a single disk.

Code: Select all

[19.02.2017 18:00:40] <104> Info     *******************************************************************************************************************
[19.02.2017 18:00:40] <104> Info     |--------------------------------------------------------------------------------------------------|
[19.02.2017 18:00:40] <104> Info     | Resource name                  | Usage | Id                                                      |
[19.02.2017 18:00:40] <104> Info     |----------------------------------------------------------------------------------- Repositories -|
[19.02.2017 18:00:40] <104> Info     | DE-WOP-B01-E01                 | 1     | {2d0ffff7-3c6b-4927-b71b-a288a3c42799}                  |
[19.02.2017 18:00:40] <104> Info     | DE-WBE-C02-E02                 | 1     | {7abc6422-27a5-4a6f-8573-0d844569efcb}                  |
[19.02.2017 18:00:40] <104> Info     | DE-WOP-C02-E01                 | 1     | {1116a408-542e-41ad-8c80-a8e2da5f9ed1}                  |
[19.02.2017 18:00:40] <104> Info     | DE-WBE-B01-E02                 | 1     | {cb55554f-ce18-4e2d-9d8a-e7d75c845350}                  |
[19.02.2017 18:00:40] <104> Info     |--------------------------------------------------------------------------------------------------|

[blithespirit]

Hey Pirx,

What is the hypervisor you using and where do these VMs reside, are they RDMs, in one datastore or split, what version and update are you using?
How many jobs are you running or how many in that job, backup job or backup copy job, how many proxy serves are you running and what's the resource on the proxies. Your SoBR repositories, what resources are assigned to each repo?

That's a start and let's see where we go from there

[pirx]

VMware ESXi 6.0, Veeam 9.5U1, no RDM's , VMDK's sometimes spread over different DS, 4 copy jobs with 1100 VM's, 4 physical proxies each with 20 cores and 20 parallel tasks, 2 source SOBR's with 2 extends, each extend with 30 parallel tasks and 50TB, destination SOBR with 5 extends and 100 TB, each one with 20 parallel tasks.

[blithespirit]

Hmm, your resources seem good, are you doing per vm or backup chain to the target repos?

The original backup, does this backup in parallel?

[pirx]

Per VM chain. Can I see parallel processing somewhere in the Logs? Backup jobs usually run during night so I can not watch the progrress. The last VM in the job with 5,4 TB and the large number of disks was not copy during the weekend interval. After 49 hours the job ended while HD9 was still copied at 63%. No disks where copied in parallel during the last couple of hours.

[Delo123]

multiple vDisks in single VM's are always copied seqentially in copy jobs (as in replicas i believe), i don't think this was ever possible. Parallel processing is "just" for multiple vm's (but would be very nice of course, especially with restores!)