backup copy"waiting for another task to complete" what task?

[vmos]

Hello, we've got veeam 7 runing a bunch of jobs across two servers. Some of the jobs also have secondary destinations. There's a couple of them have this particular problem.

In these cases, the secondary destination is a linux server and the copy mode is "by job" and direct. It'll run along, do a few VMs and then sit there for ages, on all the pending VMs, the status is either "latest restore point is already copied" or "Waiting for another task to complete" but what task?

There are no other jobs running, the other veeam server may be doing it's thing, but it's not interacting with this one. The concurrency on the linux repository is 6 jobs although increasing it to 20 doesn't seem to make much difference.

Looking at the throughput graph, it seems to wait about ten minutes between VMs. How can I find out what this is waiting on and can I speed it up?

[Vitaliy S.]

Hello,

It'll run along, do a few VMs and then sit there for ages, on all the pending VMs, the status is either "latest restore point is already copied" or "Waiting for another task to complete" but what task?

These are two different messages, do you see them at the same time? Each task processes one virtual disk, so the the second message tells you that there are no slots available to continue processing your VMs. As to the latest restore is already copied, then you should see a restore point created on the target repository. Is this the case?

There are no other jobs running, the other veeam server may be doing it's thing, but it's not interacting with this one.

How many tasks do you have configured on the source repository?

Thank you!

[vmos]

I've got two tasks configured to use this repository, at the time I'm getting these messages, usually one of them is idle. To be sure I've gone and disabled the other task but it doesn't make much difference

[Vitaliy S.]

I've got two tasks configured to use this repository, at the time I'm getting these messages, usually one of them is idle.

Unfortunately, it is still not clear whether you have two jobs using this repository as a source or as a target. Since concurrent tasks limit doesn't make any difference I was trying to understand what is the limit for source repository, the one which is used by regular backup jobs.

[vmos]

On each windows veeam server I've got half a dozen jobs, on each of my source hosts there are one or two jobs, on each linux repo (target) there are two jobs.

BUT (big butt) in order to try and get to the root of this issue, I've disabled everything except this one job and it still sits there saying "waiting for task to complete"

So what's it waiting on if all the other tasks are disabled?

[Vitaliy S.]

Let's ask our support team for assistance with logs review, as it should be easy to find out what resource is busy and why.

[JWester]

Any news on this?
We have the same issue (case 00524953):
Currently only one backup copy job is running. It copies a VM (app. 30-45 secs), then takes a nap of 3-4 minutes, copies the next VM, nap, ...
No load on cpu, network, sql, disk...

[foggy]

I cannot check the OP's case since he haven't provided the ID here, so please continue working with support on this.

Btw, what are the limitations on the number of concurrent tasks on your repositories?

[JWester]

No limits on tasks and bandwidth.

[foggy]

I was talking about the maximum number of concurrent tasks setting for backup repository (available in the repository wizard). Do you mean everything is at its default values?

[JWester]

Edit Backup Repository / Repository / Load Control
-> "limit maximum concurrent tasks" is not checked

[foggy]

Got it. And there are no other jobs running involving the same repositories, right?

[JWester]

Got it. And there are no other jobs running involving the same repositories, right?

Correct.

[palw]

I've been having a similar issue to this. We've been testing our Veeam backup solution for the last couple of months. I've
decided on using reverse incremental to multiple windows server repositories for our initial backups but only want to keep 2
weeks and then keep a second copy with retentions offsite.

I have initially setup 3 backup copy jobs with different GFS retention periods depending on the requirements and placed
backup jobs in the copy job. However when each copy job runs it's only copying 1 VM at a time with the next VM Action of
"Waiting for another task to complete", I have about 200 VM's so this will take too long.

Is this the expected behaviour with backup copy jobs? Or should a single copy job be able to process multiple VM's at the
same time?

The offsite repository it set to handle 6 jobs concurrently, and all the proxies can handle at least 4 concurrent tasks. I
have 1Gb network connection to the offisite repository.

Thanks

[Gostev]

Yes, it is expected. What is the bottleneck in your case? Since Backup Copy does nothing but copy data blocks from one storage to another, processing multiple VMs in parallel will not have any impact in most cases (and especially so on 1Gb network connection that can easily be saturated by the single stream - and you effectively have 3 due to 3 backup copy jobs). Thanks!

[palw]

Hi Anton, Thanks for your response, if its just copying data then that makes sense.

Bottleneck is the Target (Source 5% > Proxy 11% > Network 4% > Target 99%) The Target is a dedup appliance, certainly not slow disk but not as fast as the initial backups.

It may have just been the initial seed that seemed to be taking forever, its running OK now, although nowhere near as fast as the backups. What is the best practice when setting up backup copy jobs? If I have 100 VM's that I would like a on a 6 month retention, should I have them all in 1 backup copy job or separate them?

[Vitaliy S.]

Hi Paul,

Yes, dedupe storage is not a good target for jobs using random I/O compared to sequential writes. Having target as a bottleneck, means that configuring additional backup copy jobs to transfer VM data will not increase the overall performance. Improving target storage performance will allow you running more backup copy jobs at a time.

Thanks!

[Wector]

Hi,
we also try to use a "backp copy Job" to send the backup to another location. beacause the Job works sequential there ist mutch time gone. please have a look: http://www.directupload.net/file/d/3842 ... xw_jpg.htm
This could be twice as fast when the copy Job uses parallel proccessing to.

regards