Backup Copy sequence of operations causes interval to expire

[tfelice]

Running B&R v7 patch 3. I have an installation with seven backup copy jobs, each of which is set for a one-day copy interval. The WAN link operates at 10 mbps, and this is the best we can to at present. Six of these jobs have only one VM each, and they generally work fine. The other job copies five VMs, and it often fails due to expiration of the copy interval, so the last couple of VMs in that job aren't getting backed up. I'm using WAN acceleration, with the WAN accelerators set to use 8 data streams. The target repository is set to allow 2 concurrent jobs and to limit data ingestion to 2 MB/s.

The reason for having several one-VM jobs is that as VMs were added, I wanted to avoid having to re-seed the entire remote backup repository; it seemed a lot easier to just seed the new machine in a new job (or in the case of a very small VM, just run it without a seed).

For reasons I don't understand, the backup copy logic seems to be to copy a VM from one job, then to leave the rest of that job pending and go grab a VM from another job, and then another. So although there would have been plenty of time for the 5-VM job to run straight through, it can actually spend many hours just waiting between the processing of its VMs while the resources are used to copy VMs from other jobs. Any way to change this behavior? Anyone know any other way around my predicament?

[tsightler]

If possible, adding a second target WAN accelerator at the target site and using that only for the 5 VM job would probably do the trick (a target WAN accelerator can only process a single backup copy job at a time).

Other than that, do you have the interval start times configured correctly to coincide with the start times of the backup jobs?

Also, regarding your concern for new VMs being added, are these usually fresh VMs primarily with only an OS because, if so, the WAN accelerator already keeps some additional cache space just for this use case, i.e. it's expected that users will sometimes add new VMs so we have special handling for the OS portion of the VM that should make seeding not required for the most common use case.

[foggy]

It is actually the source WAN accelerator that can process a single task at a time only, so adding a second source WAN accelerator used specifically by the larger job will allow to run it independently from the others.

Currently, in case of starting several jobs simultaneously, tasks are assigned to available slots on the "first come, first serve" basis, but probably in future versions tasks belonging to the job that has started first will have priority over others.

[tsightler]

It is actually the source WAN accelerator that can process a single task at a time only, so adding a second source WAN accelerator used specifically by the larger job will allow to run it independently from the others.

Indeed. Sorry about that. Thanks for the correction!