Copy job selecting vbk of reverse incremental chain

[timmi2704]

Hey forum.
I came across an issue and I ask myself whether I might have a failure in my line of thought or whether this is absolutely expected.

I have a backup job running in reverse incremental mode processing backups daily.
Additionally there is a backup copy job set up with an interval of 2 weeks using the above mentioned backup job as a source.
A single copy cycle of that copy job takes at least a few days.

As soon as a new copy interval starts, the copy job selects the most recent restore point as it's source. (Makes sense)
Unfortunately, that's a vbk which needs to be "transformed" as soon as a new daily backup starts. That's why the backup job locks the restore point for exclusive access which cancels the copy job. (Also makes sense somehow)

Is there an easy way to prevent this from happening?
Is it possible to not cancel the copy job but simply pause it during the transformation of the backup job chain?

Thanks for your input

[PTide]

Hi,

How long is your chain? Do you have periodic Fulls?

[timmi2704]

Hey Pavel!
The backup chain contains up to 20 restore points, once a month we have an active full being processed.

[PTide]

A few more questions: what is shown as a bottleneck during backup copy? How big is your .vbk? What kind of storage do you use? What do you use as a backup copy job repo?

[timmi2704]

The bottleneck according to the statistics is either source or target WAN but usually related to the network connection.
The vbk's size is usually something around 1.3TB. In this case, the repositories for source and target are Disk based storage devices registered as a local drive on Windows hosts with a proxy on site.

[veremin]

You can add a simple script as pre-job activity for a backup job. A script will check whether a backup copy job is performing something at the moment (not just sitting in the idle state). Based on the results, the script will allow or not the backup job to proceed. Thanks.

[timmi2704]

Hi Vladimir,
unfortunately, that would result in the backup job not being executed for a week during the runtime of the backup copy job.

Is it technically possible to either stream from a backup file currently being transformed or to pause a currently running copy job without the need to retransfer/recopy the vmdk which was being processed before the lock of the backup file?

[foggy]

Copy job should not re-transfer anything it has already transferred, only new blocks required to build the restore point corresponding to the VM's latest state.

Btw, what it the size of the data it transfers during each cycle comparing to the full backup file and what kind of link do you have?

[timmi2704]

That's true. The job does not re-transfer anything it transferred before. But it looks like it starts with the last processed vmdk and goes through all of the blocks again to see whether they have been transferred for this vmdk.

The link is a low bandwidth WAN connection. The incremental changes are differing with each cycle, but I'd say it's a bit below 100GB compressed.

[foggy]

That's true. The job does not re-transfer anything it transferred before. But it looks like it starts with the last processed vmdk and goes through all of the blocks again to see whether they have been transferred for this vmdk.

Right, it needs to calculate the differences.

The link is a low bandwidth WAN connection. The incremental changes are differing with each cycle, but I'd say it's a bit below 100GB compressed.

Seems the link is indeed extremely slow. What I can suggest is having another local backup job for the same data set running once in two weeks and using it as a source for the backup copy job, so that nothing prevents it from doing its job as long as it needs.

[timmi2704]

What I can suggest is having another local backup job for the same data set running once in two weeks and using it as a source for the backup copy job, so that nothing prevents it from doing its job as long as it needs.

Thanks foggy. I see two problems with that
1. That would need at least additional space for a .vbk and for two .vib's
2. I could imagine, that this would make the use of CBT a bit more difficult, wouldn't it?

[alanbolte]

No problem with CBT, you can have as many backup or replication jobs for the same VM as you want without causing any CBT problems. If you mean that running it less often means it has to take longer to create an increment, then yes, I suppose that's unavoidable.

Do you necessarily need to run the copy only every two weeks? If you ran it every day (and increased retention to match), it might, after a few sessions, get fast enough that you don't have to disable the backup job.

Have you tried running the copy job without WAN accelerators, and checking both bandwidth used and the difference in time for the copy? If the WAN accelerator isn't providing much of an improvement, you can open a support case - it's possible that the WAN accelerator isn't providing you with the performance you need, either because of your hardware, misconfiguration, or data that doesn't deduplicate well. With bottleneck tending to source or target WAN, my first guess would be that your cache isn't on sufficiently fast storage, or that your target-side repository is slow.