Occasional very slow replication.

[readie]

I have a replication job which runs every night taking about an hour and a half. Occasionally, it takes nearly a day . .
Figures - normal job listed first, then very slow job.

Duration 1:32:55/19:31:08
Processing rate MB/s 68/48
Processed TB 8.8/8.8
Read GB 64.5/53.8
Transferred GB 32.2 (2.8x)/18.8 (2.9x)
Bottleneck
Source 95%/95%
Proxy 10%/5%
Network 14%/14%
Target 71%/67%

I can see very little difference in the figures, apart from the Duration!!

[foggy]

Bob, have you checked the job session log more closely? Are there any tasks that took extremely long compared to the normal job run (e.g. snapshot removal)?

[readie]

Do you mean the B&R console details in the lower pane? I have studied this and can't see anything particularly odd - except that each VM seemed to take a lot longer than usual.
There are a lot of 'Hot add is not supported, failing back to network' . . . but I do see this quite frequently for some reason in a variety of jobs.
Or did you mean look at a more detailed log file somewhere?

[foggy]

Yes, I mean that session log, however, you can select the particular VM in the list to the left for its detailed processing log.

[alanbolte]

The main thing to look for there would be whether individual disks took much longer than normal, or if they processed in the same amount of time.

If you think you need a more detailed look at the problem, you should open a support case to submit the debug logs.

[readie]

We have diagnosed the problem! Very strange, when Veeam is trying to hotadd VHD to the Proxy <servername> the Vcentre console shows that it is actually trying to attach the VHD to <servername>_replica!! (Which of course is a waste of time - takes around 20 - 30 minutes for each VHD until it decides it cannot use hotadd and falls back to network mode.)
I have opened a support call 01764626 and am now waiting for some one to take a detailed look at it.

[veremin]

Can you elaborate on your issue a bit? By VHD you mean virtual hard disk, not the MS virtual disk format, right? And you issue is that a backup server tries to attach disks (hot-add) to proxy replica instead of attaching it to the proxy server itself? Thanks.

[alanbolte]

Why are you replicating a proxy in the first place?

[foggy]

We have diagnosed the problem! Very strange, when Veeam is trying to hotadd VHD to the Proxy <servername> the Vcentre console shows that it is actually trying to attach the VHD to <servername>_replica!!

QA were able to reproduce this behavior and this is planned to be fixed in the next update. Thanks for the heads up!

[readie]

Alan - not sure. Are you suggesting that relying on backup (rather than replica) would be OK?
Foggy - that is good to know. At least it is nothing quirky about our environment/infrastructure (except that we replicate our Proxies, which perhaps others don't).
I was delaying upgrading with U1 until this problem was sorted, but now I'm perhaps waiting for U2?? For the moment I will stop replicating our proxies, and delete from disk, then the job can't do the same again.