We have a scenario where 1 job (out of about 10 scheduled) will hang for 40 minutes at "Processing <VM name>". Then after 40 minutes, the job starts working - snapshot, perform backup, remove snapshot, indexing all processes in about 4 minutes. This happens every single time. Other jobs on the same Veeam server backing up the same infrastructure do not do this. During the time we have been working on this issue, we have upgraded Veeam, vCenter and ESXi to the latest versions, so we are fully patched. There are no tasks running against the VM or host during this pause.
Veeam support case: 2080676
The one "quirk" of the VM being backed up is that it has 16 VMDKs attached to it. While I don't think this is good practice, I haven't seen anything anywhere to suggest this is the issue. We are using NBD for transport anyway. Trawling the Veeam logs, the only thing I can see happening during the pause is this (in the Agent log):
[16.03.2017 13:11:05] < 2756> cli| Number of sessions: 5. Interval: 299 sec.
[16.03.2017 13:16:05] < 2756> cli| Number of sessions: 5. Interval: 599 sec.
[16.03.2017 13:21:05] < 2756> cli| Number of sessions: 5. Interval: 899 sec.
[16.03.2017 13:25:59] < 8560> cli| WARN|MTA invoke thread : timed out for backup events. Wait cycle will be resumed for '336' hours.
[16.03.2017 13:26:05] < 2756> cli| Number of sessions: 5. Interval: 1199 sec.
[16.03.2017 13:31:05] < 2756> cli| Number of sessions: 5. Interval: 1499 sec.
[16.03.2017 13:36:05] < 2756> cli| Number of sessions: 5. Interval: 1799 sec.
[16.03.2017 13:41:05] < 2756> cli| Number of sessions: 5. Interval: 2099 sec.
[16.03.2017 13:46:05] < 2756> cli| Number of sessions: 5. Interval: 2399 sec.
[16.03.2017 13:47:10] < 8372> srv| retrieved command: 127 (Invoke(127))
2399 seconds is 40 minutes, so this tallies but I don't know what it means or what it is counting against. There are no other jobs running (apart from a VM copy job which is currently waiting for its next window). The only thing Veeam support has come up with so far is they suggested that the Veeam proxy couldn't communicate with the VM over RPC but we tested and found that not to be the case. If this were the case then wouldn't the job just fail anyway?