Replications Failing-host failed to process [sendSignature]

[emcdonald@arcbus.com]

I thought I'd finally post my comments and issues encountered here, after upgrading to v6 in early December and experiencing nothing but issues with replication. My current support ID is 5160665, which I've been working on for over a month.

Current environment:

2 ESX 4.1 hosts in main datacenter
1 ESX 5i host in remote datacenter
Connected via VPN 5Mbps upload to remote site - network is very stable, 100% uptime

Replication jobs are consistently failing. These happen at different times on the day, on different vms, there is no consistency to a specific vm, host, datastore that is pointing to the exact cause. Error messages received for every failure are the same:

Code: Select all

- Error: Client error: An existing connection was forcibly closed by the remote host Failed to process [sendSignature].
-- Error: Client error: An existing connection was forcibly closed by the remote host Failed to replicate content of the disk [vddk://<vddkConnSpec><viConn name="arc-edm-veeam01.corp.arcbus.com" authdPort="443" vicPort="443" /><vmxPath vmRef="vm-16" datacenterRef="datacenter-2" datacenterInventoryPath="ARC" snapshotRef="snapshot-520" datastoreName="arc-edm-vmhost01:storage2" path="arc-edm-dc01/arc-edm-dc01.vmx" /><vmdkPath datastoreName="arc-edm-vmhost01:storage2" path="arc-edm-dc01/arc-edm-dc01.vmdk" /><tran

All hosts are updated to latest patch levels.
I have two proxy servers at each site.
I have rebuilt my vCenter server, Veeam Server, Veeam proxies (all twice). I have deleted the remote datastores that were upgraded from an ESX 4.1 to ESX 5i environment, recreated as pure VMFS5 datastores.
I have tried replicating bypassing the vCenter Server, still jobs fail.
I have sent in the logs to Veeam support on numerous occasion.
I have tried disabling CBT on all vms (source and remote), no effect.
I have recreated all jobs on numerous occasions, enabled seeding, mapped to existing vms, all with no consistent success after initial replication. All initial seeds work, but subsequent replications inconsistently succeed/fail.
Testing replications of small jobs - 10 MB in size appeared to consistently replicate with no errors.
I have patched Veeam servers with patch provided by support.

I need to resolve this issue very soon as this puts us into a risk state for DR failover. If anyone has any suggestions or input, I would really appreciate your help!

Thank you.

[tsightler]

Are you using hotadd on the target? If so, I would suggest setting the target proxies to network mode and seeing if that helps. Also, are you explicitly selecting your proxies for the jobs?

[emcdonald@arcbus.com]

Explicit proxies are set. We (with Veeam Support) just set the target proxies to network mode, changed from automatic. I will know within the next 24 hours if this has made a difference.

[emcdonald@arcbus.com]

Setting to network mode resolved the issue. After 3.5 weeks I can now gladly say replications are working once more. Thank you for your assistance.

[jyarborough]

Is the network mode fix still allowing for successful/consistent replication for you? I am seeing the same problem and hope this is the answer.

[tsightler]

It should work. This is a VMware issue with hotadd and how CBT is handled with a snapshot chain which causes the proxy to become unresponsive for several seconds, interrupting the Veeam servers connection with the proxy. This causes various failure messages, but the [sendSignature] error is common. If you're using dedicated proxies on the target side disabling CBT on the proxy VM should also work.

[Threonine]

I'm also getting this error at one customer site with a new replication job. We re-seeded the job from scratch while troubleshooting this... the initial seed completed 100% successfully and then the next run of the job came back with the same result:

Error: Client error: An existing connection was forcibly closed by the remote host
Failed to process [sendSignature].

We've already set both proxies explicitly to Network mode... and we're still seeing this issue. Never had any issues until upgrading to version 6 and adding a source proxy... Time to open a new support case.

[Threonine]

Update on troubleshooting this... After restarting the management agents on the source server last week we had several successful replications. Unfortunately the last replication attempt failed again with the same error:
Error: Client error: An existing connection was forcibly closed by the remote host Failed to process [sendSignature].

We're going to try restarting the management agents again and possibly rebooting the host.

[Andresllorca]

Hi everyone,
Now I've got the same issue.
I'm working trough a VPN too between 2 ESXi. I'm trying to replicate a W2003 server, all the proxies are set and the infraestructure I think is ok too. I related this issue to Veeam suport too, but I don't have the Support ID now.
This Replication job was running correctly since a few days ago when we move the source system to another location. The VPN runs ok, I do a ping and the target answers.
I changed the transport mode to network in target proxie, and on the source proxie I continue with the automatic selection (it changes to network mode alone).
Can someone help me about this??? If you need additional information ask me about it.

Thanks in advance.

[Andresllorca]

oh sorry, the ESXi version are 5.1, and the latest one on Veeam B&R

[foggy]

oh sorry, the ESXi version are 5.1, and the latest one on Veeam B&R

Please be noted, that the current version of Veeam B&R does not support vSphere 5.1.

[Andresllorca]

Sorry, but yes, over a LAN connection it works. I do some tests over a LAN connection and I've got the logs of this REPLICATION jobs. I try to restore the replicated VM, and runs without problems. In some post of many peopple, they talk about the backup jobs and they talk about an API of the ESXi that makes this kind of jobs don't run, but they don't say anything about the replication jobs....
I don't know what happens trough a WAN conecction.

[foggy]

Sorry, but yes, over a LAN connection it works.

Looks like environmental issue then. With all technical issues it's always better to contact our support team directly - but, as I already stated above, current version of Veeam B&R does not support vSphere 5.1 and I'm not sure whether such cases can be addressed before it is supported officially.