Comprehensive data protection for all workloads
Post Reply
unluckynoob
Novice
Posts: 5
Liked: 1 time
Joined: Dec 15, 2014 10:49 am
Contact:

Client error: An existing connection was forcibly closed ...

Post by unluckynoob »

Error: Client error: An existing connection was forcibly closed by remote host.
Error: Client error: ChannelError: ConnectionReset

hi all,

any ideas/pointers on how to troubleshoot this. it happens for one particularly large guest on hyper-v cluster, all other backups are fine.

it happens every time the backup runs, but at seemingly random points through the process from a couple of minutes to several hours.

target is over a WAN/vpn connection, however VPN is stable and has been ruled out.

backup job has been recreated from scratch and have also tried preseeding, still happens....

struggling to find any relevant errors in any server eventlogs that might indicate whats going on.

any help would be greatly appreciated
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Client error: An existing connection was forcibly closed

Post by Gostev »

Hi, according to your description this issue looks to be outside B&R, and rather pure networking problem. Perhaps with the help of some networking professional and via network traffic capture/analysis (I doubt you will be able to get assistance with that via the forum posts).

May be worth opening a case with our support anyway, as they have seen it all by now, and may have some suggestions for you where to start.

Thanks!
cby
Expert
Posts: 109
Liked: 6 times
Joined: Feb 24, 2009 5:02 pm
Contact:

Re: Client error: An existing connection was forcibly closed

Post by cby » 2 people like this post

Hi

This error, along with "Error: Client error: End of file Unable to retrieve next block transmission command." are typical of backup/replication of large VMs over a WAN that may be throttled or the bandwidth reduced for some reason (QoS etc.). The 'invisible' devices and infrastructure (e.g CWDM vs DWDM) in the network chain can additionally hamper/interrupt data transfer. Also don't exclude proxy/ESX host/SAN performance at the target end.
unluckynoob
Novice
Posts: 5
Liked: 1 time
Joined: Dec 15, 2014 10:49 am
Contact:

Re: Client error: An existing connection was forcibly closed

Post by unluckynoob » 1 person likes this post

thanks for all of the suggestions.

I did some further digging on the forums and found a couple of pointers to the 'use multiple tcp/ip streams' tick box on the backup proxy.

I've unticked this and all of the issues have gone away, not a sniff of a problem. there's also no noticeable difference to backup times so not really sure what the benefit was...

anyway, for me this is problem solved.
cby
Expert
Posts: 109
Liked: 6 times
Joined: Feb 24, 2009 5:02 pm
Contact:

Re: Client error: An existing connection was forcibly closed

Post by cby »

Good find on the proxy. I'll bear that one in mind. Thanks.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Client error: An existing connection was forcibly closed

Post by Gostev »

I've encountered both VPN servers and networking hardware that started to drop connections after reaching certain number of concurrent TCP/IP connection before, so this must have been the case here... strange though, as the default is just 5 concurrent connections, which is very low. This means other workloads can easily saturate the limit, and cause similar issues in future.
unluckynoob wrote:there's also no noticeable difference to backup times so not really sure what the benefit was
The benefit can usually be seen on high latency links, or when certain networking hardware throttles individual TCP/IP connections down, for example for QoS.
Post Reply

Who is online

Users browsing this forum: divertisity, rweis, ybarrap2003 and 132 guests