Upgraded from 6.1 to 6.5 - Error "Error: Client error: Chan"

[xadamz23]

Support case ID: 00170750

Two days ago I upgraded from 6.1 to 6.5 (and installed Patch 1). Now I have jobs randomly erroring with:

Error: Client error: ChannelError: ConnectionReset

Anyone else experiencing this? I didnt see anything in the forum.

[Vitaliy S.]

No, I haven't seen this error message either, looks like a connection issue to me. What does our support team say on this?

[xadamz23]

I just recently opened the case via the web so I have not heard back from support yet.

[ZachW]

Vitaliy is correct in this. This error is related to a connection being reset by the peer for whatever reason. This is most commonly related to an issue with the firewall(s) / IDS / IPS / integrity of the connection between the source and target. My recommendation would be to utilize something along the lines of IPerf or Wireshark to look into possible issues fitting up with the common causes of this. Aside from this you can work with the support engineer to assist in a better fashion.

[xadamz23]

It is not a firewall issue or integrity of the connection between things. Bottom line is I never got this error before I upgraded, upgraded to 6.5, now I get the error. Everytime I upgrade the product new issues arise and I'm always pointed in the direction of our san or our network. I dont mean to sound like a jerk, but nothing has changed in my environment. The absolutely only thing that has changed is Veeam was upgraded from 6.1 to 6.5.

[ZachW]

I understand, but I also hope that you can understand that you can see where I am coming from.. You have provided us with an error "ChannelError: ConnectionReset" which is quite commonly a network related error that is not specific to any certain software. I mean just from my side of things I don't necessarily see why B&R would be resetting its own connection and if that were the case I don't think that Veeam would have as many users utilizing 6.5 successfully at this current time if it were not an environmental issue. Either way if you could please keep us updated throughout the duration of the ticket that would be great!

[xadamz23]

Still no where with this. All Veeam support has had me do is mess with the throttling options. Again, Tuesday, on version 6.1, everything was fabulous. Been running months and months no problems. I upgrade the software and on Wednesday, the next set of job runs, I start getting the error. Have gotten it everyday since then. So the only thing that changed in my environment is the Veeam software. So of course support has been no help so far and is immediately pointing to my network. Just like my last upgrade from 5.x to 6.x I started getting errors and they wanted to blame my SAN. It is not my infrastructure! So frustrating.

[xadamz23]

So my next question is can I downgrade to 6.1? Veeam B&R is installed in a VM and I created a snapshot before the upgrade. Database is also in the snap. After I revert I'll have to remap the backup files to my jobs since they have changed. How do I downgrade my backup proxies? (Transport, vNFS, etc...)

Any other gotchas when downgrading?

[veremin]

In the way I see it, there should be no problems with the scenario you’ve described. However, I do believe that after performing downgrade, along with remapping backup files it’ll be necessary to re-add proxy servers and repositories.

Hope this helps.
Thanks.

[Gostev]

Judging on the error, downgrade will be waste of time.

[xadamz23]

Why do you say that Gostev? I can guarantee you 100% without a doubt that if I downgrade those errors I am now getting on a daily basis will go away. Saying it's a waste of time is not very helpful and not even accurate.

[xadamz23]

But you know what....arguing with support is a waste of time. Never gets me anywere. What happaned to the customer is always right? so frustrating.

[ZachW]

If you must go about downgrading then of course feel free to. But I would also appreciate if you could keep this thread updated with what the results of the downgrade were to provide us with more information.

[Ben Milligan]

I wanted to post the resolution to this particular problem in the case any other customers run into it. The error itself definitely lends to connectivity problems hence some of the time it took to fully diagnose the root cause in this instance.

The problem ended up being a snapshot that was created prior to the update being applied, and this additional disk contention caused enough latency to drop connections within the Veeam Backup guest. Consolidating/removing the snapshot allowed jobs to complete correctly again as they were in 6.1.

[Gostev]

Why do you say that Gostev? I can guarantee you 100% without a doubt that if I downgrade those errors I am now getting on a daily basis will go away. Saying it's a waste of time is not very helpful and not even accurate.

Yet, I was right.

What happaned to the customer is always right?

Does not apply in troubleshooting situations, because generally customers do not know the product well, and often times are wrong in their suppositions on why something does not work.

[snowmizer]

Anyone else get this and how did you resolve it? Like ZachW the only thing I did was upgrade from 6.1 to 6.5. I ran restores last week without issue. Today I try to run a FLR and I'm getting this message. If there was a network issue I'd definitely know about it because our users would be screaming.

[ZachW]

If you look further back through the thread it was actually not me who was running into this issue and it was an admin that I was assisting. Within his case it was not only an upgrade from 6.1 to 6.5 that changed, he had taken a snapshot of the Veeam box prior to the upgrade and had not removed it before attempting to perform certain functions within the software. I can say once again that I have never seen a case where this hasn't been related to some network related issue ( and might not be very apparent through other applications ), even if it comes down to a misconfiguration in jobs to best suit the network that Veeam is running in. My recommendation would be to open a support case up and we will take a look at it.