All jobs failing

[stevepasq]

Yesterday we needed to reboot our virtual center server and since then all veeam jobs are failing with the following error which points to our virtual center server.

Analyzing object "group1" (group-v120
346), host "server name
" The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. The
re are no objects to backup

I feel like this is probably an easy fix but I don't have much exposure to veeam. The guy who usually administers it is on vaca this week. I rebooted both veeam servers and restarted jobs and they are still failing. From what I can see all services are started.

Veeam version 3.1.1
virtual center server running vsphere client
ESX hosts are a mix between 3.5 and 4.0

Any help would be appreciated. Thanks.
-Steve

[Gostev]

Steve, looks like there's something wrong with vCenter server after reboot? Please send all log files from Help | Support Information to support@veeam.com, we may give you additional information based on those.

[stevepasq]

Thanks. I actually opened a case for this so I will reference that case number in the email.

[mangle@4thebank.com]

We seem to be having the same issue as after a reboot all jobs starting failing. We are also getting the same error message when trying to connect to the VirtualCenter server in Veeam.

"The underlying connection was closed. A connection that was expected to be kept alive was closed by the server."

Did this issue ever get resolved? Any information would be appreciated, thanks!

[Gostev]

Unfortunately the original posted never updated this thread with resolution, or case ID. I don't recall this issue either - it was almost half-year ago and with previous product version... Do you mind contacting our support in order to troubleshoot this?

[Vitaliy S.]

Hello,

It seems like you've faced a connectivity issue while establishing the connection to your vCenter, as for the original error please log a support ticket with us, and we will be able to assist you.

Thank you!

[Sagan2009]

We seem to be having the same issue as after a reboot all jobs starting failing. We are also getting the same error message when trying to connect to the VirtualCenter server in Veeam.

"The underlying connection was closed. A connection that was expected to be kept alive was closed by the server."

Did this issue ever get resolved? Any information would be appreciated, thanks!

I have opened a support ticket regarding the issue above. I will post the resolution here once one has been reached.

[Sagan2009]

We seem to have this narrowed down to the vCenter database. I collected a network dump for Veeam support but haven't heard back on it yet. If anyone has any further information it would be appreciated, thanks!

support ticket # 520180

[Sagan2009]

This issue has been passed on to VMware support.

http://communities.vmware.com/thread/254922?tstart=0

[Sagan2009]

This ended up being an issue with the vCenter database. We created a new database and everything has been fine ever since.

[Gostev]

Jim, thanks for updating the topic with the resolution.

[stevepasq]

We had this issue arise again last week. We ended up rebooting the SQL Server hosting the remote vcenter database. All jobs started working after rebooting the database server. There is no need to create a new database. It appears that SQL keeps connections open even when the vcenter server has been rebooted so after the server comes back up it cannot make a new connection. This was the resolution the last time we had this issue as well. I suggest you tell customers to reboot their database server before going through the trouble of creating another vcenter database.

[Gostev]

Thanks Steve, this is useful information.

Actually, the suggestion to create the new vCenter database was from VMware support directly this is not something we have recommended our customers to do. As you can see in the above referenced thread, this issue affected other functionality too (such as connecting to vCenter remotely with PowerShell), and was reproducible without Veeam.