Host-based backup of VMware vSphere VMs.
Post Reply
phokay
Service Provider
Posts: 12
Liked: 2 times
Joined: Nov 07, 2013 9:48 am
Full Name: Michael Han
Contact:

Replication jobs stuck on V8 Patch1

Post by phokay »

Support call logged and the case ID is #00831799

I am not getting good responses from support, therefore, I am here to seek some help.

We are replicating VMs over the dedicated VPN connection and the link between the site is 100Mbps.
We have a Veeam Management Console and a dedicated target proxy at the target site. Veeam Management Console is not being used as a backup proxy.
At the source site, there is a dedicated proxy server. All Veeam VMs at both source and target sites run on VMware ESXi 5.x

In the beginning, it seemed to work fine but since over the last 4 weeks, we are seeing the replication jobs being stuck in the queue and doing nothing. At the moment, there are 20 jobs stuck in the queue for 2 days. Most of the queued jobs are at the "initial" status with "resource not ready:VMware backup proxy". However, the proxies at both sites are not running any jobs. The "VMware backup proxy" has been removed and it's not in the proxies list.
Now those jobs are still stuck with 0% Completed status for the jobs but in the job statistics, the VMs failed with the error "Error: Exception of type 'Veeam.Backup.AgentProvider.AgentClosedException' was thrown".

In the past few weeks, I have to kill all the jobs, restart the proxies at both sites and rerun the jobs. It would work for a couple of days and the same thing happen again.

I would appreciate if you could share your similar experiences and/or have a permanent fix for it.

Thanks you.
phokay
Service Provider
Posts: 12
Liked: 2 times
Joined: Nov 07, 2013 9:48 am
Full Name: Michael Han
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by phokay » 1 person likes this post

Updates...
- Support looked at the logs and confirmed that agent process hung.
- Advised to deploy a second proxy server. We deployed another proxy server at source site and it appeared to be getting better now.
- Will monitor for a few days and update the thread.
phokay
Service Provider
Posts: 12
Liked: 2 times
Joined: Nov 07, 2013 9:48 am
Full Name: Michael Han
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by phokay »

The same issues of proxies hanging happened again to both old and new proxies. I have confirmed that while the jobs started hanging, I could log in to the servers and they were responsive. I left the hanging jobs running and also left schedules as it was so that all other jobs could run over night. This morning, I found that both proxies were unresponsive...could not even login to the servers.
The "Traffic" tab of replication settings use "WAN Target" for Storage Optimization and Compressional level is "Optimal (recommended)".
Latest patches of Windows on the proxy was done over the last weekend. Since the issues remain the same before and after the patches, I assumed that the patches did not make any difference on the issues.
phokay
Service Provider
Posts: 12
Liked: 2 times
Joined: Nov 07, 2013 9:48 am
Full Name: Michael Han
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by phokay »

No resolution yet after 2 weeks of lackluster troubleshooting...
phokay
Service Provider
Posts: 12
Liked: 2 times
Joined: Nov 07, 2013 9:48 am
Full Name: Michael Han
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by phokay »

Installed Wireshark onto all Veeam boxes to catch a whole episode of the issue.
ale.com
Novice
Posts: 7
Liked: 3 times
Joined: Jul 15, 2016 9:59 am
Full Name: Alessio Comai
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by ale.com »

hi, did you ever solved it? thank you!
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by foggy »

Hi Alessio, the OP's case was closed as his issues were confirmed to be network-related. I recommend you opening your own case for investigation.
ale.com
Novice
Posts: 7
Liked: 3 times
Joined: Jul 15, 2016 9:59 am
Full Name: Alessio Comai
Contact:

Re: Replication jobs stuck on V8 Patch1

Post by ale.com » 2 people like this post

Just for future reference, I solved the issue expanding RAM on proxy server (and I'm on V9). thank you.
Post Reply

Who is online

Users browsing this forum: Baidu [Spider], Semrush [Bot] and 71 guests