Stuck job - no progress

[egroeg]

Hello community.

I have a regular issue that appears to crop up in my environment.

Every so often, a VM within a job will get completely stuck on backing up one of it's disks. I've tested leaving it there for hours/days to no progress, so then have to stop the job, drop the backup service and try again.

The error code provides little assistance.

The VMs DO backup, I have most of the health check in order > VMTOOLS version, VSS WRITER, LATEST patches, etc etc - but this issue appears to crop up regularly.

Can anyone provide any insight?



Veeam server - 7.0.0.871
Physical Server, Intel Xeon, 64 GB RAM,
4 GB iSCSI, MS initiator, Least Queue Depth,
EqaulLogic MPIO
1GB iSCSI fabric
ACLs on target LUNS available
VCenter 5.5, vSphere 5.5

[foggy]

George, what kind of error code are you referring to? If the job stucks, then I guess it does not throw out any error? Are you talking about some event logged into Event Log inside VM guest OS? Is it the same VM and disk every time and if yes, what kind of disk it is? Have you already contacted support with this? They will be able to review job logs and define the possible reason of this behavior.

Also, changing the proxy server could help.

[egroeg]

Hi Foggy.

Just realised that my image didnt format correctly - yes, that's right, the job actually never completes. See the below image whereby the disk in question is showing at 0.0 kb read at 0 kB/s - the disk is never accessed. Now, if we drop that job and restart the service, next time, that disk may well backup just fine....

No, I've not raised a call with Veeam support as of yet

[Vitaliy S.]

If it's always the same VM, then I would recommend to review its configuration. The reason for this "stun" will most likely be logged in the job debug files, our technical support team should be able to assist you with logs review. Thanks!

[annabennet]

I have same problem. two of my customers have this problem and i tried many ways to solve it but it appears once every two or three days. i think Veeam BR has a problem with new update and patch of vcenter and esxi. because after update infrastructure this appeared.

[foggy]

Anna, have you ever contacted our technical support with this issue?

[annabennet]

No, i think my customers don't have support contract. i didn't install Veeam BR on their environments. i only support and update them.

[foggy]

Then you can contact support directly yourself.

[annabennet]

I asked my customers, they don't have support contract. it is not only my problem, please help me to resolve it.

[foggy]

To investigate the reasons of this issue, job debug logs review is required, which is performed by technical support staff. In order to get their assistance, you need to have active product maintenance.

[Sloan]

I had these same symptoms a few weeks ago. I noticed that random disks on random VM's were hanging with zero progress occurred around the same time that the vcenter vm was attempting to up. And specifically at about the time that the VM's snapshot was being released. It turns out that we had a new vcenter server built as 2012 R2 and somehow in our template for 2012R2 VM's, the Microsoft Software Shadow Copy Provider service had gotten disabled resulting in the VSS snapshot failing. We were seeing VMware Shapshot Provider timeouts in the event logs and VSS timeouts in the backup jobs as well.

Once we got the the vcenter VM backing up successfully with no warnings we haven't seen any more hard disks hang.