6.1 Hot Add issue

[lobo519]

I have noticed an issue with the hot add – it hangs (10-15 minutes) when trying to mount the snap shot to the proxy server for several minutes. There is no task in vCenter to reconfigure the virtual machine while its hanging. I see the same type of delay in the cleanup tasks ie. reconfiguring to remove the snapshot and Removing the snapshot from the VM

Support ID:5199329

[Gostev]

Well, removing snapshot does take long time due to data commit to VMDK, this is just how VMware snapshots work.
Hot add taking 1-2 minutes is almost normal (it is pretty slow operation), but 10-15 minutes is not normal.

Anything obviously wrong about VM which disks are being attached (many existing snapshots), or the issue is for every VM?

I am willing to bet the issue is not 6.1 specific, but let's see what support finds.

[lobo519]

Not the time it takes to run - The task in vcenter literally doesn't show up for 10 - 15 minutes - its happening on every vm

I might be on to something - possibly this behavior starts after doing a file level restore.
I was able get it running normally again after a reboot.

I will do more testing tomorrow and I too am interested what support has to say

[lobo519]

I did a few tests today - The issue definitely presents itself after a file level restore and a restart fixes it.

With a replication job, it only affects the source Proxy/vCenter (My Veeam Server also). The destination runs all the vsphere tasks in a timely fashion. I have sent an update to support but they haven't even looked at my logs yet.

[Gostev]

Then you should be able to easily work around this by setting up a proxy on a different VM, and then disabling the proxy on your backup server.

[lobo519]

Then you should be able to easily work around this by setting up a proxy on a different VM, and then disabling the proxy on your backup server.

Working around it isn't really an issue - The much easier solution (for me) is to just restart the server after a file level restore (not a frequent task).

Regardless - I felt it deserved attention.

[Gostev]

It definitely does worth investigation.

[lobo519]

The latest from support is that VM Tools not being up to date is causing this issue?? They requested that I try doing the tools upgrade on a VM that is in an affected job.

No sure I understand how this could really be the issue..

[Vitaliy S.]

Hmm...If server reboot resolves the issue, then I do not understand how VMware Tools version might affect HotAdd operation either.

As a further troubleshooting step could you please create a brand new "HotAdd" proxy server (any existing Windows server will work fine) and run a test backup job with one of the affected VMs? It would help us to narrow down this issue either to the backup proxy, backup server or infrastructure itself.

[lobo519]

When I tried to move one of my jobs to new proxy (windows 7) I got the following error " Error: Client error: VDDK thread terminated." after "Preparing replica VM"

Moved it back to my original proxy (2008 R2, Backup Server)and it works fine. There's nothing special about the new Windows 7 proxy, it just running SpiceWorks and I turned the firewall off.

Any thoughts?

[Vitaliy S.]

Have you informed your support engineer about this error? Unfortunately, it's hard to say what could have caused this error message without looking at the related VM processing steps. Could you please update your ticket with the latest info and let our support team investigate it for you?

[lobo519]

will do - thanks

[Jamie Pert]

Did you ever fix this lobo?

[lobo519]

still working on it - just got off the phone with level 2 - looks like it maybe anti-virus related...

[Jamie Pert]

Interesting - I'm not sure if it's related but a very low-end server is being used as a proxy and is throwing-up that error, we're trying to determine whether this is the sole reason for the VDDK error, or if something else may be causing it.

The proxy runs McAfee, but it's disabled.

[lobo519]

Interesting - I'm not sure if it's related but a very low-end server is being used as a proxy and is throwing-up that error, we're trying to determine whether this is the sole reason for the VDDK error, or if something else may be causing it.

The proxy runs McAfee, but it's disabled.

sorry - I didnt know you were referring to the VDDK error - I was referring to the original issue. No I did not get anywhere with that error.

[RPatel]

Hi,
My Veeam backup is no longer running "Client error: VDDK thread terminated"
Please advise

[Vitaliy S.]

Rajesh, with all technical issues please contact our support team for assistance. It is almost impossible to troubleshoot technical issues via forums without reviewing full debug log files. Thanks!

[RPatel]

How do i go about logging technical issue?

[foggy]

You can open a case using our support portal (step-by-step instructions are also provided when you click New Topic).

[RPatel]

I have logged the support call since yesterday and still waiting for Support:(

[foggy]

Rajesh, as I can see from your support case, you already have been contacted. Please note, that support is provided according to our support policy, where response times are specified depending on your support level (Standard or Premium) and case severity. Thanks.

[maikelfalcon]

We've had this error come up. The problem here was that the Veeam proxy server couldn't reach our Veeam server on FULL dns name.
On the proxy server we could ping HOSTNAME but not HOSTNAME.DOMAIN.COM. (We recently did a rendom on our DNS servers).
After adding HOSTNAME.DOMAIN.COM IP-adrress to the hosts file on our Veeam proxy server, the backup started functioning again.