Discussions specific to the VMware vSphere hypervisor
Post Reply
lux209
Novice
Posts: 8
Liked: never
Joined: Feb 16, 2012 10:07 am
Full Name: Lucas Plancherel
Contact:

Re: Repeated VDDK error 1 and 2

Post by lux209 » Apr 28, 2016 10:13 am

Hello,

apparently the host reboot fully solved the problem for me. At least it did not came back since I did it 4 days ago..

campus-torsten
Lurker
Posts: 1
Liked: never
Joined: May 12, 2016 10:31 am
Full Name: Torsten Fröhlich
Contact:

Re: Repeated VDDK error 1 and 2

Post by campus-torsten » May 12, 2016 10:33 am

Any updates concerning the root cause or a bug fix for this?
Kr,
Torsten

chris_lalala
Novice
Posts: 5
Liked: 2 times
Joined: Jun 26, 2013 8:12 am
Full Name: Chris
Contact:

Re: Repeated VDDK error 1 and 2

Post by chris_lalala » May 27, 2016 6:03 am

Same issuse here. vSphere 5.5, Veeam V9.

Only Workarround is moving the VM or Datastore or reboot the Host.

Ticket #01794584

picaroon
Influencer
Posts: 13
Liked: 5 times
Joined: Feb 12, 2016 1:44 pm
Full Name: Jeltsen Haagsma
Contact:

Re: Repeated VDDK error 1 and 2

Post by picaroon » Jun 01, 2016 11:52 am

A case with VMware SDK has been created. I sent them the exported logs from an affected host with 'triva' mode enabled and I'm in the progress of setting up the VixDiskLib sample program.

chris.collins
Lurker
Posts: 1
Liked: never
Joined: Jun 02, 2016 2:58 pm
Full Name: Chris Collins
Contact:

Re: Repeated VDDK error 1 and 2

Post by chris.collins » Jun 02, 2016 3:33 pm

We had the same issue - restarting the management agent on the host where the VM was housed resolved the issue (this only causes the host to be disconnected from vCenter for a minute or two but doesn't cause any downtime to the other VMs)

Backups have been fine since (2 days and counting)

Ice-Dog
Influencer
Posts: 11
Liked: 2 times
Joined: Feb 01, 2016 10:39 am
Full Name: Gunnar Olafsson
Contact:

Re: Repeated VDDK error 1 and 2

Post by Ice-Dog » Jun 06, 2016 8:12 am 1 person likes this post

Hi all,

OP here.

This has happened to me about 4-5 times since orginally posting. I have narrowed it down to 4 of my 14 hosts that this happens to repeatedly.

I contact my account manager at Veeam to put some weight on this - but the support answers where still poor and not escalated to a the top tier.. "try using another proxy", "try using hot-add". When I am using new transport servers and using hot-add and NFS v3 causes up to 60 sec stun time.

I dragged VMware on a call and the support guy form them was at first very adamant that this could never be a VMware issue if Veeam is able to contact the API to issue snapshot commands and rejected taking any look at my vSphere settings. However I sent them logs from the latest hosts that failed and vCenter with accurate details of the times when the backups error-ed.

They VMware actually came back to me with a possible solution. They see nothing being is being logged about the backups as the backups API is not reaching the ESXi. They think that maybe during a patch that the VDDK may not have installed correctly. Meaning the API calls are not being received.

To attempt to fix this can I will update the host to the latest build. By doing this it will install a fresh VDDK that should allow the API calls in.

I will update once that is completed and report 3 months down the line if the issue will have cropped up again.

Regards,
Ice-Dog

StefanSpecht
Influencer
Posts: 16
Liked: never
Joined: Aug 17, 2010 12:21 pm
Full Name: Stefan Specht
Contact:

Re: Repeated VDDK error 1 and 2

Post by StefanSpecht » Jun 07, 2016 9:19 am

Same here.
Some of our VMWare hosts randomly get this issue. Restarting the management agents is fixing it temporarly, but this happens every few days.
We are running Veeam 8 latest update and ESXi 5.5.0 3568722. Only sites that use network mode are affected and espacially those with a lot of VMs on it (= a lot of network traffic)
I suspect this to be a bug somewhere in the ESXi network stack (full buffers etc?) and will open a VMWare support case in the next days.

pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Repeated VDDK error 1 and 2

Post by pidthepiper » Jun 07, 2016 10:36 am

Ok I have this issue now too.

ESXi 5.5u3b EP10 3568722 and Veeam 8 Update 3

Its nbd related too

Support Case[ID# 01820005]

pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Repeated VDDK error 1 and 2

Post by pidthepiper » Jun 07, 2016 11:52 am

esxcfg-advcfg -g /BufferCache/MaxCapacity
Value of MaxCapacity is 16384
~ # esxcfg-advcfg -g /BufferCache/FlushInterval
Value of FlushInterval is 30000

Are the hosts current settings, which I assume are defaults

mgratla
Influencer
Posts: 19
Liked: never
Joined: Feb 04, 2014 11:13 pm
Full Name: Mark
Contact:

Re: Repeated VDDK error 1 and 2

Post by mgratla » Jun 12, 2016 12:46 pm

Adding my voice to this one, we've got the same issue. I don't have a case with Veeam as of yet. We're on the latest build of 5.5.

Interestingly if we leave it go on for long enough the host itself eventually restarts it's management adapters.

rayzhang
Lurker
Posts: 1
Liked: never
Joined: Jun 13, 2016 1:19 am
Full Name: rayzhang
Contact:

Re: Repeated VDDK error 1 and 2

Post by rayzhang » Jun 13, 2016 1:35 am

Thanks for your discussions, Now I sloved the same problem like that. I found the vms of veeam failure in a same host ,so I thought maybe is the host problem . Because my hosts had HA,so I migrated the veeam vms to the other host ,and then veeam is backup successfully . I don't know why .Maybe the host need to reboot . I will choose a good time to test if it done .

If your host in HA, you can try to slove it by migration too :D

pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Repeated VDDK error 1 and 2

Post by pidthepiper » Jun 13, 2016 1:52 pm

mgratla wrote:
Interestingly if we leave it go on for long enough the host itself eventually restarts it's management adapters.
Could you explain this a bit more?

pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Repeated VDDK error 1 and 2

Post by pidthepiper » Jun 13, 2016 1:52 pm

Also any update on this from Veeam, i have a ticket open with you as its happened to 2 hosts currently and both you and VMware arent really helping

mgratla
Influencer
Posts: 19
Liked: never
Joined: Feb 04, 2014 11:13 pm
Full Name: Mark
Contact:

Re: Repeated VDDK error 1 and 2

Post by mgratla » Jun 13, 2016 4:34 pm

pidthepiper wrote:
Could you explain this a bit more?
Absolutely.

If left unchecked, after about 48 hours of receiving VDDK failures consistently through Veeam (We backup almost 400 machines and they all retry three times) - We see the host loose connection in vCenter momentarily and come back up. From what I've been able to tell, the host self restarted it's vpxa and vpxd processes. After this happens the jobs run fine for a while

pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Repeated VDDK error 1 and 2

Post by pidthepiper » Jun 13, 2016 4:48 pm

Ahhh now that is interesting, so eventually the host sorts itself out, but it takes a few days heh. There has to be something that triggers, but neither Veeam or VMware seem to know. Both Support teams are acting like it s the first time they have heard of this issue, even though I point them to this thread, showing peoples support case numbers

Post Reply

Who is online

Users browsing this forum: vmJoe and 29 guests