Repeated VDDK error 1 and 2

VMware specific discussions

Re: Repeated VDDK error 1 and 2

Veeam Logoby lux209 » Thu Apr 28, 2016 10:13 am

Hello,

apparently the host reboot fully solved the problem for me. At least it did not came back since I did it 4 days ago..
lux209
Novice
 
Posts: 8
Liked: never
Joined: Thu Feb 16, 2012 10:07 am
Full Name: Lucas Plancherel

Re: Repeated VDDK error 1 and 2

Veeam Logoby campus-torsten » Thu May 12, 2016 10:33 am

Any updates concerning the root cause or a bug fix for this?
Kr,
Torsten
campus-torsten
Lurker
 
Posts: 1
Liked: never
Joined: Thu May 12, 2016 10:31 am
Full Name: Torsten Fröhlich

Re: Repeated VDDK error 1 and 2

Veeam Logoby chris_lalala » Fri May 27, 2016 6:03 am

Same issuse here. vSphere 5.5, Veeam V9.

Only Workarround is moving the VM or Datastore or reboot the Host.

Ticket #01794584
chris_lalala
Lurker
 
Posts: 1
Liked: never
Joined: Wed Jun 26, 2013 8:12 am
Full Name: Christoph Hochsticher

Re: Repeated VDDK error 1 and 2

Veeam Logoby picaroon » Wed Jun 01, 2016 11:52 am

A case with VMware SDK has been created. I sent them the exported logs from an affected host with 'triva' mode enabled and I'm in the progress of setting up the VixDiskLib sample program.
picaroon
Influencer
 
Posts: 13
Liked: 5 times
Joined: Fri Feb 12, 2016 1:44 pm
Full Name: Jeltsen Haagsma

Re: Repeated VDDK error 1 and 2

Veeam Logoby chris.collins » Thu Jun 02, 2016 3:33 pm

We had the same issue - restarting the management agent on the host where the VM was housed resolved the issue (this only causes the host to be disconnected from vCenter for a minute or two but doesn't cause any downtime to the other VMs)

Backups have been fine since (2 days and counting)
chris.collins
Lurker
 
Posts: 1
Liked: never
Joined: Thu Jun 02, 2016 2:58 pm
Full Name: Chris Collins

Re: Repeated VDDK error 1 and 2

Veeam Logoby Ice-Dog » Mon Jun 06, 2016 8:12 am 1 person likes this post

Hi all,

OP here.

This has happened to me about 4-5 times since orginally posting. I have narrowed it down to 4 of my 14 hosts that this happens to repeatedly.

I contact my account manager at Veeam to put some weight on this - but the support answers where still poor and not escalated to a the top tier.. "try using another proxy", "try using hot-add". When I am using new transport servers and using hot-add and NFS v3 causes up to 60 sec stun time.

I dragged VMware on a call and the support guy form them was at first very adamant that this could never be a VMware issue if Veeam is able to contact the API to issue snapshot commands and rejected taking any look at my vSphere settings. However I sent them logs from the latest hosts that failed and vCenter with accurate details of the times when the backups error-ed.

They VMware actually came back to me with a possible solution. They see nothing being is being logged about the backups as the backups API is not reaching the ESXi. They think that maybe during a patch that the VDDK may not have installed correctly. Meaning the API calls are not being received.

To attempt to fix this can I will update the host to the latest build. By doing this it will install a fresh VDDK that should allow the API calls in.

I will update once that is completed and report 3 months down the line if the issue will have cropped up again.

Regards,
Ice-Dog
Ice-Dog
Influencer
 
Posts: 11
Liked: 2 times
Joined: Mon Feb 01, 2016 10:39 am
Full Name: Gunnar Olafsson

Re: Repeated VDDK error 1 and 2

Veeam Logoby StefanSpecht » Tue Jun 07, 2016 9:19 am

Same here.
Some of our VMWare hosts randomly get this issue. Restarting the management agents is fixing it temporarly, but this happens every few days.
We are running Veeam 8 latest update and ESXi 5.5.0 3568722. Only sites that use network mode are affected and espacially those with a lot of VMs on it (= a lot of network traffic)
I suspect this to be a bug somewhere in the ESXi network stack (full buffers etc?) and will open a VMWare support case in the next days.
StefanSpecht
Influencer
 
Posts: 16
Liked: never
Joined: Tue Aug 17, 2010 12:21 pm
Full Name: Stefan Specht

Re: Repeated VDDK error 1 and 2

Veeam Logoby pidthepiper » Tue Jun 07, 2016 10:36 am

Ok I have this issue now too.

ESXi 5.5u3b EP10 3568722 and Veeam 8 Update 3

Its nbd related too

Support Case[ID# 01820005]
pidthepiper
Enthusiast
 
Posts: 65
Liked: 5 times
Joined: Tue Aug 11, 2015 9:10 am
Full Name: Bilal AHmed

Re: Repeated VDDK error 1 and 2

Veeam Logoby pidthepiper » Tue Jun 07, 2016 11:52 am

esxcfg-advcfg -g /BufferCache/MaxCapacity
Value of MaxCapacity is 16384
~ # esxcfg-advcfg -g /BufferCache/FlushInterval
Value of FlushInterval is 30000

Are the hosts current settings, which I assume are defaults
pidthepiper
Enthusiast
 
Posts: 65
Liked: 5 times
Joined: Tue Aug 11, 2015 9:10 am
Full Name: Bilal AHmed

Re: Repeated VDDK error 1 and 2

Veeam Logoby mgratla » Sun Jun 12, 2016 12:46 pm

Adding my voice to this one, we've got the same issue. I don't have a case with Veeam as of yet. We're on the latest build of 5.5.

Interestingly if we leave it go on for long enough the host itself eventually restarts it's management adapters.
mgratla
Influencer
 
Posts: 19
Liked: never
Joined: Tue Feb 04, 2014 11:13 pm
Full Name: Mark

Re: Repeated VDDK error 1 and 2

Veeam Logoby rayzhang » Mon Jun 13, 2016 1:35 am

Thanks for your discussions, Now I sloved the same problem like that. I found the vms of veeam failure in a same host ,so I thought maybe is the host problem . Because my hosts had HA,so I migrated the veeam vms to the other host ,and then veeam is backup successfully . I don't know why .Maybe the host need to reboot . I will choose a good time to test if it done .

If your host in HA, you can try to slove it by migration too :D
rayzhang
Lurker
 
Posts: 1
Liked: never
Joined: Mon Jun 13, 2016 1:19 am
Full Name: rayzhang

Re: Repeated VDDK error 1 and 2

Veeam Logoby pidthepiper » Mon Jun 13, 2016 1:52 pm

mgratla wrote:
Interestingly if we leave it go on for long enough the host itself eventually restarts it's management adapters.


Could you explain this a bit more?
pidthepiper
Enthusiast
 
Posts: 65
Liked: 5 times
Joined: Tue Aug 11, 2015 9:10 am
Full Name: Bilal AHmed

Re: Repeated VDDK error 1 and 2

Veeam Logoby pidthepiper » Mon Jun 13, 2016 1:52 pm

Also any update on this from Veeam, i have a ticket open with you as its happened to 2 hosts currently and both you and VMware arent really helping
pidthepiper
Enthusiast
 
Posts: 65
Liked: 5 times
Joined: Tue Aug 11, 2015 9:10 am
Full Name: Bilal AHmed

Re: Repeated VDDK error 1 and 2

Veeam Logoby mgratla » Mon Jun 13, 2016 4:34 pm

pidthepiper wrote:
Could you explain this a bit more?


Absolutely.

If left unchecked, after about 48 hours of receiving VDDK failures consistently through Veeam (We backup almost 400 machines and they all retry three times) - We see the host loose connection in vCenter momentarily and come back up. From what I've been able to tell, the host self restarted it's vpxa and vpxd processes. After this happens the jobs run fine for a while
mgratla
Influencer
 
Posts: 19
Liked: never
Joined: Tue Feb 04, 2014 11:13 pm
Full Name: Mark

Re: Repeated VDDK error 1 and 2

Veeam Logoby pidthepiper » Mon Jun 13, 2016 4:48 pm

Ahhh now that is interesting, so eventually the host sorts itself out, but it takes a few days heh. There has to be something that triggers, but neither Veeam or VMware seem to know. Both Support teams are acting like it s the first time they have heard of this issue, even though I point them to this thread, showing peoples support case numbers
pidthepiper
Enthusiast
 
Posts: 65
Liked: 5 times
Joined: Tue Aug 11, 2015 9:10 am
Full Name: Bilal AHmed

PreviousNext

Return to VMware vSphere



Who is online

Users browsing this forum: ilya.konopak and 22 guests