Discussions specific to the VMware vSphere hypervisor
Post Reply
DonZoomik
Expert
Posts: 226
Liked: 57 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Community digest: vSphere 6.5 performance bug with PVSCSI

Post by DonZoomik »

I was wondering if the issue was huge IO latency spikes (500ms to 1s range) in Windows when UNMAP is invoked (and for a while after)?
I noticed them with my previous employer in April but left before this issue got anywhere. Then I fired up a brand new vSphere cluster last week and the same issue appeared. I was about to write a support case to VMware on Monday when the community digest arrived...

6.5, pvscsi, UNMAP - the stars seem to align. :D
This time I also took a look at esxtop and only symptom is that VM IOPS goes down considerably during and after UNMAP (corresponding with in-guest IO spikes), while other parameters see no major changes...

johnlong
Enthusiast
Posts: 74
Liked: 11 times
Joined: May 12, 2012 3:37 pm
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by johnlong » 1 person likes this post

With each one of the Community Digests that I read with some major flaw in vSphere 6.x, I'm constantly reminded why I'm still on 5.5. Though I know I'll eventually have to upgrade. :shock:

DonZoomik
Expert
Posts: 226
Liked: 57 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by DonZoomik »

True, but I was hoping that Gostev could shed some more light on the issue as I could find no other information other the basic fact in the newsletter.

poulpreben
Veeam Vanguard
Posts: 1003
Liked: 431 times
Joined: Jul 23, 2012 8:16 am
Full Name: Preben Berg
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by poulpreben » 3 people like this post

Hi guys. I shared the info with Gostev, and my customer just received the official, private patch. They have been running successfully on a debug version of the patch for the past 10 days.

I will report back in this thread with the KB article, once it has been published by VMware.

DonZoomik
Expert
Posts: 226
Liked: 57 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by DonZoomik » 1 person likes this post

VMware support said that the public fix will probably be released on the last week of July.

poulpreben
Veeam Vanguard
Posts: 1003
Liked: 431 times
Joined: Jul 23, 2012 8:16 am
Full Name: Preben Berg
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by poulpreben » 1 person likes this post

The final patch has been issued to our customer, and it has been deployed on ESXi 6.5 EP2 (build 5310538). The patched build number is 5620045. As a final step in the troubleshooting process, the patch was installed on both a clean install of ESXi with patches, and an existing, previously upgraded ESXi host. The patch works successfully in both cases.

According to VMware Support, the patch is now confirmed to be included in ESXi 6.5U1 GA. We are still awaiting the KB article for detailed information about the actual root cause, and how to systematically detect the symptoms, but at least we have now successfully stabilized the environment.

poulpreben
Veeam Vanguard
Posts: 1003
Liked: 431 times
Joined: Jul 23, 2012 8:16 am
Full Name: Preben Berg
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by poulpreben » 3 people like this post

I just received the following response, before they closed the ticket.
VMware Support wrote:[...] they are reviewing a number of cases at present to ensure all observations are captured and correct.
The KB number will be 2150591 once published. As of now I do not have an ETA for publishing externally.
I'll keep an eye open for the published article.

mwvme
Expert
Posts: 163
Liked: 33 times
Joined: Dec 05, 2015 10:19 pm
Full Name: Michael White
Location: Calgary, Alberta Canada
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by mwvme »

Someone accidentally share with me a comment that end of July might be a good time to expect a nice upgrade from VMware.

Michael
Michael White
Field Product Manager
https://notesfrommwhite.net
@mwVme

poulpreben
Veeam Vanguard
Posts: 1003
Liked: 431 times
Joined: Jul 23, 2012 8:16 am
Full Name: Preben Berg
Contact:

Re: Community digest: vSphere 6.5 performance bug with PVSCS

Post by poulpreben »

Hi everyone.

vSphere 6.5 Update 1 is now available.

Read more here > vSphere 6.5 Update 1 – Under the Hood

The KB article is also live > Performance issues on Windows virtual machine with hardware version 13 after upgrading to ESXi 6.5 (2150591)

Post Reply

Who is online

Users browsing this forum: another_fine_maze, rdmtech and 35 guests