Discussions specific to the VMware vSphere hypervisor
m0ps
Novice
Posts: 6
Liked: 2 times
Joined: Dec 15, 2011 7:17 am
Full Name: Maxim Kursin
Contact:

vSphere 6.5U2 support

Post by m0ps » May 04, 2018 2:24 pm 2 people like this post

[UPDATE] Jan 31st, 2019
vSphere 6.5 U2 patch level ESXi650-201811002 or later is officially supported with Veeam Backup & Replication 9.5 Update 4


Is there any official position of Veeam regarding just released vSphere 6.5U2? Is it safe to upgrade vCenter and ESXi hosts, or we need to wait for new Veeam B&R release?

Gostev
SVP, Product Management
Posts: 24293
Liked: 3330 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vSphere 6.5U2 support

Post by Gostev » May 04, 2018 2:52 pm

We have not had a chance to deploy and test it yet, as it was literally just released. Normally updates do not require any code changes from us, as typically they bring nothing but bug fixes and minor enhancements. Nevertheless, we recommend that you wait until we update our QC labs and run it through the paces. Thanks!

m0ps
Novice
Posts: 6
Liked: 2 times
Joined: Dec 15, 2011 7:17 am
Full Name: Maxim Kursin
Contact:

Re: vSphere 6.5U2 support

Post by m0ps » May 04, 2018 3:06 pm

Ok, and when you will update your QC lab? Is there any ETA, so we can plan to update our installation to 6.5U2?

foggy
Veeam Software
Posts: 17931
Liked: 1512 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: vSphere 6.5U2 support

Post by foggy » May 04, 2018 3:10 pm

We plan to update the lab next week. Cannot provide any ETA at the moment, just keep an eye on this thread - we need to let it roll for some time and will let you know if we see any issues.

jamerson
Expert
Posts: 311
Liked: 18 times
Joined: May 01, 2013 9:54 pm
Full Name: Julien
Contact:

Re: vSphere 6.5U2 support

Post by jamerson » May 05, 2018 8:56 pm

AS i understand now, we cannot update to the U2 6.5 neither to 6.5 until VEEAM tested this ?

Gostev
SVP, Product Management
Posts: 24293
Liked: 3330 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vSphere 6.5U2 support

Post by Gostev » May 06, 2018 5:31 pm 1 person likes this post

It's up to you. Some people choose to jump these minor updates the day they are released. For obvious reasons, there's some risk involved with upgrading to new platform versions before they've been thoroughly tested and declared as officially supported. Thanks!

Gostev
SVP, Product Management
Posts: 24293
Liked: 3330 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vSphere 6.5U2 support

Post by Gostev » May 10, 2018 2:26 pm

UPDATE: Our auto-tests have exhibited one potentials issue with 6.5 U2 which may reduce reliability of a number of Veeam Backup & Replication features dependent on the corresponding protocol. Having said that, the issue is only reproducible in our stress testing lab where we emulate a heavy environment load, meaning for those who already upgraded these issue will likely exhibit "random" behavior which is in reality tied up to peak load in the environment. We're opening a support case with VMware - and I will keep you posted on the progress. Thanks!

Saad
Novice
Posts: 8
Liked: never
Joined: Sep 26, 2015 6:01 pm
Full Name: Saad
Contact:

Re: vSphere 6.5U2 support

Post by Saad » May 10, 2018 8:03 pm

i am one of the people who jumped to that release .. and yes since i have upgraded vcenter and esxi hosts to 6.5u2 i've been having issues with replication jobs with veeam proxies not being able to hot add vmdks and its random and it falls back to network mode . proxies are a mix between 2008 R2 and 2012 R2 fully patched. and in this case i'm not sure if its related to the vcenter 6.2u2

bmcdonald
Lurker
Posts: 2
Liked: never
Joined: Feb 22, 2017 2:53 pm
Full Name: Brian McDonald
Contact:

Re: vSphere 6.5U2 support

Post by bmcdonald » May 14, 2018 5:44 pm

Gostev wrote:UPDATE: Our auto-tests have exhibited one potentials issue with 6.5 U2 which may reduce reliability of a number of Veeam Backup & Replication features dependent on the corresponding protocol. Having said that, the issue is only reproducible in our stress testing lab where we emulate a heavy environment load, meaning for those who already upgraded these issue will likely exhibit "random" behavior which is in reality tied up to peak load in the environment. We're opening a support case with VMware - and I will keep you posted on the progress. Thanks!
Is this only with the ESXi update or does it also happen with the vCenter update? We're looking at updating vCenter to 6.5U2 for now and holding off on completing the update to the ESXi hosts until Veeam gives an all clear but if issues have been seen with the vCenter update as well then we'll hold off.

cdbarnes
Lurker
Posts: 2
Liked: never
Joined: May 10, 2018 11:13 pm
Full Name: Christopher Barnes
Contact:

Re: vSphere 6.5U2 support

Post by cdbarnes » May 14, 2018 9:34 pm

Hi all,

Unfortunately, we also jumped to 6.5U2 and are now encountering issues with random failures with our SQL transaction log backups. Currently working with Veeam Support but we haven't really pinned down a definitive answer to what is causing the job to fail.

rgreen83
Service Provider
Posts: 14
Liked: 19 times
Joined: Mar 30, 2017 4:00 pm
Full Name: Ryan Green
Contact:

Re: vSphere 6.5U2 support

Post by rgreen83 » May 15, 2018 3:43 pm 4 people like this post

This really needs to be a priority for Veeam to release a fix and not wait until whatever next update that supports 6.7, the 6.7 upgrade is a manual deliberate process, going to 6.5U2 isn't. If you simply use VUM to install security patches etc you will end up with 6.5U2 so Veeam is asking folks to forgo security patches to be supported on their product. I understand what VMware does is outside the control of Veeam and time is needed for testing and fixing if necessary, I'm just requesting that fixes not languish waiting for other things. The distance between patches for product support and fixes has grown far too long in the past couple years while Veeam holds things back waiting for new features to be ready. I do though greatly appreciate the communication from Gostev and others here so at least we aren't in the dark.

niteb
Lurker
Posts: 1
Liked: never
Joined: Apr 08, 2013 7:49 pm
Full Name: Przemek
Location: Poland
Contact:

Re: vSphere 6.5U2 support

Post by niteb » May 16, 2018 2:24 pm

Also there is probably an issue with adding vcenter to veeamone. It shows credentials problems. The same cred on vcenter 6.5 u2 works fine.

marcvanhoutum
Lurker
Posts: 1
Liked: 1 time
Joined: May 17, 2018 7:40 am
Full Name: Marc van Houtum
Contact:

Re: vSphere 6.5U2 support

Post by marcvanhoutum » May 17, 2018 7:48 am 1 person likes this post

We've noticed some issues with application aware processing when a VM has the VMWare tools installed that come with vSphere 6.5u2 (version 10305). Veeam seems unable to deploy the Vix guest helper application inside the VM.

Downgrading VMWare tools seems to resolve the issue.

andrie
Influencer
Posts: 10
Liked: never
Joined: Nov 20, 2015 12:37 pm
Full Name: Andy
Contact:

Re: vSphere 6.5U2 support

Post by andrie » May 18, 2018 6:45 am

I can confirm problems with SQL / Exchange.

On Exchange servers our customer gets the following error message:
Unable to truncate Microsoft Exchange transaction logs. Details: Failed to call RPC function 'Vss.FinishSnapshot': Error code: 0x800401fd. Failed to invoke func [FinishSnapshot]: Object is not connected to server.

On SQL servers, the error message is:
Enumerating SQL Server databases
Failed to call RPC function 'Vss.ExploreInstances': Error code: 0x800401fd. Failed to invoke func [ExploreInstances]: Object is not connected to server.

Support case is opened, Case #03010946

spiritie
Enthusiast
Posts: 76
Liked: 7 times
Joined: Mar 01, 2016 10:16 am
Full Name: Gert van Niekerk
Location: Denmark
Contact:

Re: vSphere 6.5U2 support

Post by spiritie » May 19, 2018 5:33 pm 2 people like this post

rgreen83 wrote:This really needs to be a priority for Veeam to release a fix and not wait until whatever next update that supports 6.7, the 6.7 upgrade is a manual deliberate process, going to 6.5U2 isn't. If you simply use VUM to install security patches etc you will end up with 6.5U2 so Veeam is asking folks to forgo security patches to be supported on their product. I understand what VMware does is outside the control of Veeam and time is needed for testing and fixing if necessary, I'm just requesting that fixes not languish waiting for other things. The distance between patches for product support and fixes has grown far too long in the past couple years while Veeam holds things back waiting for new features to be ready. I do though greatly appreciate the communication from Gostev and others here so at least we aren't in the dark.
The problem with 6.5U2 is that VMware has back-ported a few features from 6.7 into this update.
So this isn't just another "small" update, but actually quite big one.

Gostev
SVP, Product Management
Posts: 24293
Liked: 3330 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vSphere 6.5U2 support

Post by Gostev » May 20, 2018 11:15 pm

We would like to investigate all issues that customers who have already upgraded to 6.5 U2 are seeing, so please don't be shy to open support cases even if 6.5 U2 is not officially supported yet. This should help us to accelerate its support by identifying all issues that this update introduced sooner. Thank you!

cdbarnes
Lurker
Posts: 2
Liked: never
Joined: May 10, 2018 11:13 pm
Full Name: Christopher Barnes
Contact:

Re: vSphere 6.5U2 support

Post by cdbarnes » May 21, 2018 2:20 am

andrie wrote:I can confirm problems with SQL / Exchange.

On Exchange servers our customer gets the following error message:
Unable to truncate Microsoft Exchange transaction logs. Details: Failed to call RPC function 'Vss.FinishSnapshot': Error code: 0x800401fd. Failed to invoke func [FinishSnapshot]: Object is not connected to server.

On SQL servers, the error message is:
Enumerating SQL Server databases
Failed to call RPC function 'Vss.ExploreInstances': Error code: 0x800401fd. Failed to invoke func [ExploreInstances]: Object is not connected to server.

Support case is opened, Case #03010946
This is one of the few errors we have seen in our environment during SQL transaction log runs. We would see a couple of successful runs then a failure or two, then another failure. This seems to have tapered off in the last few days with no change in the environment. Support case is closed, due to no recent failures, Case #02899195.

nd39475
Influencer
Posts: 22
Liked: 2 times
Joined: May 05, 2016 6:28 pm
Full Name: n d
Contact:

Re: vSphere 6.5U2 support

Post by nd39475 » May 21, 2018 1:33 pm

bmcdonald wrote: Is this only with the ESXi update or does it also happen with the vCenter update? We're looking at updating vCenter to 6.5U2 for now and holding off on completing the update to the ESXi hosts until Veeam gives an all clear but if issues have been seen with the vCenter update as well then we'll hold off.
I would also like a definitive answer on this. I can report we are running VCSA 6.5.0.20000 307201 with ESXi 6.0.0 7967664 and we have not seen any backup failures; However, we have not had any restore opportunities or testing of the backups.

mazcredi
Influencer
Posts: 11
Liked: never
Joined: Dec 13, 2011 5:34 pm
Full Name: Mazen Credi
Contact:

Re: vSphere 6.5U2 support

Post by mazcredi » May 21, 2018 3:40 pm

What is the latest release of vSphere that VEEAM recommends below 6.5U2?

DGrinev
Veeam Software
Posts: 1641
Liked: 201 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: vSphere 6.5U2 support

Post by DGrinev » May 21, 2018 4:24 pm

Hi Mazen,

You can stick with any supported vSphere version before 6.5U2. Thanks!

cambiumphil
Service Provider
Posts: 8
Liked: 1 time
Joined: Jul 08, 2015 9:30 pm
Full Name: Phil Brutsche
Contact:

Re: vSphere 6.5U2 support

Post by cambiumphil » May 21, 2018 8:07 pm

nd39475 wrote:I would also like a definitive answer on this. I can report we are running VCSA 6.5.0.20000 307201 with ESXi 6.0.0 7967664 and we have not seen any backup failures; However, we have not had any restore opportunities or testing of the backups.
I don't think you'll have a problem with vCenter 6.5U2 as long as you aren't using ESXi 6.5U2. My (limited) understanding is Veeam B&R uses vCenter to figure out where in your vSphere cluster stuff is so that it can talk to ESXi and VMware Tools in your VMs. ESXi 6.5U2 and the associated VMware Tools is where a lot of people are having trouble.

Henrik.Grevelund
Service Provider
Posts: 58
Liked: 2 times
Joined: Feb 13, 2017 2:56 pm
Full Name: Henrik Grevelund
Contact:

Re: vSphere 6.5U2 support

Post by Henrik.Grevelund » May 22, 2018 1:31 pm

DGrinev wrote:You can stick with any supported vSphere version before 6.5U2. Thanks!
Hi,

Note sure you you would link to a documentation page that doesn't mention anything about which updates are supported ?
Have nice day,
Henrik

DGrinev
Veeam Software
Posts: 1641
Liked: 201 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: vSphere 6.5U2 support

Post by DGrinev » May 22, 2018 2:15 pm

Hi,

It is vSphere 6.5 any update or patch before vSphere 6.5 Update 2 that Veeam B&R supports. Thanks!

ChuckS42
Expert
Posts: 106
Liked: 13 times
Joined: Apr 24, 2013 8:53 pm
Full Name: Chuck Stevens
Contact:

Re: vSphere 6.5U2 support

Post by ChuckS42 » May 22, 2018 5:48 pm 1 person likes this post

Personally, folks really need to wait for Veeam (and any other third-party provider's software you use that integrates with vSphere/vCenter) to test before installing anything from VMware labelled as an "Update", as opposed to a "Patch". Patches are generally safe, Updates need testing. You're aiming a loaded gun at your foot every time you jump the gun. (How's that for a mixed metaphor?)

cambiumphil
Service Provider
Posts: 8
Liked: 1 time
Joined: Jul 08, 2015 9:30 pm
Full Name: Phil Brutsche
Contact:

Re: vSphere 6.5U2 support

Post by cambiumphil » May 22, 2018 11:42 pm

Agreed! My only 6.5U2 environment (VC 6.5U2 / ESXi 6.0) is my test lab where if it doesn't work it's a "oh. I guess I'll wait for the patch" type situation

We will absolutely hold off on updating anything production until there's a Veeam B&R update that supports it!

dgapinski
Enthusiast
Posts: 55
Liked: 2 times
Joined: Dec 05, 2013 8:09 pm
Full Name: Dan Gapinski
Contact:

Re: vSphere 6.5U2 support

Post by dgapinski » May 23, 2018 4:37 pm

We are at vSphere 6.5 5973321 and ESXi, 6.5.0, 6765664, and are experiencing issues with the bug that causes our esxi hosts to lose ability to manage VMs & can only be fixed by reboot. So we're anxious to upgrade to U2, and I'll keep watching this thread. Please post progress? (thank you!)

Gostev
SVP, Product Management
Posts: 24293
Liked: 3330 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vSphere 6.5U2 support

Post by Gostev » May 23, 2018 9:15 pm 1 person likes this post

No material response from VMware Support for 12 days now, so we keep digging the issue on our end and trying to find possible workarounds. It's quite unlikely we will be able to fix it reliably from our side though, as the issue appears to sit in the ESXi itself. So in case VMware confirms this regression, you will likely have to wait for them to ship ESXi 6.5 U2a (or something) before upgrading.

dgapinski
Enthusiast
Posts: 55
Liked: 2 times
Joined: Dec 05, 2013 8:09 pm
Full Name: Dan Gapinski
Contact:

Re: vSphere 6.5U2 support

Post by dgapinski » May 24, 2018 2:34 pm

Gostev wrote:No material response from VMware Support for 12 days now, so we keep digging the issue on our end and trying to find possible workarounds. It's quite unlikely we will be able to fix it reliably from our side though, as the issue appears to sit in the ESXi itself. So in case VMware confirms this regression, you will likely have to wait for them to ship ESXi 6.5 U2a (or something) before upgrading.
Thanks for your update Gostev. So to be specific, U2 & Veeam only have problems with SQL/Exchange app-aware jobs? Regular jobs are ok? Given our pain with the current version, I'm wondering if we can update vSphere and ESX to U2 on all the hosts where we don't have Exchange and SQL running, and pin those VMs (with host affinity rules) to non-upgraded hosts. Would that work?

Gostev
SVP, Product Management
Posts: 24293
Liked: 3330 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vSphere 6.5U2 support

Post by Gostev » May 24, 2018 4:55 pm

No, you misunderstood the thread. The main issue is this one, and if you upgrade to vSphere 6.5 U2 now, almost all Veeam functionality will start behaving unreliably during periods of heavy environment load because of a regression in one of the cornerstone ESXi APIs that we're using.

There are no other big known issues at the moment that we know about, AFAIK. We've been investigating all reported issues, but so far all of them were either unrelated to the 6.5U2 upgrade, or caused by the API issue above.

Sorenemig
Influencer
Posts: 12
Liked: never
Joined: Jan 05, 2013 4:16 pm
Full Name: Søren Emig
Contact:

Re: vSphere 6.5U2 support

Post by Sorenemig » May 24, 2018 7:56 pm

I upgraded to 6.5U2 by mistake. So far, I have not experienced errors (not what I realize). All backup jobs seem to work fine. All my jobs are simple backup jobs without application integration.

Unfortunately, it is not possible to remove an update. My option is to reinstall all my hosts!

Gostev:

If I understand correctly, it is only under heavy load the issue(s) manifest itself? Will it then be reasanble safe to stay on the current version?

Also, what do you mean by heavy load?

Post Reply

Who is online

Users browsing this forum: Baidu [Spider], Google [Bot] and 29 guests