Veeam v10 Upgrade woes

[micoolpaul]

Evening all,

Case: 04106828

Has anyone else had multiple bad experiences with the Veeam v10 upgrade? I don't say it lightly, but I've never had any real issues in previous Veeam upgrades, I've had multiple colleagues all have Veeam 9.5 4b's to v10's go bad. Each one has had to result in a rebuild.

Today I had a planned v9.5 U4b to v10 upgrade. Took a config backup, ran the upgrade, after some messages about services not stopping and me manually stopping them before retrying the operation, it got to the end of the upgrade progress bar, then slowly moved its way back, finally prompting it had failed the upgrade.

It didn't just fail the upgrade, it uninstalled Veeam...

A reboot later and I thought I'd retry, now Veeam only thought I had the console installed, nothing else.

So I uninstalled the console, rebooted again, installed a fresh Veeam v10 install, attempt to restore the config backup when I get a lovely error, telling me that the backup was created in a newer version of Veeam.

Spoke to support and they couldn't get the config backup to be read and the advice was, use a fresh v10 install or uninstall v10 and revert back to 9.5 u4b and then try the v10 upgrade at a later date.

I'm more than a bit surprised that Veeam won't let you import older config backups of even the one version difference.

I've just finished uninstalling Veeam v10, rebooting the server atm before going to reinstall Veeam v9.5 U4b and hope I can import this config, otherwise this is going to be a very long night...

[foggy]

Hi Michael, looks totally unexpected, including the fact that configuration backup import didn't work. We always maintain the ability to import configuration created several versions back. Let's see if 9.5u4b chews it up, a deeper investigation would be required otherwise. Do you maybe have a conf backup created some days earlier to try in this case?

[micoolpaul]

Hey!

Appreciate the fast response, found that the customer in question didn't have configuration backup scheduled so I took a config backup specifically for this purpose (and enabled the schedule to do so of course).

Currently it's deploying the SQLServer 2016 Express Instance, will update this post with any relevant updates.

Glad to hear it's unexpected though!

[Gostev]

Hi, Michael -

I'm sorry for your bad experience with v10, it's unfortunate you had a bad luck.

In general, feedback on v10 upgrades has been quite stellar so far, and hands down better than with every other release in the history of Veeam. To be honest, your post is actually the first negative post in 2 months and 115000 downloads. This fact alone tells a lot, I think!

Of course, environment-specific issues are not impossible. Perhaps your and your colleagues' environments have something in common that throws the setup off. I hope our support engineers will be able to identify the actual issue as opposed to just recommending clean install or rollback, as only in this case we will be able to address your scenario and ensure future upgrades work fine in your environments. I will ask support management to pay a closer attention to your case.

Thanks!

[micoolpaul]

Hi Gostev,

Thanks for this and please don't read this the wrong way, I love Veeam, it's awesome! I only raise this because I want Veeam to stay #1.

One was a vSphere environment and the other two were Hyper-V environments that come to mind. But thanks for ensuring the support is there

SQL still deploying at present...

[micoolpaul]

Gonna pick this one up in the morning with support, taken multiple reboots and attempts and manual intervention but finally got 9.5 u4b back on the system, though there are still some issues from the downgrade I need to iron out. Feel it's more appropriate to pass this to support and let them feed back to everyone what the take aways are from this failed upgrade.

[micoolpaul]

Just a quick update here. After a lot of TLC from support, I'm now on v10 pending a reboot. There were some issues downgrading back to v9.5 such as the installer service which should all be within the support case to help others in future migration works but to be honest, in a world where upgrading is a one way street normally, kudos to Veeam for handling it so cleanly.

[shawnperez]

My first lengthy reply was lost so I will keep it short. I have seen this exact issue 2 or 3 times with v10 upgrade. I was also surprised as something like this has never happened before in many years of upgrades. Fix was to uninstall all components then install and point to existing database.

This is likely a larger issue that people are just working through.

[veremin]

Hi, Shawn,

And welcome to our community forums.

Big part of QA team is inactive today due to public holiday in Czech Republic, but if you provide us with the ticket number, we will double check the issue tomorrow.

Thanks!

[aich365]

Hello Michael
We are a VCSP an have upgraded to V10 and had EM installed.
The EM install failed until we rebooted the Veeam server and started again.
EM updated OK but the VCC install failed with an NFSDatastore error.
The licensing was confusing because we had V10 licence for EM and V9.5 for VCC
The VCC install was referencing an old Datastore on a T: drive that had been removed and deleted. Unfortunately we were unable to remove this Mount reference from one of the datastores. In the end we removed the customer access to the Backup Repository on the datastore and removed the repository.

We still kept getting the Datastore error.
There was no entry under
HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\NFS

In the end we had to create a local datastore and edit the registry key HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Veeam\Veeam NFS to
C:\ProgramData\Veeam\Backup\NfsDatastore\

Once this completed the upgrade finished successfully

=========
In addition we have upgraded some of our larger customers.
As noted by Veeam -
Backup Veeam Database, Backup Config, Upgrade VeeamOne First, then upgrade VBR.

We had issues where Veeam could not find the old installation to uninstall/upgrade and support had to assist manually copying the old install files to the server.
We got stuck with parts of 9.5.4b and parts of V10

With customers with Veeam Agent we have had to install Agents and Transports manually in some cases.

Regards