Case # 01877898 - ESXi 6.0.0, 4192238 breaks SureBackup

[Christer]

Hi all,

I have a SureBackup job that has worked fine for a long time. After I upgraded our ESXi hosts to 6.0.0, Build 4192238, SureBackup stopped working.
None of the VMs included managed to boot because of a lack of access to storage. When you use remote console to the VMs all you can see is either a _ underscore character, or a big VMware Logo with a message about no storage.

To test where the problem was, I did the following:

Set up a new Veeam 9.0.0.1715 installation
Created a new storage repository, on another storage device
Created a new virtual lab, using a different host, but at the same build level
Set up a new application group
Set up a new SureBackup job

After running the SureBackup job I got the same problem. VM did not get access to storage.

Running a regular Instant VM recovery works fine.

After that I set everything up again, but this time I used a host with ESXi 6.0.0, Build 3825889.
Now the SureBackup job works perfectly.

I have an open supportcase #01877989. The support guy I talked to did not understand what was wrong, so I have asked him to move the case to QA for more testing.

Would be nice if someone could test running SureBackup with Veeam 9.0.0.1715, ESXi 6.0.0, Build 4192238 and vCenter 6.0.0, Build 3634793 (all the latest versions)
We are using iSCSI in our environment. This is not the first time VMware has broken SureBackup, so I believe that the problem lies there.

Kind regards,
Christer Johannessen
Institute of Marine Research

[nothan]

Christer,

I upgraded to ESXi Build 4192238 yesterday (already on the latest versions for the rest of the components) and my Surebackup jobs are failing now too. My issue sounds different than yours though. My application group (DC only) fails to complete the "Starting" status/Power on phase. When I log in and poke around the VM it is trying to start VMWare Tools are not showing in the task tray. When I look at the Windows services the VMWare Tools service is not even listed... so no chance on passing the Heartbeat, Ping, etc phases.

Brett

[Christer]

Maybe I was a bit unclear, but what happens in your case is EXACTLY the same that happens in my case:

All application groups fails to complete the "Starting" status/Power on phase.

Code: Select all

            15.08.2016 14:38:01          dc-troms - Registering
            15.08.2016 14:40:16          dc-troms - Configuring DC
            15.08.2016 15:10:34 Error    dc-troms - Powering on
            15.08.2016 15:10:34          Error: Results: OS did not boot in the allotted time
            15.08.2016 15:21:37          dc-troms - Powering off - This step takes a very long time. In this case 11 minutes.
            15.08.2016 15:21:38          dc-troms - Unregistering
            15.08.2016 15:21:49          dc-troms - Cleaning up redo logs
            15.08.2016 15:21:50          dc-troms - Unpublishing

When I log in and poke around the VM it is trying to start VMWare Tools are not showing in the task tray. - Reason = No access to VM disk files, only snapshot that has 0 data
When I look at the Windows services the VMWare Tools service is not even listed... so no chance on passing the Heartbeat, Ping, etc phases. - Reason = No access to VM disk files, only snapshot that has 0 data

Thanks for testing for me

[nothan]

Please post back what support concludes on your open case.

Thanks,
Brett

[nothan]

Christer,

I updated the VMWare tools on my Domain Controller last night. Today the Surebackup jobs are working, include VM's that are not on the latest version of the VMWare tools.

Are the VMWare tools on your DC up to date with what come with the ESXi 6.0.0 4192238 updates? If not, you may want to give that a try.

Brett

[Christer]

Hi Brett

I will post back what Veeam support comes up with. We are patching the servers tonight, so I will try the job again on monday.

-Christer

[Christer]

Hi

After upgrading VMware tools on the VMs to the latest version, I still have the same problem. Using a host with version 6.0.0, 3825889 works fine, but not 6.0.0, 4192238.
With help from Veeam support I have tried turning off automatic snapshot created on the Surebackup VMs, and then It works:

Code: Select all

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\SureBackup
[DWORD]SnapshotVmRetryCount=0

All VMs in my surebackup job starts, and everything looks ok, but If I try taking a snapshot of one of the VMs, it breaks.

We are waiting for the developers to figure out what is going wrong.

-Christer

[hyvokar]

Any news on this one? Is this a common problem? I'm about to upgrade to patch 3.
I'm using sas and fc.

[Vitaliy S.]

OP's case is still in the research phase.

[Christer]

I am still waiting for this problem to be resolved

[ExpertTrigger]

Veeam, is their any update on this?

[Vitaliy S.]

Robert, please contact our technical team for assistance if you're experiencing the same issue. Do not see any resolution in the OP's case.

[rhiem]

Same Problem here. On Version 4192238 Surebackup won´t work anymore. Is there any Resolution yet?

Best regards.

[Christer]

Hi everyone

I was informed that QA wasn't able to reproduce the issue on the VMware ESXi build 4510822.
After I was told I updated the ESXi host to build 4510822, and this solved the issue for me.
As QA did no testing on ESXi 4192238, I am unsure what the root cause of the problem was, and which VMware patch fixed the issue.

Best regards,
Christer

[rhiem]

Thanks Christer. We will give 4510822 a try.

http://kb.vmware.com/kb/2147329

Best regards.

[lando_uk]

Hi

I'm setting up some new Dell hosts but 4510822 is too new for a customised ISO, so should I just use "VMware-VMvisor-Installer-6.0.0.update02-4192238.x86_64-Dell_Customized-A03.iso" and then install the update, or wait a while for a new Dell ISO?

[Pat490]

use Update Manager. Not sure if Dell will ever update the ISO for 6.0 at all.

[rhiem]

With 4510822 SureBackup works fine now