Backing up a Windows Failover Cluster with Shared vhdx?

[Mike Resseler]

Hi Acolada,

First: Welcome to the forums
Second: Have you created a support case for this? Honestly I doubt that it is Veeam that causes the problem but since we are involved, I wouldn't mind that our engineers look into this further. What you describe is not normal in my mind and besides us looking into this, I hope you can let MSFT look into it also.

Thanks
Mike

[nmdange]

Sounds like there are still problems https://blog.workinghardinit.work/2017/ ... rver-2016/

[Mike Resseler]

Yes,

And while we have a support case with MSFT, we always encourage others to do the same... Hoping that if there are more people, there is more pressure to get this solved faster...

[Kage]

hi guys,

I proposing to my end user to use Microsoft Failover Cluster with 2 Windows Server 2016 servers to make high availability. inside will have vm as file servers and also vm as application server. Basicly it is a 2 nodes with hyper-v clustering wit one shared storage connecting via iscsi.

Here is my architecture:

Main site:
2 Windows 2016 with Hyper-v roles

Backup infrastructure:
Veeam Backup & Replication for Hyper-V 9.5 Update 3
Veeam agent for windows 2.1

My goal :
to backup all vm with image-level backup. certain vm will running SQL with 2008 r2 and 2012 r2 guest OS.


1) Is it possible?
2) Shared storage of file servers can be a VHDX? do i need to setup vm as vhd set?
3) Should i use RDM for file server?
3) VAW 2.1 is now support failover cluster, as i read. so its need to install on host or inside guest vm?
4) if yes (install inside guest vm), then i need to have enough agent for all vm?

please advise

[Mike Resseler]

Hi Kage,

You won't be able to backup all those VM's with image-level backup. The problem resides in the guest clusters with shared VHDX (VHD set). Because of issues we still can't protect it. This is tracked with MSFT and they already brought updates for it but it still not solved. Everyone is having this issue and we are all hoping that this will get solved soon. See here for more details: https://blog.workinghardinit.work/tag/vhd-set/

This means that in the current time, you can protect all VM's without shared disk image-level, and the other ones you can install VAW in-guest. That will allow you to protect those failover clusters.

Hope it helps
Mike

[nmdange]

I saw this in the update 3 release nodes, would it allow the vhd set to be skipped?

Hyper-V 2016 VMs with non-snapshottable disk such as disks connected via in-guest iSCSI initiator can now be processed by backup and replication jobs, with unsupported disks skipped and their content not included in a backup or replica. If you’d rather prefer the job to fail on such VMs (legacy behavior), create HyperVIgnoreNonSnapshottableDisks (DWORD) value under HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication registry key on the backup server.

[spgsit5upport]

So basically 2016 is still unusable, right?

I am on 2012 R2 where at least I can have shared vmdx backed up in crash consistent mode (for backup & as workaround for DR - due to ne replication possible)

I simply can not move to 2016 in that case!

[RGijsen]

So basically 2016 is still unusable, right?

I am on 2012 R2 where at least I can have shared vmdx backed up in crash consistent mode (for backup & as workaround for DR - due to ne replication possible)

I simply can not move to 2016 in that case!

Avoid it at all cost. We made the move to 2016 and after one and a half year of tickets, gigabytes of useless logs, tiresome idiotic questions and so on we are STILL unable to backup our guest-clusters. Something that worked at least reasonably in 2012R2. I can't understand how MS think in these things. They invent a complete new mechanism for shared disks, VHDS, with some nice feautres and some limitations, fair enough. But the ability to properly backup should be one of the main concerns imho. VMs are backed up in block-level, no-one smart would consider to backup from within the guest itself, unless a very good reason. Hence that should just work. In addition, we are being promised fixes for this for over a year now. Still no cigar.

Don't build your production environment on that. We are even considering moving back to 2012R2.

[bplancic]

Hi,

my setup is this - hyper-v 2016 cluster with two WS 2016 VMs which make up SQL cluster. SQL cluster uses shared disks. What would be the way to backup this SQL cluster? If I'm not mistaken application aware processing won't work because of snapshot problems with shared disks but I would like to have some kind of backup - SQL databases will be backed up using scripts. Should I backup only system drives of cluster servers and leave out vhds disks (which seems kind of pointless...), could I use Veeam Agent...

BTW I opened a case with MS (case 118042018042101) and got an answer that they are working on it and that should be relased with WS 2019

Thank you in advance

[spgsit5upport]

@RGijsen

Thanks for this. Sadly that is what I expected. But at least this allows me to plan accordingly

Seb

[Jan1987]

Hello

Is the bug still there?

We have a virtual server 2016 SQL Failover Cluster with VHD Set disk.

The backup fails because ConfigStoreRootPath is not set. But I have set that.

The SQL Cluster is running on a Hyper-V 2016 cluster.

Best Regards,
Jan

[jzilak]

Here is info how to backup VHD Set .VHDS (not shared VHDX).
- VM config need to be 8.0
- ConfigStoreRootPath parameter on "Virtual Machine Cluster WMI" Cluster resource group is configured to shared folder writable to all Hyper-V cluster nodes
(CSV or SOFS Share). VM Group info is stored at subfolder %ConfigStoreVhdSetsLocation%\Groups
- grant FullControl to VHDS Folder for All VMs virtual account shared this VHD Set (NT VIRTUAL MACHINE\VmID)

Aftrer that configurations steps VMs with VHD Sets can be backuped with Veeam 9.5 Update 3.
To delete orphaned VM Groups from crashed Veeam backup jobs, log to HV hosts whre VMs with VHD Set are hosted and use Get-VMGroup | Remove-VMGroup cmdlet.

[gmajestix]

Looks like now also a KB exists https://www.veeam.com/kb2709 and https://support.microsoft.com/en-us/hel ... ack-up-vms. Can anyone confirm this is working?

[Mike Resseler]

Hey gmajestix,

Yes, but take notes at all of the requirements in our KB for it to work.

[chris-au]

Hi,

I've tried setting this up again, ensuring that all requirements as pointed out in the KB are met, however I'm getting a different error than those listed. When the backup job runs on Veeam I am getting the following error on the first node

Error: [WMI] Empty result. Failed to add VM to collection.

Does anyone have any ideas what might be causing this error? WMI appears to be functioning correctly between the Veeam server and the guest cluster nodes.

[foggy]

Hi Chris, assistance from technical support is definitely required here, please open a case. Thanks!

[spgsit5upport]

Where are we with this?
On Server 2016 HOST does it work to backup VM shared vhdx ?

I assume no point to ask about Server 2019

[foggy]

Hi Seb, please review this post above. Thanks!

[spgsit5upport]

But I am NOT talking about sets. I need to move existing VMs, and they are shared vhdx
Later maybe I can convert them to sets, but not instantly the same moment

Seb

[spgsit5upport]

Well, moved to ALL Server 2016 (to be all good) and good I am NOT.
Just like here - https://blog.workinghardinit.work/2017/ ... rver-2016/

all I get is this:

The other scenario we run into is that the backups do work but the recovery checkpoints never ever merge. Not even when you shut down the all the guest VM cluster nodes and start them. With frequent backup that turns into a disaster of a never ending chain of recovery checkpoints.

So after ONE backup I just gave UP on using Veeam for the backup (I really do not care that it is Microsoft fault)
Client backup seems to be the only sensible way here

Yes, of course I DID follow - https://www.veeam.com/kb2709

[GabesVirtualWorld]

Would the VEEAM Agent be of any help in shared VHDX backup?