Backing up Linux Virtual Machines

[Itdweeb99]

We are backing up some windows virtual machines and also three linux virtual machines that reside on a Hyper V server. Backups for all 3 linux virtual machines are failing with the following error message:

Failed to create VM recovery checkpoint (mode: Veeam application-aware processing). Production checkpoints cannot be create for SERVER1. Error Code '32770'.

I can't seem to find any information on this error and if I do - the person is getting the error for a windows vm and not a linux vm. Any help would be appreciated.

[Mildur]

Can you create a checkpoint by yourself of this vm in the hyperv console?

That is always the first test I do, when a hyperv production checkpoint is failing in a veeam backup job.
Veeam is not responsible for doing the checkpoint. Veeam tells the hyperv server todo the snapshot and wait for a successfull message, before the backup job runs the next step.

[Itdweeb99]

Yes, I can create a successful checkpoint in Hyperv for all 3 machines.

[Shaokat]

please create checkpoint for all 3 machine and try again. let us the what's the outcome.

[markt]

I've just managed to solve this issue in my own case - running Hyper-V Server 2019 with a variety of Linux VMs. It seems that, although the Linux Integration Services for Hyper-V are now included in most Linux Kernels, the daemons responsible for VSS (Volume Shadow Copy Service) and Key-Value Pair Exchange are not.

You can confirm this by running the following command in PowerShell on your Hyper-V Host:

Code: Select all

Get-VMIntegrationService -vmname <yourvmname>

You will see "No Contact" under the PrimaryStatusDescription next to "Key-Value Pair Exchange". You will see "OK" next to VSS but, because the daemon is not installed, VSS will not work.

The consequence of not having the VSS Daemon is that backups will fail if you have checked Hyper-V Guest Quiescence but unchecked Take crash-consistent backup under the job > Storage > Advanced > Hyper-V. Checkpoints will also fail in Hyper-V Manager, but only if you have unchecked "Create standard checkpoints if the guest does not support creation of production checkpoints" in the VM settings under Checkpoints.

The consequence of not having the Key-Value Pair Exchange is that Veeam will not be able to determine the IP address of the virtual machine if you enable application-aware processing, enter linux credentials and "Test Now".

The fix seems to be to install a package that provides hv-vss-daemon and hv-kvp-daemon.

On CentOS, it's hyperv-daemons (e.g. sudo yum install hyperv-daemons).
On Ubuntu, it's linux-cloud-tools-common (e.g. sudo apt install linux-cloud-tools-common).

Once installed, reboot the Linux VM and try running the Get-VMIntegrationService command above, again. You should see "OK" next to "Key-Value Pair Exchange".

With any luck, your backups should now complete successfully.

[QLink]

Had the same problem with Hyper-V 2019 Host and Ubuntu 22.04 LTS guest.


install azure optimized kernel did the trick for me:

Code: Select all

sudo apt-get install linux-azure

then reboot:

Code: Select all

sudo shutdown -hr 0

maybe it helps someone...

best regards

[Mildur]

Thanks for sharing your solution, QLink.

[QLink]

Update on this:

if you update ubuntu 22.04 LTS with following commands:

Code: Select all

sudo apt-get update
sudo apt-get upgrade
sudo reboot

you may break Linux Integration Services for Hyper-V again.
As a result production snapshots are no more possible to take and application aware backups will fail.

In order to get it working again you have to install latest linux-cloud-tools.
To check which version is recommended type

Code: Select all

sudo hv_kvp_daemon

which will tell you to also install latest linux-cloud-tools.

In my case:

Code: Select all

sudo apt-get install linux-cloud-tools-5.19.0-32-generic
sudo reboot

After reboot production snapshots worked again and so does veeam application aware backup.

best regards

[QLink]

two months later, same problem again after an update,

this time following commands did the trick for me:

Code: Select all

sudo apt-get install "linux-cloud-tools-$(uname -r)"
sudo reboot