Failed to create VM recovery checkpoint

[kunniyoor]

Is this issue has not fix yet!!!

We have many customer site facing this issue specially if the setup is having the following configuration

1. Hyper-V 2016 host with updated to the latest.
2. VM's are clustered (C drive as vhdx and other disks for the cluster are vhds and shared between cluster vm's guests.)
3. Whether AAP is enable or disabled.

Initial active full backup run successfully and some incremental too... but later all backups for the vm's failed with error "failed to dispatch pending task" "failed to analyze snapshot availability for task" "failed to create production checkpoint" and final error (Job failed "failed to create check point on collection 'Hyper-V Collection (ID....). Operation failed.") error code 32768.

Any help.... its totally annoying

Krish, BH

[wishr]

Hi Krishnakuamr,

Could you please try creating the production checkpoint manually? Here is the procedure.

Also, this KB might help as well.

Thanks

[kunniyoor]

Cant create checkpoint as the OS vhdx is now showing as .avhdx and getting error "Production checkpoint cant create in the current state"

As mentioned in the KB

we have the following from KB and our settings as follows
------------------------------------------------------------------------------
Please make sure the following requirements are met:

each guest VM should have the cluster feature installed ----- Yes
all members of guest cluster should be on-line ---- Yes
VHDSet shouldn't be used as Cluster Shared Volume (CSV) ---- No CSV attached as disk
folder on CSV containing VHDSet files should have proper permissions settings ----- Valid permission inplace
guest operating system should be Windows 2016 ----- Yes
All VHDSet files must reside on CSV or SMB shares ---- Resides on CSV
Microsoft supports legacy shared VHDX on Windows 2016, but this feature cannot be used like a workaround, because it cannot be included in application-consistent checkpoints, therefore Veeam doesn't support backup of such disks on Windows Server 2016
Otherwise backup job might fail on checkpoint creation with one of the errors listed below.

Solution
Error code: '32768'. Failed to create checkpoint on collection 'Hyper-V Collection'
This issue occurs because Windows can't query the cluster service inside the guest VM. To fix this issue, make sure that the cluster feature is installed on all guest VMs and cluster service is running. ----- Cluster service is installed and running ok.
------------------------------------------------------------------------

Regards

Krish

[wishr]

Hi Krish,

Were you able to create the production checkpoint manually as I mentioned above?

Thanks

[kunniyoor]

No.

it shows error "Cannot take checkpoint for "Server Name" because one or more shareable vhdx are attached and this is not part of a check point collection "vm id"

Krish

[kunniyoor]

But get-collection shows that veeam has created a checkpoint collection called "hyper-v collection" and all disks are member of the said collection.

Krish

[wishr]

No.

it shows error "Cannot take checkpoint for "Server Name" because one or more shareable vhdx are attached and this is not part of a check point collection "vm id"

Krish

Hi Krish,

This means that the root cause is related to Hyper-V itself. You should be able to create production checkpoints manually because B&R utilizes this approach when backing up Hyper-V 2016 VMs. If it does not work, the backup won't work either. In this particular situation, the best way to go would be to create a ticket with Microsoft support, but I'm pretty sure there is some issue with cluster feature inside of the guest OS. Btw, are both the guest OS and Hyper-V components fully patched?

Also, looking at the Windows Event logs on the guest VM and Hyper-V side can possibly shed some additional light on the situation. If you are sure that all the requirements in the aforementioned KB article are met and this is double-checked, feel free to reach out to our support team (do not forget to post your support case ID here for referencing purpose) - our engineers will make sure everything has been done correctly and check the logs as well.

Regards,
Fedor

[alg16ist]

I experienced this issue on one of our SQL servers (2012R2 OS, host is 2016 hyper-v, 32768 error upon failure to create a checkpoint, VSS SQL Writer death afterwards). I couldn't manually create a production checkpoint on it either. Through trial and error, I discovered that I could create a production checkpoint when the MSSQLSERVER service wasn't running. This was both good and bad: good because it meant that there wasn't some problem with the VHDX files, Veeam or hyper-v, bad because that service needs to be running for db/log enumerating and guest processing. Our SQL instance was running 4000+ databases -- we pruned this down to around 1300. The Veeam backup job was then able to complete. I hope this helps someone. I wrestled with this 32768 error for several weeks.

[atmarx]

Still seeing the original issue trying to back up a SBS 2011 server on a Windows 2019 Hyper-V host. Oh well -- AAIP disabled and backing up fine now...

[wishr]

Hi Atmarx,

What errors you are facing? Do you have SP1 installed for your WServer 2008 R2? It's a minimum requirement. Please note that we've not tested B&R 9.5 with SBS 2011 bundle specifically.

Thanks

[johannesr]

Greetings,

I have migrated a customers 2008R2 based Hyper-V server to Server 2019 (migrated like reinstalled from scratch). Everything is fine, the two machines which are hosted are running well, but the SBS backups are creating me headaches. I get the following error:

20.01.2020 20:39:55 :: Failed to create VM recovery checkpoint (mode: Veeam application-aware processing) Details: Job failed ('Error with checkpointprocess for "sbs2011". (ID of virtual computer: insert-my-computer-id-here)
Productioncheckpoints could not be created for "sbs2011". (ID of virtual computer: insert-my-computer-id-here)'). Error code: '32770'.
Failed to create VM recovery snapshot, VM ID 'insert-my-computer-id-here'.

I found articles from years ago where people are having issues like me, but without solutions. I am running the very latest build of VBR.

Any help would be highly appreciated.

Thanks
Johannes

[HannesK]

Hello,
I merged your question to the existing thread. It seems to be a known Microsoft issue. If the answers above do not help, please open a support case and post the case number for reference.

I also edited your title to SBS2011 as the error message seems to be about a 2011 server.

Best regards,
Hannes

[infinexus]

Support Case id #: 04100869

This is my first time setting up Veeam B&R community edition. I downloaded the latest version available online yesterday (v10.0.0.4461 P1) and installed it on my fresh copy of windows server 2019 standard edition, which is solely running the Hyper-V manager role. I fully patched and updated the server prior to installing Veeam B&R CE. The installation was successfully and I proceeded to setup backing up my window server 2019 VM on this same host. I added the backup repository (separate NAS via SMB) and Hyper-V host to the program. The VM host is just a standalone server that will be used to host a SQL database server. The server is empty has doesn't have any software in it yet, as I wanted to test the backup before installing the SQL server software. The VM consists of 2 SCSI hard drives. The 1st drive is a 500GB boot drive (dynamic expanding VHDX) and the 2nd is 2TB (fixed size VHDX).

The backup repository that I have is a NAS that is setup via SMB share. I configured all the share locations and credentials during the wizard and didn't give any errors. This NAS has 24TB of free space. It has read/write permissions, as I see that Veeam created a folder on there to do the backup.

When I run the backup, I received the following error:

Code: Select all

Failed to create VM recovery checkpoint (mode: Crash consistent) Details: Failed to create VM (ID: 9f3a5c6d-d3a5-4135-a667-afcbd718655d) recovery checkpoint. Job failed ('Checkpoint operation for '2019-SQL1' failed. (Virtual machine ID 9F3A5C6D-D3A5-4135-A667-AFCBD718655D)'). Error code: '32768'. Retrying snapshot creation attempt (Failed to create production checkpoint.) Task has been rescheduled Queued for processing at 4/2/2020 5:49:39 PM Unable to allocate processing resources. Error: Failed to create VM (ID: 9f3a5c6d-d3a5-4135-a667-afcbd718655d) recovery checkpoint. Job failed ('Checkpoint operation for '2019-SQL1' failed. (Virtual machine ID 9F3A5C6D-D3A5-4135-A667-AFCBD718655D)'). Error code: '32768'. 

So, I tried the following from searching google/forums with error code 32768 and also "Failed to create VM recovery checkpoint". I couldn't find anyone else with the same setup, so I followed others who used server 2016/2012R2 and had a similar error message. I tried what others suggested:

1. Tried creating a manual checkpoint (no issues and messages at all)
2. Changed production checkpoint to standard checkpoint (still doesn't work)
3. Backing up only the 500GB drive only (still fails)
4. Verified that com+ services on the host 2019 server to see if it's running (it's started and working)
5. Spinning up a brand new 2016 server and setting up the backup on it, still fails with the same error.
6. Removing the NAS backup repository and use the host's own local hard drive to backup the boot drive hard drive (still fails)
7. Uninstall B&R CE 10 and installed B&R CE 9.5u4 (still same error)
8. Tried adding an external 12TB hard drive, set it as a new backup repository and backup to that, still doesn't work.

I've already submitted a case with Veeam yesterday but haven't heard back, so I figure I post on here to see if anyone else has encountered this issue. I've noticed when running the backup, the job keeps stating "waiting for next task" and it does this twice before it fails.

Any help would be appreciated..

[HannesK]

Hello,
and welcome to the forums. I merged your question with one of the existing "error 32768" threads. As you mentioned, there are also some other threads in this forum about that error.

Best regards,
Hannes