Discussions specific to the Microsoft Hyper-V hypervisor
aj_potc
Influencer
Posts: 13
Liked: 2 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Failed to create VM recovery checkpoint

Post by aj_potc » Sep 19, 2018 11:22 am

I'm running the free edition of Backup and Replication 9.5 update 3a on a Windows Server 2016 Hyper-V host. I'm trying to back up two guests (both also Windows Server 2016).

One of the two guests is intermittently throwing the following errors:

Code: Select all

Failed to create VM recovery checkpoint (mode: Hyper-V child partition snapshot) Details: Job failed ('Checkpoint operation for VM failed.

Code: Select all

Unable to allocate processing resources. Error: Job failed ('Checkpoint operation for 'VM' failed. 

Production checkpoints cannot be created for 'VM'. Error code: '32770'.
Failed to create VM recovery snapshot, VM ID '927335d9-05c4-4936-b25a-405fcb6fd0da'.  

In the Event Log of the Hyper-V host, I can also find the following entry when the backup process is attempted:

Code: Select all

Production checkpoints cannot be created for 'VM'. 
I've checked here on the forums and have found a number of reports of Hyper-V issues (mostly related to non-Server 2016 guests), but I haven't found any concrete solutions.

Does anyone know why I would have an issue with production checkpoints, and why this would cause Veeam to fail?

Thank you very much for any help.

wishr
Veeam Software
Posts: 216
Liked: 22 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Failed to create VM recovery checkpoint

Post by wishr » Sep 19, 2018 2:31 pm

Hi Aj_potc,

In certain circumstances, this issue can appear due to a bug in Hyper-V checkpoint creation processes. Could you please check if you are able to create a checkpoint for this machine using Hyper-V Manager? If this doesn't help I would recommend reaching out Microsoft support for assistance.

If you are be able to create a checkpoint manually using Hyper-V Manager, then here is a suggestion by one of the Veeam users that might be helpful, but I would definitely recommend you to raise a support case instead of digging into the issue on your own.

Thanks,
Fedor

aj_potc
Influencer
Posts: 13
Liked: 2 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Failed to create VM recovery checkpoint

Post by aj_potc » Sep 19, 2018 4:20 pm

Hi Fedor,

Thanks for your kind reply. I do indeed believe it to be a bug in Hyper-V -- I never really suspected Veeam was the problem.

Unfortunately I wasn't able to try your suggestion to create a checkpoint manually before I rebooted the VM. After the reboot, the backups now work correctly. While I'm happy that resolved the problem, I really hate that kind of "fix," because I have no idea where the real issue was, or if it will appear again. I thought that Windows Server 2016 and Hyper-V were very stable by now, but maybe not!

Thanks again for your help.

Best,

A.J.

Mike Resseler
Veeam Software
Posts: 5000
Liked: 530 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Failed to create VM recovery checkpoint

Post by Mike Resseler » Sep 20, 2018 5:37 am

A.J.

What I mostly see when this occurs, is that one of the VSS writers got stuck. A reboot then resolves it. It probably isn't even the Hyper-V writer but it could be the SQL writer or a 3rd party VSS writer. If this happens again, before rebooting, run vssadmin list writers (elevated command prompt) in the VM to see if one of the writers is stuck. Mostly that will give you a better view of what is causing issues

My 2 cents
Mike

aj_potc
Influencer
Posts: 13
Liked: 2 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Failed to create VM recovery checkpoint

Post by aj_potc » Nov 18, 2018 9:21 pm

Hi Mike and Fedor,

I've since had this issue come up again, and this time I've decided to dig into it more deeply. I've opened support case 03291211.

I have tried creating a checkpoint manually -- that works fine. I also tried Mike's suggestion of checking on the VSS writers in the VM itself. All of them report no errors.

I didn't try the suggestion in Fedor's link, as my guest is Server 2016, and this suggestion discusses taking files from a Server 2012 ISO.

Thanks for any further thoughts/ideas!

wishr
Veeam Software
Posts: 216
Liked: 22 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Failed to create VM recovery checkpoint

Post by wishr » Nov 19, 2018 12:09 pm

Hi AJ,

Thanks for replying.

This looks strange, though. Our support team will check the logs first and reach you back, we'll track it as well.

In the meanwhile, could you please make sure that your Guest OSs are updated to the latest version? Also, It might be useful to know which Integration Services version the failed VMs are running. There is a how-to guide on that by the aforementioned link.

Thanks

aj_potc
Influencer
Posts: 13
Liked: 2 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Failed to create VM recovery checkpoint

Post by aj_potc » Nov 21, 2018 10:38 pm

Thanks for your reply.

So far I haven't had much luck troubleshooting the problem.

I've checked the version of Integration Services on the Windows Server 2016 guest VM, and it shows:
10.0.14393.2007

I believe this should be the latest version, but it's not clear to me how that could be updated (maybe automatically via Windows Update?).

Thanks for any other help or feedback.

wishr
Veeam Software
Posts: 216
Liked: 22 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Failed to create VM recovery checkpoint

Post by wishr » Nov 22, 2018 11:05 am

Hi AJ,

As far as I see the manual checkpoint creation operation suggested by our support team has failed with the same error message. In that case, we recommend reaching Microsoft Support for assistance - probably some specific reinstallations/reconfigurations will be required on the VM/Hyper-V end.

We will really appreciate if you could share the final solution by Microsoft that will be helpful, so everybody experiencing the same issue would be aware of it.

Thanks

collinp
Expert
Posts: 133
Liked: 10 times
Joined: Feb 14, 2012 8:56 pm
Full Name: Collin P
Contact:

Re: Failed to create VM recovery checkpoint

Post by collinp » Nov 22, 2018 10:17 pm

Is the error 32768 and are you using smb?

mark14
Novice
Posts: 9
Liked: 1 time
Joined: Nov 23, 2018 9:38 am
Full Name: mark14
Contact:

[MERGED] Backing up Linux VM's

Post by mark14 » Nov 23, 2018 10:03 am

Hi,

Currently we have Veeam Backup & Replication for Hyper-V and it is running as Virtual Machine on Hyper-V server. I want to backup linux virtual machine with MySQL database.

I just follow article:
https://helpcenter.veeam.com/docs/backu ... tml?ver=95

and create pre-freeze and post-thaw scripts:
https://bp.veeam.expert/applications/mysql

Backup Job has been created:
check enable applicaiton-aware processing
In Applications -> Scripts I added Pre-freeze and Post-thaw script
added guest os credentials (linux root credentials over 22 SSH)

But it is not working. Logs below:

machine ID A26BC399-7BEE-4172-A174-669AD0056C97) Production checkpoints cannot be created. Error code: '32770'. Failed to create VM recovery snapshot.
Retrying snapshot creation attempt (Failed to create production checkpoint.)
Task has been rescheduled
Unable to allocate processing resources. Error: Job failed.Production checkpoints cannot be created.Error code: '32770'. Failed to create VM recovery snapshot.

wishr
Veeam Software
Posts: 216
Liked: 22 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Failed to create VM recovery checkpoint

Post by wishr » Nov 23, 2018 10:34 am

Hi Mark,

Welcome to Veeam Community Forums and thank you for posting.

I've merged your post into a similar topic. Could you please take a quick look at it and let me know if you have any additional questions.

Thanks

aj_potc
Influencer
Posts: 13
Liked: 2 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Failed to create VM recovery checkpoint

Post by aj_potc » Nov 26, 2018 3:30 am

I've made a little more progress researching the checkpoint error 32770.

On the Server 2016 guest VM, I noticed that there are a number of services related to Hyper-V. I suppose these are the Integration Services. All of them are set to Manual startup, and several were not running. One of these was:

Hyper-V Volume Shadow Copy Requestor

After I manually started the service, I could then create Hyper-V checkpoints and run Veeam backups as expected from the host system. Very interesting.

I suppose I'll have to investigate why this service was stopped, or how I can monitor it in the future.

Does anyone have any experience with this odd situation?

wishr
Veeam Software
Posts: 216
Liked: 22 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Failed to create VM recovery checkpoint

Post by wishr » Nov 26, 2018 11:10 am

Hi AJ,

Thanks a LOT for sharing this. Personally, I haven't seen that before.

@Mark, could you please check your services state? Maybe your root cause is similar...

mark14
Novice
Posts: 9
Liked: 1 time
Joined: Nov 23, 2018 9:38 am
Full Name: mark14
Contact:

Re: Failed to create VM recovery checkpoint

Post by mark14 » Nov 28, 2018 12:55 pm

@wishr
As I said I am trying to backup Linux Virtual Machine (Debian 8.0 with MySQL). Do I need extra drivers/packages installed on machine? Also I can create now checkpoint Linux VM from Hyper-V

wishr
Veeam Software
Posts: 216
Liked: 22 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Failed to create VM recovery checkpoint

Post by wishr » Nov 28, 2018 1:34 pm

Hi Mark,

Hyper-V should embed all the required packages automatically in any guest OS. From Veeam perspective you do not require to install any additional packages either.

Could you please try to check the VM settings in Hyper-V and make sure that the checkbox called "Create standard checkpoints if it's not possible to create a production checkpoint" underneath the "Production checkpoints" option is unchecked (if not, please do so), then try to create a checkpoint manually one more time and let us know the result? Please do not forget to revert the setting back once done.

Thanks

Post Reply

Who is online

Users browsing this forum: spgsit5upport and 3 guests