Host-based backup of VMware vSphere VMs.
Post Reply
benasse
Lurker
Posts: 2
Liked: never
Joined: Jul 12, 2017 9:45 am
Full Name: benasse
Contact:

veeam backup freeze may debian linux

Post by benasse »

Hi,

About one night out of three, my virtual machines running Debian 8 are freezing during the backups execution.

According to the logs of the kernel, this would be due to rcu_sched.

I use the following kernel :

Code: Select all

Linux srv-xivo1 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux
Here is the stacktrace :

Code: Select all

Jul  8 22:35:04 srv-xivo1 kernel: [105129.524054] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 2, t=10028 jiffies, g=3028935, c=3028934, q=4)
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524119] sending NMI to all CPUs:
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524130] NMI backtrace for cpu 2
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524133] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G           O  3.16.0-4-amd64 #1 Debian 3.16.39-1+deb8u2
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524134] Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1506250318 06/25/2015
Do you encounter similar problems, and how do you remedy it?

Regards
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: veeam backup freeze may debian linux

Post by PTide »

Hi,

The hypervisor where you run the VMs - is it highly populated? Also, do all of the VMs are backed up at the same time, or by portions?

Thanks
benasse
Lurker
Posts: 2
Liked: never
Joined: Jul 12, 2017 9:45 am
Full Name: benasse
Contact:

Re: veeam backup freeze may debian linux

Post by benasse »

Hi,

There is about 30 vm on the hypervisor, and all the backup are in the same time.

Regards
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: veeam backup freeze may debian linux

Post by PTide »

Do you have any quiescence/AAIP options enabled for Debian VMs? Since it's VMware who guides snapshot creation process, I suggest you to open support cases with both VMware and Veeam. Please don't forget to post your case ID.

Thank you!
iwik
Influencer
Posts: 20
Liked: 4 times
Joined: Apr 02, 2014 11:10 am
Contact:

Re: veeam backup freeze may debian linux

Post by iwik »

Hi,

I have just found this thread, when searching after Debian 8 freeze during snapshot:

Code: Select all

An error occurred while saving the snapshot: 
msg.snapshot.error-QUIESCINGERROR.
vmware.log

Code: Select all

2017-07-26T04:34:44.253Z| vmx| I120: SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Failed to quiesce the virtual machine (40).
Guest is running kernel

Code: Select all

3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)
CPU load graph after snapshot shows 100%. We are probably hitting some bug, which is very difficult to reproduce. :?
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: veeam backup freeze may debian linux

Post by PTide »

Hi,

Do you have quiescence or AAIP enabled? If so then have you installed VMware Tools in the guest OS?

Thank you
iwik
Influencer
Posts: 20
Liked: 4 times
Joined: Apr 02, 2014 11:10 am
Contact:

Re: veeam backup freeze may debian linux

Post by iwik »

PTide wrote:Hi,

Do you have quiescence or AAIP enabled? If so then have you installed VMware Tools in the guest OS?

Thank you
Yes, quiesce is enabled. It seems this vm have older vm tools 8.6.11.26309 (build-1310128), we can try upgrade (or better replace by open-vm-tools)
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: veeam backup freeze may debian linux

Post by PTide »

Yes, please try to upgrade vm tools and if that won't help please contact our technical support team.

Thank you
Post Reply

Who is online

Users browsing this forum: Gostev and 75 guests