veeam backup freeze may debian linux

Discussions specific to VMware vSphere hypervisor

veeam backup freeze may debian linux

Veeam Logoby benasse » Wed Jul 12, 2017 10:05 am

Hi,

About one night out of three, my virtual machines running Debian 8 are freezing during the backups execution.

According to the logs of the kernel, this would be due to rcu_sched.

I use the following kernel :
Code: Select all
Linux srv-xivo1 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 GNU/Linux


Here is the stacktrace :
Code: Select all
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524054] INFO: rcu_sched detected stalls on CPUs/tasks: { 0} (detected by 2, t=10028 jiffies, g=3028935, c=3028934, q=4)
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524119] sending NMI to all CPUs:
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524130] NMI backtrace for cpu 2
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524133] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G           O  3.16.0-4-amd64 #1 Debian 3.16.39-1+deb8u2
Jul  8 22:35:04 srv-xivo1 kernel: [105129.524134] Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.0.B64.1506250318 06/25/2015

Do you encounter similar problems, and how do you remedy it?

Regards
benasse
Lurker
 
Posts: 2
Liked: never
Joined: Wed Jul 12, 2017 9:45 am
Full Name: benasse

Re: veeam backup freeze may debian linux

Veeam Logoby PTide » Wed Jul 12, 2017 10:34 am

Hi,

The hypervisor where you run the VMs - is it highly populated? Also, do all of the VMs are backed up at the same time, or by portions?

Thanks
PTide
Veeam Software
 
Posts: 3300
Liked: 274 times
Joined: Tue May 19, 2015 1:46 pm

Re: veeam backup freeze may debian linux

Veeam Logoby benasse » Wed Jul 19, 2017 9:58 am

Hi,

There is about 30 vm on the hypervisor, and all the backup are in the same time.

Regards
benasse
Lurker
 
Posts: 2
Liked: never
Joined: Wed Jul 12, 2017 9:45 am
Full Name: benasse

Re: veeam backup freeze may debian linux

Veeam Logoby PTide » Wed Jul 19, 2017 10:39 am

Do you have any quiescence/AAIP options enabled for Debian VMs? Since it's VMware who guides snapshot creation process, I suggest you to open support cases with both VMware and Veeam. Please don't forget to post your case ID.

Thank you!
PTide
Veeam Software
 
Posts: 3300
Liked: 274 times
Joined: Tue May 19, 2015 1:46 pm

Re: veeam backup freeze may debian linux

Veeam Logoby iwik » Wed Jul 26, 2017 11:37 am

Hi,

I have just found this thread, when searching after Debian 8 freeze during snapshot:

Code: Select all
An error occurred while saving the snapshot:
msg.snapshot.error-QUIESCINGERROR.


vmware.log
Code: Select all
2017-07-26T04:34:44.253Z| vmx| I120: SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Failed to quiesce the virtual machine (40).


Guest is running kernel

Code: Select all
3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)


CPU load graph after snapshot shows 100%. We are probably hitting some bug, which is very difficult to reproduce. :?
iwik
Influencer
 
Posts: 17
Liked: 2 times
Joined: Wed Apr 02, 2014 11:10 am

Re: veeam backup freeze may debian linux

Veeam Logoby PTide » Fri Jul 28, 2017 9:53 am

Hi,

Do you have quiescence or AAIP enabled? If so then have you installed VMware Tools in the guest OS?

Thank you
PTide
Veeam Software
 
Posts: 3300
Liked: 274 times
Joined: Tue May 19, 2015 1:46 pm

Re: veeam backup freeze may debian linux

Veeam Logoby iwik » Mon Jul 31, 2017 7:11 am

PTide wrote:Hi,

Do you have quiescence or AAIP enabled? If so then have you installed VMware Tools in the guest OS?

Thank you


Yes, quiesce is enabled. It seems this vm have older vm tools 8.6.11.26309 (build-1310128), we can try upgrade (or better replace by open-vm-tools)
iwik
Influencer
 
Posts: 17
Liked: 2 times
Joined: Wed Apr 02, 2014 11:10 am

Re: veeam backup freeze may debian linux

Veeam Logoby PTide » Mon Jul 31, 2017 8:24 am

Yes, please try to upgrade vm tools and if that won't help please contact our technical support team.

Thank you
PTide
Veeam Software
 
Posts: 3300
Liked: 274 times
Joined: Tue May 19, 2015 1:46 pm


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 1 guest