-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
msg.snapshot.error-QUIESCINGERROR
Hi there,
I have a Linux SLES11 VM that has been backing up successfully via B&R for some time. Last week we added an application to the VM; GlusterFS to be precise and now every time a backup is performed it bombs out with the VMware error "msg.snapshot.error-QUIESCINGERROR". Disabling VMware tools quiescence allows the backup to be performed successfully. This is on vSphere 5.5 with the latest VMware tools installed and B&R v8p2.
I'm just wandering if anyone else has come across similar issues when quiescing a Linux guest OS? All of the VMware KB's with similar errors point to Windows VSS issues so aren't helpful.
Thanks
I have a Linux SLES11 VM that has been backing up successfully via B&R for some time. Last week we added an application to the VM; GlusterFS to be precise and now every time a backup is performed it bombs out with the VMware error "msg.snapshot.error-QUIESCINGERROR". Disabling VMware tools quiescence allows the backup to be performed successfully. This is on vSphere 5.5 with the latest VMware tools installed and B&R v8p2.
I'm just wandering if anyone else has come across similar issues when quiescing a Linux guest OS? All of the VMware KB's with similar errors point to Windows VSS issues so aren't helpful.
Thanks
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Are you able to reproduce this issue by taking manual snapshot with quiescence enabled via vSphere Client?
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Its a production box so I cant test it. Just rigging up a test box now to see what happens.v.Eremin wrote:Are you able to reproduce this issue by taking manual snapshot with quiescence enabled via vSphere Client?
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
So, you can backup VM in question which also requires snapshot to be created, but cannot create that snapshot manually? The tests conducted on similar testing VM might necessarily show VM specific issue. Thanks.
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
I forgot to mention that the quiescence snapshot causes Linux to crash and requires a hard reboot, which is why im now trying to recereate the issue on a test box.
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
So, if the quiescence snapshot conducted outside of VB&R results in VM crash, may be it's time to contact VMware support team and let investigate it. Thanks.
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
I haven't tried a snapshot outside of VB&R, as I said its a production box so I'm working on recreating a test environment. I havent approached VMware support since they only offer break/fix assistance not fault finding. I'm not really looking for any support, just curious if anyone else has had similar experiences and under what circumstances. Thanks.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
You seem to be the first one to report such issue, so if you want to find the reasons and prevent this behavior, contacting support looks to be the most effective way of further action.
-
- Lurker
- Posts: 1
- Liked: never
- Joined: May 28, 2015 2:21 pm
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
I just wanted to chime in that I am also having this issue with SLES 11 VMware virtual machines on using a different backup solution. Attempting to create a snapshot within the vSphere client and using the quiesce option does not appear to reproduce the issue. I've been working with VMware technical support for close to two months to resolve this issue. Technical support suggested to update ESX 5.5 to the recently published Patch 6, however, QUIESCINGERRORs still occur seemly at random after patching the hosts.
I'm curious to know too if anyone else has had this issue occur to them and if they were able to go about resolving it. I'm also curious to know if anyone hears anything differently from VMware technical support.
I'm curious to know too if anyone else has had this issue occur to them and if they were able to go about resolving it. I'm also curious to know if anyone hears anything differently from VMware technical support.
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Hi kirchk, thanks for the input. Its interesting that you are also seeing the same issue on SLES. Would you mind letting me know what applications and filesystems are running on the affected VM's?
On a side note, I have set up a test VM and been able to recreate the issue. Interestingly the issue occurs when performing a Veeam backup but not when taking a snapshot directly through the vSphere client.
On a side note, I have set up a test VM and been able to recreate the issue. Interestingly the issue occurs when performing a Veeam backup but not when taking a snapshot directly through the vSphere client.
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Probably, I have misinterpreted your answer. If the issue is reproducible only while VB&R is present, please, open a case with our support team and let them analyze the problem directly. Thanks.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Guys, could you please check if the affected VMs have disk.enableUUID parameter set to true in the vmx file and if yes, try to backup with it set to false?
And yes, we would be grateful if you open cases with our support as well and provide their ID's for reference. Thanks!
And yes, we would be grateful if you open cases with our support as well and provide their ID's for reference. Thanks!
-
- Novice
- Posts: 3
- Liked: never
- Joined: Jun 02, 2015 7:39 am
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Hello,
i found this thread while i was searching for a similiar Problem.
In Our case is also a Sles 11 SP3 (up to date patched) affected.
We are running a Vsphere 5.5 (patches up to date) but we are NOT using veeam, we use a different Product (i am not sure if saying which one is here allowed).
Most vesphere Backup solutions rely on VDDK so the process is mostly the same, that means to me, it is somehow a Sles (kernel) + Vsphere VM Tools Problem (probably VDDK related).
We activated vmtools debug log unfortunatly it is not telling usefull informations (while the vm dies) The only thing that i can guess off is, that the related Sles VM has an Sybase DB running wich Sqlanywhere performs some scheduled DB Jobs.
Every Time the VM dies while performing the backup, these jobs are running, so it looks like an overload on the VM (just a guess)
We are still searching for a solution
regards
i found this thread while i was searching for a similiar Problem.
In Our case is also a Sles 11 SP3 (up to date patched) affected.
We are running a Vsphere 5.5 (patches up to date) but we are NOT using veeam, we use a different Product (i am not sure if saying which one is here allowed).
Most vesphere Backup solutions rely on VDDK so the process is mostly the same, that means to me, it is somehow a Sles (kernel) + Vsphere VM Tools Problem (probably VDDK related).
We activated vmtools debug log unfortunatly it is not telling usefull informations (while the vm dies) The only thing that i can guess off is, that the related Sles VM has an Sybase DB running wich Sqlanywhere performs some scheduled DB Jobs.
Every Time the VM dies while performing the backup, these jobs are running, so it looks like an overload on the VM (just a guess)
We are still searching for a solution
regards
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Just out of curiosity - does setting disk.enableUUID parameter to false allow your backup solution to proceed any further?
-
- Novice
- Posts: 3
- Liked: never
- Joined: Jun 02, 2015 7:39 am
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
disk.enableUUID is currently not within the .vmx file, what is the default value for disk.enableUUID.
Our Vm is not dying (during backup) ervery time, i can try that setting, but i need some days to proof that this setting helps.
I will test that and will report back.
Our Vm is not dying (during backup) ervery time, i can try that setting, but i need some days to proof that this setting helps.
I will test that and will report back.
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Interesting that a few people are having the same issues on SLES.
Foggy,v.Eremin, Ive just tried a few test runs with disk.enableUUID=false and it the backup went fine without any issues. When setting disk.enableUUID=true we encounter the error and the VM crashes. I imagine this setting gives us a crash consistent backup so is the equivalent of having quiesence disabled?
Foggy,v.Eremin, Ive just tried a few test runs with disk.enableUUID=false and it the backup went fine without any issues. When setting disk.enableUUID=true we encounter the error and the VM crashes. I imagine this setting gives us a crash consistent backup so is the equivalent of having quiesence disabled?
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Right, "true" (default) means application-consistent backup, "false" - crash-consistent, however the setting applies to Windows guests only, so no need to bother with Linux VMs. We are still investigating the issue.
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Are you sure about that? The test shows very different results in Linux when toggling the flag. I can't find much documentation around disk.EnableUUID but I'm guessing it enables ID disk presentation to the OS. Can anyone elaborate?foggy wrote:Right, "true" (default) means application-consistent backup, "false" - crash-consistent, however the setting applies to Windows guests only, so no need to bother with Linux VMs. We are still investigating the issue.
-
- Novice
- Posts: 3
- Liked: never
- Joined: Jun 02, 2015 7:39 am
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
In our case, the test failed, we had an crash this weekend during backup.
The Test was made with the vmx setting disk.enableUUID, false
The Test was made with the vmx setting disk.enableUUID, false
-
- Enthusiast
- Posts: 44
- Liked: 8 times
- Joined: Oct 01, 2014 12:04 am
- Full Name: Warwick Ferguson
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Hello,
I'm also experiencing this issue in a hosted Datacentre on 7 RedHat servers after upgrading VMware Tools to 9.0.5.21789 (build-1065307). Perviously we were on open vmware tools 8.3.7.4937 (build-381511).
The hoster sent me this error:
An error occured while taking a snapshot: msg.snapshot.error-QUIESCINGERROR
An error occured while saving a snapshot: msg.snapshot.error-QUIESCINGERROR
Environment:
Vmware: 5.5 - I don't know the patch level as it's a hosted server
OS: RedHat Enterprise Linux Server release 6.6 (Santiago)
Backup Software: CommVault 10
Manual Tests I have performed with VMWare tools logging on:
(http://kb.vmware.com/selfservice/micros ... Id=1007873 )
1. Snapshot through vCentre (no Quiesce) = no error or crash
2. Snapshot through vCentre (with Quiesce) = no error or crash
3. Snapshot through CommVault = no error or crash
I could not reproduce the error, but we are still experiencing it randomly over our 7 servers at least one server per night, sometimes up to 3.
When the issue occurs, I can still PING the server, I can SSh to it and enter in root and my password. The prompt never comes back after entering the password. A hard reset is the only way to resolve this.
Interesting, on our internal UAT environment we have not seen this issue:
Vmware: 5.1.0 no patches
OS: RedHat Enterprise Linux Server release 6.6 (Santiago)
All Yum Updates are the same between Dev, Test, UAT and Production
Backup Software: vRanger 6.1.0.35402 (still trying to get mgmt approval for my Veeam solution)
I have found the following articles for Linux, this talks about changing the "disk.enableUUID=false" as perviously mentioned.
http://kb.vmware.com/selfservice/micros ... Id=2038606
http://kb.vmware.com/selfservice/micros ... Id=2079220
I'm about to make these changes today.
I'm also experiencing this issue in a hosted Datacentre on 7 RedHat servers after upgrading VMware Tools to 9.0.5.21789 (build-1065307). Perviously we were on open vmware tools 8.3.7.4937 (build-381511).
The hoster sent me this error:
An error occured while taking a snapshot: msg.snapshot.error-QUIESCINGERROR
An error occured while saving a snapshot: msg.snapshot.error-QUIESCINGERROR
Environment:
Vmware: 5.5 - I don't know the patch level as it's a hosted server
OS: RedHat Enterprise Linux Server release 6.6 (Santiago)
Backup Software: CommVault 10
Manual Tests I have performed with VMWare tools logging on:
(http://kb.vmware.com/selfservice/micros ... Id=1007873 )
1. Snapshot through vCentre (no Quiesce) = no error or crash
2. Snapshot through vCentre (with Quiesce) = no error or crash
3. Snapshot through CommVault = no error or crash
I could not reproduce the error, but we are still experiencing it randomly over our 7 servers at least one server per night, sometimes up to 3.
When the issue occurs, I can still PING the server, I can SSh to it and enter in root and my password. The prompt never comes back after entering the password. A hard reset is the only way to resolve this.
Interesting, on our internal UAT environment we have not seen this issue:
Vmware: 5.1.0 no patches
OS: RedHat Enterprise Linux Server release 6.6 (Santiago)
All Yum Updates are the same between Dev, Test, UAT and Production
Backup Software: vRanger 6.1.0.35402 (still trying to get mgmt approval for my Veeam solution)
I have found the following articles for Linux, this talks about changing the "disk.enableUUID=false" as perviously mentioned.
http://kb.vmware.com/selfservice/micros ... Id=2038606
http://kb.vmware.com/selfservice/micros ... Id=2079220
I'm about to make these changes today.
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Kindly, keep us updated about the results you get. Based on the experience of previous posters, the mentioned setting should be a way to go, indeed.
-
- Enthusiast
- Posts: 44
- Liked: 8 times
- Joined: Oct 01, 2014 12:04 am
- Full Name: Warwick Ferguson
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
A follow up from my post above.
After making the required changes to the .vmx files by our hosting provider and myself making the changes in the VMWare Tools (tools.conf) file. We have not had a server become unresponsive during a snapshot for the past 5 nights. The above KB2079220 from VMWare has resolved the issue.
After making the required changes to the .vmx files by our hosting provider and myself making the changes in the VMWare Tools (tools.conf) file. We have not had a server become unresponsive during a snapshot for the past 5 nights. The above KB2079220 from VMWare has resolved the issue.
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Thank you for confirming that setting disk.EnableUUID parameter in the .vmx file to false resolves the problem for you; much appreciated.
-
- Influencer
- Posts: 19
- Liked: 1 time
- Joined: Dec 17, 2014 10:47 am
- Full Name: Tom Vernon
- Location: England
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
Thanks for the update Wocka. Interestingly KB2079220 claims to be fixed in ESXi5.5u2 however we still see the issue running on the latest version. Setting disk.EnableUUID to false or disabling VMware tools quiesce does work as a fix though.
-
- Service Provider
- Posts: 17
- Liked: 1 time
- Joined: May 09, 2013 7:49 pm
- Full Name: Jason
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
we're seeing the same issue with vcenter/veeam. esxi 5.5.0 3568722
We're getting random issues as described in this thread. When can keep retrying the jobs, and eventually get the job to complete all the vm's. Was any resolution found for anyone?
We're getting random issues as described in this thread. When can keep retrying the jobs, and eventually get the job to complete all the vm's. Was any resolution found for anyone?
-
- Expert
- Posts: 149
- Liked: 34 times
- Joined: May 01, 2012 11:56 am
- Full Name: Stuart Green
- Contact:
Re: msg.snapshot.error-QUIESCINGERROR
I had this issue and was given a fix by VMware. Here is the environment and versions I am running.
VM Guest OS: SUSE Linux Enterprise Server 11 SP4
ESXi: 5.5 U3
VirtualCenter Server: 5.5 U3e
The VM's were updated to VMware Tools 10.x from 9.x and got a load of VM backup failures "msg.snapshot.error-QUIESCINGERROR"
But the VM's would sometimes backup, and then sometimes fail. VERY inconsistent but consistent to fail on the v10.x tools compared to v9.x
I reverted back my VM's to v9.x
Logged a case with VMware to report the issue.
Several VMware Tools in the 10.x stream have been released and all applied to a sample VM but exhibited the same "msg.snapshot.error-QUIESCINGERROR" every now and again.
Now these VM's had configured in the OS via /etc/vmware-tools/tools.conf the following setting:
[vmbackup]
enableSyncDriver = false
This was set after a recommendation from VMware in v9.x tools as without it this HUNG the whole filesystem of the OS - even more severe, than the current predicament of just not quiescing and at least you got the OS back!
VMware recommended to install VMware Tools 10.x and set tools.conf
Now we are getting successful VM backups without any QUIESCE errors.
VM Guest OS: SUSE Linux Enterprise Server 11 SP4
ESXi: 5.5 U3
VirtualCenter Server: 5.5 U3e
The VM's were updated to VMware Tools 10.x from 9.x and got a load of VM backup failures "msg.snapshot.error-QUIESCINGERROR"
But the VM's would sometimes backup, and then sometimes fail. VERY inconsistent but consistent to fail on the v10.x tools compared to v9.x
I reverted back my VM's to v9.x
Logged a case with VMware to report the issue.
Several VMware Tools in the 10.x stream have been released and all applied to a sample VM but exhibited the same "msg.snapshot.error-QUIESCINGERROR" every now and again.
Now these VM's had configured in the OS via /etc/vmware-tools/tools.conf the following setting:
[vmbackup]
enableSyncDriver = false
This was set after a recommendation from VMware in v9.x tools as without it this HUNG the whole filesystem of the OS - even more severe, than the current predicament of just not quiescing and at least you got the OS back!
VMware recommended to install VMware Tools 10.x and set tools.conf
Code: Select all
[vmbackup]
enableSyncDriver = true
Who is online
Users browsing this forum: AdsBot [Google] and 33 guests