VMware Tools Quiescence question

[Supraman]

Hello everyone,

Perhaps someone could answer this question for me and that base on this article publish by Veeam Help Center in regards VMware Tools Quiescence and the combine approach (https://helpcenter.veeam.com/docs/backu ... ml?ver=110). If I am understanding this this statement,

"If some VMs cannot be quiesced with application-aware processing or application-aware processing is disabled for some VMs in the job (the Disable application processing is set for VMs in the job settings), Veeam Backup & Replication uses VMware Tools quiescence to prepare these VMs for backup or replication."

So in the statement provided my understanding if I have a backup job that contain several VMs and I have a mix of Windows and Linux machine I should enable both VMware Tool Quiescence and Application awareness at the same time and if Application awareness should fail it should automatically attempt a second time to use VMware tools quiescence correct?

[Mildur]

It will not automatically failover to vmware tools squiscing. You have to manually disable veeams application aware processing (AAP) for the vms you don‘t can or want to use that feature.

You can use Application Aware processing for linux vms too. You have to provide a linux user account for that.

The only vms, where I disable AAP, are appliances like Citrix Netscaler or the vcenter VM.

[Supraman]

Hello Mildur,

But the statement implies it does unless I misunderstood the wording (OR) there.

Furthermore, since you indicated that AAP can be enable for Linux VMs, but according to this Veeam Article, "https://helpcenter.veeam.com/docs/backu ... ml?ver=110," Linux machines should have VMware Tools Quiescence enable as it does not support Microsoft VSS. If that truly is the case should I enable AAP on Linux machines or should I enable only VMware Tools Quiescence.

[Mildur]

Yes, you are right. I am sorry.

You need to use „Ignore application processing failures option“ todo an automatic failover to vmware tools squiscing.

I will test something today. I thought, that most of my linux vms are using AAP instead of VmWare Tools Guest Squiscing. Give you a feedback.

[Supraman]

Hello Mildur,

May I ask for a follow up on your testing?

[Mildur]

I‘m really sorry for the delay.
Was a hectic week. I‘l try to do it tomorrow.

[Mildur]

Ok, I have enabled AAP, but it is not used for Linux VMs.
If I disable "VMware Tools Quiescence" in the backup job, the linux vm will be backed up with the same veeam job log entries as without "VMware Tools Quiescence".
If I find the time next week, I'll try to see how this is reflected in the veeam debug logs and vmware logs. I need to understand this better what happens in the background for my own interest.

[HannesK]

Hello,
unfortunately there was some misunderstanding from my side. I will update the post once I have clarification.

EDIT: removed misleading information

Best regards,
Hannes

[Regnor]

It would be great Hannes if you could get some clarification on that topic. I always thought that VMware Tools quiescing was the way to go with Linux VMs; AAP for Linux is only for Oracle DBs.

Some time ago through a post in the Veeam community I've found this compatibility matrix: https://docs.vmware.com/en/vSphere-Repl ... ng-support
Although it's about vSphere replication it stated that file system quiescing was only supported for SLES and RHEL.

[Supraman]

According to the VMware documentation if I am not mistaken it appear that in order for Vmware Tools to quiesce an application it require a setting that needs to be enable when the virtual machine is power off. Therefore the question then are we require to enable it on the VM if we only enable from within the Veeam job setting?

[HannesK]

Hello,
I realized that one of my Linux VMs had an issue. Even from VCenter it was impossible to create a quiesced snapshot for that specific VM.

After I tried a a new VM, it works fine and as described in the user guide.

1) If AAIP is enabled in general and application processing is disabled: Windows -> AAIP. Linux -> VMware tools quiescing
2) If AAIP is enabled in general and application processing must be successful and credentials are specified for the Linux VM: Windows -> AAIP. Linux -> VMware tools quiescing
3) If AAIP is enabled in general and application processing is "try, but ignore failures" and no credentials are specified for the Linux VM: Windows -> AAIP. Linux -> VMware tools quiescing
4) If AAIP is enabled in general and application processing must be successful and no credentials are specified for the Linux VM: Windows -> AAIP. Linux -> backup job fails because of wrong credentials. Failover is not allowed, because success is required.

Best regards,
Hannes

[Supraman]

Hello Hannes,

Sorry for the confusion, but I meant are we require to enable Vmware quiescing from within the VM configuration as stated below within the vmware article. The reason why I ask is because If I am not mistaken Veeam calls upon the VMware tools to quiesce a VM just as it would call upon windows VSS when AAIP is enable, but from what I have read we would need to enable that feature from within the Esxi. If mixing this up with something else please let me know.

https://kb.vmware.com/s/article/1028881

[HannesK]

Hello,
hmm, I don't find anything about Linux in that article.

https://kb.vmware.com/s/article/2044492 - this one describes in option 1-3 what VMware does depending on kernel version and availability of a script.

Veeam calls upon the VMware tools to quiesce a VM just as it would call upon windows VSS when AAIP is enable

I'm not sure what this means, but Veeam's implementation of VSS quiescing (AAIP) is much more advanced / reliable than what VMware-tools do. That's why we have it and always recommend it over VMware-tools quiescing for Windows.

but from what I have read we would need to enable that feature from within the Esxi.

I have not heard of a situation where that would be needed.

Best regards,
Hannes

[Supraman]

Hannes,

Sorry for the confusion, what I meant by looking at the article and the requirement for the use of VMware VSS component for server 2008 and higher, would I have to have those condition met if I were to only enable the option for Vmware Quiescing within the Veeam backup job. As it state there are some condition that are required as listed by the article, hopefully that makes more sense.

Application-consistent quiescing. For this to be available, several conditions must be met:
Virtual machine must be running on ESXi 4.1 or later.
The UUID attribute must be enabled. It is enabled by default for virtual machines created on 4.1 or later. For details see Enable Virtual Machine Application Consistent Quiescing.
The virtual machine must use SCSI disks only and have as many free SCSI slots as the number of disks. Application-consistent quiescing is not supported for virtual machines with IDE disks.
The Windows VM must not use dynamic disks. If the VM has dynamic disks, the snapshot will be file-system consistent.

[HannesK]

sorry, I still cannot follow what Windows VSS has to do with Linux backups.

[Supraman]

@Hannesk

My question was not concerning Linux box, but the question revolve around window OS. The concern is if I did not AAIP enable, but used VMware Tool quiescing would that provide a backup that would be Application-consistent quiescing or would I need to employ a script to stop lets say SQL Services and would I need to enable UUID.

[soncscy]

Maybe I'm mistaken, but as far as I know, open-vm-tools supports file system quiescence: https://kb.vmware.com/s/article/2073803

At least, it's listed in their article, and it's the entire point of open-vm-tools. I know in the past we had issues with file recovery from systems without open-vm-tools, and those with it were stable.

Vmware's application aware is far different from Veeam's, keep in mind. VMware has a distinction for "file system quiescence" but the before-mentioned article doesn't include this for the VMware tools it seems, while open-vm-tools suggest otherwise.

You will never get individual applications into a quiesced state with any VMware tools (open or proprietary). You will at least have a consistent system with open-vm-tools.

[HannesK]

for Windows, we always recommend Veeam AAIP for reliability and feature reasons. If you want to go the VMware tools way, then you need to follow what VMware says. I understand the article as "yes" (to your question), but that's more a question to ask on a VMware forum. We just trigger VMware and anything else it out of Veeam scope.

Shutting down applications is a way to be sure that they are consistent, yes.

[Supraman]

Thanks Hannesk, the reason as to which I was asking was that a third party service provider recommended enabling both of these options AAIP and VMware Tool Quiescing which I thought that was strange as for Windows OS. Furthermore, I will proceed to ask that question on the VMware forums as I would hate to enable an option that does not fully work as because we did not meet the windows requirement they have listed. Thanks for the clarification I just want to make sure that I am not just enabling stuff randomly without knowing the affect and meeting the requirement in order for it to work properly.

@Harvey is open-vm-tools differ than VMware Tool? I did realize that Vmware AAIP was different as before I was primarily using just Veeam AAIP (Microsoft VSS). I had always thought that VMware Tools quiescence was more for Linux and device that did not support Microsoft VSS.

[soncscy]

As far as I know, the answer is "yes", there are differences. The Vmware tools quiescence on windows __does__ touch VSS, just not in a way a backup application might be aware of. I presume it's just the tools don't return any data to the application, but it's been awhile.

But if you check the project page, it's pretty clear it quiesces the __filesystem__: https://github.com/vmware/open-vm-tools

So your applications aren't quiesced (e.g., if there are pending writes in buffers, they're lost). But if you prep the applications prior to snapshot, then the filesystem and the applications are application aware.