Application Aware Processing vs VMWare Tools Quiescence

[pber]

I'm curious what people are using out there. We are currently using the Application Aware Processing. We end up running into a lot of issues due due to the fact that we can't disable UAC and are forced to use the "Administrator" account (Most of our environment can't communicate RPC and have to use VIX). We set it to Copy only because we don't have Exchange and our SQL/Oracle do their own backups.

If the VMWare Tools Quiescence invokes VSS, is using Application Aware Processing a waste of time?

[PetrM]

Hello Patrick,

Yes, VMware tools also can be a VSS requestor but Application-Aware Image Processing is not a wasting of time at all.

There are some profits like:
1. AAIP deploys the runtime process directly in the guest VM instead of leveraging VMware tools and you won't need to troubleshoot potential issues with that
2. The runtime process collects an auxiliary data about guest OS which might be required in different cases: for example if you're going to use Veeam Explorer for Microsoft Exchange

May be guest interaction proxy would help to avoid VIX in your case?

Thanks!

[pber]

Thanks for the info. I'll look into the guest interaction proxy. We are currently using the AAIP, but end up with most of our jobs with warnings due to UAC issues and having to use the actual administrator account because of our security zoning and VIX. We also want to get away from using the actual Administrator account because that is just a garbage solution if we want to keep the UAC on.

[pber]

Further to that, I would be totally happy if they just had a permanent agent that just sat on every VM, instead of the way they do the on demand install each time the backup is run.

[PetrM]

Hi Patrick,

By the way, with guest interaction proxy it should be enough to use an account which is a member of Local Administrators group, take a look at our best practices.

Not sure I fully understand a benefit of having a persistent guest agent as long as the upload operation does not take time and UAC problem can be eliminated (take a look at this thread).

Thanks!

[pber]

Thanks for the info.

The port requirements between the backup server and the Guest Interaction Proxy are a concern because of our heavily firewalled environment due to compliance reasons. The 6190/6290 ports are fine, but the 445 port is a deal breaker (https://helpcenter.veeam.com/docs/backu ... =100#guest).

Currently the vast majority of our jobs use AAIP and for the most part, it works well. But we have about a steady 5% of the jobs that give warnings because the AAIP couldn't start for various reasons. We fix specific issues, they clear up, then a new one pops up in another job... wash, rinse, repeat. With the number of servers we have, that 5% represents a significant amount of daily effort.

We also have a smaller environment (~75 servers that are void of DCs, Exchange, SharePoint, SQL and Oracle) that we only use vmware tools quiescence and we have never had issues like we have with AAIP. We do quite a bit of restores in that environment and have no issues.

Since we don't have Exchange, SharePoint and our SQL/Oracle servers do their own backups, with the exception of domain controllers, it seems we have little use for what the guest processing offers. Is there another benefit to using AAIP over quiescence for our environment? Does AAIP do a better job of invoking VSS than quiescence does?

[PetrM]

Hi Patrick,

I don't think we should use definitions like "better" or "worse" when we're talking about technical details of implementation. I would say that there is no difference from VSS perspective only.

On the other hand, our best practice is still to use AAIP because of mentioned advantages.
Moreover, with AAIP the entire process of interaction with guest OS is fully controlled by Veeam and there is no need to entrust such crucial task like quiescence to 3rd party tools.

I would recommend to detect the source of warnings related to AAIP, may be all of them have the same root cause? Feel free to contact our support team to investigate all of these issues.
Anyway, don't forget about SureBackup job, this one allows you to validate backups and be sure about successful restore regardless of chosen quiescence mechanism.

Thanks!

[paps40]

I have read mixed things about this from old forum posts and wanted to ask what is the recommendation here for Veeam 11.

Should you Enable Application-Aware Processing on windows servers 2012, 2016 and 2019 that do not have SQL, SharePoint or Exchange installed on them? Do file servers, application servers, and print servers need Application-Aware Processing?

Can VMware Tools quiescence be used instead to limit backup job failures related to application aware processing?

What is the best practice here?

Thanks,

Peter

[Mildur]

Do file servers, application servers, and print servers need Application-Aware Processing?

There isn‘t any disadvantage for you, if you have them enabled on every server.
The only thing you have to look out for is, if you have vss incompatible applications like mysql or other database, you have to use a script from the database provider or stopp the database services, because veeam cannot create a consistent backup of such databases with vss.

[paps40]

See older post below from Foggy in 2017 about Application Aware Processing. Is this still the Veeam recommendation in 2021 Version 11. Use AAP for all windows vm's?

Thanks,

Peter


veeam-backup-replication-f2/is-applicat ... 44610.html
This is not true, since AAIP is a bit more than just VSS quiescence. AAIP ensures that each VSS-aware application state is transactionally consistent by performing some additional application-specific steps to prepare the application for further VSS-aware restore. It is strongly recommended to enable AAIP on all Windows VMs, unless there are reasons to not do that.

[PetrM]

Hi Peter,

I've merged the thread into the existing topic. Yes, Foggy's recommendations are still valid.

From my point of view, it would be best to let our support team to help you with troubleshooting of backup failures related to AAIP while VMware Tools Quiescence can be used as a temporary workaround.

Thanks!

[TWuser]

Sorry to turn this thread into a zombie, but if there are more recent (good) threads on this specific subject I failed to find them. The suggested "best practice" around this subject seems to be getting outdated with modern server hardening.

A note/thought for those of us trying to secure our environment - App Aware Processing requires admin credentials on the OS.
- We are trying to minimize any unnecessary holes in security that could allow lateral movement, so are disabling App-Aware on VM's lacking a database/AD/sharepoint/etc.
- We are working on using GMSA's for all database servers to help protect us a little bit more.
- GMSA's do have some annoying limitations, and not having a local admin also means no indexing, which is used in more recent editions for virus scans.

Using a persistent agent could be helpful if we wanted to get rid of the Admin credential, but those servers would still be at the whim of the VBR server (which could run scripts), so still seems safer to not use AAIP when not necessary.
Not to mention you'd still have to install AND update the agent somehow still, that process is already annoying just for physical servers where Veeam has full admin creds.

I am wondering if it's worth enabling VMware Quiescing for VM's with AAP disabled, or if that's unnecessary overhead. A file server for example. Enabling VMWare Tools Quiescence in a job enables it for "all" VMs with AAIP disabled in that job.

TLDR; I think best practice for App-Aware processing should only be enabled for servers running compatible apps (SQL, AD, etc) since it requires a local admin credential. I may enable VMware Quiescing for other servers, but not sure it's worth the additional overhead.

[PetrM]

Hello,

We've published this FAQ and included this question there. AAIP is still the recommended approach, even for VMs that are not running applications like SQL Server, Oracle, SharePoint, etc. This is mainly because we do not depend on VMware tools functioning and rely on our persistent or runtime agents to trigger VSS activity inside the guest OS.

Would you be so kind as to elaborate a bit more on the potential security pitfalls you see with the persistent agent? Normally, it enables guest processing without needing to open admin share and removes the extra port requirements of the runtime process injection.

Thanks!