Agent Hangs on Backup Exchange DAG

[LBreiter]

Hi Community,

I'm posting here because our Exchange Backup (DAG) is not working anymore. I've already opened a case (#02705854) with Veeam Support but after 2 Weeks and lot of Logs send to Veeam no successful solution yet. So I'm hoping one of you has experienced same and has probably solution to my Problem

We've configured a Cluster Backup (https://www.veeam.com/kb2463) and this was running smoothly over the last several month. But one Day (Monday, 11 of April) it suddenly stops working. The first Error Message we got was "Transmission Pipeline hangs. Aborting Process". So we waited two days before we opened a case, sometimes a Backupjob "repairs" it self, but no joy.
When the Backup is running, there are a few VeeamAgent.exe created and suddenly one or more Agents stopped working. Even when you try to kill the VeeamAgent.exe Process in the Task Manage you get a "Permission Denied" (which is the Problem, Veeam Support suggested). The only option to get rid of this hanging Agent is to reboot the Exchange Server! And even the Reboot hangs, you have to reset the Machine in VMWare. And if you try to cancel the Backupjob it also hangs in Status "Stopping", Reboot Backupserver is required.

We also have several other Backups are using the Agent (SQL Server Backup, Fileserver Backup) which running without Problems. And also VMware Backups are running without Problems.

What we have done so far:
- Updated Veeam to the latest Patch (P20220302)
- Updated the Windows Servers with the latest Patches (both Windows Server 2016 with Exchange 2016)
- Uninstalled and reinstalled the Agents on the affected Servers (several times)
- Deleted the BackupJob and create a new one, even created a new Repository for this Job
- Changed the Agent Backup Proxy for this Job
- Changed the Backup Mode from "Entire Computer" to "Volume level Backup" and added the Volumes with Exchange

Thanks for reading

Lars

[Mildur]

Hi Lars

I assume, the correct Case Number is 05388065.
Do you had already the chance to try out the suggestion with removing the antivirus for a test run?
Are there any changes to windows or exchange updates on April 11th or day before?

If you are unhappy how the case is going, you can always escalate the case to a higher tier. But I would first try to remove the antivirus and try the backup again.

[LBreiter]

Hi Mildur,

yes, already done that. Removing the antivirus software hadn't any effect, same error.
And frankly, yes I'm unhappy how the case is progressing. Even after 2 1/2 Weeks no viable solution But I'm already on tier 2, so escalate to tier 1 means exchange doesn't work anymore and that is not the case ... I'm "just" unable to Backup my Exchange Server

[Mildur]

I'm "just" unable to Backup my Exchange Server

If "unable to backup" violates your RPO Policy for two weeks, then for me it would be a Tier 1 case.
Backups are important too. Not only the running exchange server. Please try to escalate the case.

Additionally, can you tell your veeam support engineer about the antivirus test? I see that he asked (the mail was send to your colleagues who has opened the case) about it yesterday.

Update:
Can you use VM Backup Jobs as a Workaround to have at least a backup in case you need to restore the data? Please read first this KB article: https://www.veeam.com/kb1744

[LBreiter]

Hey Community,

with the Veeam Support we could solve this issue. I had to configure a few more exclusions in our Antivirus Software. There are some exclusions for the Veeam Agent which we hadn't configured yet. But with these Exclusions the Backup is working again (https://www.veeam.com/kb2034).
But to be honest, I don't know why the Backup suddenly stops working after month of successful backups.

Thanks
Lars

[Mildur]

Hi Lars
Thanks for the update.