Exchange 2010 DAG backup ?

[Dima P.]

Hi,

Have you seen this thread, its related to Exchange 2010 DAG backup, however all the advises remain valid. Thanks.

[conextra]

hi,

this is all about backup. i can backup both nodes in a lab succesful - there is no database switchover during snapshot creation and deletion. my unsecureness is, what happens in a Desaster Scenario, when i recover just one node, either active or passive. i am not sure, if this works (indeed i am doing application Aware backups).
i know, the Option with recovery Switch during exchang install, but i want to avoid this, since i Need to build a complete Server first, with all Patches, ... it would be much more efficient, if i can just recover the broken node with Veeam and then power it on.

[pronto]

Hi Community,

we are planning to migrate from an Exchange 2010 single server setup to a two server active/passive DAG environment (Exchange 2010). Both on vSphere 5.1 hosts. We are in touch with an engineer of our service partner for Exchange and he want to know if Veeam Backup & Replication can process an incremental backup of the passive node?

I read already the KB article https://www.veeam.com/blog/how-to-backu ... ation.html and learned that Veeam can do a backup of the passive node but I didn't found any hint if it works with incremental backups on the passive node also.

I'm not aware how a DAG works in detail right now but please allow the question if the behavior of incremental backups changed in a DAG environment compared to a single server setup? Normally an incremental backup deals with the transaction logs and should be no problem if the copy of the database is in use or not but this question let me took notice that this behavior could be changed in a DAG environment, especially on the passive node.

We have a meeting today with the service partner and it would be great if I could answer his question.

Thanks in advance & Bye Tom

[nielsengelen]

This depends on the backup job configuration but if you use incremental backups (forever forward incremental or incremental with synthetic full) and disable active full backups we will only make 1 active full backups (first time we backup the machine) and from there on only incrementals. So the answer to your question in short is: yes it is possible to do this.

Some additional tips & tricks: https://www.veeam.com/kb1744
And the following blog post by one of our solution architects: http://andyandthevms.com/exchange-dag-v ... plication/

[Guido]

Hello

We are doing backups of 2 Exchange 2010 servers fine for a long time. The VSS application-aware processing is on.
The truncate was done by another backup product (HPE DP). Now we want to enable the truncation in Veeam.
Do I have to enable the truncation on both servers within the backup job?

Thank you

[DGrinev]

Hi Guido,

You can enable truncation on the passive Database and it will be replicating to the second node through DAG.
For additional information, review this thread above.

Thanks!

[Guido]

Thank you DGrinev for merging my post into a this large threat with 5 pages.

So I can enable it on the passive Database. But I think you mean that I can enable it on the server running the passive database.
The question was if I have to enable truncation on both dag servers within the job. Is you answer a NO?

[Andreas Neufert]

You should truncate logs on both DAG members. Yes, truncation is replicated to the other node, but this replication is done by exchange after the backup job, so you do not have any monitoring of this process within Veeam.
When you enable it on both sides, it is done for sure and you have the feedback in the job if it didn´t ran well.

So for both Servers:
- Enable Veeam IngUest processing. Enable "Require successful process (recommended)" and "process transaction logs with this job (recommended)"
https://helpcenter.veeam.com/backup/80/ ... ob_vss.png

Increase Cluster Heart beat if not done:
http://andyandthevms.com/exchange-dag-v ... plication/

[Guido]

Thank you Andreas
Your answered my question.

[Andreas Neufert]

I got a PM that this topic is all about passive only backup.

Logfile Truncation get´s replicated within the Exchange cluster in any way if you backup one or multiple DAG members.

When you have the chance, you should backup all DAG members including Logfile Truncation at all of them.

When you want to backup only one DAG member (with all databases) then you need to enable Logfile Truncation there. As the DAG based Logfile Truncation replication runs our of Veeam control, you should monitor this process (Windows Event IDs) to avoid situations when the replication had an error and you server will fill the Logfile disk 100% => Exchange down.

[leithm]

hi,

i have 3 dag members. two of which i want to back up in the same job. when i add 2 dag nodes to a job one will always fail. checking event logs on the server indicates that vss failed due to one of the databases already being backed up (by the veeam job). for some reason i thought veeam would of handled this automatically.

active databases are split across the 2 nodes i wish to back up. application aware processing enabled, truncate logs enabled for both vms.

any advise would be greatly appreciated

cheers

[veremin]

What particular error did you get? What AAIP settings are specified for those VMs?

[leithm]

Just the default settings under AAIP indicating that VSS must complete successfully and to truncate logs. this was based on the recommendation above

'When you have the chance, you should backup all DAG members including Logfile Truncation at all of them.'

error was vss failed at prepare snapshot. according to the application logs on the server vss failed due to the database already being backed up on the other server. basically..veeam tried to simultaneously back up the same databases

[Andreas Neufert]

that should not be a problem at all. Do you use Circular Logging ?
What Microsoft event ID do you see ?

[toa1970]

Hi,

Setup: Two Exchange mailbox servers. One with all active mailboxes and one with all passive mailboxes. Exchange version is 2010 with all updates on Server 2008R2 with all updates. Within 3 months it's Exchange 2016 on Server 2016 - Veeam backup is version 9.5.0.823

Question: Let's say that Veeam backup the passive Exchange server only. How to truncate Exchange log file on the active Exchange server after backup? Can Veeam after a backup on the passive Exchange server truncate log file on the active Exchange server? Or is best practice to backup the active Exchange server? Or another solution?

[foggy]

Hi Torben, logs truncation is replicated between DAG nodes, so if you truncate logs on the passive node, this will automatically happen on other nodes as well.

[albertwt]

People,

I've got 4 nodes DAG that contains mixed of Active & Passive mailboxes, as per this article: https://www.veeam.com/kb1744 it mentions on the below:

Use Network (NBD) mode setting on Source Backup Proxy as opposed to Appliance (hotadd) mode for your backup and/or replication jobs in Veeam.

Which is quite slow ?
So why the faster mode HodAdd is not suggested to backup the DAG node?

Regards,

Al

Case# 02215952

[foggy]

Have you tried to backup via hotadd? Have you actually seen any of the mentioned symptoms? Steps outlined in the KB are just suggestions to eliminate the issues, which are environment-specific, so chances are you do not need to follow them. This topic contains other considerations as well.

[Andreas Neufert]

Yes, only if you face the described error.
If you need more background infromations, you can read my blog post:
http://andyandthevms.com/exchange-dag-v ... plication/

[jochot]

Hey,

so in this post: https://www.veeam.com/blog/how-to-backu ... ation.html
It says:
If you have enough storage space, create a single backup job for all of your DAG nodes. This will most likely result in better deduplication ratio.
In the next line:
If you want to backup DAG nodes in separate backup jobs, make sure that the schedules don’t overlap and also that all VMs are not being snapshotted at the same time.

So I'm wondering, why it is good to put them into one job, if it shouldn't be done at the same time.

We have 3 Exchange Servers, which are replicating the databases.
So if I start the backup job (when all of those are in one Job), does Veeam know that the data on all the servers is basically the same and therefore backs up only one of those databases?
Or are all 3 Databases backed up seperately?
I would think Veeam backs up one of them and sees that the data is the same (deduplication), but I couldn't find an answer anywhere.

[pesos]

Curious what the reason is for backing up more than one DAG node if you're just going to rely on the duplicated DB data being deduped? Veeam can only dedupe per job afaik so you'd have to have them in the same job to even get this to work... and deduping the DB would kind of defeat the purpose of backing up more than one node from what I can see.

If it's the system drives you're after (which could be handy) then I'd put them all in one job and exclude the data drive(s) from all but one of the nodes...

[jochot]

So when I only backup one, and all of them burn down, I can only restore one, right?
Thats why I want to backup all of them

So When I backup the DB from one node and the other drive of all the other nodes, I can restore one full Exchange Server and 2 Exchange servers without the Databases, right?
Is there a possibility to restore the database separately for those 2?

[foggy]

So I'm wondering, why it is good to put them into one job, if it shouldn't be done at the same time.

Different jobs will try to snapshot the same VM twice, which is not recommended.

So if I start the backup job (when all of those are in one Job), does Veeam know that the data on all the servers is basically the same and therefore backs up only one of those databases?

No, it will back everything up but similar blocks will be then deduped within a backup file.

Please see recommendations on DAG backup in this thread.

[Andreas Neufert]

I would say that best practices is to put the DAG members in different Jobs and to schedule on different times.

If for whatever reasons the DAG Cluster heartbeat detect the VMware Snapshot release as a downtime of one of the DAG members it will failover to the other DAGs.
The index service will peak CPU and do a lot of stuff on the disks then, which lead into a high change rate on disk at that time and it will fill (if you backup at same time) the snapshot of the other DAG members. Now when the backup ends it will commit again the VM snapshot and because of all the changes, it is more likely that the other DAG member run into the same issue and DAG detect as well that this server is down and failover again.

This is not bad from User view as the database is online on their view. Double cluster failovers in a short timeframe can lead into long time 100% CPU usage of Exchange Server (Index Service and others) which in the end can lead into disconnects and timeouts at the clients. This should be avoided.

As well a failover during backup lead into the situation that the Exchange Log can not be committed. Depending on the setup this is an issue and can lead into downtimes (disk full).

So there are good reasons not to use the same job to get the deduplication benefits. External deduplication with Microsoft or Storage can help here if the data amount is a problem.

If you only want to backup the OS disks and a single DAG member you can place the disks of the OS only VM into independant, so that they do not gather changes for those disks in the snapshot. => No problem to commit the snapshot. Important here is to disable the Log Truncation in the Inguest processing for the OS only backup Servers.
Veeam support to backup a single DAG member with inactive, active or mix of both databases and our Veeam Explorer can restore mails and objects from this (as well from inactive databases). Logfile truncations are replicated from the server that backup the databases to all other servers. Important is to monitor this replication within the Windows Event log.

[rohin.kurian]

Support Case ID: 02253325

When doing backups for my exchange DAG environment the databases failover. All nodes are on separate jobs at different times not overlapping. The max timeout has been applied and actually worked for some time last year. At this point, it has been over a year and we have no solutions to resolve this issue. Has anyone else been experiencing this issue? Any help on this would be appreciated.

Thank you,
Rohin

[DGrinev]

Hi Rohin and welcome to the community!

Please review this existing discussion related to backup of DAG environments.
Also, check this blog-post written by one of our Solution Architects, I hope it should give you some thoughts on how to resolve the issue. Thanks!

[andrewfox68]

Hello

Looking for some advice on backing up our Exchange 2016 infrastructure with Veeam.

OS: Windows 2012 R2
Exchange 2016 CU 4
Veeam 9.5 U2

Exchange Servers and DBs (DBs are in a DAG)

EX01 EX02 EX03

DB01 - Active DB01 - Passive DB01 - Passive
DB02 - Passive DB02 - Active DB02 - Passive
DB03 - Passive DB03 - Passive DB03 - Active

Veeam documentation recommends backing up a server which has holds only copy Exchange DBs. As you can see from above our Exchange Servers hold at least one active DB.

My questions are:
If I only backup the server EX01 shown above will I get application aware backups of all of the DBs (DB01, DB02, DB03)?
If I create a server named EX04 which holds only passive DBs and only backup this server with Veeam, how does log truncation work? WIll active DBs have their log truncated?
Do you have any particular recommendations for backing up our infrastructure with Veeam?

Kind regards


Andrew

[Andreas Neufert]

If I only backup the server EX01 shown above will I get application aware backups of all of the DBs (DB01, DB02, DB03)?

Correct, this is by design of Exchange VSS writer.

If I create a server named EX04 which holds only passive DBs and only backup this server with Veeam, how does log truncation work? WIll active DBs have their log truncated?

Log truncation get´s replicated through the DAG cluster by design of Exchange.

Do you have any particular recommendations for backing up our infrastructure with Veeam?

I have written a blog entry about all this with a ton of details and tips and tricks. The blog entry include as well enhanced details for question number 1 and 2.
http://andyandthevms.com/exchange-dag-v ... plication/

[tore]

Our Exchange setup is as follows:
3 Exchange 2013 servers
6 mailbox databases distributed on the servers like this:
DB - Active copy - Passive copy
DB1 - Server 1 - Server 2
DB2 - Server 2 - Server 3
DB3 - Server 3 - Server 1
DB4 - Server 1 - Server 2
DB5 - Server 2 - Server 3
DB6 - Server 3 - Server 1

As you see, all three servers hold both active and passive copies.
The server first processed in the backup job get a successful "Truncating Exchange transaction logs" event. The two next servers instead get the event in the bottom of this post.
If I change which server is processed first, this server will have a successfull VSS log trunc event.
All logs actually get truncated even if Veeam has the warning event, and Exchange show correct "Last successful full backup". I can also restore from the backup.

I have narrowed this down to be a problem when hosting both active and passive databases in a DAG. Is this not supported by Veeam?
Is there a way to prevent this warning with our current DAG setup?

20.11.2017 21:58:06 :: VSS: Backup job failed.
Cannot notify writers about the 'BACKUP FINISH' event.
A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{46f5306e-c9f3-4aa7-a989-125bd1bd0257}]. Writer's state: [VSS_WS_FAILED_AT_BACKUP_COMPLETE]. Error code: [0x800423f3].

[foggy]

It is recommended to backup a server where only passive copies reside. Please review this thread for details.