Exchange 2010 DAG backup ?

Availability for the Always-On Enterprise

Re: Exchange 2010 DAG backup ?

Veeam Logoby Cokovic » Fri Nov 23, 2012 2:11 pm 3 people like this post

Maybe i can bring up some additional light into these questions based on my very own experience with a quite large Exchange DAG environment (actually running 3 nodes with about 6000 mailboxes on it).

Question 1:
Should both nodes be backed up, isn't this just duplicating information and backup window?

It depends. If you can afford the addtional storage space for backing up both nodes then you are good to go with backing up both nodes. But there are other things you have to take into consideration when planning your backup strategy. It has to deal with your other questions.

Even after changing the values you've mentioned under question 5 there is still the possibility that you experience a database failover from the active node to the passive node when snapshot creation or commit takes place. This is something that happend to me in our environment. While Job 1 on Node 1 was running (database failover due to snapshot creation has happened) the job for the second node started too cause my backup windows overlaped. Now while backup of Node 1 was still running and the backup of the second node was intiated all the active databases from Node 2 made a failover back to node 1. Backup job on Node 1 finished and as snapshot commit was performed the databases failed over again to Node 2. Now after snapshot commit on Node 1 finished the job on Node 2 began his snapshot commit. Now the active databases again started to initiate a failover back to Node 1. They never failed over to Node 3 as we've setup database preferences. This is why i ended up in a situation where my databases flapped around between Node 1 and Node 2. Additionally because of the amount of load and mailboxes the Indexing service on these two nodes went wild resulting in VMs that were under 100% load and thus not responding anymore to any requests for quite a while. Not really funny having thousands of users complaining about a non working Outlook anymore :wink:

Question 2:
If only a single node is recommended should it be Active or Passive?

In the end we've setup a complete passive node (Node 3) and just backing up this one. That way we are preventing active databases from doing a failover and additionally offloading the backup processing to this single node. This is by the way the preferred method by Microsoft too in an Exchange DAG environment. Here's an article from a Microsoft blog where you can read something about it (scroll down to VSS Backup of Passive Copy). It's written for Exchange 2007 but still applicable for Exchange 2010 i think.

Just in case of VSS based restores it is a bit troublesome when backing up a passive node . But this is not applicable for Veeam as your are not performing VSS based restores with it. So it doesn't really matter if you backup an active or a passive node with Veeam.

Question 3:
If passive will truncate logs still work?

Yes. See the link to my post Vitaly referenced. But you should always keep an eye on your backup and see if log truncation really happens. In case you've got any issues with the Replica writer log truncation can fail. So again: watch it! :)

Question 4:
If node has mixed active and passive databases, should this be altered so all databases are active on a single node and passive on alternate node?

See answer to question one. If you can do backups from an active node without initiating a database failover you are good to go with a mixed environment. If not you should consider creating a passive node an just doing backup of this single passive node.

Question 5:
I have read about snapshot timeouts resulting in DAG failovers, should I increase cluster timeouts (SubnetDelay/SubnetThreshold) from offset or wait and see if this issue affects me?

These settings are intended to prevent a database failover when vMotion or snapshot takes place. You have to try it. You can safely increase the timeout values even when not needed.


Take a look into this article too. There are some recommended hotfixes for an Exchange 2010 DAG cluster you may want to implement in your environment.
Cokovic
Expert
 
Posts: 295
Liked: 59 times
Joined: Tue Sep 06, 2011 8:45 am
Full Name: Haris Cokovic

Re: Exchange 2010 DAG backup ?

Veeam Logoby deduplicat3d » Sun Dec 09, 2012 9:11 pm

I'm about to go live on Exchange 2010 (with DAG). I have about 300 (active) users on 2 mailbox servers both active. Should I be concerned about this timeout issue or is this relatively rare? What is the default timeout?
deduplicat3d
Enthusiast
 
Posts: 98
Liked: 11 times
Joined: Fri Nov 04, 2011 8:21 pm
Full Name: Corey

Re: Exchange 2010 DAG backup ?

Veeam Logoby Andreas Neufert » Mon Dec 10, 2012 7:48 am

yes, you need to adjust heartbeat dag settings to max as described here. With get-... you can read out your default settings.
Andreas Neufert
Veeam Software
 
Posts: 2209
Liked: 363 times
Joined: Wed May 04, 2011 8:36 am
Location: Germany
Full Name: @AndyandtheVMs Veeam PM

Re: Exchange 2010 DAG backup ?

Veeam Logoby Cokovic » Mon Dec 10, 2012 12:18 pm

deduplicat3d wrote:I'm about to go live on Exchange 2010 (with DAG). I have about 300 (active) users on 2 mailbox servers both active. Should I be concerned about this timeout issue or is this relatively rare? What is the default timeout?


It depends mainly on your underlying storage. If for example snapshot commit takes too long it's possible that you could exceed the cluster tresholds and a failover will be initiated. So nobody can really tell you what will happen in your environment. You will have to test it.

To verify your cluster settings open up a command prompt and type

cluster /prop

This will show up the cluster settings. Take a look at "SameSubnetDelay" (default 1000, max 2000) and "SameSubnetThreshold" (default 5, max. 10). You can set this paramter with

cluster /prop SameSubnetThreshold=10
cluster /prop SameSubnetDelay=2000
Cokovic
Expert
 
Posts: 295
Liked: 59 times
Joined: Tue Sep 06, 2011 8:45 am
Full Name: Haris Cokovic

[MERGED] How to backup Microsoft Exchange 2010 DAG

Veeam Logoby dreic » Wed Dec 12, 2012 12:52 pm

Hi,

we have 2 HUB/CAS servers in NLB cluster and 2 mailbox servers in DAG cluster. What should i backup to be able to do item level restore?
dreic
Influencer
 
Posts: 10
Liked: never
Joined: Sun Aug 19, 2012 5:45 pm
Full Name: Damir Reic

Re: Exchange 2010 DAG backup ?

Veeam Logoby Cokovic » Wed Dec 12, 2012 2:33 pm

You only need the mailbox server if you are doing item level restore with Veeam Explorer for Exchange. If you want to use it in conjunction with SureBackup then you would need at least one HUB/CAS server backed up too.
Cokovic
Expert
 
Posts: 295
Liked: 59 times
Joined: Tue Sep 06, 2011 8:45 am
Full Name: Haris Cokovic

Re: Exchange 2010 DAG backup ?

Veeam Logoby Andreas Neufert » Wed Dec 12, 2012 5:05 pm

in case you want to use veeam explorer for exchange only one mailbox server which hold all DBs needed. If you want to use Surebackup / VirtualLab or old U-AIR exchange wizard, you need:

all dag members
the server who host whitness quorum share or if whitness is configured as auto you need one cas server which hold no mailbox role.
At minimum one cas.
Andreas Neufert
Veeam Software
 
Posts: 2209
Liked: 363 times
Joined: Wed May 04, 2011 8:36 am
Location: Germany
Full Name: @AndyandtheVMs Veeam PM

Re: Exchange 2010 DAG backup ?

Veeam Logoby Daveyd » Mon Jun 17, 2013 3:08 pm

I am going to be running 2 mailbox servers. The 2 servers will be in 2 different ESX clusters in 2 different DataCenters connected via dark fiber. Each mailbox server will run 2 active databases with DAG copies of the other MB server.

So the setup is like this:

MB1-DC1:
1.3TB vmdk for DB1
1.3TB vmdk for DB2
1.3TB vmdk for DAG copy of DB1 on MB2-DC2
1.3TB vmdk for DAG copy of DB2 on MB2-DC2
200GB vmdk for DB1 logs
200GB vmdk for DB2 logs
200GB vmdk for DAG copy of DB1 logs on MB2-DC2
200GB vmdk for DAG copy of DB2 logs on MB2-DC2


MB2-DC1:
1.3TB vmdk for DB1
1.3TB vmdk for DB2
1.3TB vmdk for DAG copy of DB1 on MB1-DC1
1.3TB vmdk for DAG copy of DB2 on MB1-DC1
200GB vmdk for DB1 logs
200GB vmdk for DB2 logs
200GB vmdk for DAG copy of DB1 logs on MB1-DC1
200GB vmdk for DAG copy of DB2 logs on MB1-DC1

In this setup, would backing up each Mailbox VM and excluding the drives that contain the active DBs be sufficient, or would I need to backup everything on both Mailbox servers? Since there are multiple 1TB+ drives presented to each Mailbox server, the Full backups would take a considerable time. I am look for the best way to backup each Mailbox server while making it the most efficient and the most reliable to restore.
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

Re: Exchange 2010 DAG backup ?

Veeam Logoby Daveyd » Tue Jun 18, 2013 1:22 pm

Anyone?
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

Re: Exchange 2010 DAG backup ?

Veeam Logoby Cokovic » Wed Jun 19, 2013 9:04 am 4 people like this post

Seems like noone wants to give you an answer ;) I would still recommend you to search through this forum as most of the things regarding Exchange 2010 DAG backup were discussed in detail here in the past. OK now lets go on.

I'm not sure what you are trying to achieve but if it's that you want just to backup the passive copies on every node then this wont work. Disk exclusion wont prevent snapshot creation of these disks (only if you configure them as independent disks). And also VSS processing would still happen for the active databases as the VSS call from Veeam cant be performed for a specific database. It will always be done on every database on your server.

Restore shouldn't be a problem no matter which mailbox server you are going to backup. With Veeam Explorer for Exchange you would be able to restore any single item from active and passive copies of your database. Even a full database restore would be possible. I've tried this myself and it works. If you are going to restore a passive copy you would have to do some manual steps too but in the end a full database restore with Veeam FLR worked.

So i would recommend you the following:

1. If not already done you should adjust cluster timeout setting to prevent a possible database failover when vMotion or snapshot takes place (depending on your environment this still could happen even after adjusting these settings). For example see this link here.

2. There are some recommended patches from Microsoft for an Exchange DAG environment. They are not automatically installed with automatic updates. Even if the description is not applicable for your environment they still should be installed (know it from my very own experience). See here.

3. Test your backup first with only one mailbox server and see what happens. Best case would be that backup is successfull and no database failover or log truncation error will happen ;)

If everything works fine with you could

a) backup both mailboxserver. In this case i would ensure that backups are not overlapping and log truncation is actived only for one of the mailbox servers. Log truncation will also happen for passive databases. SO nothing to care about.

b) backup only one of your servers. You are still able to restore a single item or a complete database too (no matter if from active or passive database). In case a complete mailbox server fails you could restore this node from scratch with the recovery option from the Exchange setup (as described here).

4. If you have problems with database failover while backup is performed (cause of snapshot creation or removal task) then you should think about moving all your databases to just one server so you could backup the passive node (and not have any impact on your production server where the active databases reside) or keep your existing setup with spread active databases and add a third DAG member as a passive copy and then backup this one. This is what we have done in the end as we had problems with failing databases in our environment while backing up active nodes (see my post above). Backing up the passive node was even possible while working hours as the load was only on the passive node. Log truncation still happened so everything was fine.

Hope this answers your question a bit.

Cheers
Haris
Cokovic
Expert
 
Posts: 295
Liked: 59 times
Joined: Tue Sep 06, 2011 8:45 am
Full Name: Haris Cokovic

Re: Exchange 2010 DAG backup ?

Veeam Logoby Daveyd » Wed Jun 19, 2013 3:06 pm

Haris,

Thank you for the detailed reply!

Since we have both DatcCenters connected via dark fiber and have ESX hosts in both locations, we are trying to split up resources, hence 2 active databases in one DC and 2 active DBs in the other DC. In each Datacenter, our back end disks (3TB NL SAS on EMC CX4 array), which we had sized with the help of EMC. So, I do not think our spindle count would be great enough for 4 active DBs on 1 server versus 2 active 2 DBs and 2 passive DAG DBs.

Since each Mailbox server will have 6TB of presented disks, I am looking to the most efficient backup and restore scenario, if I run daily incrementals and weekly non synthetic fulls. Even though I will be backing up to a dedupe appliance (Exagrid) I do not want to backup redundant data or have the full backups, being so large, take a tremendous amount to time, if I do not have to.

We do have a 3rd mailbox server just for Archive Databases. I was thinking about putting the DAG copies on that Mailbox server, however we will not be backing the archive mailbox server up every day.
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

Re: Exchange 2010 DAG backup ?

Veeam Logoby Andreas Neufert » Thu Jun 20, 2013 2:05 pm

@Cokovic: +1. Thank you very much for your great help.


With v7 there is a new option that addresses Snapshot commit based DAG cluster failover problems.
With Enterprise Plus you can use storage Snapshots for HP Lefthand and 3 PAR to reduce VM Snapshot time to seconds.
With HP Lefthand VSA this can be used with other Storage Systems.

Basically it is. Consistency => VM Snapshot => Storage Snapshot => VM Snapshot release => Backup from Storage Snapshot ...
Andreas Neufert
Veeam Software
 
Posts: 2209
Liked: 363 times
Joined: Wed May 04, 2011 8:36 am
Location: Germany
Full Name: @AndyandtheVMs Veeam PM

Re: Exchange 2010 DAG backup ?

Veeam Logoby janatecx » Fri Jul 12, 2013 9:29 am 2 people like this post

I have A 2 node DAG with active / passive database copies on each node.
Initially I created a backup job with the 2 exchange servers of the primary datacenter (CAS-HT-MB and HT). I cleared the HT server from this job and created a separate job for this vm backup.

For me the following steps resolved the issue:

Adjust Microsoft settings for failover sensitivity (in bold, run from command line): 1.cluster /prop SameSubnetDelay=2000:DWORD (Default: 1000)
2.cluster /prop CrossSubnetDelay=4000:DWORD (Default: 1000)
3.cluster /prop CrossSubnetThreshold=10:DWORD (Default: 5)
4.cluster /prop SameSubnetThreshold=10:DWORD (Default: 5)
5. To check settings, use: cluster /prop

Next I configured a new backup Proxy on a member server and configured like this:
Use Network (NBD) mode setting on Source Backup Proxy as opposed to Appliance (hotadd) mode for your backup and/or replication jobs in Veeam.

I then configured the backup job of the HT-CAS-MB to use the newly created backup proxy.

These steps did the trick for me.
Hopefully it will also help others.
janatecx
Veeam ProPartner
 
Posts: 7
Liked: 3 times
Joined: Thu Oct 25, 2012 2:26 pm
Full Name: Jan De Smet

Re: Exchange 2010 DAG backup ?

Veeam Logoby foggy » Fri Jul 12, 2013 10:15 am

Jan, thanks for sharing this with the community, much appreciated!
foggy
Veeam Software
 
Posts: 14904
Liked: 1096 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Exchange 2010 DAG backup ?

Veeam Logoby Daveyd » Fri Jul 12, 2013 1:16 pm

janatecx wrote:I have A 2 node DAG with active / passive database copies on each node.
Initially I created a backup job with the 2 exchange servers of the primary datacenter (CAS-HT-MB and HT). I cleared the HT server from this job and created a separate job for this vm backup.

For me the following steps resolved the issue:

Adjust Microsoft settings for failover sensitivity (in bold, run from command line): 1.cluster /prop SameSubnetDelay=2000:DWORD (Default: 1000)
2.cluster /prop CrossSubnetDelay=4000:DWORD (Default: 1000)
3.cluster /prop CrossSubnetThreshold=10:DWORD (Default: 5)
4.cluster /prop SameSubnetThreshold=10:DWORD (Default: 5)
5. To check settings, use: cluster /prop

Next I configured a new backup Proxy on a member server and configured like this:
Use Network (NBD) mode setting on Source Backup Proxy as opposed to Appliance (hotadd) mode for your backup and/or replication jobs in Veeam.

I then configured the backup job of the HT-CAS-MB to use the newly created backup proxy.

These steps did the trick for me.
Hopefully it will also help others.



Thanks for the info!

We are running a 2 node DAG as well with active ad passive copies on both. Do you backup both DAG nodes or just 1 of them? I would think backing up just one of them since each contain a passive copy of the other's active DB would be sufficient?
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google Feedfetcher and 36 guests