Exchange Server - Restore diff DSVR/DB

[dhayes16]

Hello.
This is kinda urgent so I would appreciate any feedback.
We are currently working on a major issue with an exchange server restore process and I have been trading emails with Veeam on this issue (Case # 05271954). They have been very responsive trying to help us. Basically, we have an exchange 2016 server with a very fragile database and it keeps crashing at various times. We spun up an Exchange 2019 server and have a new database on that box ready to go. We tried the usual mailbox migration that is part of exchange mailbox moves and the server crashes with a corrupt database during this process. So we MAY need to resort to restoring the veeam backups (which are backing up fine) to the new exchange database. It appears that we can restore mailboxes to the other server (although we are having issues getting the process to work...Working with Veeam on that).

But the question we have is assuming we can actually restore the mailboxes to the new server and new database how do we tell active directory and/or exchange that the users database is on the new server and database? The mailbox move process does this automatically but after the mailbox is restored to the new server and database how do we update everything to tell AD where it lives now? I do not see any documentation on this. Does Veeam restore somehow make that charge for us?

Thanks for any info.
Dave

[Mildur]

Hi Dave

Code: Select all

Get-Mailbox -Database oldDB2016 | Set-Mailbox -Database NewDB2019

This command will change the database for each mailbox, but not moving any data.
Your user will be able to connect again without any data. But they can work. Receive mails, send mails.
And you have time to restore the items as soon restore is working.

For the arbitration mailboxes, you should recreate them. I don‘t know if a restore works. Never restored this type of mailboxes.

[dhayes16]

THANK YOU for your reply. So much appreciated. This would work and allow them to continue. But I have the following questions.

1. I would just need to do it for a few users. I assume I can use the -identity switch on this command?

2. In order for this to work doesn't the old database need to be up and mounted.? Or could I run this from the new server?

Basically when this crashes I can not mount the old database.

Thank you for your response.
Dave

[Mildur]

Your welcome.
I had a similar case, where two mailboxes always get corrupted. We used the command for both of them to change the database.

I would just need to do it for a few users. I assume I can use the -identity switch on this command?

Yes, that will work.

Code: Select all

Get-Mailbox user@domain.tld | Set-Mailbox -Database NewDB2019

In order for this to work doesn't the old database need to be up and mounted.? Or could I run this from the new server?

You should be able to run it from the new server.
It changes the config (stored in AD) from the mailbox to a new database. Doesn‘t depend if old database is mounted or not.

[dhayes16]

Excellent. Thanks very much. I will give this a try and see how it goes. My issue with the restore of a mailbox to the NEW database from the veeam backup is it wants to have a user mailbox already existing in the NEW database there to restore TO. I was unaware it did that.

Does the command you sent create the mailbox in the new database automatically so I can target it from the restore process?

Again so many thanks.

[Mildur]

Yes, veeam does not create mailboxes for you.
I‘m positive that you will be able todo a restore after the database configuration for the mailbox was altered to the new database. The mailbox object should be created in the new database with that command and the old one gets disconnected.

My users had always used their outlook first with the new mailbox, before I started the restore. But it should work without using Outlook first.

[dhayes16]

I can not thank you enough. My heart is beating out of my chest when this stuff happens.

I have another site with the exact same configuration (2 exchange servers). I am going to test this at that site right now. At the moment the problem database is up and mounted so I do not want to touch it hoping it makes it to the weekend. But I need a plan B and this is it.

Thanks and I will post back.
Dave

[Mildur]

Your welcome, Dave
For the corrupted database, is it possible that it gets dismounted because 1-2 user mailboxes are corrupted (in quarantine)?
If yes, you can check it with this command after it crashes again:

Code: Select all

Get-Mailbox -ResultSize unlimited | Get-MailboxStatistics | Where-Object { $_.IsQuarantined -eq $True } | Select DisplayName,IsQuarantined

[dhayes16]

Funny you say this. I received an email from someone at this site that said exactly that...He emailed one of the people and it bounced saying the mailbox is in quarantine. I checked it and it is for sure. So I am thinking I will rehome his mailbox using the commands you sent me and restore from backup. Sound reasonable?

By the way I tested the commands on the other site you mentioned on a test database and it definitely created the empty database. I am just waiting for the database to mount in veeam and I am going to attempt a restore to that test mailbox. If it works then we are good with a plan B.!

Thank you!
Dave

[dhayes16]

I ran the command and it appears that there is only one user in quarantine. I am worried that this message is saying it will let him log back in when he gets into work and could potentially crash the system.

I will tell you that the crash yesterday happened when I initiated a mailbox move on several mailboxes. So I suspect one of those mailboxes is corrupt at some level.

WARNING: The user hasn't logged on to mailbox 'domain.com/Users/HealthMailboxVQCR281'
('1730238c-753e-45b9-adc0-df508e198b20'), so there is no data to return. After the user logs on, this warning will no
longer appear.
WARNING: The user hasn't logged on to mailbox 'domain.com/Users/DiscoverySearchMailbox
{D919BA05-46A6-415f-80AD-7E09334BB852}' ('8c13ab7f-4a60-4cfc-80fa-027c671d915b'), so there is no data to return. After
the user logs on, this warning will no longer appear.
DisplayName IsQuarantined
----------- -------------
John Davidson True

[dhayes16]

Oddly enough when I attempt to restore I get the 401 unauthorized. I see you responded to someone else with this issue below.

veeam-backup-for-office-365-f47/not-abl ... 76353.html

But I am logged in as the domain admin and I have full rights to the target database...I will poke around to see if I can resolve.

Thanks again
Dave

[Mildur]

That‘s the same issue I have faced with my customer.
Don‘t move the quarantined user mailbox.
Move the healthy mailboxes to a new database, and restore the corrupted mailboxes from the backup.

The user, which is doing the restore, needs FullAccess Permission on the target mailbox.

[dhayes16]

Thanks....It looks like the login format needed to be adminuser@domain.local instead of domain\adminuser. Once I did that my test restore worked!!! Yes!

OK...I will definitely not try to move that mailbox. My concern at the moment is the database is up and running and if I do the set-mailbox command to rehome the user database it will crash the database during the "disconnect" part. Then they will all be down. I will get a snapshot and do it at lunch I guess and roll the dice.

I have a feeling there are more corrupted mailboxes in this database.

At least I know now I have a plan B..THANK YOU

I will post back.
thx
Dave

[Mildur]

and if I do the set-mailbox command to rehome the user database it will crash the database during the "disconnect" part.

This should not happen. I don't expect any issues here
Disconnecting a mailbox is done by removing active directory attributes on a user object and not in the mailbox database.

Your welcome.
Don't hesitate to ask any other question if something goes wrong with the restore

[dhayes16]

Once last question...When you say "Restore the corrupt" databases from the backup is it safe to use the restore option to the new database from the store? Or should I export to PST and import? I am not sure if the restore option to a new database will restore corrupt data

thx
Dave

[Mildur]

Code: Select all

When you say "Restore the corrupt" databases from the backup is it safe to use the restore option to the new database from the store? 

Restore the corrupted "mailbox", not the entire database. It should be safe, to do a direct restore. I don't think, corrupted items will be accepted by the target exchange when they are written to the target mailbox over the EWS API.

[dhayes16]

Great. Well I will be doing this after 5pm EST today and will post back the results. Thanks for your help on this. Do you have a donate link or something? I would love to send you something in appreciation.

[Mildur]

You don't have to donate anything. I won't accept it. Everything ok.
It was my pleasure to help you in this matter

[dhayes16]

THANK you very much for your help! So I was able to re-home the user to the new database using your instructions in the thread and restore it from the backup to the new database. It worked perfectly. Just a couple things I needed to do for others who might have this issue.
#1) On the test I did at the other site doing a test restore I needed to use administrator@domain.com but at the problem site I needed to use domain\administrator for the credentials during the restore.
#2) I needed to setup an applicationimpersonation role as defined here..https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Once I did that the restore worked perfectly.

Again thanks for your help. I still have an unstable database to contend with but it is mounted and I am hoping it will make it to the weekend so I can take it down and migrate. At least the veeam backups are working perfectly.

[Mildur]

Your welcome.
I‘m glad it worked and thanks for the summary with the permission