Urgent Help! Exchange restore not working

[samuk]

Hi

I have 300 users who have not been able to use Exchange as the restore has completed with a corrupt database.

Veeam backup 4.1 used with VSS to backup exchange 2003 wich has always completed ok.

I have restored, however the exchange databases are not mounting!

I have a very angry customer who wants answers as why all the support they pay for said:

Sorry sir, if your Databases are corrupted after a restore, you need to recreate your mailbox stores.

We have near 2000+ mailboxes..

HELP

[vbussiro]

That's the reason for surebackup. Hope I'll never run into this kind of trouble.
Anyway, have you tried the classic eseutils repair tools ?
http://technet.microsoft.com/fr-fr/libr ... G.65).aspx

First make a copy of edb/log files...

[Vitaliy S.]

Hello Sam, could you please tell me your ticket number? Also could you tell me what was the exact error when you've tried to mount the database? Additionally, could you tell me if you've tried to restore the database from a different restore point and what the result was.

[Gostev]

That's the reason for surebackup.

That's absolutely right... any backup simply captures the production server image, including whatever issues it has. So if your production server has any issues, then you can be 100% sure that you will get these on the server restored from the most recent backups. Obviously, they will have the same issue preventing mailbox stores mount captured - backup does not really do miracles, it simply mirrors the virtual disks data and stores the exact copy. This is why backup recoverability must be tested regularly, and this is why regular integrity check is not enough.

The only resolution from Veeam side I see here, as Vitaly correctly suggested, is to go back and restore older backups, which were taken before the corruption occured on the production Exchange server. Hopefully, it did not happen weeks ago...

But of course, better option would be trying out the native repair tools.

[samuk]

Ref Number: 529066

This is for a customer who has bought support.

The original exchange server was rebooted prior to the corruption due to hardware failure on the SAN and it came up no issues.

Eseutil repair works, though from looking at the size 200GB will take near 40+ hours. We did a smaller store of 10GB and this was stopped after 1.5 hours. so the decision was made to restore from backup.

I was only able to restore from the last backup as the restore time is 4 hours due to the size of the server, and do not have time to go back further.

So i have done the following:

Restored the Server
The server logged in and auto rebooted
The information store service started but none of the stores are on-line.


I will be removing the logs and running the eseutil / r command now.




This is one of the errors

Event Type: Error
Event Source: MSExchangeIS
Event Category: General
Event ID: 9518
Date: 28/07/2010
Time: 10:28:30 AM
User: N/A
Computer: EXCHANGE01
Description:
Error Read verification error starting Storage Group /DC=uk/DC=org/DC=teachfirst/CN=Configuration/CN=Services/CN=Microsoft Exchange/CN=Teach First/CN=Administrative Groups/CN=First Administrative Group/CN=Servers/CN=EXCHANGE01/CN=InformationStore/CN=First Storage Group on the Microsoft Exchange Information Store.
Storage Group - Initialization of Jet failed.
For more information, click http://www.microsoft.com/contentredirect.asp.

Event Type: Error
Event Source: MSExchangeIS
Event Category: General
Event ID: 9518
Date: 28/07/2010
Time: 10:28:30 AM
User: N/A
Computer: EXCHANGE01
Description:
Error Read verification error starting Storage Group /DC=uk/DC=org/DC=teachfirst/CN=Configuration/CN=Services/CN=Microsoft Exchange/CN=Teach First/CN=Administrative Groups/CN=First Administrative Group/CN=Servers/CN=EXCHANGE01/CN=InformationStore/CN=First Storage Group on the Microsoft Exchange Information Store.
<NULL>.
For more information, click http://www.microsoft.com/contentredirect.asp.




if the store was corrupted from before, this caused no issues with server restarts. The server was still backed-up ok acording to veeam..when restoring the server maybe it looks at the store differently as sees the corruption? maybe??


Appreciate the help...

[Gostev]

when restoring the server maybe it looks at the store differently as sees the corruption? maybe??

That is correct, the full VM restore process is different from simple server reboot, as our restore logic performs additional steps required for correct VSS-aware restore. This includes automatically dismounting the stores, performing VSS-aware restore, and then mounting the stores back.

I am afraid eseutil is the only option here, if you cannot go back and restore earlier backups.

[samuk]

Hi Gostev,

point taken, but going back would not 100% fix the problem, as this is the first time they have had to restore a full exchange server since going virtual. If this problem existed from before un-noticed we would never known until a disaster occurs..like now..

Sam

[gopher_49]

I had a similar problem.. Do you know if the disks where set to non-persistent mode?

[gopher_49]

Also,

When I did the restore (after taking the disks out of non-persistent mode) I had to make sure none of the existing logs where applied. When I applied any of the existing logs the db would not mount. Once it commited NEW logs the db would mount. This is an older ticket and probably resolved but I ran into simliar problems.

[joergr]

now let´s calm down and have this checked right: a 9518 with a jet init error especially after a restore points in 90% of all cases to a wrong drive letter after the restore (which happens quite often in the wild, honestly), that means: the drive letters are not exactly the same as before. please check that first. as long you don´t see a 1018 you should be able to get this working again, especially with eseutil (my experience). Also, you could try to move the edbs/logs away, let exchange create completely new ones, check if exchange comes up and then replace these with your previously moved. There a A LOT of kb articles regarding these cases and again, as long you don´t see a 1018 my guess is you get all your data back for good.

regards,
Joerg