Unable to make use of imported backup

[Fiskepudding]

I have a situation here, where I am asked to get some old mail from backups. Back to around January this year.

I took our offsite disk backup archive, imported two DC's and the exchange server to Veeam.
By selecting the .VBM files. So far all is good, they all imported without hickups.

The DC's works fine I can do an instant VM restore, and FLR from othem.. no problem.

(Below our NAS is replaced with xxxx, and exchange server with EX.)

When I try to do anything with the imported exhange backup I get errors.
If i try to restore guest files I get this error:

Code: Select all

31.10.2013 13:40:32          Mounting restore point
31.10.2013 13:40:32 Warning    Storage version [3701403] is not supported for read-only access.
                               --tr:Failed to open storage for read access. Storage: [\\xxxx\BACKUP\TEMP\exc04.2013\EX12013-04-20T180142.vbk].
                               --tr:Cannot apply patches stored in folder [HostFS://\\xxxx\BACKUP\TEMP\exc04.2013\EX2013-04-20T180142.vbk[41e6829d-533e-480f-8b38-3726d3fa47a2 (vm-4628)]].
                               --tr:Failed to build restore point.
                               Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://Tar2Text]. CHM
31.10.2013 13:40:32 Error    MountRestorePoint failed, mountId '51b979b9-776b-4ab8-960e-18801912c33c'
                             FlrManagementService: Failed to mount restore point (mountId:51b979b9-776b-4ab8-960e-18801912c33c)
                             TextFromTar failed
                             Client error: Storage version [3701403] is not supported for read-only access.
                             Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://Tar2Text]. CHMOD mask: [0].
                             
31.10.2013 13:42:23 Error    Job has failed unexpectedly

If I try to do an instant VM restore I get this:

Code: Select all

31.10.2013 12:03:20 Error    Publishing VM
31.10.2013 12:03:20          Canceling backup file lock
31.10.2013 12:03:20 Error    Failed to publish VM EX Error: Client error: Storage version [3701403] is not supported for read-only access.
                             Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://Tar2Text]. CHMOD mask: [0].

So i removed the imported backup, and tried to add directly from the .VBK file.
That gives an error too.. Witch probably is the reason for errors above too:

Code: Select all

"Dir failed, storageFileName '\\xxxx\BACKUP\TEM\exc04.2013\EX2013-04-20T180142.vbk' Unsupported storage version: [3701403].

All these backup filse were taken offsite on a disk encrypted with TrueCrypt.

We mouoted the disk and shared it first, but after the errors (same as above), we copyed the backups to our NAS.
Same errors.

The DCs were on the same disk as the exchange backup, and they worked fine.
We have aslo tried to import another exchange backup from April, same issue with that one.
Recent backups of the exchange, that are not taken offsite and encrypted, works.

The user account used to import DC's and Exchange to veeam are the same, and the secrurity on the NAS on these folders are the same.

I have no idea why this is happening.Any ideas?
Could the backups be corrupted?

[foggy]

Espen, remember to include support case ID, please.

[Fiskepudding]

Sure..sorry.
caseID: 00470121

[Vitaliy S.]

Recent backups of the exchange, that are not taken offsite and encrypted, works.

Just to make sure we are on the same page, local backups encrypted with TrueCrypt do work, but the backup files that were also taken offsite do not, right?

[Fiskepudding]

No, local backups are not encrypted. They work.
We encrypt them when we copy backup to external disk, and take them offsite.

We brought some backups back.
First we mounted the disk up and shared it, then import it to Veeam. Did not work on the large exchange backup.
From the same disk, backups from small DC's are no problem. (Also encrypted)

Support had a webex 2 days ago, and they told me they were going to put some other tecs on the case.

Just to avoid any encryption issues with the restore prcess, we copied the files from the encrypted disk to the NAS.
Then tried to open them from there, same result. (Small backups seems ok, large do not)
We will do some more testing with other large servers today or tomorrow.

However we have now managed to open 1 of 3 exchange backups. (all encrypted)
First two did not work, then I created this post and case, just recently we tried a third one, and it worked.

I guess there could be some corruption issues with the files when they grow large (800GB), especially with encryption in mind..?..

[Fiskepudding]

Support told me today that they have reviewed the logs again, and conclude that the backup must be corrupted.

We have started to test all of our external backups. (All were tested with surebackup prior to being offloaded to external disk and encrypted. So they should be at a good state at this point.)

If it is interesting, we have used TrueCrypt for about 1 year. So far the testing shows.
2 of 2 disks tested prior to TrueCrypt = 100% Success rate
2 of 6 disks tested after TrueCrypt = 66% Success rate

If it is TrueCrupt or something else that is the root to this is too soon to conclude with.
Need to test more backups, but it raises some suspicion towards TrueCrypt
Note that even on the same disk where we cannot open the exchange VBK other VBKs from other servers works fine. Si I guess size plays a large part in this as well.

So it seems Veeam is in the clear on this one.

[RMullis]

Just throwing this out there, could it be a similar issue to this:

http://forums.veeam.com/viewtopic.php?f=2&t=19157

Rick

[Fiskepudding]

Thanx for the tip Ric, but I dont think they are related.

[yizhar]

Hi.

1. I think that the above link might be related and worth investigation around the issue with restoring from large VBK weather on tape or disk. Read this:
> ... The real problem is the restoration of any large server from tape. If I restore a large server (800GB+) I get either ‘Script Timeout Error” or “An existing connection was forcibly closed by the remote host ...

It might not be the exact same issue or environment, but might still be related to similar cause or problem.

2. Regarding Exchange restore - were you able to restore windows guest files from the backups?
If so, have you tried to restore only the exchange EDB files?

Yizhar

[Fiskepudding]

First, just to make it clear, i don’t get "‘Script Timeout Error” or “An existing connection was forcibly closed by the remote host ..."

I am able to import the VBM file (you know the little "index" file). From there I cannot do anything with the imported backup. Because veeam obviously cannot read the VBK.

If I try to import the VBK directly it get an error immediately. So i can NOT restore windows files, VMware files and so on, because veeam cannot read the VBK.
Hence, no EDB files restored.

The first two backups that we took back from disk, veeam could not read. I test many other backups (different dates off cause) that worked fine, newer and older, bigger and smaller VBK's
So I am pretty sure these first tested VBKs are corrupted somehow. Size does not matter... it seems. Some might argue with that one

No worries though, all other backups we tested have worked.
Most important we got the data we was looking for.

[Fiskepudding]

Just wanted to say this was defiantly due to corrupted files.

We have now tested at least 100 backup files, encrypted with TrueCrypt and not encrypted. And the only one that had issues were the two exchange VBK's mentioned earlier..
What are the odds.

[foggy]

Thanks for the update. No reasons to blame TrueCrypt then.

[Fiskepudding]

Thats correct, TrueCrypt is in the clear.

[RMullis]

Just throwing this out there, could it be a similar issue to this:

http://forums.veeam.com/viewtopic.php?f=2&t=19157

Rick

This issue is not related to the resolved issue above. Above was restore from tape with large files only.

Thanks,
Rick