Very slow initial loading of FLR, normal?

[GreenEnvy]

Hi all,
We have 3 sites each with Veeam BR 6.5. All 3 sites have vSphere 5.1 deployments. All three have vSphere connecting to an Equallogic SAN (one per site) via iSCSI (jumbo enabled, dedicated VLAN) for the VM's.
Veeam is running on a Windows 2012 VM so it can restore files from our Windows 2012 file servers (that have dedupe enabled on them).

We backup 24 VM's on our main site, reversed incrememntal, and keep 10 copies. The backups themselves seem to perform well. The backups are stored on a Overland Storage Snapserver, it is mapped via iSCSI initialtor in Windows to the VM Veeam is running on.

The question is about when we go to do file level restores (Windows). We pick our restore point, enter in the notes, and click Finish. Then it takes 45 minutes to an hour before the window to browse the files actually appears. Once we find the files we want and do a Copy-To (we don't restore back in place generally), we get about 12 MB/s transfer rates, which don't seem to line up with the very slow opening of the backup.

I'm not looking for support here, just wondering if this is normal or not.

Thanks
Lee

[Gostev]

No, normally it should takes under 10 seconds. Try backing up some VM to a local disk on the backup server itself, and doing FLR from that backup instead.

[GreenEnvy]

Ok thanks.

I took a look in log files and see thousands of these errors each time we try a FLR:

Code: Select all

[16.04.2013 15:30:11] < 8> Warning  rp: update: error = 5, id = 562949954140319.
[16.04.2013 15:30:11] < 8> Error    ERROR: fid = 0505: step = 0002, error = 5: 00000005
[16.04.2013 15:30:11] < 8> Error    Access is denied.

[16.04.2013 15:30:11] < 8> Warning  rp: update: error = 5, id = 562949954140320.
[16.04.2013 15:30:11] < 8> Error    ERROR: fid = 0505: step = 0002, error = 5: 00000005
[16.04.2013 15:30:11] < 8> Error    Access is denied.

I'll open a support case.

[veremin]

I'll open a support case.

Furthermore, please update the topic with support case ID for further reference and also kindly share found solution for the convenience of future readers.

Thanks.

[GreenEnvy]

CaseID 00198431

[emachabert]

I am seeing the same behaviour on a 4TB Windows 2012 file server.
I started a FLR job 30 minutes ago and it's still in "please wait".
Have you any feedback ?

[GreenEnvy]

Support told me its a known issue in 6.5 involving reparse points. My log files are full of errors about reparse points.
This affects doing a windows FLR of a deduped Win2012 volume.
They say it will be fixed in veaam 7.

[foggy]

Eric, from the OP's support case I can tell that his particular problem was caused by the known issue in v6.5 that is to be addressed in the future release. The issue itself consists in the large number of NTFS reparse points in the guest OS, which requires significant time to process by FLR Backup Browser.

As a workaround you can use Other OS FLR Wizard, however, take in mind, that deduplication volumes will not be recognized by the Linux FLR appliance. So probably the best choice for restoring from this VM (if Data Deduplication feature is enabled on its disks) is to use Instant Recovery.

Also, I suggest opening your own case with our technical support to assure that you experience the same issue.

[emachabert]

Hi Alexander,

Ok this is exactly what I was seeing in the VeeamShell.log, many reparse points.
I have also the same log entries as shown above: Error ERROR: fid = 0505: step = 0002, error = 5: 00000005.

For the moment, it takes 45 minutes to open the FLR window and generate 1GB VeeamShell logfile (flooded by the access denied error code). After that, the restore speed is good ( 30MB/s).
So I think we will wait for v7 to be released.

One last question, can I safely remove the VeeamShell_1.log which is 1GB in size or you have a log rotation/cleaning process.

[foggy]

Some of the logs are truncated and compressed automatically and there is also a retention for those compressed logs, but you can delete any of them manually if you'd like to free up space.

[GreenEnvy]

As a followup, as Veeam support said the issue was caused by reparse errors, a known but in 6.5. I've installed 7.0 and now our restores open immediately.

[foggy]

Thanks for confirming, much appreciated.

[emachabert]

Nice.
I'll update then