Availability for the Always-On Enterprise
Post Reply
cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Slow file level restores from dedupe appliance

Post by cookiejc » Nov 30, 2016 11:15 am

Hi, we have some large file servers which have Veeam backups stored on a dedupe appliance (NetApp Altavault). Mounting the backup for file level restores takes between 1-2 hours. Veeam support have previously advised that this is due to the time it takes to scan the entire backup file set. I noticed the following in the Svc.VeeamMount.log:

Source.DisableFullBlockRead = false

I was hoping this implied there's a way to set this value to "true" and stop the full block read prior to performing a restore?

PTide
Veeam Software
Posts: 4247
Liked: 349 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Slow file level restores from dedupe appliance

Post by PTide » Nov 30, 2016 2:02 pm

Hi,

Please provide some details about your setup. What are the job settings, is encryption enabled? Do you have "Decompress backup data blocks before storing" check box selected in the repository settings?

Thanks

csinetops
Expert
Posts: 113
Liked: 15 times
Joined: Jun 06, 2014 2:45 pm
Full Name: csinetops
Contact:

Re: Slow file level restores from dedupe appliance

Post by csinetops » Nov 30, 2016 2:13 pm

That is odd. Are the backup sets sitting in local cache or are they out in the cloud and need to pull back down? I also have a AltaVault and just tested my biggest VM on there @ 3.8TB. A cached backup doing Windows FLR came up in under 5 minutes. I have dedupe and compression turned off and let the AltaVault handle that.

foggy
Veeam Software
Posts: 16691
Liked: 1343 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Slow file level restores from dedupe appliance

Post by foggy » Nov 30, 2016 2:16 pm

Elaborating on this, you could try to enable partial block reading (which is basically a default setting) via registry, provided backups are not encrypted and compression is not enabled.

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Slow file level restores from dedupe appliance

Post by cookiejc » Nov 30, 2016 3:45 pm

Hi, thanks for your replies.

Compression, dedupe and encryption is turned off in the jobs as the Altavault does that. The "decompress backup data blocks before storing" setting is unticked in the repositories (as per the NetApp best practice they gave us)

These backups are still stored in cache as we haven't had to evict any data yet

These file servers are about 20TB each which doesn't help

foggy can you point me towards that registry setting?

thanks again

foggy
Veeam Software
Posts: 16691
Liked: 1343 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Slow file level restores from dedupe appliance

Post by foggy » Nov 30, 2016 4:02 pm

Try setting the DisableFullBlockRead value to 3 on the Veeam B&R server and restart the Veeam Backup Service.

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Slow file level restores from dedupe appliance

Post by cookiejc » Nov 30, 2016 4:06 pm

Foggy, can you advise where that registry value is, or needs to be created?

foggy
Veeam Software
Posts: 16691
Liked: 1343 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Slow file level restores from dedupe appliance

Post by foggy » Nov 30, 2016 4:09 pm

The standard Veeam B&R hive is HKLM\SOFTWARE\Veeam\Veeam Backup and Replication. And the value type is DWORD.

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Slow file level restores from dedupe appliance

Post by cookiejc » Nov 30, 2016 4:10 pm

I have looked in there and that key does not exist... does it need to be created to enforce this setting?

foggy
Veeam Software
Posts: 16691
Liked: 1343 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Slow file level restores from dedupe appliance

Post by foggy » Nov 30, 2016 4:11 pm

Correct, you need to create it.

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Slow file level restores from dedupe appliance

Post by cookiejc » Dec 01, 2016 2:23 pm

Thanks I've now implemented this change and can see it has taken effect in the log but so far it looks like mount times are still slow. The log has got to this point and just stops... I expect it to take about an hour from this point:

Code: Select all

Source.DisableFullBlockRead = true\n  (EString) Processing.Part[0].Type = DynGroup\n  (EString) Processing.Part[0].GptDiskObjectPath = DynGroup-b2528697-0783-4072-8efd-95674944f8fb\n  (EString) Processing.Part[1].Type = OracleAsm\n  (EString) Processing.Part[2].Type = DiskIds\n}\n'
Any more ideas?

foggy
Veeam Software
Posts: 16691
Liked: 1343 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Slow file level restores from dedupe appliance

Post by foggy » Dec 01, 2016 3:11 pm

Actually partial block read doesn't affect mount process, it is effective during actual restore. While mount itself requires random reads from guest OS MFT and expected to be slow for large volumes with large amount of files and fragmented MFT, especially on a dedupe device.

rreed
Expert
Posts: 354
Liked: 72 times
Joined: Jun 30, 2015 6:06 pm
Contact:

Re: Slow file level restores from dedupe appliance

Post by rreed » Dec 01, 2016 4:19 pm

What is your block size (job settings, Advanced, Storage, Storage Optimization)? In v8 we found cranking that up to Local Target 16+TB (largest block size) made a massive improvement on restores from a dedupe device. Compression off but inline dedupe on (you're only moving around the dedupe but the end device still finds some to do). I think they've made some improvements in v9 but we never went back to turn that down to confirm - it's always been working so we continue to leave well enough alone for now.
VMware 6
Veeam B&R v9
Dell DR4100's
EMC DD2200's
EMC DD620's
Dell TL2000 via PE430 (SAS)

foggy
Veeam Software
Posts: 16691
Liked: 1343 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Slow file level restores from dedupe appliance

Post by foggy » Dec 01, 2016 4:27 pm

The block size doesn't really matter if blocks can be read partially.

Post Reply

Who is online

Users browsing this forum: No registered users and 21 guests