Very slow file level restore and problem with permissions

Availability for the Always-On Enterprise

Re: Very slow file level restore and problem with permission

Veeam Logoby tsightler » Wed Sep 24, 2014 3:44 pm

I'm going to do a blog post on this but suffice it to say that the Veeam file browser, while it has improved significantly over the years, still has a ways to go to close the performance gap with tools like robocopy and xxcopy. Fortunately you can still use those tools with Veeam by access the C:\VeeamFLR folder and copying files from there instead. I expect we'll continue to enhance the performance of the file browser over time, but it's pretty hard to be these tools for now and robocopy has been a native part of Windows since 2008R2.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Very slow file level restore and problem with permission

Veeam Logoby davidkillingsworth » Mon Feb 09, 2015 12:57 pm

Hi,

I am using Veeam 8 (patched) and I'm trying to restore guest files from a linux box and store them on the same Windows server on an attached NAS drive where the Veeam backup jobs are configured.

It ran for 3 hours and nothing happened. I then canceled, restarted the FLR restore and copied just a few folders and it works, but it's very very slow. There are hundreds of subfolders in hundreds of subfolders.

It's just unusable as a solution.

Is there anything I can do to speed this up. I'll try to FTP them from the linux box to Windows and see if that helps.
davidkillingsworth
Enthusiast
 
Posts: 29
Liked: 4 times
Joined: Mon Nov 21, 2011 5:41 am
Location: Hong Kong

[MERGED] File level restore slow

Veeam Logoby gktechnology » Wed Apr 29, 2015 6:47 am

Just had to restore 71GB of data to a Windows Server VM from a backup repository. This was in a single folder and consisted of around 120K subfolders & files. Speed was showing at 2MB/s and it took nearly 10.5 hours to complete. Just let it run after hours and it was successful but it doesn't sound right to me. I can download from our online backup service provider quicker than this!

Any suggestions as to where I might start looking for a bottleneck?
gktechnology
Novice
 
Posts: 3
Liked: never
Joined: Tue Apr 28, 2015 8:55 pm
Full Name: George Bradley

Re: Very slow file level restore and problem with permission

Veeam Logoby Shestakov » Wed Apr 29, 2015 8:45 am

Hello George,
As you can see in the thread, it`s a known issue when copied a lot of small files.
What version of VBR are you at?
Thanks.
Shestakov
Veeam Software
 
Posts: 4856
Liked: 394 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Very slow file level restore and problem with permission

Veeam Logoby gktechnology » Fri May 01, 2015 8:17 pm

7.0.0.841 if I remember correctly.
gktechnology
Novice
 
Posts: 3
Liked: never
Joined: Tue Apr 28, 2015 8:55 pm
Full Name: George Bradley

Re: Very slow file level restore and problem with permission

Veeam Logoby Shestakov » Fri May 01, 2015 9:34 pm

Thanks for the reply George. Try to update to the newest version. There were a some improvements in FLR performance tehre.
Thanks!
Shestakov
Veeam Software
 
Posts: 4856
Liked: 394 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Very slow file level restore and problem with permission

Veeam Logoby lando_uk » Tue Jul 21, 2015 5:16 pm

Hi, just to revive this thread.

Using file level (copy) task I'm currently restoring 25GB files (47,000 files, 5300 folders) and I'm getting 2 MB/s. So its going to take about 4hrs.

This is to recover a ransomware attack. We use both Veeam and Commvault, Commvault is much quicker restoring so this isn't a OS specific thing, its a Veeam performance thing.

Looking at the data flow, the restore accesses the vbk and vib on the backup repository, it then gets sent to the B&R master server that the file explorer is running from, and then it gets sent back to the guest server. Wouldn't it be better if the files went straight from the repository/proxy to the guest, rather than going around the houses?

I was thinking maybe its the slow RAID6 repository with forward incrimentals, but reading the forums it seems 2 MB/s is the norm for these kinds of restores, and even if I had more IO performance it wouldn't matter.

m
lando_uk
Expert
 
Posts: 236
Liked: 17 times
Joined: Thu Oct 17, 2013 10:02 am
Full Name: Mark

Re: Very slow file level restore and problem with permission

Veeam Logoby Gostev » Tue Jul 21, 2015 10:39 pm

lando_uk wrote:Looking at the data flow, the restore accesses the vbk and vib on the backup repository, it then gets sent to the B&R master server that the file explorer is running from, and then it gets sent back to the guest server. Wouldn't it be better if the files went straight from the repository/proxy to the guest, rather than going around the houses?

This will be an option as a part of ROBO enhancements in v9. Please stay tuned to v9 feature announcements for more information, in fact these enhancements will be the part of next announcement I believe.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Very slow file level restore and problem with permission

Veeam Logoby dellock6 » Wed Jul 22, 2015 6:52 am

You remember correctly Anton, just wait one week ;)
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5047
Liked: 1330 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Very slow file level restore and problem with permission

Veeam Logoby lando_uk » Wed Jul 22, 2015 10:03 am

You know, what we also need is a permissions only restore option on the File Explorer - was this feature we have used a lot with Commvault, when people have accidently destroyed permissions structures, but not files.
lando_uk
Expert
 
Posts: 236
Liked: 17 times
Joined: Thu Oct 17, 2013 10:02 am
Full Name: Mark

Re: Very slow file level restore and problem with permission

Veeam Logoby Erhardt » Mon Dec 14, 2015 9:09 pm

Had the same issue today (Veeam 8 U3), recovering 38GB and tons of folders and subfolders.
First the recovery were very slow (200kb/s), then it increased to 3-4 MB/s.

Next time i will try the "VeeamFLR" folder. I look forward to Veeam 9 :P
Erhardt
Influencer
 
Posts: 18
Liked: never
Joined: Fri Aug 23, 2013 2:25 pm
Full Name: Erhardt

Re: Very slow file level restore and problem with permission

Veeam Logoby infused » Mon Mar 21, 2016 11:52 pm

Bumping as I had to do this today.

60,000 files in 40gb of data.
200kb/s

This is on version 9. In the end, I just did a complete vm recovery which took 5mins or so at 250mb/s.

So is there an issue still?
http://www.infused.co.nz - My Blog.
infused
Enthusiast
 
Posts: 93
Liked: 3 times
Joined: Sat Apr 20, 2013 9:25 am
Full Name: Hayden Kirk

Re: Very slow file level restore and problem with permission

Veeam Logoby Gostev » Tue Mar 22, 2016 12:31 am

Need to investigate the specific case, for example the issue can be due to restore going over VIX instead of network.
Other than that, I've already seen people reporting 5-10 times faster FLR with v9 comparing to v8 here on the forum.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Very slow file level restore and problem with permission

Veeam Logoby infused » Wed Mar 23, 2016 11:19 pm

This was VIX as it's impossible to do network.

What I did want to ask though is how I can use the other os with segregated networks?

For example, I tried doing this. The VM started, data was mounted, I could ping the device on the network it was deployed to (separate distributed switches), but because Veeam couldn't talk to it, it timed out.

Is there a way to get this to work, or am I doing it wrong?
http://www.infused.co.nz - My Blog.
infused
Enthusiast
 
Posts: 93
Liked: 3 times
Joined: Sat Apr 20, 2013 9:25 am
Full Name: Hayden Kirk

Re: Very slow file level restore and problem with permission

Veeam Logoby foggy » Thu Mar 24, 2016 2:53 pm

You can configure routing to allow access from Veeam B&R console to FLR appliance.
foggy
Veeam Software
 
Posts: 14741
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Majestic-12 [Bot], mark.planger, soylent and 28 guests