-
- Expert
- Posts: 213
- Liked: 26 times
- Joined: Feb 01, 2012 7:24 am
- Full Name: Espen Dykesteen
- Contact:
Re: Very slow file level restore and problem with permission
I did a instant VM recovery, and copied the InetPub folder over the network. The folder is still containing 1,68 GB of data (59137 files, 6324 folders).
The transfer speed was much better. We got 27 Mbps.
However, this is not a practically way to restore files "on demand".
In addition, in a production environment doing a instant VM recovery is not possible. Then we would needed to setup and start a SureBackup job.
Any other ideas?
If not i guess i have to open a support ticket.
The transfer speed was much better. We got 27 Mbps.
However, this is not a practically way to restore files "on demand".
In addition, in a production environment doing a instant VM recovery is not possible. Then we would needed to setup and start a SureBackup job.
Any other ideas?
If not i guess i have to open a support ticket.
-
- Expert
- Posts: 117
- Liked: 4 times
- Joined: Mar 03, 2011 1:49 pm
- Full Name: Steven Stirling
- Contact:
Re: Very slow file level restore and problem with permission
Could you please explain what changes you made to your setup? I'm having similar slow restore issues, but restores are going across the LAN as far as i know...
-
- Expert
- Posts: 213
- Liked: 26 times
- Joined: Feb 01, 2012 7:24 am
- Full Name: Espen Dykesteen
- Contact:
Re: Very slow file level restore and problem with permission
Are you refferig to my post?..
I made no changes, just used different methods for restoring.
Instant VM recovery is not always a good option in a production environment, sure backup takes long time to get up and running.
I would lilke the normal "Restoring Guest files (windows) " to just work at decent speeds.
I made no changes, just used different methods for restoring.
Instant VM recovery is not always a good option in a production environment, sure backup takes long time to get up and running.
I would lilke the normal "Restoring Guest files (windows) " to just work at decent speeds.
-
- Expert
- Posts: 213
- Liked: 26 times
- Joined: Feb 01, 2012 7:24 am
- Full Name: Espen Dykesteen
- Contact:
Re: Very slow file level restore and problem with permission
I recieved an answer from Veeam suport regarding this issue (case 00203915)
Support confirmed that they were aware of the problem, and provided a workaround: OtherOS FLR with FTP
Although this is a bit more timeconsuming to set up, the restore/copy speed was much faster, and totaly acceptable.
Complete answer from Support:
Support confirmed that they were aware of the problem, and provided a workaround: OtherOS FLR with FTP
Although this is a bit more timeconsuming to set up, the restore/copy speed was much faster, and totaly acceptable.
Complete answer from Support:
We are aware of the low performance while restoring multiple small-sized files via Windows FLR. This behaviour will definitely be improved in the future versions of Backup&Replication.
As a workaround here I suggest using OtherOS FLR with the FTP enabled. I've just tested it in our test environment and was able to get decent speed and restore some files with Norwegian symbols in their names. Basically, you just need to connect to the FLR aplliance's IP through FTP protocol using Windows Explorer or any FTP client you like and copy the files to original or any other server.
Could you please try this method and tell me if it works for you?
-
- Influencer
- Posts: 16
- Liked: never
- Joined: Nov 08, 2012 1:14 pm
- Full Name: Knut R.
Re: Very slow file level restore and problem with permission
Same answer as I got, though I use "CopyTo" instead of "Other OS FLR with FTP". Easier and decent speed for me.
Here's my short thread by the way: Windows FLR "Restore", max number of files?
Here's my short thread by the way: Windows FLR "Restore", max number of files?
-
- Expert
- Posts: 213
- Liked: 26 times
- Joined: Feb 01, 2012 7:24 am
- Full Name: Espen Dykesteen
- Contact:
Re: Very slow file level restore and problem with permission
I tried CopyTo as well, that was also way to slow. Only thing that gave me decent speed was with FTP.
This seems like a serrious issue with a basic function that needs be addressed.
We dont restore VM's on dayly basis, but files and folders gets restored almost every day.
This seems like a serrious issue with a basic function that needs be addressed.
We dont restore VM's on dayly basis, but files and folders gets restored almost every day.
-
- Enthusiast
- Posts: 40
- Liked: never
- Joined: Jul 21, 2011 10:10 am
- Full Name: Alex Whittaker
- Contact:
[MERGED] : Restore Copy to
I wonder if anyone can help. i am trying to restore some files to a new file server using "copy to" when this is running it seems to be really slow. it will stick on a word document for a long time then carry on. it is at about 1 - 2 MB/s and can drop to 362KB/s. At 138gb that is going to take forever.
thanks for any help
thanks for any help
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Restore Copy to
Hi, Alex. Are you running “Windows OS” restore? If so, can you kindly run the “Other OS” type of restore and see what speed numbers you get there.
Thanks.
Thanks.
-
- Enthusiast
- Posts: 40
- Liked: never
- Joined: Jul 21, 2011 10:10 am
- Full Name: Alex Whittaker
- Contact:
Re: Restore Copy to
i am running a windows restore as it is from a windows server to another windows server. i will try that now thanks
-
- Enthusiast
- Posts: 40
- Liked: never
- Joined: Jul 21, 2011 10:10 am
- Full Name: Alex Whittaker
- Contact:
Re: Restore Copy to
I just had a look at that but i need to preserve the Ownership and file access permissions
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Very slow file level restore and problem with permission
For now this is the best strategy, indeed.i will try that
It has been confirmed that there might be some issues while restoring number of small files though Windows OS restore.
According to plan, this problem will be addressed in the upcoming product release.
Thanks.
-
- Enthusiast
- Posts: 40
- Liked: never
- Joined: Jul 21, 2011 10:10 am
- Full Name: Alex Whittaker
- Contact:
Re: Very slow file level restore and problem with permission
there are far to many files to reset all the permissions. i will just try robocopy from the old server.
thanks
thanks
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Very slow file level restore and problem with permission
I just had a look at that but i need to preserve the Ownership and file access permissions.
While doing Other OS restore there is a tick box called “Preserve permissions and ownerships”. So, using it in order to keep original permissions settings for recovered files.
Thanks.
-
- Enthusiast
- Posts: 40
- Liked: never
- Joined: Jul 21, 2011 10:10 am
- Full Name: Alex Whittaker
- Contact:
Re: Very slow file level restore and problem with permission
there is a tick box but it is greyed out on my restore on the "other OS" on the windows it is fine apart from it taking 24+ hours to restore
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Very slow file level restore and problem with permission
I believe this option is available for Linux hosts only.AlexWhit wrote:there is a tick box but it is greyed out on my restore
-
- Enthusiast
- Posts: 40
- Liked: never
- Joined: Jul 21, 2011 10:10 am
- Full Name: Alex Whittaker
- Contact:
Re: Very slow file level restore and problem with permission
that will be why i will just have to put up with the slow restore. saying that it is back up to 7MB/s now
-
- Enthusiast
- Posts: 86
- Liked: 2 times
- Joined: Jan 20, 2011 5:45 pm
- Full Name: Mike Schlottman
- Contact:
[MERGED] Slow windows file level restore with a large ammoun
Are there any known issues with windows file level restore being slow when recovering hundreds of thousands of files? I ran into this once before and after working with support it looked like it may be related to having an orphaned user with permissions to the files. In this case there were no orpaned users. The restore speed is about 1MB/S. I can restore the entire VM very quickly from the same backup so I know it is not a problem with network or disk.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Very slow file level restore and problem with permission
Mike, yes, this looks like a known issue that will be addressed in v7, please review this topic above for details. Actually, this was announced by Anton in his weekly digest less than a month ago:
Speaking about networkless file level recovery (FLR) for Windows, the latest weekly report from R&D brough some excellent news that I am eager to share. We've had quite a few complaints on the forums that networkless restore of a large number of small files can be very slow. Our developers took a closer look at the issue, identified primary bottleneck, and redesigned the way how multiple small files are restored in v7. According to the most recent build notes, this change resulted in up to 40 times restore performance increase! Before somebody asks, Linux FLR had similar issues restoring a bunch of small files over the network, but those were caused by a different problem, and this was addressed in v7 as well (up to 50 times file restore performance increase). Pretty amazing stuff!
-
- Enthusiast
- Posts: 86
- Liked: 2 times
- Joined: Jan 20, 2011 5:45 pm
- Full Name: Mike Schlottman
- Contact:
Re: Very slow file level restore and problem with permission
This is good news. Without this improvement, I've got people telling me I need to dump Veeam for netbackup. Will V7 restores run at good speeds using v6 backups or only with backups done with V7?
Right now my only workaround is to do an instant recovery and robocopy the files to the original location. Not exactly the one click restore I need to allow operators to restore files when I am on vacation.
Thanks for the quick reply.
mike
Right now my only workaround is to do an instant recovery and robocopy the files to the original location. Not exactly the one click restore I need to allow operators to restore files when I am on vacation.
Thanks for the quick reply.
mike
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Very slow file level restore and problem with permission
Does not depend on the version the backup was created with.mschlott wrote:Will V7 restores run at good speeds using v6 backups or only with backups done with V7?
-
- Expert
- Posts: 213
- Liked: 26 times
- Joined: Feb 01, 2012 7:24 am
- Full Name: Espen Dykesteen
- Contact:
Re: Very slow file level restore and problem with permission
Have done some more testing with V7. Same Inetpub folder, where the windows restore failed (slowed down to x KB/s and practily stopped).
This is now much better.
I have a stable sped of about 2 MB/s, so there are major improvements. My guess however is that it would be faster to instant restore the entire VM and retrieve the files.
But if time is not pressing, at least the "Restore guest fiels (windows)" actually works.
6336 folders
59138 files
1,5 GB
took 16 minutes to restore.
This is now much better.
I have a stable sped of about 2 MB/s, so there are major improvements. My guess however is that it would be faster to instant restore the entire VM and retrieve the files.
But if time is not pressing, at least the "Restore guest fiels (windows)" actually works.
6336 folders
59138 files
1,5 GB
took 16 minutes to restore.
-
- Enthusiast
- Posts: 86
- Liked: 2 times
- Joined: Jan 20, 2011 5:45 pm
- Full Name: Mike Schlottman
- Contact:
Re: Very slow file level restore and problem with permission
This is not good news to me. A restore speed of 2MB/s is not acceptable when recovering 100G of data. What this means to me is that this type of recover is no longer an operator level action. It will involve escalating the recover to someone with more experience.
Is this considered to be the final resolution or is there on going work being done on this issue? Will this be logged as a bug, or is this to be accepted as the way Veeam does large restores?
Is this considered to be the final resolution or is there on going work being done on this issue? Will this be logged as a bug, or is this to be accepted as the way Veeam does large restores?
-
- Chief Product Officer
- Posts: 31816
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Very slow file level restore and problem with permission
Yes, this is the final resolution. We are talking up to 50x times improvement from previous version here, by the way.
If you ask me, 5000 files a minute is excellent performance, especially considering you are doing the restore from compressed and deduped image level backup file. I can hardly get the same speed copying files on my netbook. And 5000 files a minute should be plenty for most file restore scenarios.
By the way, you cannot really assume you will get 2MB/s on 100GB of data, as the speed will depend on the amount of files primarily. You will get much faster speed restoring fewer of larger files. Because honestly, 60000 files in 1.5 GB are really small files. Now, if you really are in need of restoring 4 million of files (extrapolating Espen's kind of data to 100GB) on a frequent basis, that's a very interesting server workload and disaster scenario there...
If you ask me, 5000 files a minute is excellent performance, especially considering you are doing the restore from compressed and deduped image level backup file. I can hardly get the same speed copying files on my netbook. And 5000 files a minute should be plenty for most file restore scenarios.
By the way, you cannot really assume you will get 2MB/s on 100GB of data, as the speed will depend on the amount of files primarily. You will get much faster speed restoring fewer of larger files. Because honestly, 60000 files in 1.5 GB are really small files. Now, if you really are in need of restoring 4 million of files (extrapolating Espen's kind of data to 100GB) on a frequent basis, that's a very interesting server workload and disaster scenario there...
-
- Enthusiast
- Posts: 86
- Liked: 2 times
- Joined: Jan 20, 2011 5:45 pm
- Full Name: Mike Schlottman
- Contact:
Re: Very slow file level restore and problem with permission
Thanks for the quick reply Gostev. I'll have to give this a try with my data. In my case the average file size is about 1/2 MB, so perhaps my performance will be much greater than 2MB/S. Once I get V7 configured I'll run a test restore to test my performance.
-
- Expert
- Posts: 213
- Liked: 26 times
- Joined: Feb 01, 2012 7:24 am
- Full Name: Espen Dykesteen
- Contact:
Re: Very slow file level restore and problem with permission
I think that restore of so many small files in 16 minutes is not bad.
Did test restore of c:\Windows (2003) and got 10 MB/s
16,5MB
Folders: 5903
Files: 43021
TIME: 29min 15 sec.
So things have improved in V7 no doubt.
Did test restore of c:\Windows (2003) and got 10 MB/s
16,5MB
Folders: 5903
Files: 43021
TIME: 29min 15 sec.
So things have improved in V7 no doubt.
-
- Influencer
- Posts: 16
- Liked: 1 time
- Joined: May 13, 2013 2:37 pm
- Full Name: Will Pulsifer
- Contact:
Re: Very slow file level restore and problem with permission
Is this still and open issue? i have the same problem with the latest patched version of v7 . i cant get over 2 mb no matter how many files i try to restore. I need to have NTFS rights preserved so other OS is a BAD option.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Very slow file level restore and problem with permission
Will, as it is stated in Anton's post above, everything depends on the number of small files you're trying to restore. Are you stating that the speed for the small number of files is also that slow? Btw, could you please describe your setup briefly, especially regarding the location of Veeam B&R management server and repository. Thanks.
-
- Service Provider
- Posts: 327
- Liked: 23 times
- Joined: Oct 09, 2012 2:30 pm
- Full Name: Maso
- Contact:
Re: Very slow file level restore and problem with permission
Is there a general issue in Veeam when you restore alot of small files? We just did a restore of 170 000 files ~ total 4 GB with Guest files Windows over lan (7.0.0.839). Average speed aroung 300 - 400 KB/s. No issue when restoring bigger files to the same server (25 - 30 MB/s). I know that big files restore better than small ones but should we not expect more?
\Masonit
\Masonit
-
- Chief Product Officer
- Posts: 31816
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Very slow file level restore and problem with permission
It's a general issue with and without Veeam. Try to perform a regular copy of these files over the network (not from Veeam backup), and you will see that it will not be much faster.
-
- Service Provider
- Posts: 327
- Liked: 23 times
- Joined: Oct 09, 2012 2:30 pm
- Full Name: Maso
- Contact:
Re: Very slow file level restore and problem with permission
Tried copy exact same files from production vm to another server. Then there is much better performance (9 MB/s). Also tried using "copy to" during restore but the perfomance then is same as with normal restore (300 - 400 KB/s)
\Masonit
\Masonit
Who is online
Users browsing this forum: Bing [Bot], stephen.mintrom and 29 guests