Incredibly slow Direc SAN restore

VMware specific discussions

Re: Incredibly slow Direc SAN restore

Veeam Logoby PG5150 » Tue Sep 08, 2015 3:49 am

I was wondering if there was any feedback in relation to the post from Taran and MrDave.
PG5150
Lurker
 
Posts: 1
Liked: never
Joined: Wed Jul 16, 2014 5:26 am
Full Name: Philip Gardiakos

Re: Incredibly slow Direc SAN restore

Veeam Logoby SyNtAxx » Fri Sep 11, 2015 11:05 am

I was wondering if their is a solution to the problem yet.
SyNtAxx
Expert
 
Posts: 127
Liked: 14 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: Incredibly slow Direc SAN restore

Veeam Logoby Gostev » Sat Sep 12, 2015 1:31 am

I overheard recently that support now has a patch that switches thick disk restore mode to eager zeroed.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Incredibly slow Direc SAN restore

Veeam Logoby SyNtAxx » Sat Sep 12, 2015 11:55 pm

Gostev wrote:I overheard recently that support now has a patch that switches thick disk restore mode to eager zeroed.

Fantastic! is it general release for public consumption, or do you have to ask nicely for it :) ?

Thanks.
SyNtAxx
Expert
 
Posts: 127
Liked: 14 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: Incredibly slow Direc SAN restore

Veeam Logoby Gostev » Sun Sep 13, 2015 1:16 am

With rare exceptions, you don't have to be nice to get hot fixes which are already in support's hands ;)
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Incredibly slow Direc SAN restore

Veeam Logoby Taran » Sun Sep 13, 2015 10:32 pm 1 person likes this post

Here's an update for everyone. The hotfix supplied by support improved the restore speeds to 200MB/s. There is some further work happening, however this is a significant improvement.

Asking support nicely is the way to go to get the hotfix (being nice never hurts). It might be a good idea to reference this forum post.
Taran
Veeam Software
 
Posts: 65
Liked: 8 times
Joined: Tue Jul 16, 2013 5:05 am
Full Name: Taran

Re: Incredibly slow Direc SAN restore

Veeam Logoby MrDave » Mon Sep 14, 2015 12:09 am

Worked a treat. We do believe however that we should be able to achieve speeds faster than 200MB/s as we have some SSD trays in our SAN. There is further investigations happening there.
MrDave
Novice
 
Posts: 3
Liked: never
Joined: Wed Sep 02, 2015 6:50 am

Re: Incredibly slow Direc SAN restore

Veeam Logoby Lasr » Fri Oct 09, 2015 12:13 pm

Hi,

is this hotifx included in B&R 8.0 Update 3?


Regards
Lasr
Lasr
Influencer
 
Posts: 16
Liked: 4 times
Joined: Tue Dec 23, 2014 2:22 pm

Re: Incredibly slow Direc SAN restore

Veeam Logoby foggy » Fri Oct 09, 2015 12:23 pm

No, chances are this will be addressed in v9.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Incredibly slow Direc SAN restore

Veeam Logoby Gostev » Fri Oct 09, 2015 5:05 pm

Based on hot fix testing, for v9 we are simply planning to make this a default behavior.
Right now, there is simply not enough time to build and test delivering this as an option.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Incredibly slow Direc SAN restore

Veeam Logoby SyNtAxx » Wed Oct 14, 2015 10:10 pm

What is the path/hotfix number so I may request it?

-Nick
SyNtAxx
Expert
 
Posts: 127
Liked: 14 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: Incredibly slow Direc SAN restore

Veeam Logoby v.Eremin » Thu Oct 15, 2015 10:06 am

Within an existing support ticket just refer to the Taran's post. I think it should be enough to get the said fix. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Incredibly slow Direc SAN restore

Veeam Logoby Anders » Fri Oct 16, 2015 8:23 am 1 person likes this post

With a EMC VNX5300 attached with 8G FC I got these numbers
The VM I've testet this with is 500 GB in size. (1x 100 GB, 2x 200 GB VMDKs)
Direct SAN backups run at around 150 MB/s
Network backup runs at 70 MB/s (1G Network)
Restore via Network runs at about 70 MB/s (1G Network)
Restore via DirectSAN without patch runs at 25-45 MB/s
Restore via DirectSAN with the patch runs at about 99 MB/s
The problem with the patch is that it takes ages before the restore starts because it creates an empty zeroed VMDK file first, and that took longer than the entire VM Restore does via Network. We are pretty sure this is because we have VAAI disabled due to a bug in the EMC VNX firmware.

In the end we uninstalled the patch.

So making the patch the default in v9, without a way to disable this, will make people with VAAI disabled cry!

Also, remember to include your Veeam build number when requesting the patch via support - will speed up getting the patch by a day or two!
Anders
Enthusiast
 
Posts: 36
Liked: 7 times
Joined: Thu Sep 09, 2010 9:45 am
Full Name: Anders Lorensen

Re: Incredibly slow Direc SAN restore

Veeam Logoby foggy » Fri Oct 16, 2015 1:59 pm

There will be a registry value to switch the behavior. Btw, thanks for this feedback, much appreciated.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] Slow restore san mode - testing, weird result

Veeam Logoby m1kkel » Fri Dec 18, 2015 11:26 pm

Hello.

Just finishing setting up restore and backup via san mode. Had some issues with restore, because of lun's were write protected on my proxy server, which is windows server 2012R2, because of default san policy.

Anyway, the restore of an entire VM with only 1 restore point, was slow, around 77 MB / sec. Tried Hotadd, which was faster, around 90 MB / sec.

Here is the really weird part. Server is 2012R2 running storage spaces, 12 disk presented as jbod without raid controller, SATA 7,2K Mirror with deduplication. CPU is xeon something with 4 cores, and 32 GB memory. Now, i am copying a file internally on the server, from the V: drive (for veeam :-) ) to the C drive, which is a SSD disk (in mirror on another controller)
Copying a VBK file from the backupupjob folder called "DirectSan_test_Hostingtest" to the C: drive, i get a speed rating around 30-130 / sec. (size is around 30GB) Copying some VIB files from other folders around 17 GB in size, i get speed around 500-700 MB pr. second which is as expected. And some of the other VIB files from some of the other backup jobs are REALLY slow just copying internally from V: to C: and some are extremely fast.
Volume is NTFS, formatted in 64 KB cluster size, and formatted with the option for large drives.

This does not make any sense to me, is something fragmented, and maybe some of the chunks are placed randomly on the volume, and maybe some are aligned up nicely?

Settings in veeam for storage is local target, and optimal compression.
In veeam, i am not using the option for "Align backup file datablocks"

Suggestions?
m1kkel
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Thu Nov 06, 2014 8:01 pm
Full Name: Mikkel Nielsen

PreviousNext

Return to VMware vSphere



Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Majestic-12 [Bot] and 6 guests