Incredibly slow Direc SAN restore

VMware specific discussions

Re: Incredible slow Direc SAN restore

Veeam Logoby foggy » Thu Jun 25, 2015 12:56 pm

You can check whether other restore modes work better for you. I would say that we're aware of the issue and currently monitoring the demand for implementing eager zeroed disks restore.
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Incredible slow Direc SAN restore

Veeam Logoby WimVD » Thu Jun 25, 2015 2:00 pm

+1 for implementing eager zeroed disk restores :)

And since we're feature requesting it would be nice to use storage integration (3par, netapp,...) to present the destination restore disk to the backup server 'on demand' when initiating a restore.
Now we have to manually present it first and bring it online when doing direct san restore...
WimVD
Service Provider
 
Posts: 48
Liked: 10 times
Joined: Tue Dec 23, 2014 4:04 pm

Re: Incredible slow Direc SAN restore

Veeam Logoby SyNtAxx » Mon Jun 29, 2015 1:02 am

foggy wrote:You can check whether other restore modes work better for you. I would say that we're aware of the issue and currently monitoring the demand for implementing eager zeroed disks restore.



Maybe i'm off base here, but if you have to test the winds to see if anybody 'wants/demands SAN restores to work as expected i think that is a problem. I resent the fact I have to take a snapshot of a lun I backed up via SAN and mount it to a vm as an RDM in order to restore Gigantic vms via hot add in order to get decent restore speeds. If its not obvious, I vote for SAN restores to work properly.


-Nick
SyNtAxx
Expert
 
Posts: 127
Liked: 14 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: Incredible slow Direc SAN restore

Veeam Logoby chjones » Mon Jun 29, 2015 11:46 pm

The case I opened is still open with Veeam and has kind of been taken over by VMware in their own case, so we are waiting on them. The last update I received was last Wednesday and the Support person said he was in discussion with the engineering team and haven't come to any conclusion as to the cause of slow restore performance with Lazy Zeroed Disks and SAN Restore Mode.

I had been told earlier that they were able to reproduce the issues, so that's something I guess.

dmitri-va wrote:
chjones wrote:Veeam have now opened a case with VMware, as they see the same results internally, and I've given them permission to hand over my details to VMware if they wish to contact me regarding the issue. Fingers crossed.


Just came across the same issue with my Direct SAN restore testing. Is there any update on your case?
chjones
Enthusiast
 
Posts: 83
Liked: 25 times
Joined: Tue Oct 30, 2012 7:53 pm
Full Name: Chris Jones

Re: Incredible slow Direc SAN restore

Veeam Logoby foggy » Fri Jul 03, 2015 1:38 pm

SyNtAxx wrote:Maybe i'm off base here, but if you have to test the winds to see if anybody 'wants/demands SAN restores to work as expected i think that is a problem. I resent the fact I have to take a snapshot of a lun I backed up via SAN and mount it to a vm as an RDM in order to restore Gigantic vms via hot add in order to get decent restore speeds. If its not obvious, I vote for SAN restores to work properly.

Nick, I cannot recall the initial reasons for the current implementation, probably the need for disk preallocation that takes quite long for large eager zeroed disks and can seriously affect job duration (while data transfer for lazy zeroed disks starts right away). However we did not have much complaints regarding that until recently, that's why I was talking about demand. Anyway, we're performing some research in this regard. Thanks again for your feedback.
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Incredibly slow Direc SAN restore

Veeam Logoby doggatas » Thu Jul 09, 2015 11:25 pm

Hi

Just wondering if there is anyway to use network mode as opposed to SAN mode for restores only? or is the mode dependant on which mode is set for the backup proxy used? What if I only have/need one backup proxy, i.e. the Veeam server is a physical server? Can we force restore to be completed of network mode?

Regards,
David
doggatas
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Tue Jul 24, 2012 1:15 am
Full Name: David O

Re: Incredibly slow Direc SAN restore

Veeam Logoby Vitaliy S. » Fri Jul 10, 2015 9:49 am

Hi David,

You're correct that the restore mode depends on the proxy configuration. If you install a physical proxy server and do not mount any LUNs network restore mode will be used all the time.

P.S. in order to restore over the SAN mode, I believe read/write access is required, so if you could configure your proxy server to operate in read-only mode which is sufficient for doing backups, then you will achieve your goal as well.

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19570
Liked: 1104 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Incredibly slow Direc SAN restore

Veeam Logoby b.vanhaastrecht » Thu Aug 27, 2015 10:02 am

Do we still need to put the presented LUN's online in Windows? As we can't restrore over SAN while LUN's are presented read/write, but are Offline (as desired).
========================================
Veeam ProPartner and Cloud Connect Provider
b.vanhaastrecht
Service Provider
 
Posts: 337
Liked: 66 times
Joined: Mon Aug 26, 2013 7:46 am
Location: The Netherlands
Full Name: Bastiaan van Haastrecht

Re: Incredibly slow Direc SAN restore

Veeam Logoby Delo123 » Thu Aug 27, 2015 11:04 am

Yes, they need to be online for direct SAN restore to work...
Delo123
Expert
 
Posts: 351
Liked: 97 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Incredibly slow Direc SAN restore

Veeam Logoby b.vanhaastrecht » Thu Aug 27, 2015 11:19 am

And is this marked as a bug by Veeam? Why are backups posible, but not restores in Offline mode?
========================================
Veeam ProPartner and Cloud Connect Provider
b.vanhaastrecht
Service Provider
 
Posts: 337
Liked: 66 times
Joined: Mon Aug 26, 2013 7:46 am
Location: The Netherlands
Full Name: Bastiaan van Haastrecht

Re: Incredibly slow Direc SAN restore

Veeam Logoby Delo123 » Thu Aug 27, 2015 2:55 pm

Not sure, maybe Veeam can answer.
However as you of course know, do not initialize the disks, just put them online :)
Delo123
Expert
 
Posts: 351
Liked: 97 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Incredibly slow Direc SAN restore

Veeam Logoby Vitaliy S. » Fri Aug 28, 2015 10:36 am

b.vanhaastrecht wrote:Do we still need to put the presented LUN's online in Windows? As we can't restrore over SAN while LUN's are presented read/write, but are Offline (as desired).

Bastiaan, I have double-checked with the QA team and LUNs do not need to be online when using direct SAN restore. If you see another situation, then please contact our support team for investigation.
Vitaliy S.
Veeam Software
 
Posts: 19570
Liked: 1104 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Incredibly slow Direc SAN restore

Veeam Logoby Taran » Wed Sep 02, 2015 7:13 am

One of my customer's is encountering this, I've asked them to post the support case number here. Here are my notes:

Checked:
    Proxy with DAS Repository
    Proxies have been configured to use SAN mode
    Restores log indicates SAN mode is being used
    Proxies have 12 cores, but are configured for 20 concurrent tasks
    Backup jobs are using “Local storage” as the storage optimization. (There aren’t any huge 16TB+ backup files)
    Local backup job in Melbourne then backup copy to the repository in Sydney.
    Restores are from the repository in Sydney to the Sydney DataStore, the vCenter is in Melbourne
    Melbourne is production and the network is far busier.
    Backup speeds are 125+ MB/s
    Full VM restores, the plan is to use these restored VMs for replica mapping.
    Eventual plan is to use the backup copy job as a source for the replication jobs.
    Quick view of Veeam One reports for the datastores did not yield any issues on the datastore side, and alarms in the infrastructure didn’t seem related.
    Restore jobs are selected use the same disk type
    Disks are Thick provisioned with mostly Lazy Zeroed Disks, a few eager zeroed disks, Restored disks are the same type.

Tests:
All tests were done with Lazy Zeroed Disk VMs only
    Restore in Sydney with the current setup: speed 26MB/s
    Change the Sydney proxy to network mode: speed 76MB/s
    ***Network mode speed over a 10GB, non-management interface was 3 times faster to do the same restore***
    Try to restore a VM from a regular backup job, instead of a backup copy job: speed 75MB/s
    Restore in Melbourne in SAN mode: 60MB/s
    Restore in Melbourne in network mode: 35MB/s

Problem:
Restores are functioning, however the speed is not as expected.
Taran
Veeam Software
 
Posts: 65
Liked: 8 times
Joined: Tue Jul 16, 2013 5:05 am
Full Name: Taran

Re: Incredibly slow Direc SAN restore

Veeam Logoby MrDave » Thu Sep 03, 2015 5:10 am

Hi my case is what Taran is talking about. We are experiencing extremely slow restores as well. Case number 01006622.

We have also created a VM with Thick Provisioned Eager Zero and tried to restore as the same type with the same slow results ~25MB/s.

Our SAN is a Dell Compellent with dual SC8000 controllers.

Hope someone can help. Thanks
MrDave
Novice
 
Posts: 3
Liked: never
Joined: Wed Sep 02, 2015 6:50 am

Re: Incredibly slow Direc SAN restore

Veeam Logoby Taran » Thu Sep 03, 2015 5:37 am

The customer is having difficulty posting, so I thought I'd pop the case number in: 01006622
Taran
Veeam Software
 
Posts: 65
Liked: 8 times
Joined: Tue Jul 16, 2013 5:05 am
Full Name: Taran

PreviousNext

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 15 guests