Feature Request - SureBackup staging area.

Availability for the Always-On Enterprise

Feature Request - SureBackup staging area.

Veeam Logoby SBarrett847 » Mon Nov 13, 2017 1:53 pm

I love the Idea of Surebackup, however my main Backup repo is a StoreOnce who's Random I/O performance is abysmal. This prevents the VM being able to boot within any reasonable time frame. However it does have a reasonable restore rate (still not great though).

I would like if Surebackup could fully restore the VM to local storage or primary storage first, then boot and test the VM from there.
SBarrett847
Service Provider
 
Posts: 186
Liked: 29 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Feature Request - SureBackup staging area.

Veeam Logoby foggy » Tue Nov 14, 2017 4:43 pm

Hi Stephen, what SO firmware version does your storage run? StoreOnce restore performance was significantly increased in 3.15.1, specifically to allow SureBackup/Instant Recovery possible. So make sure you're running the recent one.
foggy
Veeam Software
 
Posts: 15394
Liked: 1142 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Feature Request - SureBackup staging area.

Veeam Logoby SBarrett847 » Tue Nov 14, 2017 4:46 pm

Hi Foggy - I'm on 3.16.3-1730.1 - however it is a CIFS share, not catalyst if that makes any difference.
SBarrett847
Service Provider
 
Posts: 186
Liked: 29 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Feature Request - SureBackup staging area.

Veeam Logoby foggy » Tue Nov 14, 2017 5:15 pm

It could. But also the overall setup might not be optimal. Some hints can be found here. Btw, what kind of VM's are you restoring - do they have a single or multiple disks?
foggy
Veeam Software
 
Posts: 15394
Liked: 1142 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Feature Request - SureBackup staging area.

Veeam Logoby SBarrett847 » Wed Nov 15, 2017 9:06 am

Testing now and I'm Getting about 15-20 MB/s per stream from the my StoreOnce (4500) I've never ever seen anything like 275MB/s restores as in that link. I have been able get up to 250/300 MB/s going into the StoreOnce, but coming out it is very slow.

From the Link it would seem that if you aren't using catalyst, you should probably not be using incrementals, as the performance is better with Full backups. Also it recommends using primary storage snapshots to allow for lightning restores. However as I use Hyper-V, and Veeam can't Leverage native 3Par snapshots for restores on Hyper-V, I'm hamstrung and have to rely on the StoreOnce to perform all backup tasks.

StoreOnce IMO is not really suitable for Primary Backup storage, and in places I have Backups going to an old SAN for a few VM I really need to be able to use Instant recovery.

However - I digress - If I could get Surebackup to restore to a staging area (local HDD or SSD array for example) and then boot and perform tests, I could get the Surebackup jobs to complete.
SBarrett847
Service Provider
 
Posts: 186
Liked: 29 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Feature Request - SureBackup staging area.

Veeam Logoby foggy » Wed Nov 15, 2017 2:36 pm

Well, using any dedupe device as primary backup storage is actually against our reference architecture.
foggy
Veeam Software
 
Posts: 15394
Liked: 1142 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Feature Request - SureBackup staging area.

Veeam Logoby SBarrett847 » Wed Nov 15, 2017 2:45 pm 1 person likes this post

Yeah, I know that now :)

It is my fault for letting HP design the solution originally. Will consider Exagrid for the future I think.
SBarrett847
Service Provider
 
Posts: 186
Liked: 29 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Feature Request - SureBackup staging area.

Veeam Logoby foggy » Wed Nov 15, 2017 3:02 pm

I didn't mean that actually, but rather introducing some raw device in-between, to better comply with the 3-2-1 rule.
foggy
Veeam Software
 
Posts: 15394
Liked: 1142 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Feature Request - SureBackup staging area.

Veeam Logoby FedericoV » Mon Nov 20, 2017 2:27 am 1 person likes this post

Hi Stephen, StoreOnce random IO has been optimized for Catalyst only. If you are using CIFS, than IVMR is quite slower than using Catalyst and, sorry to say, it is even not supported.
If you can, I suggest you to test IVMR with a Backup Repository created on a Catalyst Store.
Regarding the Restore Performance, your 15-20MB/s per stream are quite lower then the expected performance for your SO4500. For a single stream restore I would expect about 200MB/s.
If you can, try mounting the CIFS share from a server and make a command line test such as "copy /b MyLargeFile.vbk NUL" ? Write NUL not NULL, it works similarly to the unix /dev/null.
Another point to check is the "per VM file". we know that the restore (and the dedupe) is lower when there is one file per job rather than per VM.
I hope this can help you discover the root cause of your slow full restore performance.
FedericoV
Technology Partner
 
Posts: 3
Liked: 6 times
Joined: Mon Aug 21, 2017 3:27 pm
Full Name: Federico Venier

Re: Feature Request - SureBackup staging area.

Veeam Logoby SBarrett847 » Tue Nov 21, 2017 8:38 am

Much obliged Frederico - I do indeed see ~200MB/s using the NUL copy (and Copying to a local Flash array)- however I get no where near that in Veeam. I wonder what Veeam is doing during a restore that kills performance so much.

I'll Switch to Per VM and see if that helps.
SBarrett847
Service Provider
 
Posts: 186
Liked: 29 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Feature Request - SureBackup staging area.

Veeam Logoby FedericoV » Mon Dec 04, 2017 1:16 am

I guess that per-VM will work much better. Without the per-VM there is only one writing stream for all the VM in a job. The result is that the data from a VM is interleaved with the data of all the other VMs in the job. The backup is slower and you also get less dedupe. As you have tested, even the restore is quite slower. StoreOnce is optimized for sequential IO, but when you restore a VM from a multiplexed file, Veeam has to read a chunk of data from StoreOnce, than it will seek forward to avoid reading the other VMs blocks, and so on. This causes a lot of random IO on StoreOnce and makes the internal StoreOnce read-ahead less effective. I suggest disabling Veeam own dedupe and compression as well. Finally, if you can, make a test using Catalyst over LAN, it is even faster and, by the way, IVMR has been certified with it.
FedericoV
Technology Partner
 
Posts: 3
Liked: 6 times
Joined: Mon Aug 21, 2017 3:27 pm
Full Name: Federico Venier


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], NightBird, Yahoo [Bot] and 1 guest