Comprehensive data protection for all workloads
Post Reply
cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Server 2016 Large ReFs Repo RAID options

Post by cookiejc » Aug 02, 2018 2:35 pm

Hi, We have a new HP Apollo 4510 Gen 10 with 60 x 10TB 7.2K SAS disks ready to be commissioned as a new mid / long term Veeam repository. The plan is to use Server 2016 and ReFS to make use of the new features and efficiencies.

I have read through a number of different scenarios on the forums but am unlcear on the best RAID choice. At the moment the controller is only giving me the options of RAID0,5 or 1+0. The push is to go for RAID5 to provide the extra capacity but i've read a few posts saying to avoid this due to the disk of rebuild time upon disk failure.

There is also the option to use storage spaces direct but this was not the initial intention and would be more of a dive into the unknown in terms of S2D as a backup repo

Can anyone give me any advice on a good configuration in this scenario? Is RAID5 completely unsuitable?

Many thanks

Mgamerz
Enthusiast
Posts: 73
Liked: 9 times
Joined: Sep 29, 2017 8:07 pm
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by Mgamerz » Aug 02, 2018 4:57 pm

RAID 5 on 60 10TB disks?

Absolutely not. That's 59 chances for any disk to die and the whole thing is dead once the first drive goes down. I would never recommend RAID 5 these days for anything but a low number SSD array. When RAID 5 rebuilds it has to read data from all drives to compute the data on the missing drive. So you are effectively doing 59 disks full read. On 10TB drives that will take a pretty long time - the longer it takes, the higher the chance of another failure. When another drive fails, the whole thing is dead.

Raid 10 will be a big increase in performance but every disk will have a mirror so you're gonna take a big hit on usable space. But you can lose multiple drives, as long as the mirror is not lost it will be OK. Even on that large of an array I don't know if I would do RAID 6.

And obviously don't consider raid 0.

My vote on this would be RAID 10, as its the only real option.

Regnor
Service Provider
Posts: 282
Liked: 54 times
Joined: Jan 31, 2011 11:17 am
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by Regnor » Aug 02, 2018 7:50 pm 1 person likes this post

I second Mgamerz opinion, don't go for Raid5 or Raid6 with those sizes. If you need capacity then use Raid60; if you need performance take Raid10.

I don't see an advantage in using Storage Space compared to hardware RAID; although it offeres some great features and flexibility especially with ReFS.

dellock6
Veeam Software
Posts: 5599
Liked: 1563 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by dellock6 » Aug 02, 2018 7:58 pm

I confirm what was said before, we usually discourage raid5, and we instead suggest Raid60 to our customers, as this is a good balance between space and performance, especially with large stripes like these.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com/en/
vExpert 2011-2012-2013-2014-2015-2016-2017-2018
Veeam VMCE #1

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by cookiejc » Aug 03, 2018 8:03 am

Hi all, thanks very much for your replies. I'm a bit stuck as the controller is only giving me the option for Raid 0, 5 or 1+0! which is odd as the model is a HPE P408i-a which is suggested on the following link should support RAID 60

https://support.hpe.com/hpsc/doc/public ... 26319en_us

I'll troubleshoot whether this is an issue or just simply that the controller doesn't actually support this RAID. If my only option is to use Storage spaces direct, is this strongly advised against?

Thanks again

Regnor
Service Provider
Posts: 282
Liked: 54 times
Joined: Jan 31, 2011 11:17 am
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by Regnor » Aug 03, 2018 8:44 am

Could you double check to which controller the disks are connected? There's also an E208i, which only supports 0,1 and 5.

If you want to use storage spaces with parity you would have to go with mirror-accelerated parity; otherwise performance will decrease badly.
https://docs.microsoft.com/en-us/window ... ted-parity

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by cookiejc » Aug 03, 2018 8:52 am

Hi yes I realised after that last post that the P408i is the embedded controller used for the RAID1 / OS disks. You are right that the additional controller is a E208i with the limitations of 0,1,5 and 1+0! I'm not sure why that was spec'd with this order or whether we could change it for a 408

Which puts me back to being stuck! We use mirror accelerated parity with our new Converged and Hyper-Converged Hyper-V deployments but these have SSDs acting as the cache. With this scenario where all disks are the same does it still apply?

nmdange
Expert
Posts: 394
Liked: 91 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by nmdange » Aug 03, 2018 6:06 pm

Mirror-Accelerated Parity can only be done on Storage Spaces Direct, so you would have to have 4 servers, not just 1. Your best bet is probably to replace the RAID controller if possible. I'd complain to whomever quoted you the server that they didn't do it properly!

cookiejc
Enthusiast
Posts: 63
Liked: never
Joined: Nov 06, 2013 3:15 pm
Full Name: J Cook
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by cookiejc » Aug 06, 2018 7:56 am

Thanks mmdange I came to the same conclusion on Friday as well and ordered one of these: https://www.proliantserverstore.co.uk/p ... gJnIPD_BwE

I was trying to avoid having to replace the controller as I now have to wait for the cogs of government procurement to get it replaced..

Thanks all for the advice

JaySt
Service Provider
Posts: 122
Liked: 23 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by JaySt » Aug 21, 2018 8:13 pm 1 person likes this post

I'm also looking into a Apollo 4510 gen10 configuration for a 2016 ReFS repository. I'm aiming at 24 disks initially (12TB 12Gb NLSAS disks). I think I'll place them in a RAID60 configuration with two parity groups of 10+2 disks. I can expand the repository later by adding extra parity groups or create another group with two parity groups all the way up to 60 disks.
RAID controllers used will be a p408i-a (modular chip) for boot SSDs and a p408i-p (PCIe card) for the 60 disks. Starting off with 192GB of RAM to come close to 1GB RAM:1TB Data ratio recommendation
interesting hardware.
Veeam Certified Engineer

Mgamerz
Enthusiast
Posts: 73
Liked: 9 times
Joined: Sep 29, 2017 8:07 pm
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by Mgamerz » Aug 21, 2018 10:55 pm

Isn't Raid 1 + 0 Raid 10?

Regnor
Service Provider
Posts: 282
Liked: 54 times
Joined: Jan 31, 2011 11:17 am
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by Regnor » Aug 22, 2018 7:05 am

I would say they are the same.

heavy
Influencer
Posts: 16
Liked: 3 times
Joined: Sep 27, 2016 9:03 am
Full Name: Thomas Müller
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by heavy » Aug 22, 2018 7:22 am

JaySt wrote:I think I'll place them in a RAID60 configuration with two parity groups of 10+2 disks. I can expand the repository later by adding extra parity groups or create another group with two parity groups all the way up to 60 disks.
So far i have configured very large repo's in similar fashion. I don't like the idea of having too many disks in a single array. But what exactly is too many, anyone have a good reference point?

orb
Enthusiast
Posts: 40
Liked: 5 times
Joined: Apr 01, 2016 5:36 pm
Full Name: Olivier
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by orb » Aug 26, 2018 3:41 pm

General rule of thumb for old RAID technology

Over 1.2TB, you avoid R5/50. Raid Group size for RAID6/60 ( 4+2, 8+2, 12+2 or 14+2) to keep everything align.
However, I'll be careful to go beyond 4+2 in a RAID6/60 for drive over 6TB due to rebuild time.

As mentioned already, RAID 60 is proven path for large repository

Oli

Ctek
Service Provider
Posts: 64
Liked: 9 times
Joined: Nov 11, 2015 3:50 pm
Location: Canada
Contact:

Re: Server 2016 Large ReFs Repo RAID options

Post by Ctek » Aug 27, 2018 1:40 pm

We have recently acquired physical servers for repositories and after much testing and deliberation, we chose RAID60 with 36 drives, using 2 Disk Groups and 2 hot-spares. The performance is decent enough to fill a 10Gbit link. While disk rebuild is long, at least we have N+2 redundency on each disk group.
VMCE 9 Certified - Systems Administrator

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 35 guests