Server 2016 Large ReFs Repo RAID options

[cookiejc]

Hi, We have a new HP Apollo 4510 Gen 10 with 60 x 10TB 7.2K SAS disks ready to be commissioned as a new mid / long term Veeam repository. The plan is to use Server 2016 and ReFS to make use of the new features and efficiencies.

I have read through a number of different scenarios on the forums but am unlcear on the best RAID choice. At the moment the controller is only giving me the options of RAID0,5 or 1+0. The push is to go for RAID5 to provide the extra capacity but i've read a few posts saying to avoid this due to the disk of rebuild time upon disk failure.

There is also the option to use storage spaces direct but this was not the initial intention and would be more of a dive into the unknown in terms of S2D as a backup repo

Can anyone give me any advice on a good configuration in this scenario? Is RAID5 completely unsuitable?

Many thanks

[Mgamerz]

RAID 5 on 60 10TB disks?

Absolutely not. That's 59 chances for any disk to die and the whole thing is dead once the first drive goes down. I would never recommend RAID 5 these days for anything but a low number SSD array. When RAID 5 rebuilds it has to read data from all drives to compute the data on the missing drive. So you are effectively doing 59 disks full read. On 10TB drives that will take a pretty long time - the longer it takes, the higher the chance of another failure. When another drive fails, the whole thing is dead.

Raid 10 will be a big increase in performance but every disk will have a mirror so you're gonna take a big hit on usable space. But you can lose multiple drives, as long as the mirror is not lost it will be OK. Even on that large of an array I don't know if I would do RAID 6.

And obviously don't consider raid 0.

My vote on this would be RAID 10, as its the only real option.

[Regnor]

I second Mgamerz opinion, don't go for Raid5 or Raid6 with those sizes. If you need capacity then use Raid60; if you need performance take Raid10.

I don't see an advantage in using Storage Space compared to hardware RAID; although it offeres some great features and flexibility especially with ReFS.

[dellock6]

I confirm what was said before, we usually discourage raid5, and we instead suggest Raid60 to our customers, as this is a good balance between space and performance, especially with large stripes like these.

[cookiejc]

Hi all, thanks very much for your replies. I'm a bit stuck as the controller is only giving me the option for Raid 0, 5 or 1+0! which is odd as the model is a HPE P408i-a which is suggested on the following link should support RAID 60

https://support.hpe.com/hpsc/doc/public ... 26319en_us

I'll troubleshoot whether this is an issue or just simply that the controller doesn't actually support this RAID. If my only option is to use Storage spaces direct, is this strongly advised against?

Thanks again

[Regnor]

Could you double check to which controller the disks are connected? There's also an E208i, which only supports 0,1 and 5.

If you want to use storage spaces with parity you would have to go with mirror-accelerated parity; otherwise performance will decrease badly.
https://docs.microsoft.com/en-us/window ... ted-parity

[cookiejc]

Hi yes I realised after that last post that the P408i is the embedded controller used for the RAID1 / OS disks. You are right that the additional controller is a E208i with the limitations of 0,1,5 and 1+0! I'm not sure why that was spec'd with this order or whether we could change it for a 408

Which puts me back to being stuck! We use mirror accelerated parity with our new Converged and Hyper-Converged Hyper-V deployments but these have SSDs acting as the cache. With this scenario where all disks are the same does it still apply?

[nmdange]

Mirror-Accelerated Parity can only be done on Storage Spaces Direct, so you would have to have 4 servers, not just 1. Your best bet is probably to replace the RAID controller if possible. I'd complain to whomever quoted you the server that they didn't do it properly!

[cookiejc]

Thanks mmdange I came to the same conclusion on Friday as well and ordered one of these: https://www.proliantserverstore.co.uk/p ... gJnIPD_BwE

I was trying to avoid having to replace the controller as I now have to wait for the cogs of government procurement to get it replaced..

Thanks all for the advice

[JaySt]

I'm also looking into a Apollo 4510 gen10 configuration for a 2016 ReFS repository. I'm aiming at 24 disks initially (12TB 12Gb NLSAS disks). I think I'll place them in a RAID60 configuration with two parity groups of 10+2 disks. I can expand the repository later by adding extra parity groups or create another group with two parity groups all the way up to 60 disks.
RAID controllers used will be a p408i-a (modular chip) for boot SSDs and a p408i-p (PCIe card) for the 60 disks. Starting off with 192GB of RAM to come close to 1GB RAM:1TB Data ratio recommendation
interesting hardware.

[Mgamerz]

Isn't Raid 1 + 0 Raid 10?

[Regnor]

I would say they are the same.

[heavy]

I think I'll place them in a RAID60 configuration with two parity groups of 10+2 disks. I can expand the repository later by adding extra parity groups or create another group with two parity groups all the way up to 60 disks.

So far i have configured very large repo's in similar fashion. I don't like the idea of having too many disks in a single array. But what exactly is too many, anyone have a good reference point?

[orb]

General rule of thumb for old RAID technology

Over 1.2TB, you avoid R5/50. Raid Group size for RAID6/60 ( 4+2, 8+2, 12+2 or 14+2) to keep everything align.
However, I'll be careful to go beyond 4+2 in a RAID6/60 for drive over 6TB due to rebuild time.

As mentioned already, RAID 60 is proven path for large repository

Oli

[Ctek]

We have recently acquired physical servers for repositories and after much testing and deliberation, we chose RAID60 with 36 drives, using 2 Disk Groups and 2 hot-spares. The performance is decent enough to fill a 10Gbit link. While disk rebuild is long, at least we have N+2 redundency on each disk group.

[PKaufmann]

General rule of thumb for old RAID technology

Over 1.2TB, you avoid R5/50. Raid Group size for RAID6/60 ( 4+2, 8+2, 12+2 or 14+2) to keep everything align.
However, I'll be careful to go beyond 4+2 in a RAID6/60 for drive over 6TB due to rebuild time.

As mentioned already, RAID 60 is proven path for large repository

Oli

I am starting a new repository Server with Windows Server 2019.
This server will only keep the last 5 days of backups with a immunity copy to AWS S3.
GFS Backup copy job will be in place to copy new backups immediately to our long term Data Domain Storage.

I am starting with 15x 2.4TB 10k SAS Drives and will most likely add more disks in 2021..
Server is a Dell PE R740XD, 96GB Memory (can be extended if required).

I was going to use Raid 6 but as I want to add more disks later, I guess Raid 60 is the better choice.

What would you suggest for Raid 60? 6+6 Disks and 2 Hot Spares, or 7+7 Disks and no hot spare.
We have proSupport 4hr mission critical..

I usually always use hot spares and recommend it as well, but with Raid Lv 60 I am not so sure,
especially because all backups are copied twice (immediate Copy in SOBR to AWS with immunity and Backup Copy Job to Data Domain)..

Let me know your thoughts/ideas