JBOD rackmount 12+ satdrives, recommendation?

[frankive]

I need to expand my capacity for veeam backups as it is growing fast, especially since we got the backup copy jobs with GFS retention..

I was looking for a enclosure where i could put 12+ check SATA drives with 4TB drives.

What do people in here recommend? A qnap connected through iscsi with raid6? or a jbod enclosure and storage spaces?

[Gostev]

I like JBOD and Storage Spaces, as this is most economical, no-frills solution.

[frankive]

you have any recommended jbod enclosure you have tried and is reliable?

[Gostev]

No, we don't do this kind of testing/validation.

[ryan1212]

We have been quite happy buying used proliant DL180G6s on eBay. Typical config for us is 16GB ram, windows server 2012R2 with dedup enabled, two raid 5 arrays of 6x4TB disks and dual quad core CPUs.

[dellock6]

Hi Frank, we are using with good success also several Supermicro servers, both the 12 shelves server model (2 RU) and even the 4RU version with 36-48 disks.
Whatever is your final decision, be really careful in choosing a good raid controller, it's going to be the cornerstone of your final performances. We are really happy with LSI 9260 and lately 9280.

Luca.

[Fiskepudding]

As Luca points out, a good controller is important.

We are happy with our 3 SuperMicros 4U with 36x3,5” drive trays. They are relative affordable and offer a lot of storage.
(Not all trays are occupied, but the LSI 9260 keeps up, even with RAID6 and reversed incremental.)
So JBOD 12 sata drives on the LSI 9260 would not be a bottleneck.
But you might want took at the pricing on bigger units. We ended up with the rater large 36 bay unit, so we can easy expand down the road.

[dellock6]

Hi Espen,
nice to see we ended up to the same kind of design, that's a mutual validation of the design choices
The only thing that had me think about the 2RU instead of the 4RU in some situation is the fault domain: with the small one you loose less data when it breaks, and also rebuild time of a smaller storage is less. On the other side, a smaller unit means more network ports on switches (and this has a weight in a datacenter economics) and higher total cost of acquisition to manage the same amount of disks (to manage 48 drivers I can use one dense 4RU system, or I need to have 2 * 2RU systems with 24 disk slots each)...

Luca.

[Fiskepudding]

Agree on that Luca, the performance during rebuild is not that bad it just takes a verryverryverry long time. Considering RAID6, the risk of two other disks crashing during rebuild is still very small.
They are also configured with a hot spare, so even if a disk failure happens on Friday just after work, the rebuild would already be a few % done on Monday.
We have 3 of these units, so in case of a major fault the other two can take over backup jobs temporary. They are scaled for that.

Further we have a QNAP where we do full backups every weekend to. So it is possible to restore at least 7 days old backups even if one of these big units has a disastrous failure. And off cause offsite full restore points.

There is no universal-correct way to do this. It all comes down to RTO and RPOs, we find ours acceptable.
I would not consider JBOD on a 36 disk system, probably not on a 12 drive either… unless they only were used for "temp-files" or if every restore point is loaded of to external site/system (Veeam Backup Copy job).

But having nine 12-drive units would make the solution more robust, no doubt, and to some extent more flexible. But in the process, more expensive, more rack space demanding (24u VS 12u), and more power hungry
With power comes greed… (that was a joke).
But with added power usage comes:
- The need for bigger/more UPS(maybe need for another UPS (add another 2U)
- The need for more cooling in the environment / serverroom
- A bigger power bill, obvisually
This actually sums up a bit of the reasons for our design… got a bit carried away here. Feel free to moderate the post

[Gostev]

Great discussion. Physical server with JBOD is what myself and our solution architects recommend to customers these days as the most economical, fast and reliable target for Veeam B&R you can possibly get. Sure, you can probably get even cheaper, or faster, or more reliable target - but physical server with JBOD will always be the best possible combination of all three factors.

[frankive]

anyone who can point me to a site which a few jboc enclossure they would recommend?

[dellock6]

Hi Frank,
a good starting point is the Supermicro website itself, otherwise depending on your location (there is no info in your profile) there are several OEM building complete servers with supermicro chassis. I have some links but don't want to do public promotion here in the forums...

Luca.

[Fiskepudding]

Frank, I see you are from Norway...
We bought our SuperMicro boxes from http://nextron.no/.
I can highly recommend them. They have ok prices on most things, including disks.
They also have affordable 10Gbit solutions.

Web-page allows you to play around with the configuration too

[frankive]

So you use one of there super micro jbod enclosure, connect them to to a server 2012 through a HBA, enable storage spaces and dedup, and way to go?

[dellock6]

No Frank, the jbos itself is a server. You install Win2012 directly on it and create a local disk carved out of the jbod space.

Luca.

[Fiskepudding]

I think you are both right. On some units you can install OS/2012 directly on it, but you can also connect the JBOD devices to another unit with a mini SAS cable. (That would by terms probably make it a DAS... but it is still a DAS.... with JBOD)
Nextron are very solution oriented, so you might want to present to them what you want to achieve and what current hardware you have, and they can suggest a good solution.

[bcscholl]

I'm planning on re-using an old HP-G6 that has 12x400GB drives 4GB of memory. Not sure what Raid card installed but plan to setup Raid-5, install Server 2012R2 and use iSCSI-Target server. I figure two 1-GB NIC connections directly to a Dell R620 (Veeam Server).

[dellock6]

Brian,
why don't you use this server directly as a Veeam Repository, without exposing it via iscsi? In this way Veeam can deploy the repository components on this server and give you better performances.

Luca.

[abradley]

Great Discussion - we too have some HP server hardware that would be great to re-purpose as backup repositories.

Question for the group - has anyone looked at the performance hit of RAID-5 vs. JBOD? I am concerned about the potential data loss of using JBOD with Win2012.
Our HP servers have a basic RAID card in them and there are better ones available to control external cages that are reasonably priced, but at what performance impact?

Considering going with a 10g adapter in the server, our entire infrastructure is virtualized including the Veeam server and proxies.

Anyone doing this today either with a DIY server or a purchased NAS box and has some experience with performance differences between the RAID setups?

Thanks,

Andy

[bcscholl]

Good question. Here is a list of my hardware:
HP-G5: 2x Xeon 3060@2.4GHz (Two 2-cores, 4 logical CPUs), 4GB memory, 12x400GB HDD
Dell R620: 2x Xeon X5660@2.8GHz (Two 8-cores, 24 logical CPUs) , 50GB Memory, 2x72GB HDD

So I plan to use the HP as simple storage via iSCSI directly connected to the Dell, then install Veeam on the Dell. I've already started replicating machines from the Hyper-V cluster directly into the Dell.

[itfnb]

We've gone through this recently as well. I opted to take a Dell R710 coming out of vSphere production (new hosts going in), load up the internal bays for the nightly backup jobs, buy an empty Dell MD1200 from their refurb store, and load it up with 12 4TB drives in RAID 10 for the on-site backup copy jobs. As more space is needed, I'll chain up more DAS units. Even if iSCSI is en vogue, 6Gb/s SAS is hard to beat for the money (to be fair, we do use iSCSI in our production environment).

[guitarfish]

I'm missing something...how are you connecting the server with the JBOD to the host being backed up?

I am backing up right now from host to a NAS, and there's a Win PC as the proxy. Backups times are fine but restores, yes they are slow. I have a 1GB LAN, I assume that is the bottleneck. So how would backup to a WinServer help?

[foggy]

You just add this server as Windows type backup repository into Veeam B&R console. Having I/O intensive operations (e.g. transformation) over backup files being performed locally by Veeam agent running right on this server makes it a more preferable backup target. In this case VM disks will be also processed by the local agent during restores.

[dellock6]

I feel like there is a misundesrtanding here about the term JBOD and what it means.
Reading around the forums, seems we are referring to jbod as "many disks connected directly to the repository server", as opposed to a LUN carved out from a SAN and mounted on the repository itself.
In reality, I think we are all referrring to some "internal storage" of the repository server, like those servers with 12-24 disks. Or also a disk enclosure (usually sas-based) that has no controller in it, and it's used by the server by connecting it to a disk controller.

Both this scenarios are really nice and effective, but it's NOT jbod. Jbod means "just a bunch of disks", and it means the disks are only exposed one by one to the operating system, or concatenated to form a single logical drive with NO raid protection (in windows is what is called a spanned drive). Those internal disks at the end are connected to a raid controller, and so we end up having a raid group, and not a jbod.

Sorry everyone if I too used the term jbod while in reality I was talking about an internal storage. Hope this will help to have everyone on the same page from now on

PS: Honestly, I would never place my backups in a jbod, I would always use some form of protection, being it a raid or a replicated storage.

[benwaynet]

Isn't windows storage spaces just software raid?

If you used a true jbod with storage spaces your counting on windows to do all the parity and check bits. But it sounds like storage spaces is easy to expand.

Using a disk controller in the server and letting it handle parity is what we tipically do, but expanding a raid set is never easy.

I'm looking for a solution that allows me to start with what I need and grow the storage, but not have to plan the shell game of which drive letter do I put which veeam backup job on.

I think storage spaces would allow me to create one drive letter and then keep expanding it, but software raid scares me

jb

[hans_lenze]

As long as you understand the way Storage Spaces work with columns and all, you should be fine. Columns and resiliency type dictate most of the performance
http://blogs.technet.com/b/askpfeplat/a ... nsion.aspx

Just know that you cannot just add two or three disks and have the existing volume take up the free pool space. If you start with a full chassis (12 disks) and plan to add another full chassis (another 12 disks) you should be fine. If disks get cheaper you could even buy bigger disks and add those to the same pool. It's really the number of disks that Storage Spaces cares about and not the size.

[ccspence]

Can you tell me what SuperMicro chassis you are using? If you have a pat number that would be a huge help. Thanks in advance.

[Gostev]

a disk enclosure (usually sas-based) that has no controller in it, and it's used by the server by connecting it to a disk controller

When I say JBOD, I mean this above. For internal disks in the server, I use "internal storage". No difference between the two from Veeam backup repository perspective though.

[dellock6]

Christian, you can look at some 4U Supermicro machines on their website, there are many models there. We used in the past at my previous company this one:

http://www.supermicro.nl/products/syste ... E1R24L.cfm

with additional LSI controller (the controller is where you'd like to put your money in). Now I see there are newer model with even more disk slots.

[ericcsinger]

I know the post is old, but I wanted to throw in a few suggestions.

I'm building a windows 2012 storage server and had the same need. Here is what I've gone with and why.

JBOD: I would recommend two vendors both of which use the same hw. Raid inc. or dataon. The jbods are of great quality and you can get a good warrenty with it. It's also 100% supported by MS including hw health monitoring. I personally went with 4 60 bay units and 60 drives (which I also got with a warrenty from raid inc).

Sas controllers: anything lsi is pretty good but if you're doing storage spaces just get a lsi sas hba not a raid controller.

With a dell r710 and bottlenecked by my PCI bus, I get 2GBps writes and 4GBps read( yes that's bytes not bits). If I had a newer server and a newer sas card I could easily get 4GBps write and 8 read (minus overhead). More disk would also equal more IOPS and storage.

Total for the storage minus the server was roughly 60k. (Slightly less).