Comprehensive data protection for all workloads
Post Reply
frankive
Service Provider
Posts: 895
Liked: 106 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

JBOD rackmount 12+ satdrives, recommendation?

Post by frankive » Jan 04, 2014 3:28 pm

I need to expand my capacity for veeam backups as it is growing fast, especially since we got the backup copy jobs with GFS retention.. :)

I was looking for a enclosure where i could put 12+ check SATA drives with 4TB drives.

What do people in here recommend? A qnap connected through iscsi with raid6? or a jbod enclosure and storage spaces?

Gostev
SVP, Product Management
Posts: 25107
Liked: 3677 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Gostev » Jan 04, 2014 7:35 pm

I like JBOD and Storage Spaces, as this is most economical, no-frills solution.

frankive
Service Provider
Posts: 895
Liked: 106 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by frankive » Jan 05, 2014 7:29 pm

you have any recommended jbod enclosure you have tried and is reliable?

Gostev
SVP, Product Management
Posts: 25107
Liked: 3677 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Gostev » Jan 05, 2014 8:39 pm

No, we don't do this kind of testing/validation.

ryan1212
Influencer
Posts: 16
Liked: 6 times
Joined: Feb 26, 2013 3:48 pm
Full Name: Ryan
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by ryan1212 » Jan 05, 2014 10:55 pm

We have been quite happy buying used proliant DL180G6s on eBay. Typical config for us is 16GB ram, windows server 2012R2 with dedup enabled, two raid 5 arrays of 6x4TB disks and dual quad core CPUs.

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » Jan 06, 2014 10:02 pm

Hi Frank, we are using with good success also several Supermicro servers, both the 12 shelves server model (2 RU) and even the 4RU version with 36-48 disks.
Whatever is your final decision, be really careful in choosing a good raid controller, it's going to be the cornerstone of your final performances. We are really happy with LSI 9260 and lately 9280.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Fiskepudding
Expert
Posts: 213
Liked: 26 times
Joined: Feb 01, 2012 7:24 am
Full Name: Espen Dykesteen
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Fiskepudding » Jan 07, 2014 12:47 pm

As Luca points out, a good controller is important.

We are happy with our 3 SuperMicros 4U with 36x3,5” drive trays. They are relative affordable and offer a lot of storage.
(Not all trays are occupied, but the LSI 9260 keeps up, even with RAID6 and reversed incremental.)
So JBOD 12 sata drives on the LSI 9260 would not be a bottleneck.
But you might want took at the pricing on bigger units. We ended up with the rater large 36 bay unit, so we can easy expand down the road.

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » Jan 07, 2014 10:20 pm

Hi Espen,
nice to see we ended up to the same kind of design, that's a mutual validation of the design choices :)
The only thing that had me think about the 2RU instead of the 4RU in some situation is the fault domain: with the small one you loose less data when it breaks, and also rebuild time of a smaller storage is less. On the other side, a smaller unit means more network ports on switches (and this has a weight in a datacenter economics) and higher total cost of acquisition to manage the same amount of disks (to manage 48 drivers I can use one dense 4RU system, or I need to have 2 * 2RU systems with 24 disk slots each)...

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Fiskepudding
Expert
Posts: 213
Liked: 26 times
Joined: Feb 01, 2012 7:24 am
Full Name: Espen Dykesteen
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Fiskepudding » Jan 08, 2014 6:57 am

Agree on that Luca, the performance during rebuild is not that bad it just takes a verryverryverry long time. Considering RAID6, the risk of two other disks crashing during rebuild is still very small.
They are also configured with a hot spare, so even if a disk failure happens on Friday just after work, the rebuild would already be a few % done on Monday.
We have 3 of these units, so in case of a major fault the other two can take over backup jobs temporary. They are scaled for that.

Further we have a QNAP where we do full backups every weekend to. So it is possible to restore at least 7 days old backups even if one of these big units has a disastrous failure. And off cause offsite full restore points.

There is no universal-correct way to do this. It all comes down to RTO and RPOs, we find ours acceptable.
I would not consider JBOD on a 36 disk system, probably not on a 12 drive either… unless they only were used for "temp-files" or if every restore point is loaded of to external site/system (Veeam Backup Copy job).

But having nine 12-drive units would make the solution more robust, no doubt, and to some extent more flexible. But in the process, more expensive, more rack space demanding (24u VS 12u), and more power hungry
With power comes greed… (that was a joke).
But with added power usage comes:
- The need for bigger/more UPS(maybe need for another UPS (add another 2U)
- The need for more cooling in the environment / serverroom
- A bigger power bill, obvisually :)
This actually sums up a bit of the reasons for our design… got a bit carried away here. Feel free to moderate the post :)

Gostev
SVP, Product Management
Posts: 25107
Liked: 3677 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Gostev » Jan 08, 2014 10:46 am 2 people like this post

Great discussion. Physical server with JBOD is what myself and our solution architects recommend to customers these days as the most economical, fast and reliable target for Veeam B&R you can possibly get. Sure, you can probably get even cheaper, or faster, or more reliable target - but physical server with JBOD will always be the best possible combination of all three factors.

frankive
Service Provider
Posts: 895
Liked: 106 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by frankive » Jan 13, 2014 2:55 pm

anyone who can point me to a site which a few jboc enclossure they would recommend?

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » Jan 13, 2014 3:13 pm

Hi Frank,
a good starting point is the Supermicro website itself, otherwise depending on your location (there is no info in your profile) there are several OEM building complete servers with supermicro chassis. I have some links but don't want to do public promotion here in the forums...

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Fiskepudding
Expert
Posts: 213
Liked: 26 times
Joined: Feb 01, 2012 7:24 am
Full Name: Espen Dykesteen
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Fiskepudding » Jan 14, 2014 10:23 am

Frank, I see you are from Norway...
We bought our SuperMicro boxes from http://nextron.no/.
I can highly recommend them. They have ok prices on most things, including disks.
They also have affordable 10Gbit solutions.

Web-page allows you to play around with the configuration too :)

frankive
Service Provider
Posts: 895
Liked: 106 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by frankive » Jan 14, 2014 6:37 pm

So you use one of there super micro jbod enclosure, connect them to to a server 2012 through a HBA, enable storage spaces and dedup, and way to go?

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » Jan 14, 2014 10:23 pm

No Frank, the jbos itself is a server. You install Win2012 directly on it and create a local disk carved out of the jbod space.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

Fiskepudding
Expert
Posts: 213
Liked: 26 times
Joined: Feb 01, 2012 7:24 am
Full Name: Espen Dykesteen
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Fiskepudding » Jan 15, 2014 11:22 am

I think you are both right. On some units you can install OS/2012 directly on it, but you can also connect the JBOD devices to another unit with a mini SAS cable. (That would by terms probably make it a DAS... but it is still a DAS.... with JBOD)
Nextron are very solution oriented, so you might want to present to them what you want to achieve and what current hardware you have, and they can suggest a good solution.

bcscholl
Influencer
Posts: 18
Liked: 2 times
Joined: Dec 04, 2013 3:56 pm
Full Name: Brian Scholl
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by bcscholl » Feb 09, 2014 11:08 pm

I'm planning on re-using an old HP-G6 that has 12x400GB drives 4GB of memory. Not sure what Raid card installed but plan to setup Raid-5, install Server 2012R2 and use iSCSI-Target server. I figure two 1-GB NIC connections directly to a Dell R620 (Veeam Server).

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » Feb 10, 2014 1:00 pm 1 person likes this post

Brian,
why don't you use this server directly as a Veeam Repository, without exposing it via iscsi? In this way Veeam can deploy the repository components on this server and give you better performances.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

abradley
Influencer
Posts: 12
Liked: never
Joined: Apr 19, 2011 3:55 pm
Full Name: Andrew Bradley II
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by abradley » Feb 10, 2014 1:21 pm

Great Discussion - we too have some HP server hardware that would be great to re-purpose as backup repositories.

Question for the group - has anyone looked at the performance hit of RAID-5 vs. JBOD? I am concerned about the potential data loss of using JBOD with Win2012.
Our HP servers have a basic RAID card in them and there are better ones available to control external cages that are reasonably priced, but at what performance impact?

Considering going with a 10g adapter in the server, our entire infrastructure is virtualized including the Veeam server and proxies.

Anyone doing this today either with a DIY server or a purchased NAS box and has some experience with performance differences between the RAID setups?

Thanks,

Andy

bcscholl
Influencer
Posts: 18
Liked: 2 times
Joined: Dec 04, 2013 3:56 pm
Full Name: Brian Scholl
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by bcscholl » Feb 10, 2014 1:40 pm

Good question. Here is a list of my hardware:
HP-G5: 2x Xeon 3060@2.4GHz (Two 2-cores, 4 logical CPUs), 4GB memory, 12x400GB HDD
Dell R620: 2x Xeon X5660@2.8GHz (Two 8-cores, 24 logical CPUs) , 50GB Memory, 2x72GB HDD

So I plan to use the HP as simple storage via iSCSI directly connected to the Dell, then install Veeam on the Dell. I've already started replicating machines from the Hyper-V cluster directly into the Dell.

itfnb
Enthusiast
Posts: 58
Liked: 9 times
Joined: Mar 12, 2012 8:18 pm
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by itfnb » Feb 10, 2014 3:32 pm

We've gone through this recently as well. I opted to take a Dell R710 coming out of vSphere production (new hosts going in), load up the internal bays for the nightly backup jobs, buy an empty Dell MD1200 from their refurb store, and load it up with 12 4TB drives in RAID 10 for the on-site backup copy jobs. As more space is needed, I'll chain up more DAS units. Even if iSCSI is en vogue, 6Gb/s SAS is hard to beat for the money (to be fair, we do use iSCSI in our production environment).

guitarfish
Enthusiast
Posts: 77
Liked: 2 times
Joined: Mar 06, 2013 4:12 pm
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by guitarfish » Feb 10, 2014 6:31 pm

I'm missing something...how are you connecting the server with the JBOD to the host being backed up?

I am backing up right now from host to a NAS, and there's a Win PC as the proxy. Backups times are fine but restores, yes they are slow. I have a 1GB LAN, I assume that is the bottleneck. So how would backup to a WinServer help?

foggy
Veeam Software
Posts: 18449
Liked: 1589 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by foggy » Feb 10, 2014 8:53 pm

You just add this server as Windows type backup repository into Veeam B&R console. Having I/O intensive operations (e.g. transformation) over backup files being performed locally by Veeam agent running right on this server makes it a more preferable backup target. In this case VM disks will be also processed by the local agent during restores.

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » Feb 10, 2014 10:50 pm

I feel like there is a misundesrtanding here about the term JBOD and what it means.
Reading around the forums, seems we are referring to jbod as "many disks connected directly to the repository server", as opposed to a LUN carved out from a SAN and mounted on the repository itself.
In reality, I think we are all referrring to some "internal storage" of the repository server, like those servers with 12-24 disks. Or also a disk enclosure (usually sas-based) that has no controller in it, and it's used by the server by connecting it to a disk controller.

Both this scenarios are really nice and effective, but it's NOT jbod. Jbod means "just a bunch of disks", and it means the disks are only exposed one by one to the operating system, or concatenated to form a single logical drive with NO raid protection (in windows is what is called a spanned drive). Those internal disks at the end are connected to a raid controller, and so we end up having a raid group, and not a jbod.

Sorry everyone if I too used the term jbod while in reality I was talking about an internal storage. Hope this will help to have everyone on the same page from now on :)

PS: Honestly, I would never place my backups in a jbod, I would always use some form of protection, being it a raid or a replicated storage.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

benwaynet
Enthusiast
Posts: 30
Liked: never
Joined: Apr 15, 2009 5:27 pm
Full Name: Jason Benway
Location: West Michigan
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by benwaynet » May 14, 2014 1:35 pm

Isn't windows storage spaces just software raid?

If you used a true jbod with storage spaces your counting on windows to do all the parity and check bits. But it sounds like storage spaces is easy to expand.

Using a disk controller in the server and letting it handle parity is what we tipically do, but expanding a raid set is never easy.

I'm looking for a solution that allows me to start with what I need and grow the storage, but not have to plan the shell game of which drive letter do I put which veeam backup job on.

I think storage spaces would allow me to create one drive letter and then keep expanding it, but software raid scares me :)

jb

hans_lenze
Service Provider
Posts: 17
Liked: 4 times
Joined: Sep 07, 2012 7:07 am
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by hans_lenze » May 14, 2014 6:48 pm

As long as you understand the way Storage Spaces work with columns and all, you should be fine. Columns and resiliency type dictate most of the performance
http://blogs.technet.com/b/askpfeplat/a ... nsion.aspx

Just know that you cannot just add two or three disks and have the existing volume take up the free pool space. If you start with a full chassis (12 disks) and plan to add another full chassis (another 12 disks) you should be fine. If disks get cheaper you could even buy bigger disks and add those to the same pool. It's really the number of disks that Storage Spaces cares about and not the size.

ccspence
Lurker
Posts: 1
Liked: never
Joined: May 28, 2014 10:27 pm
Full Name: Christian Spence
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by ccspence » May 28, 2014 10:37 pm

Can you tell me what SuperMicro chassis you are using? If you have a pat number that would be a huge help. Thanks in advance.

Gostev
SVP, Product Management
Posts: 25107
Liked: 3677 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by Gostev » May 29, 2014 1:42 pm

dellock6 wrote:a disk enclosure (usually sas-based) that has no controller in it, and it's used by the server by connecting it to a disk controller
When I say JBOD, I mean this above. For internal disks in the server, I use "internal storage". No difference between the two from Veeam backup repository perspective though.

dellock6
Veeam Software
Posts: 5789
Liked: 1657 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by dellock6 » May 30, 2014 3:51 pm

Christian, you can look at some 4U Supermicro machines on their website, there are many models there. We used in the past at my previous company this one:

http://www.supermicro.nl/products/syste ... E1R24L.cfm

with additional LSI controller (the controller is where you'd like to put your money in). Now I see there are newer model with even more disk slots.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

ericcsinger
Influencer
Posts: 24
Liked: 7 times
Joined: May 11, 2014 8:52 pm
Full Name: Eric Singer
Contact:

Re: JBOD rackmount 12+ satdrives, recommendation?

Post by ericcsinger » Jul 26, 2014 9:21 pm 2 people like this post

I know the post is old, but I wanted to throw in a few suggestions.

I'm building a windows 2012 storage server and had the same need. Here is what I've gone with and why.

JBOD: I would recommend two vendors both of which use the same hw. Raid inc. or dataon. The jbods are of great quality and you can get a good warrenty with it. It's also 100% supported by MS including hw health monitoring. I personally went with 4 60 bay units and 60 drives (which I also got with a warrenty from raid inc).

Sas controllers: anything lsi is pretty good but if you're doing storage spaces just get a lsi sas hba not a raid controller.

With a dell r710 and bottlenecked by my PCI bus, I get 2GBps writes and 4GBps read( yes that's bytes not bits). If I had a newer server and a newer sas card I could easily get 4GBps write and 8 read (minus overhead). More disk would also equal more IOPS and storage.

Total for the storage minus the server was roughly 60k. (Slightly less).

Post Reply

Who is online

Users browsing this forum: Bing [Bot], cosmin.ciobanu and 75 guests