Help me choose a storage solution for my media server - AVS Forum
Forum Jump: 
 
Thread Tools
post #1 of 7 Old 07-08-2014, 11:04 AM - Thread Starter
Senior Member
 
ClemsonJeeper's Avatar
 
Join Date: Sep 2003
Location: Ashburn, VA
Posts: 340
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 52 Post(s)
Liked: 28
Help me choose a storage solution for my media server

Some background: I'm very well versed in server building and maintenance. I currently run a media server which is running Ubuntu with a ARC-1220 hardware raid controller. I built this years ago using 1.5TB drives (8x1.5) in a RAID-6 configuration on a reiserfs partition. This works pretty well - I store all my important stuff (documents, etc) as well as all my media on this array and I backup offsite the REALLY important stuff. I run various services on the server for streaming/sharing my media to my network.

My array is reaching capacity, and it is about time for an upgrade anyway.

I know there is a lot of new tech out there since I designed this (ZFS, FlexRaid, etc). One of the main issues with my Raid 6 array is that when its time to upgrade (as in my current situation), I'm boned. I have to copy ALL my data somewhere, replace all the drives, create a new array, and copy it all back. And then I have to junk a bunch of good storage.

Ideally, I'd like to rack a drive bay like: http://www.newegg.com/Product/Produc...-044-_-Product

Install some SAS controllers: http://www.newegg.com/Product/Produc...-792-_-Product

And now I have 12 bays I can swap in and out drives as I please.

Now, I need to figure out what solution (if any) meet my goals:

- up to 2 drives can drop out at any given time without data loss
- drives can be added at a later date to increase the size of the pool
- drives can be replaced out of the pool if I run out of space in my enclosure (eg: max out 12 drives, I can swap out a 1.5TB for a 4TB and it will pull this into the array and fill it).

All of this being transparent to Linux would be great, but I'm open to other solutions (snapshot?)

Anyone have any suggestions/links?

My Home Theater Build: The Vortex Theater Build

Last edited by ClemsonJeeper; 07-08-2014 at 11:08 AM.
ClemsonJeeper is offline  
Sponsored Links
Advertisement
 
post #2 of 7 Old 07-09-2014, 04:54 AM
One-Man Content Creator
 
wmcclain's Avatar
 
Join Date: May 2006
Posts: 16,939
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 190 Post(s)
Liked: 322
For snapshot and other non-standard raid solutions, SnapRAID has a quick comparison table of different options: http://snapraid.sourceforge.net/compare.html

SnapRAID is the only one of those I have used, and it suits my needs. It's best suited to collections of large files that rarely change, like a media file library.

I use it on this recent build: http://watershade.net/wmcclain/curator.html

-Bill
wmcclain is offline  
post #3 of 7 Old 07-09-2014, 08:12 AM
AVS Addicted Member
 
rdgrimes's Avatar
 
Join Date: Mar 2004
Location: Nuevo Mexico
Posts: 16,019
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 105 Post(s)
Liked: 198
I gave up on RAID systems years ago. Redundancy is nice, but you can accomplish the same thing using HDDs to simply back up critical stuff. Most of my media storage on HDD is just rips of shiny discs I own, so its already redundant. So instead of using JBOD or other RAID schemes, I just use multiple drives. Collections that require a lot of work or time to create are backed up on additional HDD and stored. I have about 20TB of storage including backups. I've never trusted RAID arrays where everything is under control of one OS and controller. (and everything is powered on at once).

I currently use Zappiti to catalog everything, and just turn on whatever drive I need at the time. The drives are all in multi-bay docks and shared on the network.
rdgrimes is offline  
post #4 of 7 Old 07-09-2014, 08:56 AM
Member
 
tdallen's Avatar
 
Join Date: Nov 2006
Posts: 126
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 11
I think unRAID does everything you want except for two disk failure. Even in a two disk failure the data is still accessible on all non-failed drives, though.

http://lime-technology.com/
Getting Started
tdallen is offline  
post #5 of 7 Old 07-09-2014, 08:56 AM - Thread Starter
Senior Member
 
ClemsonJeeper's Avatar
 
Join Date: Sep 2003
Location: Ashburn, VA
Posts: 340
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 52 Post(s)
Liked: 28
Quote:
Originally Posted by rdgrimes View Post
I gave up on RAID systems years ago. Redundancy is nice, but you can accomplish the same thing using HDDs to simply back up critical stuff. Most of my media storage on HDD is just rips of shiny discs I own, so its already redundant. So instead of using JBOD or other RAID schemes, I just use multiple drives. Collections that require a lot of work or time to create are backed up on additional HDD and stored. I have about 20TB of storage including backups. I've never trusted RAID arrays where everything is under control of one OS and controller. (and everything is powered on at once).

I currently use Zappiti to catalog everything, and just turn on whatever drive I need at the time. The drives are all in multi-bay docks and shared on the network.
Yeah, I can understand that point of view. However, the time it took me to rip my blu ray library is not something I wish to invest again. Therefore redundancy is still a priority for me.

That and I use my media server for my home file server as well for photos, documents, etc. I like the security of my data being safe (and my ultra important stuff being backed up off site in case my house burns down <knock on wood>).

I'm leaning more towards FlexRaid's transparent raid for Linux solution. The only issue is they just went public Beta so I'm not quite sure how comfortable I feel about putting all my data on a beta technology.

My drives (4x 4TB) and enclosure and cards show up all by Friday so I'll be testing this solution without destroying my current Raid setup so we'll see how that goes. If it goes well, then I can just pull all my 1.5TB drives from my hardware raid into the transparent raid array and have that much more storage.

My Home Theater Build: The Vortex Theater Build
ClemsonJeeper is offline  
post #6 of 7 Old 07-10-2014, 05:58 AM
Member
 
tdallen's Avatar
 
Join Date: Nov 2006
Posts: 126
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 11
I'd spend some time on the product forums for FlexRaid, SnapRaid, and unRAID before making a firm commitment. I was inclined towards FlexRaid-F on Windows from the beginning because my background was primarily Windows, but I wasn't that impressed with the level of communication/community/support for FlexRaid. I wound up going unRAID for my recent NAS build, but felt that there was more support for both unRAID and SnapRaid than FlexRaid.

Of course, if you just install and it works who cares... but I like to be able to count on a little help along the way.
tdallen is offline  
post #7 of 7 Old 07-12-2014, 02:47 AM
Member
 
theslydog's Avatar
 
Join Date: Jun 2007
Posts: 140
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 12
Quote:
Originally Posted by ClemsonJeeper View Post
Yeah, I can understand that point of view. However, the time it took me to rip my blu ray library is not something I wish to invest again. Therefore redundancy is still a priority for me.
I go 1/2 way.
I rip to a USB3 drive plugged into the PC and then copy over the net to a non raid 4 bay NAS. When the USB3 Drive is full it gets turned off and is my back up and I plug in a new USB3 drive. So if the NAS died the worst I have to do is plug back in the USB3 drive and copy it over to the new NAS Drive. Works for me
theslydog is online now  
Reply Networking, Media Servers & Content Streaming

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off