[pandaconda]

I'll be building an HTPC soon for playing HD video on a 1080p projector. If all goes well, I'll be running it with Ubuntu (to avoid smashing the damn thing when Windows MC crashes time after time).


I'm going to use a basic MB (~$75 US), but one that has built-in RAID 0,1,&5.


My question is this: Is it worth setting up a RAID 5 array of 4 1TB disks if all I'll be doing is playing HD video?


I don't want to pay for a separate RAID controller, and if I have to I'll be fine with stand-alone disks, but the extra data protection of RAID 5 would be nice. Any additional comments on ease-of-use for a Linux array, transfer speeds, etc. would be welcome.


Thank you very much in advance!! (I don't think there has been a thread on this topic in about a year, I did check)

 

[ndoggac]

I would avoid using the motherboard "fake-raid" for anything on a server. Use Linux software RAID if anything. If your motherboard dies, you don't have to buy the exact same motherboard to rebuild the array. Using software RAID you can move all your drives to a new system and rebuild the array. Performance difference is negligible. I run a Linux media server using Ubuntu Server Edition (Jaunty). I have two 640 GB drives setup for HD movies, tv, etc. Then I have two 500 GB drives in a mirrored RAID1 for my music, pictures and other media and backups.


I guess my approach is I can always re-download the HD movies with relatively little effort. But my pictures are impossible to replace (also backed to DVD) and my music (with much time invested into editing ID3 tags) would be a real pain.


The whole system is headless (power & network only) and I use the Popcornhour networked media player hooked to my HDTV.


Some recommended addons for your Ubuntu server:


webmin (remote admin web interface)

incron (auto run scripts when folders/files change, created, deleted)

azureus (********** client you can run headlessly)


if you ever want to feed an additional HDTV from the same server:

popcornhour (wired or addon wireless usb)

yamj

llink


google mdadm for Linux raid guides, and check the ubuntu forums

 

[pandaconda]

Thanks for the reply, ndoggac. If I do the RAID 5 thing, I'll use mdadm (I had just finished reading about that).


I'm leaning toward just 4 separate drives, or 2x2 in a 0 configuration. Like you said, this isn't really where I'm storing the important stuff.


I guess the deciding factor now is:

How hard is it to rebuild the RAID 5 with mdadm after a failure of one of the disks? I'd appreciate it if you or someone else could point me to some more info on this, I haven't really been able to find it other than people just saying "it's a pain".

 

[Troubleshooter]

Seriously - use Linux software RAID -it'll perform fine and you can plug and unplug the disks over years and they'll still be recognized and work fine. I have an ancient 3ware controller in one of my old servers with some old (rather useless) data on it. That card certainly isn't being made anymore and getting another would be difficult. On the other hand my current server uses boring old Linux software RAID. It'll work for a very long time even after the controller goes away.


-Trouble

 

[pandaconda]

Oh sorry, what I meant was "What is the rebuilding process for a RAID 5 array in general?" Does it take lots of expertise or time?


If there is a significant hassle involved, I'll just set it up as two RAID 0's and be sure not to put anything too precious on it. I don't really want to lose as much data capacity as I would with RAID 1.

 

[k_ross]

Expertise? No. Time? Yes. But the array is usable while it rebuilds the new drive. You just have to hope another drive doesn't die while it's rebuilding.

 

[MichaelZ]

I bought a 4 port sata 3ware card on ebay for $75, it worked so well I found another for the same price. A lot easier to setup and maintain - IMHO - I've done it both ways.

 

[drkdiggler]

Rebuilding the array is simple:


1. Power down the machine and remove the bad drive. This can also be done hot with SATA drives, but requires more effort/complexity.


2. Insert new drive and power up.


3. Format new drive to have same partition setup as other RAID drives.


4. Use the 'raidhotadd' command to add the new disk back into the array.


I have done it a few times through my experience with Linux and its not a big deal, and to be honest, I can't think of it being much easier. The only advantage that I have heard of from hardware controllers is that they will automatically start rebuilding the drive, without you having to setup the partitions. However, in many cases, your RAID storage drive will just have one big partition on it anyway, so this isn't a concern.

 

[ndoggac]

the drive partitioning & formatting, raid configuration and raid rebuilding is made very easy through the webmin interface as well if you prefer not to use command line. I've swapped my RAID1 configuration from an old server to a new one with no problem. I also recovered it after a PATA ribbon cable went bad on me. It's quite easy, and with a quick google search you can find some great guides.

 

[drkdiggler]

quantum, can you set a read ahead buffer for the RAID array as well, e.g., /dev/md0? From the commands above, it looks like you are setting the read ahead buffer for the drives within your RAID array. I was contemplating increasing the buffer size for my RAID array because I run into issues if I am doing a heavy write while watching a BluRay. I thought that increasing the read buffer would take care of the problem.

 

[quantumstate]

I recommend this guide . It's the one I used to convert my running non-RAID HTPC to a RAID10offset2 array, and it's great. Just substitute your RAID level of choice.


And for resync please consider enabling bitmap-based write-intent log. Very nice. It keeps a bitmap of 'dirty' (changed) blocks in the array, and on any resync only those need be reconstructed, not the whole array.

After the arrays are created and mkfs:

# mdadm --grow --bitmap=internal /dev/md0

(Reverse with mdadm --grow --bitmap=none /dev/mdX)

(The /proc/mdstat page describes how to interpret the bitmap line)


Also consider enabling readahead. It's the readahead cache for the array, and can greatly speed up access if large files (like ours). First see what default is and test with no setting:

# blockdev --getra /dev/sda

256

# blockdev --getra /dev/sdb

256

# time dd if={somelarge}.iso of=/dev/null bs=256k


Then set the a value 1024, 1536, 2048, 4096, 8192, and maybe 16384, to see where peak is (reboot each time, or disk cache will lie):

# blockdev --setra 4096 /dev/sda

# blockdev --setra 4096 /dev/sdb

# time dd if={somelarge}.iso of=/dev/null bs=4096k


Put your commands with the final value in rc.local or sommut. This alone can double or triple read speed.


Now; the array will be very busy during mythcommflag and the vast majority of access will be write, so further tune for that if you can. That's why I used the Offset variant of RAID10.

 

[quantumstate]

drkdiggler (sorry, I was still modifying my post when you posted) no you have to set ra on the physical drives, and they MUST be all the same of course.


Write is always a problem with Myth. I haven't solved this yet, but I hope someone here has the time and wherewithall to solve. Meanwhile increase the read buffer on your BD drive...

 

[drkdiggler]

For some reason, my question got put ahead of your post. Same question.

 

[drkdiggler]

I will give it a shot when I get a chance. Just for clarification, I play BluRays from my RAID array on my file server over my network. The concurrent writes were file copies from another machine to the RAID array. I'm hoping that increasing the RA buffer to 4096 or so (~ 20 MB over 5 drives) will allow the system to buffer enough data to feed my HTPC.

 

[quantumstate]

Yes, on posts like mine above I often modify 10 or 15 times to complete and clarify, and I hate that "modified by' tagline so I often delete the original and repost it.


OK, so it's not a BD device but your array. Please try each of the buffer sizes I gave above (and reboot each time) to see where you get the most benefit from readahead on your particular system. Then put the commands for your drives into rc.local.

 

[bellaskid]



Quote:
Originally Posted by ndoggac /forum/post/16972460


Some recommended addons for your Ubuntu server:


webmin (remote admin web interface)

I was thinking about using eBox for my Ubuntu server (I'm done with WHS) until I saw your post. I investigated into webmin further and found this: https://help.ubuntu.com/community/WebMin


Have you had any problems with webmin?

 

[ndoggac]

not a one...

http://ubuntuforums.org/showthread.php?t=1081042


Posts 8 & 9 sum it up for me.


It was also recently in the running for sourceforge community choice awards

http://sourceforge.net/community/cca09/winners/


under "Best Tool or Utility for SysAdmins"

 

[blackoper]

i also use webmin and have been for years. It's a very nice program