or Connect
AVS › AVS Forum › Video Components › Home Theater Computers › Walkthrough: Purchase and Setup of unRAID Media Server
New Posts  All Forums:Forum Nav:

Walkthrough: Purchase and Setup of unRAID Media Server

post #1 of 65
Thread Starter 
Disclaimer: I’m definitely not an expert at unRAID, in fact I’ve just put together my first server build quite recently. I did, however, do quite a lot of research in preparation, and yet could not find a concise step-by-step guide so I always felt like I was leaving something out, or missing an “optional” step that would still be to my benefit in the future. As a result I decided to make this mini-guide representing what I’ve found through my research and what worked for me so others can hopefully save some time and/or feel confident that they are not missing a critical step in the process.

There is a good possibility there is a better guide than this on lime-tech’s website, wiki, or forums, or perhaps even on this site, and my google search skills just suck. [[EDIT - As pointed out on page 2 of this thread, HERE is the best official walkthrough from lime-tech's site. It's longer than mine, but is a good resource if anything I post below is unclear, or you want to do more than a typical basic setup.]] If not though, maybe this will help some people. Enjoy.

Step 1 – Deciding you want unRAID

If you want something as a compliment to your HTPC to host your media files, the short answer is, yes, you almost positively want unRAID. The long answer… won’t be talked about here, but can be found many other places in this forum and elsewhere. There are other options, but a personal media server for a HTPC is what unRAID was designed for.

Step 2 – Buying the Parts

For more information, see unRAID’s Hardware Compatibility Wiki which includes a recommended build section. All prices I list are current as of 1/8/11. I’ll keep this section brief as well, since there is no one right answer and more thorough discussion readily available elsewhere.

2.a – Required Parts

Case – The main consideration in any case is the number of 3.5” bays, or stacked sets of three 5.25” bays that can be converted to 3.5 bays. It is advised to think about the future expansion of your server with this purchase. My advice: If you can’t see yourself going above 12 hard drives for a long time (~22TB of storage), go with the Cooler Master Centurion 590 ($80). If you do want to prepare for 13+ hard drives, go ahead and invest in the Norco 4220 ($350 – but has been seen recently for as low as $270 I believe).

CPU – This piece is largely unimportant for a straight storage server, as long as you have around 1.8GHz or higher. A frequently recommended choice is the AMD Sempron 140 ($37) or Intel Celeron 430 for an Intel build ($42).

Motherboard – Requirements are 6+ SATA ports (the more the better), 1+ PCIe x4/x8/or x16 slot (the more the better), onboard video (doesn’t matter how bad, just that it’s there), and onboard gigabit LAN (look for 10/100/1000 Mbps). Plenty of basic options can be found on Newegg for $50-$60 or so. Also, make SURE the onboard LAN is not using the Atheros chipset – it doesn’t mix nice with unRAID. I didn’t realize this, and had to buy a NIC card after hours of headaches trying to figure out what was wrong. Intel or Realtek based LAN chipsets are recommended.

Memory – Can be done with less, but just because it is so cheap get 1 stick of 2GB value ram, ($25) in case you want to use add-ons in the future.

Power Supply – 500W Recommended, to give breathing room for most setups, up to a 650W+ if you will be approaching 20 hard drives. ~400W PSU’s should work as well for most smaller/midrange builds, but it saves headaches later to get a 500W for basically the same price.

Hard Drives – As many as required to fill your storage needs. RPM doesn’t matter as much for media, so the big, 5400 RPM green drives are a good choice. HOWEVER, do NOT buy Samsung F4 Drives (will corrupt data in unRAID over time if correct firmware not installed, see below – F3 drives are fine) or WD AAKS drives (possible problems, not worth it).

[[EDIT 1/25 - As of now Samsung F4 drives have a firmware update that will fix the problem, but the firmware revision number on the update is the same as the number of the faulty firmware, so it is impossible to know for sure the corrected firmware has been installed. Samsung F4 drives can safely be used if you be sure and install the new firmware, or wait until the revision number changes and install that firmware as needed.]]

unRAID software – For 3 drive support, the software is free. For 4-6 drives, a plus license is required ($59 with code KEY10) and for 7-21 drives, a pro license is required ($109 with same code). An upgrade from Plus to Pro can be bought later for $59.

Flash Drive – 1GB+ is needed. Any that you have on hand will work except for Verbatim Store’n’Go, Alien 1GB, or SuperTalent Pico.

2.b – Optional Parts

Expansion Cards – Required if using more HDDs than your motherboard has SATA ports. Plugs into your x4/x8/x16 PCIe slot. The SuperMicro AOC-SASLP-MV8 is the recommended one ($116) and will hold 8 additional hard drives. Also required for use of these is 1 breakout cable per 4 HDDs ($18 each).

Drive Bays – In my opinion, it is not worth the expense to get hot swap bays (~$100 per 5 drives) as by then, the Norco case (with 20 hot swap bays) more than pays for itself. The Centurion case comes with 1 bay to hold 4 drives, so for drives 5-12, 1-2 more of these bays can be bought and installed ($25 each). The Centurion case can hold 3 of these bays, or 12 total HDDs.

Uninterruptable Power Supply – Not necessary, but many swear by them in the event of possible corruption during an unclean shutdown due to power failure. APC Brands are recommended; pick a model with enough Watts to power your system ($80-$120).

Internal USB Adapter – Recommended so you don’t have your OS Flash Drive sticking out of your system. Can pick up a Header-Pin to Type-A adapter for around $10 (example).

2.c – Other Part Concerns

Gigabit Router/Switch – If you don’t already have one, make sure you get one for optimal media streaming/transfers. If you have an older 10/100 router, buy a Gigabit switch versus replacing your router as they are typically cheaper.

Cables – Keep in mind most HDDs don’t come with SATA cables and some motherboards only come with 1-2, so hit up monoprice for the difference. Also make sure you have a spare CAT 5/6 cable for the new build, and a DVI/VGA/etc video cable to go from your onboard video to your display at least for initial setup.

Step 3 – Building the System

Mostly, there is nothing special about building the server, just put it all together as you would a normal computer. However, it is highly recommended that before you install your HDDs, you write down the serial number of each HDD and make a spreadsheet as to where you are putting each HDD, by serial number, in the case, so in the future you know where each one is located.

You then need to get your flash drive configured for unRAID. For the Plus and Pro versions you have to have your flash drive’s GUID. I recommend just installing the free version first, and then you can see your flash drive’s GUID once installed on the devices list. So download/extract the unRAID software (make sure to follow all directions, like labeling your drive UNRAID) and also pick up the Pre-clear script HERE and add it to the flash drive in the same place as the bzroot and bzimage files.

With the server built and the flash drive connected, you should now be ready to turn it on and try a first boot.

Step 4 – Initial Installation

4.a – Getting unRAID up

Make sure you have a keyboard, network cable, and display connected and fire your new build up. If it doesn’t automatically boot off of the flash drive, go into the BIOS and make sure you enable booting from a USB flash drive. While in the BIOS, also make sure your hard drives are set to ACHI mode, usually found under SATA configuration. Make the changes if needed, save and exit, and restart to boot it up.

Once it boots, you should see it run a bunch of commands and stop at Tower Login. That means it’s up and running and it is time to check the portal via another computer. In your browser on the other computer (Chrome doesn’t work for me so I use FF, but YMMV) type “//tower” and the main GUI should come up. If it doesn’t, replace “//tower” with the server’s IP address. To find the IP address, go back to the server and at the login screen type “root” and press enter, and at the next prompt type “ifconfig eth0” which should bring up the IP information.

In the main GUI, for now leave everything the same and click on the Devices tab, where you can view your flash drive’s GUID. Using this you can now purchase the Plus/Pro version if needed, which is usually delivered in a couple of hours after ordering. You should also see the hard drives you have installed in a drop down menu that you can add to the array. Don’t add them, but write down the connection labels for each HDD (ex: sda, sdb, hda, etc.)

4.b – Hard Drive Pre-Clear

*** Note: This step will erase ALL data on the hard drives you do this for. If you have installed some hard drives with data and some without, you will need to do this for only the empty hard drives, then build the array using the pre-cleared drives, then transfer the data off of the drives remaining to the new array, then repeat the process for the old data drives.

This step is not technically mandatory, but you really need to do it to make sure your drives are good to go for data storage. This will prep the drive for unRAID by writing zeroes to your entire HDD, and then run a S.M.A.R.T. test to check disk integrity. Assuming you downloaded the pre-clear script onto your flash drive, go back over to your server, login using “root”, and then enter the following commands:

cd /boot
preclear_disk.sh /dev/###

where ### = the connection label for the hard drive you want to pre-clear (that you had written down above). This will pull up the information of the HDD you are about to pre-clear (so you can double check it is the correct one) and ask you to confirm by typing “Yes”. Confirm the command, and the pre-clear will start.

Be prepared, this process takes a long time, around 30 hours for a 2TB drive. Fortunately, you can do up to 6 drives at one time, as unRAID has 6 virtual consoles that you can use. To toggle between them, use Alt-F1 through Alt-F6, and repeat the commands above for each hard drive.

After a HDD is done, the S.M.A.R.T. results will display any problems it found during testing (Tip: if the results do not all fit on the screen, you can use Shift + PageUp/Down to scroll up/down screen). Most if not all hard drives will show some errors, most of which are harmless, but some may be indicators that a drive will soon fail. For interpretation, check the S.M.A.R.T. wiki and the interpretation thread on the unRAID forums HERE. If the drive has critical errors, it may be best to RMA the drive at this point (however, even though the S.M.A.R.T. wiki says Read Error Rate may be an indicator of failure, many normal drives display this error. Check the unRAID thread for more explanation and examples).

4.c – Creating the Hard Drive Array

Note: This is a good point, if needed, of shutting down the server (either through the “poweroff” command in console or through the GUI) and installing the Plus/Pro version of unRAID if you haven’t already.

Once all of your drives are pre-cleared, go back to the GUI in your browser and hit the Devices tab. You can now add them as data/parity disks (remember, the parity disk must be the biggest size disk in your array). Tip: If you will be copying lots of data onto the array initially, wait on assigning the parity drive until after you have copied the data, as it will otherwise bottleneck transfer speeds. You will not have protection until after the parity drive is assigned, but you can assign it before deleting the files from their original location.

Click over to the Main tab, and you can see the drives added listed as “Unformatted” in the disk status tab, and a Format button should be in the command area and will activate once you check the box that says you know what you are doing. Hit the format button, and in 5-20 minutes the drives should be formatted and ready to go.

Once you assign the parity, (preferably after you have put data on the drives) you can also begin the parity check with an enabled button on the main screen. This will run periodically by itself, but it’s best to check it once all your data is on manually to make sure there are no errors detected.

4.d – Creating User Shares and Managing the Network

It’s now time to add a folder structure that will be shared across the array, which is done by user shares. Click on the Shares tab, and add a user share by entering a share name (ex: “Movies”), choosing an allocation method, and putting a split level.

For allocation method, I recommend leaving “High-water” enabled (which fills disk1 until 50%, then the next disk until 50%, etc, then back to disk1 until 75%, and so on) if you are unsure, but any method you select is fine.

For split level, it’s a pretty complicated concept that’s hard to explain without pictures, but basically refers to the level of folder in the share that you want to split among disks versus keeping them all on one disk. For example, my user share will be a folder named “Movies” and within that folder I will have individual folders for each movie, holding the .mkv and .nfo files, thumbs, etc.. I want the “Movie” folder (which is level 1) to be split among all the disks, but I do not want the contents of the individual movie folders (level 2) to be split, I want them all together. Thus, for my Movie user share, I want to set it to split level 1. Better, more thorough examples can be found HERE.

Once that is done, click over to the Settings tab, and check the Workgroup entry to make sure it is the same name as the other devices on your network. To check your current workgroup name, right click on your My Computer shortcut and click Properties, and you should see it there. If the server workgroup is defaulted to a different name, change it in your GUI, hit apply, and the change should happen in a minute or two.

At that point, the server should show up in your network, and you should be able to access the user share folders you created and begin using your server just like any other shared folders on another computer. Dragging and dropping into the user share folder should work fine, and unRAID should automatically split the files according to your split level and allocation settings.

Step 5 – Server Maintenance

Adding cache drives, monitoring and replacing drives, using add-ons, setting security, and other very important things are all important topics that can be learned at a slower pace and seem pretty adequately addressed in unRAID’s FAQ HERE so I’ll leave that off this guide.

That’s pretty much it! If I left something off let me know, otherwise I hope this helps.
post #2 of 65
Awesome post ! I'm a few months out yet,but what a great guide ! Hope to see some more.
post #3 of 65
Thanks!! Great guide
post #4 of 65
There is a Samsung F4 firmware update that claims to fix issues that folks found with this drive (data loss). Are you saying that even with this firmware this drive is still a problem for UnRaid?
post #5 of 65
Thread Starter 
The last I saw the firmware update they did was not a true fix and may still cause problems down the road, but I have no first hand knowledge as I simply avoided the drives. If you have found something more recent that says differently however my info may be outdated on that.
post #6 of 65
The problem with the firmware update is that they used the same firmware revision number so there is no way to tell that it actually updated with the exception of the software that told you it updated successfully. The problem is that it corrupts data without the user having any idea it's happening until it is too late. I would stay away until they fix this issue with the rev number.
post #7 of 65
Great write up. I completely agree with everything. I just ordered everything for my unraid box and should have it up tomorrow. Props
post #8 of 65
Thread Starter 
Thanks, and yeah SUBCOB, the revision number issue is the last I remembered as well. Hope they get that fixed soon.
post #9 of 65
Lots of good info there. One point I'd like to make about the use of preclear_disk.sh. If you have a drive you'd like to add to the array and you don't want the array to be down just to preclear the disk, you can insert the drive in an empty slot and run preclear_disk.sh from the command prompt. The disk will be precleared in the background and won't affect normal operation of the array. When the preclear has completed, shut down the array, move the disk to the assigned slot, power it up and add the disk to the array.

If you're using a WD EARS drive, make sure you jumper the drive before inserting it into the array. Running preclear_disk.sh prior to setting it up in the array is an absolute must. Otherwise, you'll have all sorts of issues getting it to work.

One note about using a UPS. It is highly recommended that you do so. If the array is shut down unexpectedly it will automatically perform a parity check when it starts back up. The array cannot be written to until the parity check is done. You can, however, read from the array while the parity check is in progress.

Don't waste your money on a PCI or PCI-X SATA expansion card. They will create a huge bottleneck in the system as the PCI bus has much less bandwidth available compared to the PCI-E bus. Running parity checks and setting up new disks will take several times longer to accomplish due to the lower bandwidth. Make sure you only use PCI-E SATA controllers like the one recommended. Adaptec also makes a 4-disc controller that works well with unRAID.
post #10 of 65
Just an FYI, it appears in this post from Tom, that with the 4.6.1 release, you'll no longer need to jumper the EARS drives:

http://lime-technology.com/forum/index.php?topic=9936.0
post #11 of 65
Talk about perfect timing!
I received all my parts for my UnRaid server last week and the 2tb EARS drives I bought from Dell showed up yesterday.
So with any luck by this weekend I will find the time to put it all together and get it running.

Your post answered my biggest question.
I have 4 2TB drives and was unsure about assigning parity before or after transfering all my Data from my WHS Server box.
Now I know to assign only 3 of the drives as data drives, add my shares, move data over, then assign parity. Thanks!

One question still remains though.
Since my drives are EARS, I went ahead and ordered jumpers from ebay. They havent arrived yet, and I was hoping they would show by this weekend. But from the post Brianley linked, as long as I use UnRaid 4.6.1 I dont need the jumpers.
Is 4.6.1 available now? If I buy my UnRaid license is that what I will get?

Thanks
post #12 of 65
Awesome guide! Will definitely be bookmarking this for when I fill up the 4 TB in my HTPC.
post #13 of 65
Thread Starter 
Quote:
Originally Posted by dan4081 View Post


One question still remains though.
Since my drives are EARS, I went ahead and ordered jumpers from ebay. They havent arrived yet, and I was hoping they would show by this weekend. But from the post Brianley linked, as long as I use UnRaid 4.6.1 I dont need the jumpers.
Is 4.6.1 available now? If I buy my UnRaid license is that what I will get?

Thanks

4.6.1 is not available yet - you can check by going to the download page: http://lime-technology.com/download/...-unraid-server
It currently shows only 4.6, but when that changes you will know it is out.

You then have 2 options with EARS drives basically, you can go ahead with the install using the jumpers on 4.6, and just leave them like that forever, or you can wait until 4.6.1 and not install the jumpers.

It may be worth it to wait if you care about the possible benefit from the advanced format nature of the drives, but imo it's not a big deal either way - I'd just build the sucker. It's just a gamble on when 4.6.1 hits the stable release, it could be tomorrow, or a month from now.

Also it's worth noting that 4.6.1 and later versions of unRAID fully recognize the drives with jumpers so its not like you are limiting yourself to v.4.6 forever if you jumper them, you can upgrade unRAID at any time.
post #14 of 65
Thanks for the reply.
You are right.
I am just going to build it as long as I get my jumpers that I ordered.
post #15 of 65
Quote:
Originally Posted by dan4081 View Post

Your post answered my biggest question.
I have 4 2TB drives and was unsure about assigning parity before or after transfering all my Data from my WHS Server box.
Now I know to assign only 3 of the drives as data drives, add my shares, move data over, then assign parity. Thanks!

You might want to check the unRAID documentation on the Wiki to make sure the data won't get wiped when it initially sets up the drives. UnRAID tends to want to preclear any new drive it sees unless preclear_disk.sh has been run on it first. To be on the safe side, I'd go ahead and set up the data and parity drives before transferring any data. Parity will be adjusted as data is written to the drives.
post #16 of 65
Thread Starter 
Quote:
Originally Posted by captain_video View Post

You might want to check the unRAID documentation on the Wiki to make sure the data won't get wiped when it initially sets up the drives. UnRAID tends to want to preclear any new drive it sees unless preclear_disk.sh has been run on it first. To be on the safe side, I'd go ahead and set up the data and parity drives before transferring any data. Parity will be adjusted as data is written to the drives.

Agreed. I didn't really go into the scenario where you are installing drives with data, but you can not, to my knowledge, add data drives to the array without them being wiped. You have 2 options on transferring the data I believe:

For this I'll just assume you have 1 2TB for Parity, 1 2TB with data already on it, and 1 2TB that is clean to keep it simple.

Option 1 (better if command line transfers scare you and more fool-proof, but a little more work) - Only install the disks on your server initially that are empty (2 of the drives), pre-clear them, then assign 1 of them as a data drive, but don't assign the other as a parity yet. With the data drive in the array, format it, then start the array, create the user shares, etc, and you should be able to see it on the network. With your HDD with data on it connected to another computer, dump the files onto the array using Windows Explorer. Once that is done, assign the parity drive. After the parity check completes (so now you have data security again) shut down the server, install the old data HDD, then reboot, pre-clear it, and add it to the array.

Option 2 - install them all at once, but only pre-clear the 2 empty ones still (so make SURE you get the serial numbers right!) and add the empty one as a data disk in your array again. Once that is done, install the SNAP script from HERE to be able to mount/share your data drive that is not part of the array. You can then copy files from your data drive to the array drive by using the "cp" command, explained HERE. After that's done, you can then assign the parity drive, then pre-clear the data drive like normal.

Honestly, option 1 is what I did and while it is a pain going back in the case and installing another hard drive again, it's still my recommendation for beginners just because you can know you aren't screwing anything up.
post #17 of 65
Sorry guys...I wasnt very clear with my post.
Im not going to physically move any of the HD's from my WHS box. Once the new server is done the WHS box I have now is going to be sold with all 4 HD's left in it.

My plan is as follows:
I am going to install 4 brand new WD 2Tb EARS drives into my new server to start. (I'll pre clear them first.) I will set up 3 of them as data drives.
I will then move all my data from my WHS box via network to the new drives.
Once it is moved over then I will add the 4th drive as the parity drive.

Better?
post #18 of 65
Yes
post #19 of 65
Great idea for a thread.
Maybe I missed this, but AFAIK, the JoeL's Preclear script is an add-on, although I'd consider it to be essential. Does Tom now include it as a default?

If not, it can be downloaded at the end of the first post in this thread.

Also, the new version 5 beta 3 which will be coming out "soon" will have support for the 3TB drives that are already released.
post #20 of 65
Sorry. Redundant post.
post #21 of 65
The preclear script can be added to the boot flash drive. It is not included with the unRAID software AFAIK.
post #22 of 65
I received my jumpers finally and will put this together tonight/this weekend.

Even though it looks like tlfreeman did a great job explaining 'Shares' I will probably be back with some questions when I set mine up.
(I would like them to be the same as the various folders I had on my WHS.)
post #23 of 65
Check the documentation posted in the unRAID wiki. It should answer any questions you might have.
post #24 of 65
Quote:
Originally Posted by captain_video View Post

Check the documentation posted in the unRAID wiki. It should answer any questions you might have.

Already read it.
Between the WIKI and what was posted here I believe I should be good.
post #25 of 65
Im up and running!

Other than a couple stupid mistakes on my part, it was actually really easy.

Right now I am moving my data over and it looks like it will take a couple of days.

Thanks again for the step by step.
post #26 of 65
Just a reminder, once you get you data moved over to unRAID perform another parity check to ensure your data is good. I am sure this is part of your plan, but will say it anyway.
post #27 of 65
Thanks Subcob
I didnt add a parity drive yet...just 3 data drives.
My plan is after the data is done I will add the 4th drive as my parity drive then.
I assume there isnt anything special I have to do except add it and let it do its thing?

One question...where on the web GUI do I see how much free/used space I have left?
post #28 of 65
Thread Starter 
You are correct, once you assign the parity drive it's pretty easy, and it will just do its thing.

And on the main page of the GUI you should see "Size" and "Free" - Size is the total size of the drive, and Free is the free space left. I guess you'd have to do some subtraction to figure out the space used .
post #29 of 65
Thanks again.
Ill look there once I have some data moved.
post #30 of 65
If you install unmenu you will have much more freedom with what you see in web gui. If not already done there are threads on the unRAID forum on how to install. Highly recommended.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers
AVS › AVS Forum › Video Components › Home Theater Computers › Walkthrough: Purchase and Setup of unRAID Media Server