Building a low power, high perf NAS with iSCSI

Hi everyone,
I am starting a thread related to my quest for a NAS box build.
[apologies for yet another thread, but my requirements are rather complex]
If you look into my requirements you may notice that I want a fully featured NAS box. A simple external HDD connected to a router wont do.


My background:
I have been running multiple computers at home [all PCs] with one HTPC serving as a 'server'. I have many other PCs which use the server. I am a System Administrator by profession, work mostly with Windows servers and VMware with a bit of Linux servers and Macs thrown in.

My requirements:
1. high network throughput capability
2. low power consumption
3. Ability to backup to external device [like external HDD or tape]
4. iSCSI capable
5. RAID-5 in hardware [preferably RAID6]
6. Granular permissions control over its file shares.
7. Expandable to multi-terabyte capacity

Nice to have:
DLNA

My application:

The NAS will serve as a central data repository for stuff like recorded TV, movies, photos, music, data etc
I will use it to serve a lot of photos which I edit on another PC in Adobe Lightroom. I also do HD video editing and want to store the RAW and final videos on the NAS. That means transferring large files over the network.
This is why I need high network throughput.


Low Power I don’t want to keep a PC running 24x7. So the NAS should consume low power and also be able to sleep at night time.
However, I do want data to be available all the time. So Wake on LAN type things are needed.

Integrated backup would be very nice. Although I do understand that I could simply backup the NAS' contents over the network from another PC.

DLNA would be also nice in order to support hardware based devices like receivers but I hear DLNA is no good anyway.

iSCSI capability is very nice! Win 7 supports it natively. This will allow media center to easily record directly to NAS.

RAID:
As an IT admin, RAID is second nature to me. I work with large SANs at work too.
My experience with firmware RAID [the kind where you have it in the motherboard's BIOS, like Intel® Matrix Storage Technology] has been very good. However, I have mostly used RAID 0 for higher speeds. I have on occasion used RAID-5 but have not run any bench marks on it.

My testing:
In my VMware environment, I have already tested:

1. freeNAS

• easy setup
• Could not get SMB [windows file sharing] working properly
• iSCSI works fine but I heard it can have only 1 iSCSI target.
• Network throughput was OK but not a valid test since I was in a VM.
• I hear that freeNAS's network throughput may not be good.

2. OpenFiler

• Little more involved setup
• SMB worked
• iSCSI works fine
• Network throughput was OK but not a valid test since I was in a VM.
• I hear that I could use backup software called "Amanda" inside OpenFiler's Linux OS.

unRAID:
I have also researched into unRAID [not tested it yet] but I worry about the slow network throughput most people talk about.

Windows Home server:
This is also one option I have considered but it does not serve my needs totally. Plus I have heard about slow network throughput and such.

Information I am looking for:

1. Has anyone used an atom CPU and done any network throughput type benchmarks with any NAS OS?

I worry that the atom may not have enough power to serve large files over a GB network quickly.

2. How is the disk r/w speed with firmware RAID-5?

3. What motherboard would you recommend for this application?

4. How much power are we looking at?

In my testing with a wattmeter, I have seen that my current HTPC sucks around 230 watts from the power line. It is a core2duo on an intel motherboard, with a 2x 500 GB RAID0 and a GT6600 GPU. Running it 24x7 is simply unacceptable to me. [hence this NAS quest!]

Your recommendations and comments are welcome.

reserving for future NAS build instructions

Here is my experience with Openfiler [OF] till date.

• Everything is on enterprise grade gigabit switches.
• OF was setup on a standard Dell Optiplex 755 with a single 80 GB hdd [no RAID]

Downloaded from http://www.openfiler.com/community/download/ I used the “ISO Image (x86/64)”
Installation was pretty straightforward but at the partitioning step, you need to manually edit the root [/] partition to be 1024MB…otherwise the auto partitioning will use up all disk space and leave nothing for our use.


iSCSI setup:
one word....Works fine!

I used these instructions for setting iSCSI in openfiler:
http://www.techhead.co.uk/how-to-con...ith-vmware-esx

Note: I only used the instructions until he says “Open Filer Configuration Stage Finished!”
Then I opened up Windows control panel and searched for iSCSI. In the setup tool I entered the IP of openfiler and it immediately detected the iSCSI target. I told it to use it.
Then I went to disk management and assigned a drive letter and formatted the iSCSI “disk”.

File sharing:
Instructions for setting up file sharing was taken from here:
https://forums.openfiler.com/viewtopic.php?id=2491

Note: I struggled lot and the file share authentication did not work for me. I kept getting authentication errors. I ended up just enabling public access. [Need to spend more time] I think our FDCC secure desktop settings may be the culprit.

Anyway I went ahead, with the benchmarks.

Here are my benchmarks on OF:
These numbers were observed on a windows server 2003 VM [its drives are on our SAN]:


using fileshare:
write to OF = 280 mbps = 33.3 MB/s
read from OF = 260 mbps = 31 MB/s

using iSCSI:
write to OF = 540 mbps = 64 MB/s
read from OF = 260 mbps = 31 MB/s

Notes:
1. I think "read from OF speeds" are limited to the "write to disk" speed of the VM.
2. Notice the huge jump of writing to OF when comparing fileshare vs iSCSI.



I will also try to use some kind of backup [maybe Amanda] software on OF.

Amanda + Openfiler:
The more I search about Amanda and Openfiler I more I get convinced why we do IT the way it is right now. Keep it simple, otherwise it is simply not worth it!

For once you cannot install Amanda server on Openfiler. The people who are talking about running Amanda on openfiler are actually installing a client of Amanda.

http://forums.zmanda.com/showthread.php?t=2699
http://www.lost-it.org/drupal/?q=node/234
https://forums.openfiler.com/viewtopic.php?pid=23249

Having a simple Windows server with a Tape drive gives me almost all the features I want [not iSCSI though]. However I could get Windows Storage Server and get iSCSI.

I thought about iSCSI and my idea of recording everything to the NAS using a iSCSI mounted drive on the NAS. But then how will I give access to these recording all the other HTPCs? I could do a read only mount of that volume on the other HTPCs.

Maybe iSCSI is not that important after all. Man! It sure is fast :-)


More later :-)

reserving #3 for future NAS build instructions

reserving #4 for future NAS build instructions

reserving f#5 or future NAS build instructions

well, as high as possible :-)

For numbers.........here at work I regularly get around 25 MB/s speeds over a gigabit network while downloading large files off an old file server.

However, I get 40 - 48 MB/s [around 333 - 402 mbps] to and fro the SAN.

Something around those lines would be really nice to have.

HD Video files are pretty huge off my Canon 7D DSLR.

Interesting!

So you guys are getting pretty healthy speeds from unRaid! From the stuff I have read........I got the impression that unRaid was slow.

Unfortunately, unRaid does not do iSCSI

Anyone with experience with Openfiler here?

WOW ! that opened up my eyes
60 MBps? Are you sure?

I just looked at this page:
http://lime-technology.com/wiki/inde...t_is_unRAID.3F

they say:



Please tell me what is your config inside unraid....you are not running some RAID0 or something, are you?

BTW, what is a cache drive [I gotta read up on that]

thanks for your feedback.

Looks like everyone here is leaning towards unRAID.

I looked at it yesterday [just the specs] looks good.
I think the R/W bottleneck [actually not too bad] is due to the software RAID.

Advantages are ability to use different sized drives. Disadvantage is speed.

Over this weekend a couple of things happened which may change this project.

1. My HTPC1 which also works as a server stopped responding. It was so bad that at one point I thought I lost the disk array and I'll have to rebuild it. Miraculously it started working on its own. I never could find what actually went wrong. I suspect the external HDD.

2. I found out that backups on my HTPC1 which also works as a server were not really working properly for the past few weeks. I use Acronis True Image Home 2009. Frankly I have never been happy with the reliability and options/features of that software.

If the server had lost all data AND the backups were not really there.I'd have lost ALL my critical data. This has made backups extremely important to me now.

As an IT guy, we swear by enterprise backup stuff like tape libraries and ArcServe, Backupexec etc. That has grown to be extremely reliable in the past few years. However, it is still prohibitively expensive for home use. But as my wife put it, our digital memories are priceless.

This makes backup my first priority.

So at this point I may want to look at a Windows Server having a tape drive running Backupexec. Maybe WHS will do. especially that now the new version is coming out.

But with that [I guess] low power goes out of the picture

Presently LTO-3 tape drives are around $1500. That will take 400 GB native. My critical data is currently around 250 GB and growing. I could look for LTO-4 but that gets even more expensive.

What do you use for backups?

I have tried backup to cloud using Mozy.

Very slow uploads but the deal breaker was the restore problems.

When the time came to restore [I lost a disk] Mozy gave me a hard time. the client kept on giving me errors. I opened a ticket and the support guys kept going roung and round in circles.

However, after about 2 weeks of troubleshooting, they were finally able to restore my data.

have you tried restoring data with Crashplan yet?

WOW !
this may be the perfect thing! It even does backups! Sweet!

For example this $350 TS-219P+ Turbo NAS' specs say:

I wonder how good is its performance?? Gotta google it.


Wait wait. I think I spoke too soon.........I looked at reviews at newegg:
http://www.newegg.com/Product/Produc...82E16822107018

The backups are just a copy and I quote:


I'm not sure this will do. I want proper daily backups which go back a month or so. Not just one copy of my data. What if something gets deleted accidently. Gone forever?