What is the best raid level for HTPC? - Page 2

Fine, I highlighted the parts that are correct in RED. The rest is either completely wrong or as a minimum inaccurate.

Peter

Can you explain why you say the red highlighted areas are inaccurate?

Originally Posted by Wikipedia

A stripe on a 5 disk RAID5 array contains some 4 disks of data information and 1 disk worth of parity information. unRAID does a similar thing but takes the data part of the stripe from whatever is on each individual data disk and then puts the calculated parity on the parity drive. It doesn't combine files from different disks into a stripe or have parity drive pointers. unRAID also doesn't do any parity checking when reading data.


I guess you could argue unRAID is only good for archival storage but that's not really true. You read and write as much as you want without "breaking" the storage format and it would just be slower when doing so compared to a striped RAID array. Still, unRAID really is a storage solution geared towards doing more reading than writing. To me, an archive in computer speak is a permanent back-up copy of data (like a zip file). People aren't using these servers to archive content, they're using them to store content they access all the time.


The reads are done by just reading from a single disk so will not be slower than a single disk.

The drive rebuild would be similar to any other RAID array rebuild time. I'd expect it to take about 8-10hrs for a 2T drive. Actually, this rebuild time is much quicker than some of the times I've read for rebuilding a RAID 5 array drive (I've read about it taking 35hrs in some cases).

There are no checksums. No idea what the author is trying to describe there.

Scaling problems - Once again, no idea what the author is trying to say. Adding a 2T disk takes about 8hrs. You can add different sized drives and make full use of each drive. You can't claim either of those on your RAID5 array. Then, there is a disk burn-in and pre-clear script that you can first run on the drive. After running this script, it will take you more time to physically install the disk than to add the disk to the array.

There is no bottlenecking if you are reading drives. The bottleneck will be the network. There is a bottleneck if you write to multiple drives at once since each of these writes is using the common parity drive.

Peter

If you remove the "archival" word I think the description from Wikipedia is perfect.


If unRAID is reading the data disks directly and not doing any parity check, I agree you would have no read performance hit but anytime you have parity involved the writes would be affected.

Were the comparisons of the RAID5 vs unRAID using comparable disks?

From the scalability standpoint, isn't the parity based on the bit on every data disk? So if I had an unRAID array with 10 disks, it would use the bit from the other 9 data disks? But other RAID configs do the same, right? The difference is where the parity is stored?

I would have to disagree with the network *always* being the slowest part. With a gigabit network, your max bandwith is around 125MB per second. In an unRAID array you can put any speed disks, I have seen some very slow disks before. My WD was only reading at around 20-30MB per second. If you have newer disks I would hope that they are faster.

Personally, I would just go with a FreeNAS or OpenFiler solution with RAID5. My reasons are when I build a new system in general I match the HDDs and the above solutions are free. If I had 3 different disks though I would seriously consider unRAID, I just don't think I would pay for the software.

I would suggest you try to update Wikipedia if you think it is inaccurate.

I didn't post that the network is always the slowest part. I posted that the network will be the bottleneck when reading drives. I also never posted that you'll always hit that bottleneck.

You must be using some good networking hardware to get 125MBps...

Peter

+1000

So a hard drive fails, what do you lose? Some movies? If you really care, back them up to a separate drive. RAID exists to give maximum system uptime, not as a backup solution.

Well, 125MBps is the theoretic bandwidth limit, I don't expect to get close to that but I am hoping to get around 100MBps.


I read "will be" as always. If the drive is not a fast drive then it could be the bandwidth limiter, not the network.

With an areca 1261ml and upgraded to a 2gb stick of ram in the controller and 1.5TB drives it takes way more than 35 hours to rebuild a single disk. I cant remember offhand but its like 3-4 days on mine and rebuilding 2 disks in a raid 6 takes like over a week. It took me about 1 week to create the initial raidset and about 1 week to create the initial volume when I first built my array.

Yes, the drive may be the reason you get a certain speed. However, in my eyes a slow drive is just a slow drive because it's speed is not being artificially lowered due to some other component. To me, a bottleneck means something that is restricting the speed that is possible. The network can be a bottleneck. A PCI SATA card with a bunch of disks attached means the PCI buss is a bottleneck.





Ouch! I didn't realize the times were nearly that long. unRAID will rebuild disks or create the parity typically in <12 hours with up to 20 disks assuming you place all those disks on good controllers which are plugged into good busses. Typically, a Supermicro 8-port SATA to PCIe x4 buss controller is used these days.

Peter

RAID rebuild times of >2 days would scare me. At that point a UPS is a must have as well, otherwise you risk total data loss.

Not a big fan of RAID, software or full hardware.

Not all RAID arrays take that long. My 8x2TB RAID6 array took 6 hours to initialize on an Areca ARC-1680ix-24.

For those who build massive RAID arrays, do you test the rebuild by purposely yanking a disk out (or introducing errors) ?

I haven't had a failure yet, actually still not really using it much since I am waiting for my Ceton card to get all the pieces working correctly.

RAID can be used as a backup solution, but it's a bad one. It's not designed for backups. What if you accidentally delete on those movie files? What if a virus takes out your data? What if your friend spills beer on the computer during a party? RAID isn't going to help you in any of those situations.

My unRAID array that contains fourteen 1.5TB data discs and one 2TB parity drive takes less than eight hours to rebuild the data to a single drive. I recently upgraded from a Supermicro AOC-SAT2-MV8 8-port PCI SATA controller to an AOC-SASLP-MV8 8-port PCI-E controller and the throughput increased considerably. The old controller was a huge bottleneck to my system due to the PCI bus interface.

^ I'm surprised they make PCI SATA controller cards. PCI bandwidth maxes out at 127MB/s (and is shared among all PCI devices). PCI-Express gives 250MB/s per lane (so x1 is 250, x4 is 1024, etc.).

Since SATA v.1 is 150MB/s and SATA v2 is 300MB/s, any RAID array involving striping is quickly going to max out the PCI bus.

It's a PCI-X card, which just happens to also work in a PCI slot.

Peter

Well, nevermind then. Although sticking it in a PCI slot still limits it to 127MB/s.

The Areca ARC-1680ix-24 is fast, that's for sure. Of course that's to be expected considering the $1000+ price tag.

Good enterprise grade controllers and hard drives are fast and reliable. They're also expensive.

Unraid shines because it's inexpensive, simple, flexible, plenty fast for media, and economical.

Every data storage method has weaknesses. What one should do is look at the application before buying the solution. Failure to do so doesn't mean you won't get something to work, but you can easily spend way more than you should.

I would never buy a enterprise raid system for home, nor would I put together and unraid system for an enterprise. They are different solutions to different problems.

why is everyone building dedicated pc's for backing up their media? wouldnt a STARTECH S354UFER and 4 drives attached to the local htpc be plenty?

Yes, if you don't mind the noise, heat, and lack of expandability.

No, I've got 6 drives in my server already.

Peter

Wow, you must be using really slow disks or the initialization is set to 20%. To initialize my 10 disk RAID 6 array of Hitachi 2TB drives took less than 2 days and a rebuilt (to test) was less than 2 days as well.

I rebuilt and reinitialized parity on a 6 HDD RAID 5 array earlier today in just a few hours on a PCI-x 133mhz slot w/ a bad battery and it only took a few hours, they're velociraptor drives so that does help, and I currently have an 8 HDD RAID 5 array rebuilding on a PCI-e 8x slot for the past 2 hr that's at 50% but it's full of slow 7200.8 drives (they're not slow though working together in the array). I did not however have an actual failed drive in either one but some cable/backplane issues and events that lead to of instead of just re-enable failed array (what normally happens if you drop and drive then put it back in) which only takes seconds plus the time time to re-initialize parity (drive can still be used during this time but write speeds will only be like a single drive) to the arrays needing to be rebuilt. BTW my RAID controllers are all LSI based.

Just to make sure this is clear, you just lost all the data on your RAID5 array due to some type of cable failure or poor connection?

Peter

The term RAID basically refers to constructing a virtual storage space from multiple disks with protection against some level of drive failure. The original meaning was reliable array of inexpensive disks, although enterprise vendors don't believe in inexpensive anything, so it's gotten redefined to redundant array of independent disks. You need more than one disk or it's not an array. You need some kind of redundancy, or it's not reliable. (As you increase the number of disks, you increase the probability that one will fail.) Beyond that, it's up for grabs what kind of parity or anything else it uses. unRAID sure sounds like a kind of RAID to me, although I haven't read the documentation. The real question would be whether it can continue in operation when one drive has failed.

The real purpose of RAID is to let you continue operation even if an error occurs (drive failure or just failure of a block). RAID is *NOT* a backup scheme. If you have your drives all spinning on the same machine, there's too high a probability of software failure or even operator error taking out the whole thing. (Studies of system reliability show that today that biggest cause of downtime is actually human error. That's been our experience too.) You need at least one and preferably more than one backup in a safe location. Backup is the bedrock. Once you have that, you add RAID if you can't tolerate taking the system down to restore from the backups. Since many types of errors will take out the whole RAID, it can't always prevent you from having downtime. But drives do eventually fail (much more often the manufacturers' data suggests), and RAID should keep the system in operation while you replace a failed drive (or at least until you can get a new drive).