Planning to rebuild my 20TB WHS Flexraid server: Information requested. - Page 2

Here is a research paper Microsoft wrote on the issue of disk media errors before we had 1Tb drives
https://research.microsoft.com/apps/pubs/default.aspx?id=64599

Here is more recent paper from Dr Steve Hetzler, an IBM research scientist
http://drhetzler.com/smorgastor/2012/04/02/series-ssd-1-hard-errors-and-reliability/

For just ports, which is all you need for software solutions like flexraid and ZFS, a cheap LSI 2008 sas card = 8 Sata3 ports in a x8 slot.
It can work in open-ended shorter slots as well. You can turn closed-ended slots into open-ended slots with an exacto knife and a steady hand.

Ibm m1015 is currently the hot deal, $100 or less for card+cables.

Beware of any x1 sata 2 or 4 port card, all of them are garbage for various reasons, usually the chipset.

I second that. Definitely the best way to go.

Fyi, I just picked up one of these for my WHS 2011 machine. I got the drivers from here. It's working just fine, although I won't be using the RAID features.

Monoprice has a few other models too. Gotta love Monoprice's prices!

Perhaps

- Supermicro AOC-SAS2LP-MV8 8-port SAS/SATA 6.0Gb/s PCI Express 2.0 x8 card (Marvell 88SE9480 chip), ~$100

is the most straightforward, cheap pure HBA (i.e. non-RAID) card. I have been using it in several systems and it's pretty good. Two of this card + onboard 8 SATA = 24 SATA ports in NORCO RPC-4224 case.

As others mentioned, LSISAS2008 chip-based cards include:

- IBM ServeRAID M1015 8-port SAS/SATA 6.0Gb/s RAID 0,1, 10 and JBOD PCI Express 2.0 x8 card, ~$80 at ebay
- LSI MegaRAID SAS 9240-8i (= M1015 + RAID 5, 50), > $200
- LSI SAS 9211-8i, >$200

The last card is a pure HBA (hence no "MegaRAID") and is recommended for your purpose. IBM M1015 can be turned into 9211-8i by flashing the firmware (OK, I have no hands-on experience with this card ).
Edited by renethx - 11/9/12 at 3:35pm

I definitely want to go the ESXi route. I've already got a ESXi server setup with WHS, Win 7, and a few other things. I'm all set with the server hardware. Supermicro board with passthrough, Xeon, and ECC ram. I want to keep it in one box so I can get the speed advantage. If I separate the systems I'd have to deal with the network limitations. So for my plan I really do need the HBA. I've got some boxes I could use to setup a ZFS server to play with. I really don't have the disks though. I've tended to pass my old disks down to computers for my kids and other family members. At this point I've only got one non server in the house with a spinner so I can't upgrade to SSD for disks. My plan was to upgrade my 1 TB drives in my Unraid server to 2 TB and use the old disks. I just haven't got around to it yet. I seem to get distracted and just build another computer with my available funds. Maybe this will give me the kick in the butt I need to finally do it. I do plan on using Napp-it for the setup since I like a nice GUI.

I don't advocate ZFS for everyone. I think it's the best file system out there. It's a whole lot more work to get setup, and it certainly has some more complicated hardware requirements. Honestly on this forum I usually recommend Unraid, it's what I'm using now. I can also see the advantages of Flex Raid or Snap Raid. They all accomplish similar goals, namely providing some degree of protection for our media files. This gets the job done for the majority of use cases here. Our focus here is on HTPC's, servers are a useful part of the eco system not the focus. Usually I leave ZFS for other forums. As for other posters, I can't program like them so this is my hobby. I like building things and making them work. I can definitely see why someone would like a simpler system that just works and probably provides 95% of the functionality. I'm not trying to sell ZFS to everyone. I just saw ZFS brought up and thought I'd share what I would like to do if I could afford it.

Not so fast. A vdev can be a single drive or a set of drives with some sort of RAID-X underneath it. If you don't care about redundancy, you can just add each drive as a single vdev each and then stripe across all of them. That will get you full usage of your mixed size drives.

This is exactly what I do.

VMWare ESXi runs all my VMs, and I have a separate test machine that runs NAS4Free (BSD-based ZFS) and provides iSCSI storage for all my VMs. I am satisfied with the performace over 1GB NICs with iSCSI that I am building a single more powerful server to consolidate both machines into a virtualized utopia. The internal transfer rates between VMs over the paravirtualized NICs will run at 10GB/s so ethernet will never be the bottleneck in iSCSI. I also have a 3rd box that's been around a while that runs Server 2008 with Starwind iSCSI (free) to provide storage for a VM to test how FlexRAID performs with iSCSI. It performs admirably, though I don't trust FlexRAID to protect an ever-changing iSCSI VHD.

As long as they work stably, there is no difference, of course.

If you build a new system from scratch, I recommend a mb with 8 SATA ports (can be had < $100) + zero, one or two of SAS2LP, with 8 / 16 / 24 SATA ports in total.

Otherwise, use whatever you already have and save money.

They don't, thats the problem.

LSI has chipsets in literally every major SAN/NAS vendor out there worth buying, and get rebranded by everyone in their own servers, all for good reasons.
Personally I won't touch anything else with my data except motherboard chipset ports. (and not those "extra 2 raid ports" that cost OEMs 50 cents so they can have another feature bullet point)

Silent data problems don't show up until its painful.


Cheap things have their place, just know their limitations.
I have lots of bargain bin screwdriver and ratchet sets freighted over from china, they are still useful for casual stuff. But if I'm rebuilding an engine or whatever, out come the real tools with actual tolerances.

Data (scientific / quantitative) that back up your claim?

Exactly. I see very little relevance of "silent data corruption" problem in a home theater environment. We are not treating critical data, say, that could jeopardize our lives or a company, but just media for our entertainment. Movie can be easily retrieved from the original disc. TV programs? well, they are already a lot corrupted in my eyes, let alone downloaded media.

Personally I never listen to these server professionals, in particular in the HTPC area (somehow I always feel they lack a good balance).
Edited by renethx - 11/11/12 at 5:06am

The cheap four port cards work well until you've got to access more than one drive at a time like during a rebuild. Then you'll be seriously limited. I personally have one of these cards, and one of the Supermicro cards in my unraid server. I'd like to replace the Monoprice card, but it's functional. It's takes a big hit on my parity check, rebuild speeds though.

On the subject of the LSI cards. As previously stated they are enterprise grade. These other cards would probably work fine for your use case. The LSI cards will work for any use case. This would come in handy if you get influenced by any of the crazy talk that we started on your various threads. They also offer more bandwidth than the other options. If you wanted to be prepared for any option the LSI cards are the way to go.

Here is a good write up from the Unraid forums. It also includes the files and info on flashing to IT mode. This makes them into plain HBA's instead of raid cards.
http://lime-technology.com/forum/index.php?topic=12767.0

If your happy with what you've got there is no real need to upgrade. The easiest upgrade solution is the Supermicro card recommended by renethx. If you just stick with WHS it will work well. If you ever think you'll experiment with any other OS's, or just want a better card for possibly less money. Along with possibly of more work in the initial setup. I'd go with the LSI card. If I was building today it's what I'd go with.

Leave deep error recovery on the drive enabled (its the default) and you not have to see any of it, the manufacturers put it in for this reason.

Enabling TLER (Timed Limited Error Recovery) disables this protection, once you do that, another error recovery mechanism is needed.

The key point is, you throw away redundancy, which is kind of a big deal. If you have a 1:1 backup of your data then sure, eschew redundancy... but for most here, this is the big reason to go with these solutions in the first place. Like I said before, it is not about the amount of data you have that determines the solution you employ, but about what your risk tolerances are in conjunction with your usage requirements that determine what is a good fit. Going ZFS for the sake of going ZFS because someone said so or because you are lured by the whole "built for enterprise" line is not a good way to go about making decisions. ZFS is good for some, but not for all.

No offence but it sounds like some of you like playing with fire. There is nothing more frustrating than dealing with disk problems because a lot of it is out your control. You don't have to use ZFS at all... But if you value your time and sanity its a good option to explore, especially when you are dealing with consumer grade hardware and lots of data. It doesn't matter if is only movie files. It takes hours upon hours of your precious time to re-rip or re-acquire.

If you have other people in your household that depend on system uptime then I can only highly suggest you do the extra little things to make your systems more reliable.

+1

Video is at most 25% on my box most of that is family videos from camcorders and conversions from 16mm film, it is a backup device mostly, if you have kids with laptops, I think you know where I am headed.

I would be curious though about the number of people who build servers exclusively for ripped movie storage. Poll maybe?

The customer is satisfied with his cheapo SATA card. He is not frustrated with disk problems at all. Then why step in further? A "Hey, your system with such a cheap SATA card must have disk problems, you should replace it with a professional-grade card"-type argument sounds professionals' arrogance to me.

He is going to buy a new card and recommending a better card should be welcome, of course.
Edited by renethx - 11/12/12 at 1:55am

AOC-SAS2LP-MV8 supports both SATA2 and SATA3 HDDs with no problem.

Do you know if the AOC-SAS2LP-MV8 allows programs to read SMART data?
I've had a couple of the cheap cards in the past and I have never been able to access the SMART in programs like HD Tune. Not sure if it is a driver issue or just the card (Disks are setup as JBOD)

The proponents of ZFS forget to mention limitations such as not being able to dynamically expand/reduce pools, the almost requirement to have a fast ssd as a cache drive, complexity in managing it etc. It is no doubt an industrial strength file system, and it is probably out of place for many home uses unless the user is advanced (which many here are).

 

I also would base the decision not on the size of data (2TB or 200TB is immaterial) but on how static it is. If its mostly media which is write once, then a snapshot solution is actually better as it doesn't stripe data and is much simpler in theory and in practice, as well as being infinitely expandable. For me, simplicity is paramount - both in the technical implementation as well as the management interface.

 

If there is ever an implementation of ZFS that is easy for the end user (by which I mean no cmd line needed, ever) and which is more flexible, then it might be an alternative. Storage Spaces is dead in the water and btrfs will only be in Linux which rules it out. Meanwhile, I can take 10 2TB drives and store their parity on 2 more drives this giving me reasonable peace of mind as well as disaster recovery - I can simply pull out a drive and use it, even if there's bit rot only a few affected files are damaged. This is why I hate striping solutions.