or Connect
AVS › AVS Forum › Video Components › Home Theater Computers › HTPC - Linux Chat › External Array Recommendations
New Posts  All Forums:Forum Nav:

External Array Recommendations

post #1 of 102
Thread Starter 
I am quickly running out of space on my 2TB RAID10 array, and there's no more internal mounting in my case. Seems like the next logical step is an external array.

Now; my setup is first class, and I want to keep all parts that way. Ideally the array would carry like 6-10 drives, be 19" wide, and have a black high-grade aluminum look. My case is the Silverstone GD02-MT.

Needless to say it should be SATA not USB or firewire, and ideally it should externally expose the drives for hot-swap. I should think there are some great server solutions out there.

Silverstone has a pretty good one, but it's only for 5 drives and doesn't have all I want. Says it does all the RAIDs, but I presume that's in hardware. I need to RAID10 with the MD system in software.

Any suggestions?
post #2 of 102
Quote:
Originally Posted by quantumstate View Post

I am quickly running out of space on my 2TB RAID10 array, and there's no more internal mounting in my case. Seems like the next logical step is an external array.

Now; my setup is first class, and I want to keep all parts that way. Ideally the array would carry like 6-10 drives, be 19" wide, and have a black high-grade aluminum look. My case is the Silverstone GD02-MT.

Needless to say it should be SATA not USB or firewire, and ideally it should externally expose the drives for hot-swap. I should think there are some great server solutions out there.

Silverstone has a pretty good one, but it's only for 5 drives and doesn't have all I want. Says it does all the RAIDs, but I presume that's in hardware. I need to RAID10 with the MD system in software.

Any suggestions?

Take a look here. http://www.scsi4me.com/sata-serial-a...enclosure.html
post #3 of 102
Thread Starter 
Oooo, I like.

After a good bit of research, the only HTPC-grade external storage is the Silverstone, and there are reports that it's problematic. I've just about decided that I'll have to go with a rackmount of some sort.

One thing that's not clear is whether you can daisy-chain e-SATA?
post #4 of 102
Thread Starter 
Whelp, here's an update on the lawsuit I was prosecuting against the company which sold me the propane system to convert my car, and its manufacturer, as some here know. Yesterday after two years of research, study, and legal work in my spare time paid off. I won my lawsuit for denial of warranty on the propane injection system for my car.

For a year I tried and tried to get the simple part which had failed in my system and was causing chronic alarms and fault-over to gasoline at every single stop sign. But they delayed, and deferred, and made countless empty promises, and hoped I'd get lost.

So when I wised up after a year I filed suit in Washington =Superior= Court asking for a refund of everything, a new engine, peace in the Middle East, and that defendants be taken out and executed. They hired a New York lawyer to harass and further obfuscate, obstruct, and delay... but I took advantage of his errors and oversights and got him banned from the case, as 'not licensed to practice law in the State of Washington'. As further punishment the judge struck all their pleadings and defenses, including their original answer which is crucial to their defense.

I then noticed them to appear at a deposition (colloquially called the 'nuclear option') so I could formally question them... which they failed to do; not too surprising since one company is in New York and the other is in The Netherlands, but nevertheless failure to appear at deposition is a serious offense to the court. So I then filed a motion asking the Court to order them to appear and it did order them to appear at two second scheduled depositions (one for each defendant), which of course they each failed to do. (for a $5,000 case? Why would they?) So I filed a Motion for Sanctions, which the Court granted and also by the way found them in Contempt of Court for failing to appear at the Ordered deposition.

Well, it wasn't quite over. I then sent each of the defendants Requests for Admissions, asking them to admit the warranty, their full responsibility, product liability, failure to honor my valid warranty, responsibility for the Kennedy assassination, etc, and lo and behold their New York attorney dared to answer and sign them. He had earlier been ordered by the Court to no longer attempt to represent them, or else, remember? So technically he was in Contempt of Court, but I had bigger fish to fry. I asked that the Court deem all my Requests for Admission, to be admitted to by defendants, in full and conclusively as provided in Washington state law, since the New York attorney was not authorized to answer them and their 30 days were up since he sent them back at the last minute as usual. (I was counting on this) The Court granted my request on July 2nd, obliterating their case and deeming that defendants had admitted to everything plus the kitchen sink.

Now there was absolutely nothing left in question in the case and I was pretty sure I had it won, so I filed a Motion for Summary Judgment. I had to wait two months for it to be heard, as the docket is backed up, so it was scheduled for yesterday. However this past Monday I got a phone call from a local attorney saying he had just been appointed to represent defendants. Bad news, as I thought they would just not show and I'd win by default. Well he came out with knives blazing; Motion to Strike my summary judgment on numerous grounds, Declaration in support of same, Motion for Continuance, Motion to Extend Time, Notice for Hearing, etc. I studied all his filings and didn't see how they'd affect anything for anything. Their case was already dead, but I filed an eviscerating Response just the same.

So yesterday we both showed up for the trial. Right off the bat the judge knocked off all his filings, citing abuse of process in their waiting until the 11th hour to do all this. The attorney was visibly shocked. Then the judge considered my Motion for Summary Judgment. She asked for my testimony and I basically read off the important points of my Motion. However I had not put in a 'magic sentence' certifying that all my statements in my Declaration were true and correct, but I requested to be put under oath and certify right then, and she allowed it. She asked for his rebuttal, and he was clearly not ready but made a valiant attempt to shoot down my case; he had expected to get his motions granted I guess, and hadn't even considered this possibility. He is a good attorney as they go, but missed alot of things I would have done for the sake of rigor. Looked like a routine hourly type work-a-day dude to me, not no top-gun ferocious litigator like I'd battled in the past. I rebutted, and the judge didn't take much time to render her decision.

She used terms like, "It's time to put this case out of its misery", and the "long-suffering Mr. {me}" in reference to my dealings with defendants, and "he is a credit to pro se's". She granted me a judgment of $15,480.00 plus costs and post-judgment interest accumulating at 12%/annum. Unfortunately I hadn't had time to research the law on treble damages, or I would have probably gotten that too. The attorney then questioned the judge about whether she has jurisdiction, the fact that the defendants are out-of-state and do not do business here, and that they hadn't sold me the system, clearly angling for grounds for appeal, but I countered that all these questions had been deemed admitted conclusively in my Requests for Admission... completely destroying any hope he had.

We both thanked the judge and she retired to her chambers. I waited around while the attorney got his stuff together, then asked whether we have anything further to talk about. He was clearly angry (I guess for losing such a case so badly to a lowly pro se [self-represented plaintiff]... little does he know who I am) and said no, that he has to confer with his clients to see whether they want a motion for reconsideration or to appeal... so I left. I was a gracious winner and didn't intend to embarrass him.

I am pretty sure they won't make further filings, given their past abuse of process and reputation with the Court, so I might get paid soon. If not, I get to learn about the Collections process in a court at this high of a level, and in The Netherlands. defendants have 30 days to appeal, and if I hear nothing I begin collections with a Demand Letter to each defendant. They have been found liable jointly and severally, meaning that I can look to either one for all of it, or both. I don't care how they divide up responsibility. I'd like to listen in on that fight though...

Now see; If they had just sent me the blasted $95 part, I would have fixed it myself. Or if they had actually followed through with one of the four times they'd promised to fix my car but failed, this all would never have happened. Think they'll learn a lesson here and start treating other warranty claims more seriously? (Naaah) A few months ago I told them it looks like 'there must be something in the water' over there in New York, with all these broken promises...

Luckily my specialty and avocation is punishing businesses who routinely rip people off who are less able to defend themselves, insurance companies especially. Think an unregulated market actually works? Check out this story:
California's Real Death Panels: Insurers Deny 21% of Claims

I now have the part on order from the Dutch manufacturer's rep in Greece. Cost me 38 Euros (~$57), but it remains to be seen whether they have their act together well enough to send it to me. No other manufacturer's rep in the world would respond to my request except Portugal, which wants 210 Euros for the same part! Pffffff...
post #5 of 102
Quote:
Originally Posted by quantumstate View Post

One thing that's not clear is whether you can daisy-chain e-SATA?

Pretty sure that you can't. Congrats on your win, btw.
post #6 of 102
Thread Starter 
Thanks.

I'm gathering the same. Seems there are certain ways external storage is connected:
- In consumer systems firewire and USB
- Enterprise, SAS, Infiniband, multichannel, and SCSI
- New way, e-SATA.

I want e-SATA as it's already built in to my mobo.

Also I find that there is often a motherboard inside the DAS box, which mediates transfers and implements RAID in software. This is the case with most RAID DAS boxen. I don't want this. I want as direct a connexion as possible with only an e-SATA to SATA port replicator inside.

So after combing 10 pages of a Clusty search, I guess I've narrowed my solution-set to these:
http://www.sansdigital.com/accustor/index.php
http://www.enhance-tech.com/products.../R8_index.html
http://www.enhance-tech.com/products...RS8_index.html
http://www.aicipc.com/Category.aspx?ref=RSC2U
http://www.iostor.com/index.asp?Page...TS&Category=37
http://www.istarusa.com/rackmount_chassis/products.aspx
http://www.twinipc.com/ProductDetail...ref=RSA-2ED2-2
http://www.twinipc.com/Category.aspx?ref=RSC2U
http://www.bsicomputer.com/new/stora...8-ms_intro.htm

Mainly collected names for 2U chassis' up to this point and I haven't filtered them yet for e-SATA nor 80+ power supply. I'd like run/fail lights, silence, removable rack ears, and hot-swap but not mandatory.
post #7 of 102
Looks like most of those have a mini-SAS host connection which would give you the same performance as eSATA, but you'd have to get a card, obviously. Seems like one of them supported firewire/eSATA, though.

I could be wrong, I just glanced through them.

Let us know what you decide to go with and why, please. I'll be doing the same with my next server build. It'll be a while, but I'm going all-out with a rackmount and 4 CPU board and ext storage so I can start experimenting with some app hosting. Dom0 will prolly be NetBSD since they just reworked their threading model from scratch.

One thing that I'll do, and you may want to consider this, also, is run OpenSolaris in a headless VM as an iSCSI target so I can use ZFS on my ext storage.
post #8 of 102
Thread Starter 
Yup. The only thing that comes even close is this, but not enough drives, and it has an individual e-SATA connector for each drive, rather than an e-SATA-to-SATA2 port multiplier. What, am I going to have to jerry-rig something? Are we doing New Science(tm)?

If I just get the enclosure, I need to make sure it has a hot-plug SATA2 backplane and ~300w PSU for 8 drives, but does hot-plug also impact the port multiplier? Who knows.

As to OS, I'm pretty happy with Debian Testing. Has some freakin' annoyances, but it's safe.

Headed for Friday Harbor this evening for a couple days. (work)
post #9 of 102
Thread Starter 
Port Multipliers.

But apparently there's a frickin' kernel patch needed to make it work.
post #10 of 102
Isn't a self-built NAS like this one a better solution?
post #11 of 102
Quote:
Originally Posted by tux99 View Post

Isn't a self-built NAS like this one a better solution?

I think he'd have to connect that with GbE, and he, understandably, wants to use his existing eSATA.

@QS: I meant just running solaris in a vm to handle the drive array so you can use ZFS. You'd still do everything else in your Debian host including accessing the iSCSI target.

Also, it might be worth the $$ to get a mini-SAS adapter. It would definitely give you more options. You can always get a single eSATA drive for OS and database backups.
post #12 of 102
If your not in hurry, you might want to consider trying to wait for USB3.0.

It should be just as fast as eSATA plus supports 4m cable lengths compared to eSATAs 1m(?) limit. Plus you can easily add more devices like you requested.

Unfortunately it's probably about a year away from having products even though there's already a kernel driver.
post #13 of 102
Quote:
Originally Posted by quantumstate View Post

Port Multipliers.

But apparently there's a frickin' kernel patch needed to make it work.

SIL port multipliers have been supported in kernel since linux kernel .18 I have been using it since .16 but back then I had to roll my own kernel with patches applied.

Just get a port multiple type pci-e card that has the sil3132 or other SIL supported chipset and run the esata to an external case with 5in3 sata backplanes.

Hotswap worked for me last week when I had a drive die in the 5in3 enclosure.

Stuff I use that works:

*Port Multiplier [http://www.addonics.com/products/hos.../ad5sapm-e.asp sata 2 port multiplier 1 to 5]
*Sata PCI-E card [http://www.siliconimage.com/products/product.aspx?id=32 PCI-e 2 channel esata card SIL3132 chipset]


my setup: http://mythtv.org/wiki/User:Blackoper
post #14 of 102
Thread Starter 
tux, NAS seems to me like a technology stacked onto a technology. It is a good thing for sure and has its place, but I'd like to avoid that. With your suggestion I was trying to think of a way to have one side of the mirror at the HTPC and the other remote, in case of fire or theft, but that doesn't seem to be possible with Linux MD RAID.

mm, again a VM & ZFS is a technology stacked onto a technology. And a mini SAS would mean my disks are different flavors, and I'd like to keep consistent.

Mac I hate USB. It's always been inferior to firewire, and still will be with 3.0. Its fundamental flaw is that it is master-slave, not peer-to-peer and daisy-chainable. It's like the difference between token-ring and ethernet; or Forrest Gump and Forrest Whittaker. I only use USB under duress.

Good to know blackoper that the addonics works; I see it everywhere. Also good to know that the kernel has it built-in now. But what I'm trying to find out is whether my mobo's e-SATA supports multipliers; I've got to think it has to, or else what is the use? Point-to-point one disk? Nah, I expect more from Asus.

No room in my case for a backplane.(above) I must reexamine the external cases I linked to above. What I need is one with a hot-swap backplane that will take SATA drives AND which I can run a SATA cable from each drive to the port-multiplier. That's the problem. I can't believe I have to jerryrig that.

I'm surprised to see that you would run a pre-rolled, 'tourist' OS like Mythbuntu? You know your stuff, so why? And what do you mean by 'Apache in secure mode'? And 'secure ftp'?
post #15 of 102
what's your motherboard type and sata chipset? I'm pretty familiar with all the ones that work with port multipliers. If you have open pci-e slots those two port SIL cards work great and have external E-sata connectors and internal sata connectors (and you can jumper which to use)

Quote:
Originally Posted by quantumstate View Post

I'm surprised to see that you would run a pre-rolled, 'tourist' OS like Mythbuntu? You know your stuff, so why? And what do you mean by 'Apache in secure mode'? And 'secure ftp'?

Ease of use. Why install everything from scratch if you don't have to. Mythbuntu is pretty nice especially for it's diskless frontend booting features. I can combine the built in diskless server and my dd-wrt router for completely painless diskless frontends. I had the new zotac ion board fully working within minutes of turning it on. It's was almost plug n play except for adding shares to boxee and xbmc (and I'm sure I could make that automatic as well with some config file editing in both programs.)

secure ftp is: encrypted ftp server instead of the standard ftp that send usernames/passwords/files in clear text

secure mode apache is just https with everything passworded

I'm probably going to be moving over to linuxmce when the next version comes out that includes myth .22 I finally have all my home automation stuff ready to go. If linuxmce doesn't work well, though I'll be going the virtualization route and running multiple os's like mythbuntu, linuxmce, etc
post #16 of 102
Thread Starter 
Asus P5N7A-VM with external e-SATA port. Their support requires the board serial# just to ask a frickin' question, and that has delayed my asking it 5 days so far, just as they'd hoped.

You seem to invest alot less effort in your systems than I do.


And, yet another of my web hosters has collapsed. That makes 5 of my web hosting companies failed in 3 years. I've had no email for three days this time, before I figured it out. (been busy) I am sick and tired of paying them, and then having them disappear with my (admittedly measly) money. I am setting up my own hosting and email, in a Debian VirtualBox VM (for security) alongside my qb1ttorrent client. Don't know how many days it'll take to figure it out, but it is frickin' time.
post #17 of 102
Quote:
Originally Posted by quantumstate View Post

No room in my case for a backplane.(above) I must reexamine the external cases I linked to above. What I need is one with a hot-swap backplane that will take SATA drives AND which I can run a SATA cable from each drive to the port-multiplier. That's the problem. I can't believe I have to jerryrig that.

Wouldn't the Addonics storage rack (http://www.addonics.com/products/rai...k_overview.asp) do what you want?
post #18 of 102
Thread Starter 
Hadn't seen that, thanks. But this is an upscale home theater setup, and looking for something better than utilitarian... at least on the face. I guess I can paint the rest of it black if need be.
post #19 of 102
Where is the server/storage located at? Personally I have a problem with drive whine/fan loudness so I have them in a storage room that shared a wall with my home theater area. If you had another room to put the storage you could then probably save a lot of money by not worrying about the looks as much, plus you wouldn't hear the system
post #20 of 102
Quote:
Originally Posted by k_ross View Post

Wouldn't the Addonics storage rack (http://www.addonics.com/products/rai...k_overview.asp) do what you want?

That looks awesome and quite versatile. Thanks, k_ross!
post #21 of 102
Thread Starter 
I put special care into making a completely silent system. I've recited my setup here several times, but nobody listens. My system is absolutely silent, at least to my old ears, and it belongs in the living room not least because the case display shows comprehensive system status.

Still thinking about remoting half the array. Will investigate further when I get time.
post #22 of 102
I guess you could replace the fans in those storage racks with quieter ones, or put a speed controller on the existing fans.
post #23 of 102
This is one amazing storage server (not really suited for the living room though):

http://blog.backblaze.com/2009/09/01...cloud-storage/

67 terabyte 4U server for $7,867

Their choice of JFS for it is interesting, it confirms to me that JFS is the best choice for multi-TB filesystems.
post #24 of 102
Thread Starter 
Nice writeup on the Backblaze storage server, thanks tux.
post #25 of 102
I'm waiting patiently for the btrfs file system to be finished. Once it is, it should become the defacto standard for raid arrays.

About the backblaze storage server, I saw that earlier this week. Those 5 port backplanes are specially made by a chinese company and not available unless you order large amounts from what others who did the research were saying. Item: Chyang Fun Industry (CFI Group) CFI-B53PM 5 Port Backplane (SiI3726)

take that back here is an email about the backplanes:
Thanks your E-mail

CFI-B53PM backplane the price / quantity as follow:

50pcs US43

40pcs US43.5

30pcs US44

20pcs US44.5

10pcs US45

1pc US46

EMS freight depone the weight, so please let me know how many quantity you want, then I can evaluate the freight let you know, may 4~5 days arrive.
post #26 of 102
Thread Starter 
A high-quality (Debian) MD RAID FAQ:
http://git.debian.org/?p=pkg-mdadm/m...an/FAQ;hb=HEAD
post #27 of 102
Thread Starter 
OK, I am told by someone who seems to know (on the linux-raid listserv) that the reason my array is so darned busy when commercial-flagging is that the database is on the same array as the target video. (tho a different part)

He also says it is a fluke that I can boot to a RAID10 array, and that one drive would boot but the other won't, although I am skeptical about this because I've pulled the plug on each drive and I do get failover. But no one on the list contradicted him...

Nevertheless it seems that for performance (and ostensible boot reasons) I need a dedicated boot drive (probably a 2.5" WD Green), and the array handling /home. So my next decision was how to configure the array, and I've decided that it has to be a NAS with one side of the mirror in the HTPC and the other side remote in the garage. This way, if I ever have a theft or fire, I'll still have my library of videos and other backups. The garage unit will be down low somewhere, in case of fire everywhere) Apparently performance won't be a concern, as it will be gigabit ethernet, and the fastest the drive controllers can go is 3Gb/s; of course actual data acquisition from the drives is a small fraction of both. (far slower) I am told I can favor use of the local drive with the write-mostly option, so will investigate that.

Unfortunately with this choice I won't be able to get /home if a local array drive fails (network must be up before /home is mounted), but my goal is not 99.9999% uptime... it is data protection and performance.

So now I look for a remote case. Ideally it should be waterproof and dustproof, but I doubt I'll find that.
post #28 of 102
Waterproof and dustproof means airtight, which means you'll have a challenge keeping the components inside of it cool.
post #29 of 102
Thread Starter 
The Silverstone case has a large external passive heatsink, but it has other drawbacks. But this is what I have in mind if by some miracle it exists...
post #30 of 102
Thread Starter 
istarusa tells me that you must have a processor board inside the enclosure, as it is not possible to have a direct SATA to Gbethernet bridge; more complex than a SATA port-multiplier. This makes sense. However their fastest NAS is 100Mb, so they're out.

Oh, and aicipc and twinipc are the same company. twinipc seems to be the manufacturer.

Look at this little honey:

That's about what I need for the garage (no vents on top), and has 2x GbE (for network fail-over, load-balancing/bandwidth aggregation and multi- IP settings), 2x eSATA ports, and it'll take up to 4 IP surveillance cameras. About $750, but the processor is only 1.6GHz. There's a consumer-grade version (TS-419P, thermally-controlled fan, ~$590), but it's only 1.2GHz. Other companies offer enterprise NAS of 2.4GHz and 3.0GHz. But it is sure clear that qnap is competent.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: HTPC - Linux Chat
AVS › AVS Forum › Video Components › Home Theater Computers › HTPC - Linux Chat › External Array Recommendations