or Connect
AVS › AVS Forum › Video Components › Home Theater Computers › Mfusick's How to build an affordable 30TB Flexraid media server: Information Requested.!
New Posts  All Forums:Forum Nav:

Mfusick's How to build an affordable 30TB Flexraid media server: Information Requested.! - Page 70

post #2071 of 3342
Quote:
Originally Posted by Mfusick View Post

He offered it to me. I'm considering it.
If you want to go T, I suggest you just go hardware because the speed hit is so high.
post #2072 of 3342
Quote:
Originally Posted by spectrumbx View Post

Take the plunge already. biggrin.gif
It is well tested. It has been stable since earlier this year, but we spent more time fine tuning it.
With enough RAM, you will be pressed to notice any speed hit over RAID-F.
Transparent RAID (tRAID) is not necessarily better than RAID over File System (RAID-F). It is newer - yes. It covers more deployment scenarios - yes. Got some super cool real-time RAID features - yes. However, RAID-F's Snapshot feature remains king for static data. You just can't beat the flexibility. So, going to tRAID from RAID-F is not an upgrade but rather a migration. Both are relevant and address the problem a bit differently - and that's all there is to it.
So, if RAID-F works best for you as it should for many users on this forum, then I am more than pleased. smile.gif

It is certainly hard to shake off that "newer is better" thought.
I don't think you tuned it well enough in your setup. Everybody else is getting 50MB+ on TCQ and as fast as the source disk when OS Caching is on (so, 500MB+ possible).

The power going out with OS Caching should not be an issue. It will result in nothing more than an interrupted buffered transfer.

@Mfusick
Back to you, no question that Snapshot RAID might be the best choice for your particular needs, but I need a guinea pig real customer system for a series of video tutorials on tRAID. tongue.gif
It is rather easy to switch between tRAID and RAID-F. So, if tRAID proves less ideal, you can revert back to RAID-F. smile.gif
Power going out would be an issue if you are saving fresh data and not just copying from one to the other.
post #2073 of 3342
Hey Mfusick, kindly do a favor for me. Look at some SMART data of some of your Seagate drives. Let me know what you see in the "Raw Read Error Rate", "Reallocated Sector Count" and "Current Pending Sector Count" fields.
post #2074 of 3342
Quote:
Originally Posted by Dark_Slayer View Post

Hey brahim/spectrumbx,

I know you have a lot of pretty well doc'd info out there about tRAID, but can you do me a favor and answer my questions so I can be a lazy-good-for-nothing forum leech? smile.gif
  1. You can continually add disks to your tRAID pool, correct? (Not just replace for larger - as is the case with ZFS)
  2. When you mention the array is self-healing, what steps will it take during background scrubbing?
  3. What does tRAID do when my read error rate gets too high on a particular disk? Do you have some determination of a reasonable drive health to continue the array? Are the s.m.a.r.t. reports still just the raw data?
  4. For live data reconstruction, I need multiple parity discs correct? e.g. I have RAID-F on 1x4TB parity supporting 2x4TB and 4x3TB data disks. For live reconstruction, I'd need to add another 4TB disk correct?

Some other curiosities I have are pulling the drives. . . say I have my array with a tolerance of 2 disks and I pull one. How do I know what data is on that disk? Also, I can't really put the disk back in the array if it's been read by any OS correct? At least, it isn't safe to do so. If I had some desire to take a disk out of my array (with a tolerance of more than 1 disk) and copy it's info over to a different computer in some other location (outside my LAN - which would be the only logical reason for doing this), then I could wipe it and add it back to the array for restoration without screwing anything up?

Also, how long does live data reconstruction take if I lose one of the 3 or 4 TB drives?

Finally, does the initial parity calculation render my data unusable until it's complete?
1. Yep. You can both replace disks in the array as well as continually expand the array with new disks (unlike ZFS). Expansion can be done while the array is online or offline.

2. tRAID is not self healing. That will be a feature of NZFS, which we will not discuss for now. However, it does have automatic hot-spare rebuild.

3. Read errors are transparent from an array usage perspective as data is simply reconstructed from parity. Disk health monitoring is done separately outside of the core RAID engine. In fact, you can do SMART monitoring without configuring any array as it is an independent feature. SMART will alert you via email and/or SMS.

4. Live data reconstruction works with single or multi-parity. So, no additional disk. If your question was on data restoration, then yes, you will need a replacement disk. Conversely, you can choose to restore to the PPU and essentially un-RAID. That is, you can restore the data into the parity disk transforming the parity disk into a data disk. This can be a great option compared to running in degraded RAID mode for a long period (say, you're broke or it takes a long time to acquire a replacement disk).

5. There are various way to identify disks including generating led activities. You can also access each tRAID disk outside of the pool to explore its file content.

6. Any disk taken out of the array, read/written elsewhere, can be added back. However, you will need to run a Verify & Sync operation to rectify any parity issue.

7. Taking a disk out, copying its content somewhere else, wiping it, and then restoring it sounds like a valid way to test and play with the recovery features. No issue with that.

8. Recovering a 4TB disk could easily take 12 hours (YMMV) on a standard system, and it can be done while the array is online or offline.
Quote:
Originally Posted by DotJun View Post

If you want to go T, I suggest you just go hardware because the speed hit is so high.
Once again, you will be pressed to tell the difference between a properly tuned tRAID system and a RAID-F one.
Most performance options are off on a fresh install of tRAID as to provide a safe introduction to the system: http://wiki.flexraid.com/2013/06/27/performance-tuning-in-transparent-raid/

It is really best to be safe than sorry. I recommend that people fully test their systems for stability first before enabling the performance options.
People tend to assume their systems are stable simply because they can boot into Windows and run Prime.

Unless you are on the final release and followed the performance tuning threads, you seriously need to shake off that impression of low speed. That is unless you find writes of 50MB+ with no caching and hundreds+ with caching low. Then yeah, it is not designed to compete with striped RAIDs. smile.gif
Remember that there is no overhead on read performance. Large storage arrays tend to be "write once - read often".
post #2075 of 3342
Quote:
Originally Posted by spectrumbx View Post

Live data reconstruction works with single or multi-parity. So, no additional disk. If your question was on data restoration, then yes, you will need a replacement disk. Conversely, you can choose to restore to the PPU and essentially un-RAID. That is, you can restore the data into the parity disk transforming the parity disk into a data disk. This can be a great option compared to running in degraded RAID mode for a long period (say, you're broke or it takes a long time to acquire a replacement disk).

Thanks for all the info

Forgive me for being thick, but what is live data reconstruction if not a self-healing array? Forgetting the healing part, what is live data reconstruction in terms of files on a tRAID array?

Restoring while the array is online is a nice feature. . .

What about initializing the array? That still takes 10+ hrs during which all the data is inaccessible, correct?
post #2076 of 3342
Quote:
Originally Posted by Dark_Slayer View Post

...
Forgive me for being thick, but what is live data reconstruction if not a self-healing array? Forgetting the healing part, what is live data reconstruction in terms of files on a tRAID array?

...
Read errors and failed disks are reconstructed live (in real-time) such that you experience no downtime (you always get access to your data).
This is in contrast to RAID-F where the failed data must be recovered in order to be accessed.
Quote:
...
What about initializing the array? That still takes 10+ hrs during which all the data is inaccessible, correct?
You have the choice of initializing in the background. Essentially, you can publish the array without computing parity upfront, get normal access to the data, and run the Verify & Sync task in the background.
post #2077 of 3342
Quote:
Originally Posted by spectrumbx View Post

1. Yep. You can both replace disks in the array as well as continually expand the array with new disks (unlike ZFS). Expansion can be done while the array is online or offline.

2. tRAID is not self healing. That will be a feature of NZFS, which we will not discuss for now. However, it does have automatic hot-spare rebuild.

3. Read errors are transparent from an array usage perspective as data is simply reconstructed from parity. Disk health monitoring is done separately outside of the core RAID engine. In fact, you can do SMART monitoring without configuring any array as it is an independent feature. SMART will alert you via email and/or SMS.

4. Live data reconstruction works with single or multi-parity. So, no additional disk. If your question was on data restoration, then yes, you will need a replacement disk. Conversely, you can choose to restore to the PPU and essentially un-RAID. That is, you can restore the data into the parity disk transforming the parity disk into a data disk. This can be a great option compared to running in degraded RAID mode for a long period (say, you're broke or it takes a long time to acquire a replacement disk).

5. There are various way to identify disks including generating led activities. You can also access each tRAID disk outside of the pool to explore its file content.

6. Any disk taken out of the array, read/written elsewhere, can be added back. However, you will need to run a Verify & Sync operation to rectify any parity issue.

7. Taking a disk out, copying its content somewhere else, wiping it, and then restoring it sounds like a valid way to test and play with the recovery features. No issue with that.

8. Recovering a 4TB disk could easily take 12 hours (YMMV) on a standard system, and it can be done while the array is online or offline.
Once again, you will be pressed to tell the difference between a properly tuned tRAID system and a RAID-F one.
Most performance options are off on a fresh install of tRAID as to provide a safe introduction to the system: http://wiki.flexraid.com/2013/06/27/performance-tuning-in-transparent-raid/

It is really best to be safe than sorry. I recommend that people fully test their systems for stability first before enabling the performance options.
People tend to assume their systems are stable simply because they can boot into Windows and run Prime.

Unless you are on the final release and followed the performance tuning threads, you seriously need to shake off that impression of low speed. That is unless you find writes of 50MB+ with no caching and hundreds+ with caching low. Then yeah, it is not designed to compete with striped RAIDs. smile.gif
Remember that there is no overhead on read performance. Large storage arrays tend to be "write once - read often".
Are you saying T raid is as fast as a dedicated hardware raid card? My hardware array goes faster as I add more discs.
post #2078 of 3342
Quote:
Originally Posted by DotJun View Post

Are you saying T raid is as fast as a dedicated hardware raid card? My hardware array goes faster as I add more discs.

No, he specifically said that would not be the case. Hardware Raid usually uses Raid 5 or Raid 6, which uses striping, and striping can increase speed because one file is spread over many discs, so that you can read/write from several discs in parallel.
post #2079 of 3342
Quote:
Originally Posted by Nevcairiel View Post

No, he specifically said that would not be the case. Hardware Raid usually uses Raid 5 or Raid 6, which uses striping, and striping can increase speed because one file is spread over many discs, so that you can read/write from several discs in parallel.
I thought T raid also striped? Now I'm confused haha I thought T raid was your typical software raid.
post #2080 of 3342
No, its not typical ;-). It still leaves your files as they are on the disc, so you can access them after pulling the disc out of the system. That makes striping impossible.
post #2081 of 3342
Quote:
Originally Posted by Nevcairiel View Post

No, its not typical ;-). It still leaves your files as they are on the disc, so you can access them after pulling the disc out of the system. That makes striping impossible.
So it's snapshot but in real time?
post #2082 of 3342
Thats a over-simplification, since it works quite a bit differently, but for the sake of argument lets just go with "yes".
post #2083 of 3342
This:
Quote:
Originally Posted by DotJun View Post

If you want to go T, I suggest you just go hardware because the speed hit is so high.
Then:
Quote:
Originally Posted by DotJun View Post

I thought T raid also striped? Now I'm confused haha I thought T raid was your typical software raid.
Basically, you have no idea what the thing is, but yet you claim know something about its performance. Humans. rolleyes.gif


Quote:
Originally Posted by Nevcairiel View Post

Thats a over-simplification, since it works quite a bit differently, but for the sake of argument lets just go with "yes".
Don't give into the temptation of oversimplification as you would be taken for your exact words. wink.gif
It only adds to the confusion and specially in light of the fact that simple minds love to take things literally.

Transparent RAID is NOT Snapshot RAID in real-time mode.
Snapshot RAID is Snapshot RAID over File System. RT RAID is Real-Time RAID over File System. Transparent RAID is Transparent Real-Time RAID.
post #2084 of 3342
T RAID is not over File System?
post #2085 of 3342
Quote:
Originally Posted by spectrumbx View Post

Read errors and failed disks are reconstructed live (in real-time) such that you experience no downtime (you always get access to your data).
......................

How can I read data on a failed disk If the data has not yet been rewritten to the disk?

David
post #2086 of 3342
Quote:
Originally Posted by DavidT99 View Post

How can I read data on a failed disk If the data has not yet been rewritten to the disk?

David

Read from parity most likely.
post #2087 of 3342
Quote:
Originally Posted by spectrumbx View Post

This:
Then:
Basically, you have no idea what the thing is, but yet you claim know something about its performance. Humans. rolleyes.gif
Don't give into the temptation of oversimplification as you would be taken for your exact words. wink.gif
It only adds to the confusion and specially in light of the fact that simple minds love to take things literally.

Transparent RAID is NOT Snapshot RAID in real-time mode.
Snapshot RAID is Snapshot RAID over File System. RT RAID is Real-Time RAID over File System. Transparent RAID is Transparent Real-Time RAID.
Yep, I think I posted afterwards that I assumed it functioned like your typical software implementation of raid 5/6, which is why I suggested to go hardware due to a speed hit. Now, if what you are saying is that T-Raid does not incur any kind of speed hit on writes then I will take back what I said about suggesting hardware raid over t raid.

Edit* adding a smiley so my post here doesn't come off in a mean spirited way 😊
post #2088 of 3342
Quote:
Originally Posted by spectrumbx View Post

This:

Don't give into the temptation of oversimplification as you would be taken for your exact words.
Could someone with the knowledge of how this software actually work, please take a moment to explain it to those of us who don't? Or point to a link that does explain? I, and I'm sure others, would be grateful.
post #2089 of 3342
Thread Starter 
Quote:
Originally Posted by A9X-308 View Post

Could someone with the knowledge of how this software actually work, please take a moment to explain it to those of us who don't? Or point to a link that does explain? I, and I'm sure others, would be grateful.

You might want to PM him as he's not a consistent visitor to this website. He's often not signed in for month or two at a time. Or best email for a specific answer. Feel free to post the response here. I am interested too but I fear he's disappeared. I'm guessing unless he has a specific reason to visit this site he's likely not monitoring threads consistently.

also,

New Build project incoming:

2 x LG Black Blu-ray Drive SATA Model UH12NS30 - OEM

1 x G.SKILL Trident X Series 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 2400 (PC3 19200) Desktop Memory Model F3-2400C10D-16GTX

1 x Intel Core i5-4670K Haswell 3.4GHz LGA 1150 84W Quad-Core Desktop Processor Intel HD Graphics BX80646I54670K

1 x Seagate Barracuda 7200.14 ST3000DM001 3TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive Bare Drive

1 x SILVERSTONE Black Aluminum / Steel Grandia Series SST-GD08B ATX Media Center / HTPC Case

1 x ASRock Z87 Extreme4 LGA 1150 Intel Z87 HDMI SATA 6Gb/s USB 3.0 ATX Intel Motherboard

1 x Rosewill CAPSTONE-450 450W Continuous @ 50°C, Intel Haswell Ready, 80 PLUS GOLD, ATX12V v2.31 & EPS12V v2.92, SLI/CrossFire ...

1 x SAMSUNG 840 Series MZ-7TD250KW 2.5" 250GB SATA III Internal Solid State Drive (SSD)

Building this for a friend this weekend. I'll try to post a few pictures and thoughts. (He is addding more hard drives on his own with media)

Also on the to-do list,

Remove the Vertex 3 from my Desktop (used for dual booting) and put into my server. RMA out my server SSD ( It's busted mad.gif ) Reinstall WHS2011 and Flexraid, and pool all my drives again. Hope I don't have a HDD fail in the process. That would suck (but I would live)

Also- upgrading my Desktop to a 256GB SSD for OS (since when did 128GB stop being enough ? )
I'll use the 128GB Vector for my dual boot now. Also RAID 0 my scratch disc with SSD cache.

Lastly,

Installing a second tuner card for OTA recording. Setting up a new cable modem (upgraded to Comcast Blast) and replacing my really crappy linksys wireless N router (120 model). Plus I need to run a new Cat6 wire and tips to my HTPC under the floor because wife ripped the tip off my CAT5e cable so might as well.

Plus I need to rake leaves, mow lawn final time before snow comes, and set up IR repeater for wife to watch/ control HTPC from another room on second display. Wife is pregnant with twins and now thinks I must do everything while she watches HTPC ... tongue.gif

Wish me luck.
post #2090 of 3342
Quote:
Originally Posted by Mfusick View Post

You might want to PM him as he's not a consistent visitor to this website.
I was going to ask 'PM whom?' then remembered this thread and this link in it. Before then I didn't realise spctrumbx was the developer of FlexRAID.

I'm going to get the HTPC done on Monday and repurpose the old workstation into a backup PC with some added storage in the form of the M1015. Basically, I'll be reloading VIsta, some general purpose programs and Snapraid and adding a couple more drives as time goes on. The server has been delayed because the HDD cages I ordered from Norco (the $9 ones reneTHX suggested) took forever to get here, and when they arrived, none of my mounting screws seem quite long enough.

HTPC is:
Asrock Z77 Extreme 3
3770 with 212 Evo cooler
W7 Ultimate
4G G Skill Ripjaws X CL9
Asus 1G HD7790
Samsung 120G 840 Evo
Seagate 1TB HDD (local temporary storage)
LG CH12L28 BR drive
Corsair CX430 v3 PSU.

I have a spare Antec 900 case I think I'll put it in as it came with a number of quiet fans and should run cooler/quieter then the Thermaltake Soprano case I have. It will be just out of the room and potentially still audible.

My new TV and projector also arrived, but both were missing mounting brackets so I can't use them until they get here.
post #2091 of 3342
Thread Starter 
You are using windows Vista (seriously ?)
post #2092 of 3342
Quote:
Originally Posted by Mfusick View Post

You are using windows Vista (seriously ?)
I used it for 5 years with zero problems and I still own the license (same machine) so I see no benefit to 7. I have VIsta, 7 and 8 machines and I see little operational difference between them.
post #2093 of 3342
Thread Starter 
Makes sense ... I just haven't seen someone choose vista in quite a while smile.gif
post #2094 of 3342
Only issue I see with Vista is that it doesn't support TRIM for SSD's. Not a big issue though.
post #2095 of 3342
Thread Starter 
Quote:
Originally Posted by wiley165 View Post

Only issue I see with Vista is that it doesn't support TRIM for SSD's. Not a big issue though.

I guess only if you had a crappy older SSD (samsung, OCZ and modern drives do have their own TRIM features built in) that you wanted to use. A normal hard drive or modern SSD would likely work. I was thinking more about drivers and such, as the compatibility is growing less for older OSes over time so at this point today my concern would be the effort and time I take to set it all up might not pay back the investment before I need to do it again for some reason.
post #2096 of 3342
Quote:
Originally Posted by Mfusick View Post

I was thinking more about drivers and such, as the compatibility is growing less for older OSes over time so at this point today my concern would be the effort and time I take to set it all up might not pay back the investment before I need to do it again for some reason.
It's an existing, running machine (recently replaced by my new workstation) that I'm keeping as a spare. No issues with drivers as I already have them all. Still don't understand the Vista hate as it's given me less trouble in 5 years than W8 in six weeks.
post #2097 of 3342
Thread Starter 
I am about to clean install my desktop and move the SSD over to my server and clean install WHS and flexraid- wish me luck eek.gif
post #2098 of 3342
Quote:
Originally Posted by Mfusick View Post


New Build project incoming:
...
1 x SAMSUNG 840 Series MZ-7TD250KW 2.5" 250GB SATA III Internal Solid State Drive (SSD)

So are you moving away from the Sandforce controllers? I have a Corsair Force GT that crapped out on me in a similar fashion as your Vertex that I RMA'd and is currently a backup. My latest SSD (Samsung 840 PRO) uses a better controller IMO and the tedious process of re-installing the OS, etc. is not something I want to do often. At the time I initially purchased the GT it was one of the better performing drives, but in hindsight it lacked the stability I need in a OS drive and not to mention it was too small for a Win8 +Steam game folder + Program files, etc. Well good luck in your install endeavor and some day I'll be following along with the WHS 2011 and flexraid server approach.
post #2099 of 3342
Thread Starter 
By default I guess I am because the market has. Newer SSDs are better and that's probably the most attractive to me. Also Sandforce has a new controller due out 3xxx that supports PCI express and should exceed sata 3 capability.

But having owned 20 Sandforce controllers and having generally good luck I can't say I'm scared of them if the price was right. I think today they still have a place in the budget segment of the market.

I only paid $50 for my server SSD drive. I bought it last Black Friday on special and basically bought it because it was bigger than the 60gb SSD I had and was only $50. I would buy again for the same circumstances.
post #2100 of 3342
Quote:
Originally Posted by Puwaha View Post

Good points all above, but I still make the personal case that FlexRAID is not the best choice for that much data. I have personally outgrown FlexRAID but its a good choice if all you are protecting is mostly in hanging data like movie files. Like mention above don't forget the intel NIC. Other brands work just fine on a windows server based OS, but I typically only use those "other" brand NICs as a maintenance/service connection.

what did you choose instead of FlexRAID? I'm looking for something Windows based with RAID and pooling
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers
AVS › AVS Forum › Video Components › Home Theater Computers › Mfusick's How to build an affordable 30TB Flexraid media server: Information Requested.!