Does anyone still prefer Hardware RAID 5 for media server with Windows as the OS? - Page 4 - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #91 of 183 Old 08-14-2013, 07:18 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 21,381
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 645
Quote:
Originally Posted by aliaskary77 View Post

the tolerance levels are important, but raid1 proponents keep ignoring capacity. i dont have unlimited funds. no single drive will give me enough capacity. a 10 disk raid 1 set, even with a tolerance of 9, is absurd. great that the theory works out for you, no way will i put that in practice. SirMasters layout gives me the best options with flexraid and my disk capacity. i was going to do 9 disks with 1 parity, but now might do 8 with 2 parity.

At some point practicality must take over.

10 -12 data drives with 2 parity drives is affordable and reasonable, and yields 25TB-40TB of storage space. You can suffer two simultaneous drive failures.

Cost = $99 for 3TB drives. 14 drives (12+2 parity) is $1400. That is 36TB of storage space.

In comparison it's another $1200 to run RAID with a mirrored set up. No thanks. The inherent value in a software solution grows as your data grows until you hit a point where your leaving consumer territory and entering into PRO IT territory. No simple media server is going to get there.

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
Sponsored Links
Advertisement
 
post #92 of 183 Old 08-14-2013, 07:28 AM
AVS Special Member
 
damelon's Avatar
 
Join Date: Sep 2007
Location: Woodbridge, VA
Posts: 1,687
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 104
Quote:
Originally Posted by EricN View Post

Hotboxing 8 drives with an inadequate PSU, a crap controller, and no backups? I'm not surprised that didn't go well. If you heard a disaster story about someone buying a huge TV and securing it to a cheap, undersized mount because their budget had no room for an appropriate one, would you conclude that wall mounting is not practical for the home? The answer is to buy a smaller TV and divert some money towards supporting it properly. The same is true of storage.
RAID is not a backup. If you follow storage advice from someone who claims RAID will suffice in lieu of backups, or if you buy products from vendors claiming their RAID software will act as a backup, you are just asking for more pain. What kind of data was it anyways? Where did it come from?

Well it was a enclosure designed specifically for that... an 8 bay hot swap JBOD enclosure with PSU and fans all built in. I could have upgraded the fans or the PSU, but it came with all of that. I also had never done anything like that before, so it was a bad assumption on my part to assume something made for that purpose wouldn't be suited for it. Please don't be over-dramatic with your silly analogies that aren't even similar. No one is attacking you so no need to give a troll response.

Of course I know RAID is not a backup. I should have known better than to use a word like that involving computers, seems like everyone takes everything so literally. I of course did not have a backup, which was the point I was making, and another reason I stated it as a question. Failover would have been a better word in that case. None of the data was un-recoverable, it was just ISO images of all of my Blu Rays. So mostly I just lost all the time it took to convert them. Still really annoying. Anyway, I edited that paragraph with the reason for why I prefer the current software raids is exactly BECAUSE i don't have a backup. I don't have to start from scratch and re-rip everything if multiple drives fail, only the data which I would lose would be on the drives that did fail. It saves me money on cost of a controller, I have less work to do if I lose some data, etc. There are way more pros than cons on why I'd prefer FlexRaid/UnRaid in my media NAS over Hardware Raid. There just really isn't a good reason I could see to use it.
damelon is online now  
post #93 of 183 Old 08-14-2013, 07:32 AM
AVS Special Member
 
Puwaha's Avatar
 
Join Date: Jan 2003
Posts: 1,122
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 30
Quote:
Originally Posted by SirMaster View Post

He said 10 disks, not 4.

No his $1000 bet was with 4 disks. Go back and read it.

My 10 disk proposal (which I didn't bet anything on, btw) was under an assumption of the typical use case of 10 disks under FlexRAID with 2 of those being parity. Most poeple emulate a RAID-5 or RAID-6 with FlexRAID.

Quote:
Then lets not overlook FlexRAID's new mode: tRAID. tRAID solves the problem you mentioned of data not being available until after restore. tRAID works exaclty like RAID-F FlexRAID that has been described (from a fault tolerance standpoint), but offers real-time parity and real-time reconstruction of a failed disk. It's transparent, you don't even know the disk is gone other than the fact that it tells you, and reading data from that disk is slower since the array is degraded. But you can keep the array online and read data from the failed disk(s) even while rebuilding onto a replacement disk or hot-spare which it also supports.

How is real time parity with tRAID any different than a standard striped real time RAID-5 or 6 then? Other than the fact that it's not striped the tolerance levels are the same.

Can you refute my 4 disk RAID-1 vs FlexRAID challenge as I posed it?
Puwaha is offline  
post #94 of 183 Old 08-14-2013, 07:39 AM
AVS Special Member
 
Puwaha's Avatar
 
Join Date: Jan 2003
Posts: 1,122
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 30
Quote:
Originally Posted by aliaskary77 View Post

the tolerance levels are important, but raid1 proponents keep ignoring capacity. i dont have unlimited funds. no single drive will give me enough capacity. a 10 disk raid 1 set, even with a tolerance of 9, is absurd. great that the theory works out for you, no way will i put that in practice. SirMasters layout gives me the best options with flexraid and my disk capacity. i was going to do 9 disks with 1 parity, but now might do 8 with 2 parity.

No one is ignoring capacity at all. It's very well known what RAID-1 will net you space wise.

There is a risk in any RAID scheme but nothing will ever beat a 1:1 ratio at the most basic level. Either you achieve the 1:1 ratio through RAID-1 to provide maximum uptime value (and pretty good read performance to boot)... Or you achieve the 1:1 ratio through a backup. Parity will never be able to beat a mirror.

If you value your uptime and sanity, don't get greedy with disk space... Besides its cheap right?
Puwaha is offline  
post #95 of 183 Old 08-14-2013, 07:41 AM
AVS Special Member
 
bryansj's Avatar
 
Join Date: Feb 2004
Location: Atlanta, GA
Posts: 6,244
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 27 Post(s)
Liked: 165
Quote:
Originally Posted by Puwaha View Post

No his $1000 bet was with 4 disks. Go back and read it.

My 10 disk proposal (which I didn't bet anything on, btw) was under an assumption of the typical use case of 10 disks under FlexRAID with 2 of those being parity. Most poeple emulate a RAID-5 or RAID-6 with FlexRAID.
How is real time parity with tRAID any different than a standard striped real time RAID-5 or 6 then? Other than the fact that it's not striped the tolerance levels are the same.

Can you refute my 4 disk RAID-1 vs FlexRAID challenge as I posed it?

If you lose 2 discs in the same RAID1 set then you've lost half your data. If you lose 2 in FlexRAID then you've lost no data.

bryansj is online now  
post #96 of 183 Old 08-14-2013, 07:43 AM
AVS Special Member
 
Puwaha's Avatar
 
Join Date: Jan 2003
Posts: 1,122
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 30
Quote:
Originally Posted by Mfusick View Post

At some point practicality must take over.

{snip}

In comparison it's another $1200 to run RAID with a mirrored set up. No thanks. The inherent value in a software solution grows as your data grows until you hit a point where your leaving consumer territory and entering into PRO IT territory. No simple media server is going to get there.

I'm flabbergasted! You are talking about 40TB of storage and don't think you are already in "pro" territory?
Puwaha is offline  
post #97 of 183 Old 08-14-2013, 08:01 AM
AVS Special Member
 
Puwaha's Avatar
 
Join Date: Jan 2003
Posts: 1,122
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 30
Quote:
Originally Posted by bryansj View Post

If you lose 2 discs in the same RAID1 set then you've lost half your data. If you lose 2 in FlexRAID then you've lost no data.

What happens if you lose 3 disks?

In RAID-1 you still have 50% of your data. In FlexRAID you have a 50% chance of losing all your data (both data disks and one parity disk).

So at a two disk failure you have a 25% chance of losing data in RAID-1. It's not perfect but tolerable. In a three disk failure scenario your odds jump up to 50% that you will lose all data with FlexRAID. You are trading slightly better front-end tolerance for much worse odds at the catastrophic end with FlexRAID.

There's no one perfect solution for everyone... But the developer of FlexRAID claiming that his system is better than RAID-1 is selling snake oil.
Puwaha is offline  
post #98 of 183 Old 08-14-2013, 08:19 AM
Member
 
SirMaster's Avatar
 
Join Date: Aug 2009
Posts: 101
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 24
Quote:
Originally Posted by Puwaha View Post

How is real time parity with tRAID any different than a standard striped real time RAID-5 or 6 then? Other than the fact that it's not striped the tolerance levels are the same.

IMO tRAID is superior to standard RAID 5/6 for these reasons:
  • More than 2 parity disk support for large arrays.
  • Cheap support for Windows (real RAID6 in Windows requires somewhat expensive RAID cards that have limited number of ports, usually 8).
  • Ability to use disks of any size without wasting any space.
  • Ability to add disks with data already on them.
  • Disks can be read by themselves outside the array (good for more catestrophic failures or if the server breaks down and you just want to access some files.)
  • Only the disk that is being read from needs to spin up (saves power and lower disk wear, most people on AVS are looking for media-storage which is write once-read many)
SirMaster is offline  
post #99 of 183 Old 08-14-2013, 08:24 AM
Advanced Member
 
spectrumbx's Avatar
 
Join Date: Jun 2007
Posts: 787
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 20
Quote:
Originally Posted by damelon View Post

...
Of course I know RAID is not a backup. I should have known better than to use a word like that involving computers, seems like everyone takes everything so literally. I of course did not have a backup, which was the point I was making, and another reason I stated it as a question. Failover would have been a better word in that case. ....
You would be amazed at the fact that very few of the backup snobs actually do any backup at all let alone proper backups.

As Mfusick pointed out, practicality is everything.
Once backing up the array becomes impractical due to time or cost, people will simply forego it.
We all know what we should do, and then there is what we actually do.

This is why minimizing the losses in case of the worst case scenario matters to most.
Quote:
Originally Posted by Puwaha View Post

No one is ignoring capacity at all. It's very well known what RAID-1 will net you space wise.

There is a risk in any RAID scheme but nothing will ever beat a 1:1 ratio at the most basic level. Either you achieve the 1:1 ratio through RAID-1 to provide maximum uptime value (and pretty good read performance to boot)... Or you achieve the 1:1 ratio through a backup. Parity will never be able to beat a mirror.

If you value your uptime and sanity, don't get greedy with disk space... Besides its cheap right?

You keep chanting 1:1 without realizing the reciprocity aspect of that equation.
Yes, you get a full copy for every bit, but each duplicated bit can only restore a specific bit.

In the case of parity, rather than a 1:1, you have a *:1 (many-to-one) or *:* (many-to-many), which is much more powerful.
In such reciprocity, each parity bit can restore any bit in a set.

RAID is not backup. Even a RAID scheme using backup techniques isn't backup.
Some of you think you are getting a good backup in that 1:1 mantra, but you are not. Your backup strategy should be separate from your RAID strategy.

A one-to-one scheme is and will remain inferior to a many-to-many scheme. wink.gif
spectrumbx is offline  
post #100 of 183 Old 08-14-2013, 08:24 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 21,381
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 645
Quote:
Originally Posted by Puwaha View Post

No one is ignoring capacity at all. It's very well known what RAID-1 will net you space wise.

There is a risk in any RAID scheme but nothing will ever beat a 1:1 ratio at the most basic level. Either you achieve the 1:1 ratio through RAID-1 to provide maximum uptime value (and pretty good read performance to boot)... Or you achieve the 1:1 ratio through a backup. Parity will never be able to beat a mirror.

If you value your uptime and sanity, don't get greedy with disk space... Besides its cheap right?

I think your ZFS background, your above average understanding of Hardware RAID, and your personal value system are very different from many on this forum. While it seems right and obvious to you, it's a lot different for the average Joe AVS-er.

You say storage is cheap (and in many ways it is) but dropping another $1000 or $2000 to do a 1:1 mirror on something like a simple media server just does not fly with many. Most around here are working with budgets half that for the entire media server build. If you wanted to build an affordable media server your not going to do that on RAID or with 1:1 mirrors. I think most like the idea of 4-8 data drives and only a single parity. The performance hit is worth the value and easy/flexibility of implementation.

Arguing this further seems silly because no one is going to run it. It's an argument of Flexraid versus 1:1: RAID that is purely in "theory"

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #101 of 183 Old 08-14-2013, 08:26 AM
AVS Special Member
 
StardogChampion's Avatar
 
Join Date: Dec 2007
Location: New Hampshire, USA
Posts: 2,995
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 31 Post(s)
Liked: 117
Quote:
Originally Posted by spectrumbx View Post

Your backup strategy should be separate from your RAID strategy.

Bingo! Give that man a prize!

Though, I would add to this even before considering any strategy, consider whether you need to keep all this stuff in the first place biggrin.gif

 

 

StardogChampion is offline  
post #102 of 183 Old 08-14-2013, 08:26 AM
AVS Special Member
 
bryansj's Avatar
 
Join Date: Feb 2004
Location: Atlanta, GA
Posts: 6,244
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 27 Post(s)
Liked: 165
Just goes to show that RAID isn't backup.

bryansj is online now  
post #103 of 183 Old 08-14-2013, 08:33 AM
Member
 
SirMaster's Avatar
 
Join Date: Aug 2009
Posts: 101
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 24
Quote:
Originally Posted by Puwaha View Post

I'm flabbergasted! You are talking about 40TB of storage and don't think you are already in "pro" territory?

That's the point. That's what changed IMO with the advent of FlexRAID. Previously, a 40TB array would have likely required "Pro" solutions, and would have been out of reach for most average consumers looking for a simple array to store all their AV media. But now with FlexRAID it really doesn't have to require "Pro" level hardware anymore. You can run a normal Windows machine with a bunch of random cheap JBOD SATA port cards and build up a huge aray 1 disk (of any size) at a time, and adding parity disks as the volume and number of disks grows to maintain whatever level of tolerance you desire.

For most of us, FlexRAID is in a class of its own in terms of flexibility.

You don't have to explain RAID-1 vs FlexRAID to me smile.gif. I understand perfectly your example with a 4 disk array and I would probably not choose or recommend FlexRAID be set up with 2 data 2 parity myself. I also understand and listed the tradeoffs with RAID-1 vs FlexRAID in larger disk counts.
SirMaster is offline  
post #104 of 183 Old 08-14-2013, 08:35 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
This whole discussion started because I made this statement:
Quote:
Originally Posted by itznfb View Post

There are still people who prefer RAID1 for media. It's the most simple solution available with the fastest rebuild times and your data is (arguably) the safest. RAID5 less so. However given money not being a factor I would still prefer RAID5/6 for personal use.

spectrum vehemently disagrees with this due to a want to represent his software as something where your data is "safer" which just doesn't hold water. For me, personally software RAID is fine for my Blu-ray collection but there is no way in hell my photography collection is leaving my RAID1 array. Technically still a "media" array. My photos sit on 2 hardware RAID1 volumes with 4 - 3TB disks each. So I have 6TB available for photo storage with 8 disks. It's costly yes but it's something that software RAID can't match unless I use the same capacity loss and the same $ investment. Also, I know that I will never be at the mercy of parity calculation which isn't reliable. Not when I'm talking about my photos anyway. I still prefer hardware RAID5 for my movies (even though I'm using FlexRAID currently for testing) just because I like hardware RAID.

Bottom line is for critical data people are still going to choose RAID1 because as I've said over and over a block copy is a safer redundancy method than any algorithm used in parity calculation. And yes I still backup my photos off site.

Again... RAID1 IS the most simple solution. Say you only need 3TB for your media collection. There is no reason not to use RAID1 as there is no thought to it whatsoever.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #105 of 183 Old 08-14-2013, 08:39 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 21,381
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 645
Quote:
Originally Posted by Puwaha View Post

I'm flabbergasted! You are talking about 40TB of storage and don't think you are already in "pro" territory?

12 drives of 3TB for $99 is still consumer IMO. I bought them a couple at a time here and there... It just started adding up.

My next "rebuild" I might make the jump over. But I'd argue I am a little more advanced and bigger than average around here. I remember you were one of the first posters in my server rebuild thread explaining to me the benefit of ZFS. I was not ready for that back then, but next time wink.gif

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #106 of 183 Old 08-14-2013, 08:41 AM
Advanced Member
 
aliaskary77's Avatar
 
Join Date: Oct 2007
Posts: 581
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 7 Post(s)
Liked: 43
yep. the stuff that i cannot lose fits into external 2.5 drives. One locally every week. One in the bank lock box every month. Will add nightly backups to the htpc server and maybe even dropbox/crashplan.

note: the post above is my opinion. as such, when reading any recommendations from me, please do you research and seek out other recommendations and make up your own mind on your next course of action. i mean, most reasonable adults should know that, but it seems this should be stated anyways.
aliaskary77 is online now  
post #107 of 183 Old 08-14-2013, 08:48 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 21,381
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 645
Quote:
Originally Posted by itznfb View Post

This whole discussion started because I made this statement:
spectrum vehemently disagrees with this due to a want to represent his software as something where your data is "safer" which just doesn't hold water. For me, personally software RAID is fine for my Blu-ray collection but there is no way in hell my photography collection is leaving my RAID1 array. Technically still a "media" array. My photos sit on 2 hardware RAID1 volumes with 4 - 3TB disks each. So I have 6TB available for photo storage with 8 disks. It's costly yes but it's something that software RAID can't match unless I use the same capacity loss and the same $ investment. Also, I know that I will never be at the mercy of parity calculation which isn't reliable. Not when I'm talking about my photos anyway. I still prefer hardware RAID5 for my movies (even though I'm using FlexRAID currently for testing) just because I like hardware RAID.

Bottom line is for critical data people are still going to choose RAID1 because as I've said over and over a block copy is a safer redundancy method than any algorithm used in parity calculation. And yes I still backup my photos off site.

Again... RAID1 IS the most simple solution. Say you only need 3TB for your media collection. There is no reason not to use RAID1 as there is no thought to it whatsoever.

Your being short sighted.

Most want more than 3TB, they want it cheap and easy too. They want to grow as they go. I'd happily sacrifice some reliability if I had to do it just to get that.

Hardware raid is almost such a PITA that I'd consider saying "F-it" and just running JBOD without any back up, rather than 1:1 and spend the $ on more drives. Your still very slanted and short sighted in your argument. You might be right in "theory" but your not right in real life for average person's viewpoint.

1:1 RAID on a media server is foolish so there is little point in arguing it's validity or performance. It's like saying to someone buying a new car " but a space shuttle can go way faster". It might be true, but hardly is it going to gather a serious consideration for obvious reasons.

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #108 of 183 Old 08-14-2013, 08:51 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by Mfusick View Post

Your being short sighted.

Most want more than 3TB, they want it cheap and easy too. They want to grow as they go. I'd happily sacrifice some reliability if I had to do it just to get that.

Hardware raid is almost such a PITA that I'd consider saying "F-it" and just running JBOD without any back up, rather than 1:1 and spend the $ on more drives. Your still very slanted and short sighted in your argument. You might be right in "theory" but your not right in real life for average person's viewpoint.

1:1 RAID on a media server is foolish so there is little point in arguing it's validity or performance. It's like saying to someone buying a new car " but a space shuttle can go way faster". It might be true, but hardly is it going to gather a serious consideration for obvious reasons.

I'm not being short sighted because what people want in terms of capacity isn't the discussion. The discussion is whether data is "safer" on a block copy or on a parity calculation. The discussion is what is safer not what has the most capacity. No one is going to argue RAID1 has a capacity benefit. But there are people here arguing that data is safer on a software parity calculation which just isn't true.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #109 of 183 Old 08-14-2013, 09:10 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 21,381
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 645
Quote:
Originally Posted by itznfb View Post

I'm not being short sighted because what people want in terms of capacity isn't the discussion. The discussion is whether data is "safer" on a block copy or on a parity calculation. The discussion is what is safer not what has the most capacity. No one is going to argue RAID1 has a capacity benefit. But there are people here arguing that data is safer on a software parity calculation which just isn't true.

Does it matter ? I don't think so. 1:1 isn't even on my radar as an option. The whole conversation is uninteresting and pointless. I am decidedly uninterested in even determining the "truth" because it does't matter.

That's my point.

All this talk about RAID and 1:1 just seems like a waste of letters and typing.

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #110 of 183 Old 08-14-2013, 09:32 AM
Advanced Member
 
spectrumbx's Avatar
 
Join Date: Jun 2007
Posts: 787
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 20
Quote:
Originally Posted by bryansj View Post

Just goes to show that RAID isn't backup.
... and that very few actually backup their RAID arrays. wink.gif

I think the theme of the day is: practicality.

So, users should stay realistic and choose the approach that best fits their reality rather than stress themselves with strict ideals.
If I am in a situation where my array is too large for practical backup, I will go with a scheme where I don't lose all my data if I lose more disks than the tolerance level. smile.gif
Quote:
Originally Posted by itznfb View Post

This whole discussion started because I made this statement:
spectrum vehemently disagrees with this due to a want to represent his software as something where your data is "safer" which just doesn't hold water. For me, personally software RAID is fine for my Blu-ray collection but there is no way in hell my photography collection is leaving my RAID1 array. Technically still a "media" array. My photos sit on 2 hardware RAID1 volumes with 4 - 3TB disks each. So I have 6TB available for photo storage with 8 disks. It's costly yes but it's something that software RAID can't match unless I use the same capacity loss and the same $ investment. Also, I know that I will never be at the mercy of parity calculation which isn't reliable. Not when I'm talking about my photos anyway. I still prefer hardware RAID5 for my movies (even though I'm using FlexRAID currently for testing) just because I like hardware RAID.

Bottom line is for critical data people are still going to choose RAID1 because as I've said over and over a block copy is a safer redundancy method than any algorithm used in parity calculation. And yes I still backup my photos off site.

Again... RAID1 IS the most simple solution. Say you only need 3TB for your media collection. There is no reason not to use RAID1 as there is no thought to it whatsoever.
I too run RAID1 for my most critical data as posted in this thread.
Why? Well, my most critical data fits snugly on a single 2TB drive. So, for RAID protection, mirroring just makes sense; and then, I backup the hell out of it. smile.gif

In your case, however, you have 8 disks with 2 being used for data and 6 for redundancy.
My whole point has been that, if given 6 disks to protect 2 disks, I would go with a parity scheme supporting 6 parity drives rather than two separate RAID 1 arrays.
Why?
1. With a 6-parity configuration, I can lose any 6 disks and suffer zero data loss.
2. With 2x 4-way mirrors, losing 4 disks from the same set will cause the whole set to be lost. Losing 6 disks will certainly result in some data loss unlike with the parity configuration.

Listen, you have great protection as is with your setup, albeit over kill.
I wouldn't worry about much past the technicalities of this discussion.

I think people are fearful of parity based RAID because of the risk of losing a drive past the tolerance level during the restoration process and thereby losing everything.
It has happened, and solutions like FlexRAID address that problem to some level.
spectrumbx is offline  
post #111 of 183 Old 08-14-2013, 10:46 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by Mfusick View Post

Does it matter ? I don't think so. 1:1 isn't even on my radar as an option. The whole conversation is uninteresting and pointless. I am decidedly uninterested in even determining the "truth" because it does't matter.

That's my point.

All this talk about RAID and 1:1 just seems like a waste of letters and typing.

Then your opinion in this thread is meaningless. Why are you posting in this thread if it's uninteresting and pointless to you? It's not uninteresting and pointless to many other people.

Quote:
Originally Posted by spectrumbx View Post

... and that very few actually backup their RAID arrays. wink.gif

I think the theme of the day is: practicality.

So, users should stay realistic and choose the approach that best fits their reality rather than stress themselves with strict ideals.
If I am in a situation where my array is too large for practical backup, I will go with a scheme where I don't lose all my data if I lose more disks than the tolerance level. smile.gif
I too run RAID1 for my most critical data as posted in this thread.
Why? Well, my most critical data fits snugly on a single 2TB drive. So, for RAID protection, mirroring just makes sense; and then, I backup the hell out of it. smile.gif

In your case, however, you have 8 disks with 2 being used for data and 6 for redundancy.
My whole point has been that, if given 6 disks to protect 2 disks, I would go with a parity scheme supporting 6 parity drives rather than two separate RAID 1 arrays.
Why?
1. With a 6-parity configuration, I can lose any 6 disks and suffer zero data loss.
2. With 2x 4-way mirrors, losing 4 disks from the same set will cause the whole set to be lost. Losing 6 disks will certainly result in some data loss unlike with the parity configuration.

Listen, you have great protection as is with your setup, albeit over kill.
I wouldn't worry about much past the technicalities of this discussion.

I think people are fearful of parity based RAID because of the risk of losing a drive past the tolerance level during the restoration process and thereby losing everything.
It has happened, and solutions like FlexRAID address that problem to some level.

Again... it comes to the fact that I answered a question and my answer was correct. Do people still prefer hardware RAID over software. The answer is yes. The answer is yes because your data is safer. I don't care how many disk failures a parity configuration can theoretically recover from because a parity algorithm that complex has a higher chance of failing it's rebuild that it does succeeding (opinion (and I'm not alone in this opinion)). I've worked in the IT industry long enough to know that parity based redundancy is not reliable on the hardware or software side. Bottom line is my original answer applies to the original question and it is true.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #112 of 183 Old 08-14-2013, 10:56 AM
AVS Special Member
 
EricN's Avatar
 
Join Date: May 2002
Posts: 1,166
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 12 Post(s)
Liked: 154
Quote:
Originally Posted by SirMaster View Post

That's the point. That's what changed IMO with the advent of FlexRAID. Previously, a 40TB array would have likely required "Pro" solutions, and would have been out of reach for most average consumers looking for a simple array to store all their AV media.

(40TB) / (10GB per movie) * ($20 per movie) = $80,000 of BluRays
(40TB) / (35MB per song) * ($1 per song) = $1,200,000 of music
(40TB) / (5GB per hour) / (6 hours per day) = 3.8 years of recorded TV.

Who is actually willing to buy 40TB of media but unwilling to spend a few grand on storing it properly? Flexraid fills a market niche for torrent downloaders that acquire drives one-by-one, a niche that "Pro" vendors have ignored. That doesn't mean it measures up to them.
EricN is offline  
post #113 of 183 Old 08-14-2013, 10:57 AM
Member
 
SirMaster's Avatar
 
Join Date: Aug 2009
Posts: 101
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 24
Why is a parity calculation more unreliable? Computers are simple machines. I've never seen an XOR fail where a bit copy would succeed. If XOR is failing, then it's due to some physical hardware failure and then there is no guarentee a bit copy wouldn't fail as well.

Assuming the parity calculation of the software in question is written mathematically correct which can be verified.
SirMaster is offline  
post #114 of 183 Old 08-14-2013, 11:02 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by SirMaster View Post

Why is a parity calculation more unreliable? Computers are simple machines. I've never seen an XOR fail where a bit copy would succeed. If XOR is failing, then it's due to some physical hardware failure and then there is no guarentee a bit copy wouldn't fail as well.

Assuming the parity calculation of the software in question is written mathematically correct which can be verified.

Because it only takes one tiny inconsistency to corrupt the entire array and fail parity as a whole which can be caused by an infinite number of variables. Say a drive dies while it's updating the array. Good luck. Go ahead and pull a hot swap drive out on the fly while the FlexRAID array is updating that disk and see if you can recover it.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #115 of 183 Old 08-14-2013, 11:03 AM
AVS Special Member
 
bryansj's Avatar
 
Join Date: Feb 2004
Location: Atlanta, GA
Posts: 6,244
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 27 Post(s)
Liked: 165
I've got all my computers backed up nightly to internal storage on my Windows Server, including the server itself (on another small drive). This gives both a bare metal and file level restore. In addition to that I have my important data (docs, pictures, home videos...) copied to a different local storage drive on the server as file level backup nightly using SecondCopy. This "important" data is right under 1TB and the main portion of it is digital pictures. Now that I have a local file level copy of my files I can have one instance of CrashPlan running on just my server. This important data is then copied to their cloud and to an external HDD at my parents' house. Now my important data has four to five copies both locally and offsite. I also have the option to pull up older versions and find deleted files to restore.

Now with all true backup out of the way I have my RAID. It adds a safety net to my media collection with four data drives and one parity drive using FlexRAID(-F). If it fails then I'll be upset, but my wife won't care which is what matters here. I've already seen most of those movies anyway so I'd probably only be replacing 20% of it.

Notice all of this is automated so it actually happens without my input. I considered the external HDD to bank vault kind of backup, but I'm sure that would stop happening as initially planned.

bryansj is online now  
post #116 of 183 Old 08-14-2013, 11:10 AM
Member
 
SirMaster's Avatar
 
Join Date: Aug 2009
Posts: 101
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 24
Quote:
Originally Posted by EricN View Post


Who is actually willing to buy 40TB of media but unwilling to spend a few grand on storing it properly? Flexraid fills a market niche for torrent downloaders that acquire drives one-by-one, a niche that "Pro" vendors have ignored. That doesn't mean it measures up to them.

That niche may or many not describe my situation :P. All I was ever saying is that there are cases where FlexRAID makes the most sense. I've never been one to say it's absolutely better or anything. That was other people.

I know plenty of people who have several disks in their desktops filled toth ebrim with media. They have no backup, they have no redundancy. Their backup plan is: "redownload" when I lose a disk. This is obviously very poor and there is an easy way to improve it immensely.

That product is FlexRAID. Litterally all they have to do is install a piece of "cheap" software onto their existing copmuter, and add 1 or 2 more disks for parity, and follow the cruise control setup and boom, all their disks are now protected against 1 or 2 or more disk failures. They didn't have to build a separate dedicated server machine (that takes money, electricity, additional administration), they didn't have to learn a new operating system, they didnt have to wipe their disks first (they have nowhere to store several TB in the meantime while they wipe their disks for RAID, ZFS, unRAID, etc)

Do you really not think $60 (FlexRAID) plus $100 or $200 for 1 or 2 more disks is not worth the added redundancy it offers to users? Users who are on limited budgets, don't want another computer, don't want to learn a new OS, and want to be able to continue expanding the array affordably and easily?

You may not think so. That's fine, but there are plenty of people, me included who find that very appealing.
SirMaster is offline  
post #117 of 183 Old 08-14-2013, 11:12 AM
Member
 
SirMaster's Avatar
 
Join Date: Aug 2009
Posts: 101
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 24
Quote:
Originally Posted by itznfb View Post

Say a drive dies while it's updating the array. Good luck. Go ahead and pull a hot swap drive out on the fly while the FlexRAID array is updating that disk and see if you can recover it.

You actually can do this if you are using a UoR disk. It caches the changes to an extra disk before committing them to the array during the update. So in the event of a failed update (power failure, other hardware failure, ripping the disk out :P), you can revert back to the old parity.

If you are ripping out a disk in the middle of a parity block update, you could still recover everything in the array minus the file(s) that block makes up in RAID-F and tRAID. How would this be different from ripping out a disk in the middle of a RAID-1 block update? Would you not still have a corrupt file as well where the mirrored block was only updated partially?
SirMaster is offline  
post #118 of 183 Old 08-14-2013, 11:26 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by SirMaster View Post

You actually can do this if you are using a UoR disk. It caches the changes to an extra disk before committing them to the array during the update. So in the event of a failed update (power failure, other hardware failure, ripping the disk out :P), you can revert back to the old parity.

If you are ripping out a disk in the middle of a parity block update, you could still recover everything in the array minus the file(s) that block makes up in RAID-F and tRAID. How would this be different from ripping out a disk in the middle of a RAID-1 block update? Would you not still have a corrupt file as well where the mirrored block was only updated partially?

Well in all of my test cases it's failed. So in theory it may be able to recover but in reality it doesn't. Just check the FlexRAID forums. It's littered with posts about rebuild failures. Some are self inflicted and other are just limited of either software or parity. In the event that a parity backup is able to do a point in time restore then in that case you're better off than you would be with hardware RAID5/6 but not RAID1. You're essentially in the same boat except with RAID1 you wouldn't have run into that risk.

If you rip a disk out of a RAID1 set in the middle of the block update there is no dependency from disk to disk. So disk 1 is still fine. That's the not the case with parity. There are dependencies all over the place. With RAID1 there are not.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #119 of 183 Old 08-14-2013, 11:44 AM
Member
 
SirMaster's Avatar
 
Join Date: Aug 2009
Posts: 101
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 24
I'm quite active on the FlexRAID forums myself since I am extremely interested in it's development and how ot works since I use it myself and awnt to see my array be safe and successful. Though it is my experience as with most things that you really only see the people posting about the problems they have with such and such a product. You rarely see people post whe nthings go right smile.gif

That being said. I have been using and following FlexRAID for 2 years now and I have not had any problems with it. When I was evaluating it, I simulated many failures and was satisfied with the recovery performance of the array. I have had unforseen failures with disks since and FlexRAID has always worked to save my data thus far, so again, I am personally happy with a working product.

Recently I have been busy putting the tRAID release candidate builds through the ropes testing it in every way I can, simulating all sorts of failures and reporting any problems to the developer. He has been great so far at resolving the issues. I also want to see tRAID be a solid product since I am interested in migrating to it from RAID-F in the future.

Nevermind about the RAID-1 question, it does make sense to me now, right, there should never be a discrepency between the disks aside from bitrot.
SirMaster is offline  
post #120 of 183 Old 08-14-2013, 11:49 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by SirMaster View Post

I'm quite active on the FlexRAID forums myself since I am extremely interested in it's development and how ot works since I use it myself and awnt to see my array be safe and successful. Though it is my experience as with most things that you really only see the people posting about the problems they have with such and such a product. You rarely see people post whe nthings go right smile.gif

That being said. I have been using and following FlexRAID for 2 years now and I ahve not had any problems with it. When I was evaluating it it simulated many failures and was satisfied with the recovery performance of the array. I have had unforseen failures with disks since and FlexRAID has always worked to save my data thus far, so again, I am personally happy with a working product.

Recently I have been busy putting the tRAID release candidate builds through the ropes testing it in every way I can, simulating all sorts of failures and reporting any problems to the developer. He has been great so far at resolving the issues. I also want to see tRAID be a solid product since I am interested in migrating to it from RAID-F in the future.

Nevermind about the RAID-1 question, it does make sense to me now, right, there should never be a discrepency between the disks aside from bitrot.

Agreed you'll typically see more complaints than praise in a public forum and with my own testing under normal circumstances FlexRAID has performed more than adequately in normal operation as well as recovery scenarios. It's the cases where I wouldn't expect it to recover is where it doesn't recover. And I'm not even holding that against FlexRAID or parity based redundancy... my only argument through this entire discussion is that RAID1 IS protecting your data in those scenarios where parity based redundancy is not.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
Reply Home Theater Computers

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off