AVS › AVS Forum › Video Components › Home Theater Computers › Re-introducing FlexRAID (Flexible RAID)
New Posts  All Forums:Forum Nav:

Re-introducing FlexRAID (Flexible RAID)

post #1 of 249
12/20/08 at 12:39pm
Thread Starter 
This thread is very outdated.

FlexRAID has evolved greatly into a far more robust solution than discussed here.

Official site: http://www.flexraid.com
Forum: http://forum.flexraid.com
Reply
Reply

AVS Top Picks

post #2 of 249
12/22/08 at 9:56am
Thread Starter 
----
Reply
Reply
post #3 of 249
12/23/08 at 4:54pm
hmm I'll have to check this out. I'll give it a test on a ubuntu 8.10 kernel when I get a chance
Reply
Reply
post #4 of 249
12/25/08 at 4:24pm
Thread Starter 
Quote:
Originally Posted by blackoper View Post

hmm I'll have to check this out. I'll give it a test on a ubuntu 8.10 kernel when I get a chance

Cool.
FlexRAID is very easy test drive.
You can test it out even if using a single drive since anything can be a UoR (Unit of Risk).
A UoR can be a file, folder, disk, network share, etc.
Reply
Reply
post #5 of 249
12/25/08 at 7:29pm
Pardon my ignorance but I am new to RAID and am just learning of flexraid. If I theoretically had 5 1TB disks, could I use flexraid to create 1 large array with parity that can be accessed by other computers on my network?
Reply
Reply
post #6 of 249
12/26/08 at 1:49pm
Lets say i have 2 disks in raid0
Can flexraid:
-Automatically detect datarot, and automatically repair it?
-Restore ALL data if one of the disks failed?
Reply
Reply
post #7 of 249
12/26/08 at 3:08pm
Thread Starter 
Quote:
Originally Posted by tkole View Post

Pardon my ignorance but I am new to RAID and am just learning of flexraid. If I theoretically had 5 1TB disks, could I use flexraid to create 1 large array with parity that can be accessed by other computers on my network?

Yes, except that FlexRAID is not responsible for the presentation of the data.
Read this thread: http://www.openegg.org/forums/posts/list/28.page
It has what you might be looking for.

If you don't care for WHS, you can achieve the same thing by using Windows DFS (Distributed File System): http://www.openegg.org/forums/posts/list/29.page

Quote:
Originally Posted by tetsuo55 View Post

Lets say i have 2 disks in raid0
Can flexraid:
-Automatically detect datarot, and automatically repair it?
-Restore ALL data if one of the disks failed?

FlexRAID can and will detect datarot as you schedule it to.
However, it does not do automatic datarot repair yet.
You will be alerted, and it will be up to you to take the appropriate action (delete the corrupted file and restore it).

Yes, FlexRAID can restore all of the data on your RAID 0 if it fails.
You will have to add your RAID 0 volume as a DRU to FlexRAID.
That said, FlexRAID Basic should not be used to RAID partitions that contain a running OS or database.
Reply
Reply
post #8 of 249
12/27/08 at 5:59pm
spectrumbx, thank you for the reading material. Maybe you can tell me if I'm coming to the correct conclusions. I have a pc running vista32 buisness that I want to use as a media server. I have an 8 port sata card that currently has 5 1TB disks connected. Could I use vista to create a dynamic disk using 4 of the disks and stripe data across them to create 1 large 4TB drive, and then use the remaining drive as a parity drive for flexraid to protect the 4TB dynamic drive?

Tom
Reply
Reply
post #9 of 249
1/9/09 at 11:29am
Thread Starter 
Linux + MHDDFS vs. WHS

http://www.openegg.org/forums/posts/list/88.page
Reply
Reply
post #10 of 249
1/26/09 at 12:18pm
THANKS Spectrumbx!!!!!! Been looking forward to this release1! I have really enjoyed the beta versions!! Great program keep up the good work!!!
Reply
Reply
post #11 of 249
3/19/09 at 7:09am
Thread Starter 
FYI, FlexRAID Basic 1.1 RC1 is out.

Code:
Changes:
-----------
- Various bug fixes
- Memory optimizations (mostly for large RAIDs)
- RAID engine performance optimization (at least 15% and up to 50% performance improvement)
- I/O API re-work for better scaling
- Advanced RAID reconfiguration: Remove locked DRU / Rename locked DRU
- Option to enable read errors retry
- Support for transaction (ability to undo/rollback a failed operation automatically so that the parity data is not left in an unknown state)
- New data pickup delay feature (a delay since the last modified date specifying when a new file is ready to be added to the RAID)
Reply
Reply
post #12 of 249
4/4/09 at 8:39am
Thread Starter 
FYI, FlexRAID Basic 1.1 RC2 is out.

Code:
Changes from RC1:
-----------------
- Various bug fixes
- New multi-step parity feature (read more on this here: http://www.openegg.org/referenceFlexRAID.curi#multi-step)
http://www.openegg.org/forums/posts/list/118.page
Reply
Reply
post #13 of 249
8/13/09 at 2:47pm
Thread Starter 
FYI: FlexRAID 1.2 final is out!
www.flexraid.com
Reply
Reply
post #14 of 249
8/13/09 at 3:23pm
Wow, I wish I knew about this before I set up my WHS server. This looks a heck of a lot easier to use than unRAID.

I have a couple questions that don't seem to be answered on your home page. Maybe they're too simple.

Do you assign one or more drives as parity-only (drobo-style), or is parity info spread out across all drives (RAID5 style)?

Can you add new drives to the pool at any time? If the new drive is larger than any of the others, do you need to make that the new parity drive (like Drobo), or is parity sorted out automatically?
Reply
Reply
post #15 of 249
8/13/09 at 4:02pm
Thread Starter 
Quote:
Originally Posted by candre23 View Post

Wow, I wish I knew about this before I set up my WHS server. This looks a heck of a lot easier to use than unRAID.

Yep, FlexRAID is quite simple and flexible.
Many users use FlexRAID paired with WHS.

Quote:
Originally Posted by candre23 View Post

Do you assign one or more drives as parity-only (drobo-style), or is parity info spread out across all drives (RAID5 style)?

You assign one or more drives as parity.
The parity drives can contain other data, but it is best to keep the parity drives for parity data only if you can help it.
You can use the free space from many different sources including network shares to host the parity data.
So, yeah... drobo-style (but much more flexible) and not RAID5 style.

Quote:
Originally Posted by candre23 View Post

Can you add new drives to the pool at any time? If the new drive is larger than any of the others, do you need to make that the new parity drive (like Drobo), or is parity sorted out automatically?

Yep, you can add more drives to the pool at any time.
You don't need to make the new drive the parity drive.
However, the parity space must be as large as the largest drive.

Note that I said "parity space" as you can assign multiple smaller drives, network share space, etc as "parity space".
So, use your best and newer drives for data and your old and smaller drives for parity space.
Reply
Reply
post #16 of 249
8/13/09 at 9:59pm
Quote:
Originally Posted by spectrumbx View Post

Many users use FlexRAID paired with WHS.

Wow, I never even thought of that. It would be a nice way to keep redundancy without having to use WHS's full mirroring. Right now I have 10TB across 15 physical drives, but with mirroring I only have 5TB of usable space. If I switch to FlexRAID I could have 8TB usable and only 2TB (largest drive size) of parity. The only problem is that switching over would be a very complicated and time consuming project. Maybe the next time I'm ready to add a drive I'll try to work it out.

One more question - does FlexRAID give you some easy way to keep track of which drive is which? I have a lot of same-model drives. I'd never be able to keep track of them if it weren't for WHS's wireframe display. With FlexRAID WHS would just see one big drive, correct? I just want to be sure that if I lose a drive, I'll be able to figure out which drive I lost.
Reply
Reply
post #17 of 249
8/13/09 at 10:30pm
Thread Starter 
Quote:
Originally Posted by candre23 View Post

Wow, I never even thought of that. It would be a nice way to keep redundancy without having to use WHS's full mirroring. Right now I have 10TB across 15 physical drives, but with mirroring I only have 5TB of usable space. If I switch to FlexRAID I could have 8TB usable and only 2TB (largest drive size) of parity. The only problem is that switching over would be a very complicated and time consuming project. Maybe the next time I'm ready to add a drive I'll try to work it out.

One more question - does FlexRAID give you some easy way to keep track of which drive is which? I have a lot of same-model drives. I'd never be able to keep track of them if it weren't for WHS's wireframe display. With FlexRAID WHS would just see one big drive, correct? I just want to be sure that if I lose a drive, I'll be able to figure out which drive I lost.

It is not complicated going from WHS's mirroring to FlexRAID.
There is a tutorial on the FlexRAID forum for FlexRAID + WHS.
Also check out this guy's build: http://www.avsforum.com/avs-vb/showthread.php?t=1160305

As far the wireframe, you will still use all existing WHS features.
The only thing you are replacing is switching from mirroring to FlexRAID.
What's more interesting is that you can still have WHS's mirroring for some of the data (critical data) and FlexRAID for the rest (movies, music, etc.).
Reply
Reply
post #18 of 249
8/14/09 at 4:57am
Quote:
Originally Posted by spectrumbx View Post

FYI: FlexRAID 1.2 final is out!
www.flexraid.com

whats new in 1.2? Seemed like I had to register to find that out.

I interested in flexraid. I was currently trying to build a unraid machine, but the hold up has been buying another machine just to run it. I live in an apartment, and didn't want to have another PC running next to my media center just for storage. It sounds like I could just install this on my media center and have it handle this?

Does anyone know any good 4 bay estata enclosures? I seems like most of them are the same price as buying a cheap PC (~250) which defeats the purpose of getting one for me. I don't have any room in my media center case for more drives.

Thanks.
Reply
Reply
post #19 of 249
8/14/09 at 6:00am
Thread Starter 
Quote:
Originally Posted by mackworth View Post

whats new in 1.2? Seemed like I had to register to find that out.

I interested in flexraid. I was currently trying to build a unraid machine, but the hold up has been buying another machine just to run it. I live in an apartment, and didn't want to have another PC running next to my media center just for storage. It sounds like I could just install this on my media center and have it handle this?

Does anyone know any good 4 bay estata enclosures? I seems like most of them are the same price as buying a cheap PC (~250) which defeats the purpose of getting one for me. I don't have any room in my media center case for more drives.

Thanks.

Version 1.2 only has bug fixes (no new features).

Yeah, you could just install FlexRAID on your media center machine and not need another server.
Combining server/machine makes plenty of sense.
My VMware server doubles as my file server.
No need to maintain another machine if you don't have to.

There was a number of 4/5 bay eSATA enclosures selling for $70 not too long ago.
Check slickdeals.
Those on Newegg (specially the Rosewill ones) usually get discounted.
So, it is just a matter of waiting for them to be on sale again.
Reply
Reply
post #20 of 249
8/15/09 at 5:53am
I have what must be a dumb question... I've been thinking about building a server using FlexRAID and WHS. I understand that the current release is "snapshot" RAID, and if that means that any files that changed since the last synch are lost, I'm good with that. That should mean in theory that you only lose the last days worth of files in a failure. But what I'm having a hard time wrapping my head around is: if something changes in the array since the last resynch, doesn't that break parity?
Reply
Reply
post #21 of 249
8/15/09 at 7:16pm
Thread Starter 
Quote:
Originally Posted by Darin View Post

I have what must be a dumb question... I've been thinking about building a server using FlexRAID and WHS. I understand that the current release is "snapshot" RAID, and if that means that any files that changed since the last synch are lost, I'm good with that. That should mean in theory that you only lose the last days worth of files in a failure. But what I'm having a hard time wrapping my head around is: if something changes in the array since the last resynch, doesn't that break parity?

It depends on the nature of the changes.
If you edited or deleted some data, the RAID could be partially out of sync, and you would need to re-sync it.

In the worst case scenario where a drive fails after the RAID is out of sync, you only risk losing up to the amount of data that changed.
So, if you changed (edited/deleted) 1GB worth of data, you risk to lose up to 1GB worth of data if a drive fails before you have a chance to re-sync.

So, FlexRAID Basic is best suited for data that is fairly semi-static like DVD backups, movies, music, pictures, etc. (anything that is not continuously being edited).

FlexRAID Live!, when released, will remove this limitation.
Reply
Reply
post #22 of 249
8/16/09 at 7:11am
Whats the status of FlexRaid Live?


Also, I have a 1 TB drive with movies and music on it. If I get 2 more drives, I can create the initial array using my drive that already has data on it right? I don't need to format them or anything?
Reply
Reply
post #23 of 249
8/16/09 at 7:28am
Quote:
Originally Posted by spectrumbx View Post

In the worst case scenario where a drive fails after the RAID is out of sync, you only risk losing up to the amount of data that changed.
So, if you changed (edited/deleted) 1GB worth of data, you risk to lose up to 1GB worth of data if a drive fails before you have a chance to re-sync.

Ok, thanks, that is what I was looking for. If it's only those files that have changed since the last sync that are at risk, I can live with that. My fear was that since parity relies on differences between files that are being "combined" for the parity data, the other data that hasn't been changed could be at risk during a failure, because the parity data is partially based on data that has changed.

In other words, in a simplistic example of a four drive array: three data drives and 1 parity drive, and one huge file on each data drive, it seems to me like if you synched them all up (created parity data on the parity drive), changed the data on drive A, then had a failure on drive B, it would seem like the data on drive B would be at risk. The data on drive B would have to be rebuilt based on the remaining data on the remaining drives. But one of the pieces of the puzzle (the file on drive A) has changed, making it impossible to piece everything back together to recreate the B drive. Does that make sense?
Reply
Reply
post #24 of 249
8/16/09 at 9:40am
Thread Starter 
Quote:
Originally Posted by mackworth View Post

Whats the status of FlexRaid Live?


Also, I have a 1 TB drive with movies and music on it. If I get 2 more drives, I can create the initial array using my drive that already has data on it right? I don't need to format them or anything?

FlexRAID Live! is still in development and is coming greatly along.
I got busy with other stuffs for the past few months, which has slowed everything.
I am back to the project, and things are picking back up.

No, you don't need to format anything.
Drives with existing data can be added to a RAID without issues.
Reply
Reply
post #25 of 249
8/16/09 at 9:45am
Thread Starter 
Quote:
Originally Posted by Darin View Post

Ok, thanks, that is what I was looking for. If it's only those files that have changed since the last sync that are at risk, I can live with that. My fear was that since parity relies on differences between files that are being "combined" for the parity data, the other data that hasn't been changed could be at risk during a failure, because the parity data is partially based on data that has changed.

In other words, in a simplistic example of a four drive array: three data drives and 1 parity drive, and one huge file on each data drive, it seems to me like if you synched them all up (created parity data on the parity drive), changed the data on drive A, then had a failure on drive B, it would seem like the data on drive B would be at risk. The data on drive B would have to be rebuilt based on the remaining data on the remaining drives. But one of the pieces of the puzzle (the file on drive A) has changed, making it impossible to piece everything back together to recreate the B drive. Does that make sense?

You are correct in that existing data could be compromised, but only up to the amount of data that changed.
So, it is data that is in the "range" of the data that has changed that is at risk when the RAID is out of sync.
So again, if 1GB changes (deleted or edited), each drive in the RAID would be at risk for 1GB worth of data.
Reply
Reply
post #26 of 249
8/16/09 at 10:28am
Quote:
Originally Posted by spectrumbx View Post

So again, if 1GB changes (deleted or edited), each drive in the RAID would be at risk for 1GB worth of data.

So the short of it is, until Live is available, if a large file is deleted or changed, you should force a synch, if a small file is deleted or changed, your risk is minor. But writing new files poses no risk, because the parity is on the file level, not the disk level. Do I understand correctly?
Reply
Reply
post #27 of 249
8/16/09 at 11:09am
I have about 4 terabytes of video spread out over 3 hard drives. All three drives are almost full. 2 of my drives are 1.5TB seagates and I plan on buying 2 more.

Can I create a flexraid RAID 5 array with all my drives without wiping out the data that is stored on them? Will the raid 5 array appear as a single volume drive letter in WIndows 7 64 bit?
Reply
Reply
post #28 of 249
8/16/09 at 2:27pm
Flexraid does not do RAID5. It keeps all your drives individually and just creates a parity (recovery) area that you can use to recover a lost drive or lost files.
Reply
Reply
post #29 of 249
8/16/09 at 9:41pm
Quick question. I was running some tests and I cannot figure out something. Does Flexraid access every single drive at the same time while creating a parity or does it do it one drive at a time. My system right now has 10 drives and I am could not figure out the I/O speed while it was creating my latest parity.
Reply
Reply
post #30 of 249
8/16/09 at 10:11pm
Thread Starter 
Quote:
Originally Posted by Darin View Post

So the short of it is, until Live is available, if a large file is deleted or changed, you should force a synch, if a small file is deleted or changed, your risk is minor. But writing new files poses no risk, because the parity is on the file level, not the disk level. Do I understand correctly?

Correct.

Quote:
Originally Posted by Pastuch View Post

I have about 4 terabytes of video spread out over 3 hard drives. All three drives are almost full. 2 of my drives are 1.5TB seagates and I plan on buying 2 more.

Can I create a flexraid RAID 5 array with all my drives without wiping out the data that is stored on them? Will the raid 5 array appear as a single volume drive letter in WIndows 7 64 bit?

The only RAID engine currently enabled is a RAID 4 engine.
But yeah, you can create the RAID without wiping any data.

As far as presenting the data as one volume, you have a few options but none that is currently built in.
FlexRAID-View (which is supposed to be the built-in feature) has not been released yet.
Check the forum for your current options for presenting the data as one volume.

Quote:
Originally Posted by MickeyDora View Post

Quick question. I was running some tests and I cannot figure out something. Does Flexraid access every single drive at the same time while creating a parity or does it do it one drive at a time. My system right now has 10 drives and I am could not figure out the I/O speed while it was creating my latest parity.

It accesses all the drives at the same time unless you are doing "multi-step parity".
Your I/O system will be your bottleneck, specially with many drives.
Reply
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers

AVS Top Picks

AVS › AVS Forum › Video Components › Home Theater Computers › Re-introducing FlexRAID (Flexible RAID)