or Connect
AVS › AVS Forum › Video Components › Home Theater Computers › Flexraid help deciding
New Posts  All Forums:Forum Nav:

Flexraid help deciding - Page 2

post #31 of 164
Quote:
Originally Posted by chappy16775 View Post

It seems that if my Music and Pics are on one particular drive and my Movies on three others then when I pool that data structure stays in tact. But...after pooling, when I add, say, a new music album to the pool where is it saved physically? If I save it into my Music sub folder will it automatically save into the original Music drive? Or from the point you create the pool does it decide for itself where to save the new files depending on space (which may result in my new music album being saved onto one of the Movies drives for example...or even worst the album being scattered all over the place)?
Thanks

I have not seen scattering. What it appears to do from my experience is say I have a 2TB drive with 1.5TB taken up by movies, and then another couple drives out there with varying amounts of space. It seems to fill up that 2TB drive when I copy something into my My Movies folder. Once it is filled, it creates a My Movies folder on another drive and begins copying movies there until that one fills up. From what I can tell, it does not do any type of load balancing between drives (I add two 2TB drives a while ago and they are still largely empty, despite my other 4 being almost completely full each).
post #32 of 164
Thread Starter 
Quote:
Originally Posted by bomberjim View Post

I've been using Flexraid for quite some time and love it. I don't think however, that you're going to be able to do exactly what you want - at least not easily.

Thanks, I'm thinking the same thing, but I'll still eventually grab the free trial and take a look

I didn't realize I'd be the only one to want something done this way, but I've got pretty high hopes since flexraid is just a software jbod with parity

I mostly just want to keep all of my 1TB stuff on that drive, but I guess it doesn't really matter. I suppose I can still backup from the pool
post #33 of 164
Quote:
Originally Posted by bomberjim View Post

I've been using Flexraid for quite some time and love it. I don't think however, that you're going to be able to do exactly what you want - at least not easily. If you pool drives, Flexraid will control the pool and you won't be able to dictate which drives files are saved to. If you don't pool, it will require an "expert" mode setup and you MAY be able to control which drives you save to. This is certainly not how Flexraid (or any type of raid) is designed to work however. If this is critical to you, I'd suggest asking these specific questions on the Flexraid forums prior to purchase.


+1
Agree
post #34 of 164
If you want to control the pool so much.. your better off just deleting a drive from the pool or not adding it in the first place.

That's not really the point of flexraid.

however... I am pretty sure if it's already on a drive it will stay there. Its only new stuff you add that you don't control...
post #35 of 164
Quote:
Originally Posted by ncarty97 View Post

I have not seen scattering. What it appears to do from my experience is say I have a 2TB drive with 1.5TB taken up by movies, and then another couple drives out there with varying amounts of space. It seems to fill up that 2TB drive when I copy something into my My Movies folder. Once it is filled, it creates a My Movies folder on another drive and begins copying movies there until that one fills up. From what I can tell, it does not do any type of load balancing between drives (I add two 2TB drives a while ago and they are still largely empty, despite my other 4 being almost completely full each).

right
post #36 of 164
Thread Starter 
Quote:
Originally Posted by ncarty97 View Post

I have not seen scattering. What it appears to do from my experience is say I have a 2TB drive with 1.5TB taken up by movies, and then another couple drives out there with varying amounts of space. It seems to fill up that 2TB drive when I copy something into my My Movies folder. Once it is filled, it creates a My Movies folder on another drive and begins copying movies there until that one fills up. From what I can tell, it does not do any type of load balancing between drives (I add two 2TB drives a while ago and they are still largely empty, despite my other 4 being almost completely full each).

This is essentially what I want. More stuff on less drives -> less drive spin ups
post #37 of 164
I'm just quoting FlexRaid wiki here but it seems there are different options in FlexRaid for how this is dealt with:

Storage Pool Merge Modes

FlexRAID Storage Pool supports several merge modes:
- an automatic mode with balanced space across drives
- an automatic mode reducing folder split across drives
- a manual mode where you explicitely configure the merge

Auto Merge with Balanced Space Priority

With this mode, the configuration is automatic and there is no configuration to deal with. All folders and files already on the drives will be available in the pool, and when you add folders and files to the pool, they will be written to the different drives depending on their available space.

This mode is very similar to what you are used to with WHS with the difference being in that it balances the disk space so that the drives are used evenly.

Auto Merge with Minimized Folder Split Priority


As above, the configuration is automatic in this mode and there is no configuration to deal with. Again, all folders and files already on the drives will be shown in the pool, and when you add folders and files to the pool, they will be written to the different drives in order to minimize the folder split: rather than balancing the data based on free space, this mode tries to keep folder splits to a minimum.

This mode will fill up each drive one by one, and as such can cause certain drives to be used more than others. But it is more energy efficient.

Explicit Merge

This is the mode you are all familiar with from older versions of FlexRAID. This mode allows for a lot of flexibilities and really intricate configurations.

It allows you to treat your source data as a database, and to create Views into that data (no need to mess with symbolic/hard links).

Additionally, it provides the greatest energy saving feature.
post #38 of 164
I am too looking to expand the storage on the server. Bear with me, I am running Windows 2000 Server on a Pentium II 233 MHz, running RAID5.

Perhaps my mind is stuck in a traditional RAID where redundancy is achieved by writing in 2 locaitons.

If, for the simplicity of mathematics, I have 5x 3Tb drives, and I dedicate 4 of them to the pool, and 1 of them to the parity, then my pool should be 12 Tb, correct?

How is the parity achieved on the single 3 Tb drive? I just can't get over the notion that 12 Tb of data can squeezed into 3 Tb of space...

Should 1 of the 4 pooled drives fail, and I replace it with a blank, will FlexRAID rebuild the data in the pool using the 3 Tb parity drive?

Will the array be accessible during the rebuild (like RAID 5)?

Sorry for all the questions.

I would like to utilize my older IDE drives, which are all sub 500 Gb, which will push the drive count over 20, and UnRaid requires the "pro" license for the high drive count.
post #39 of 164
Thread Starter 
Quote:
Originally Posted by blueiedgod View Post

How is the parity achieved on the single 3 Tb drive? I just can't get over the notion that 12 Tb of data can squeezed into 3 Tb of space...

Magic
post #40 of 164
Thread Starter 
Quote:
Originally Posted by blueiedgod View Post

How is the parity achieved on the single 3 Tb drive? I just can't get over the notion that 12 Tb of data can squeezed into 3 Tb of space...

I've wondered the same thing for a while. I thought hard drives could save two levels of deleted information under the top level allowing a lot of deleted stuff to be recovered, but the amount of data that can be recovered is too high for my mind to wrap around it

I've read several stories of people losing a drive and rebuilding it from their parity, it's better than nothing
post #41 of 164
Quote:
Originally Posted by Guinness77 View Post

I have a couple of questions:
I currently just have a bunch of disks that I manage manually, and make mirror of each one.
Can I mirror each drive in Flexraid if I wanted? Or would I be better off just using Cobian backup to create a copy and mirror the changes?
If I pooled the drives, would I still be able to see the individual drives and mirror them?
Is it even worth mirroring anymore, or just use a few more parity drives for safety?
If I create a pool of my drives, and I copy an iso file to the drive, will that one file always be on 1 drive (what if there is not enough room on any of the drives, will it spread it across the drives)? I would prefer not to have files broken and spread.
If I start to organize my movies etc.. once pooled, how will the changes look when I pull a drive out of the pool?
You can pull the drive out of a pool and pop it in another computer and it will just be a normal drive correct?
Can you have multiple storage pools?
Also, can you set a certain percentage of the drive to be empty (i currently leave around 150gb). Is that even worthwhile doing?
Sorry, I got a little carried away with the questions, I am just used to manually doing everything.
Thanks

Just one correction to a previous answer. You CAN set a certain percentage. It's under the settings and preferences and called "Reserve". It is worth playing with, as it will determine when files in a folder are "split" between drives. It can also make a difference as to which drive is the largest (think parity drive) when dealing with drives that are supposedly the same size but from different manufacturers.
post #42 of 164
Quote:
Originally Posted by ncarty97 View Post

I have not seen scattering. What it appears to do from my experience is say I have a 2TB drive with 1.5TB taken up by movies, and then another couple drives out there with varying amounts of space. It seems to fill up that 2TB drive when I copy something into my My Movies folder. Once it is filled, it creates a My Movies folder on another drive and begins copying movies there until that one fills up. From what I can tell, it does not do any type of load balancing between drives (I add two 2TB drives a while ago and they are still largely empty, despite my other 4 being almost completely full each).

That's the default mode. You can change it to auto-space priority to "balance the space available on the drives".
post #43 of 164
Quote:
Originally Posted by blueiedgod View Post

I am too looking to expand the storage on the server. Bear with me, I am running Windows 2000 Server on a Pentium II 233 MHz, running RAID5.
Perhaps my mind is stuck in a traditional RAID where redundancy is achieved by writing in 2 locaitons.
If, for the simplicity of mathematics, I have 5x 3Tb drives, and I dedicate 4 of them to the pool, and 1 of them to the parity, then my pool should be 12 Tb, correct?
How is the parity achieved on the single 3 Tb drive? I just can't get over the notion that 12 Tb of data can squeezed into 3 Tb of space...
Should 1 of the 4 pooled drives fail, and I replace it with a blank, will FlexRAID rebuild the data in the pool using the 3 Tb parity drive?
Will the array be accessible during the rebuild (like RAID 5)?
Sorry for all the questions.
I would like to utilize my older IDE drives, which are all sub 500 Gb, which will push the drive count over 20, and UnRaid requires the "pro" license for the high drive count.

That's what makes UnRaid and Flexraid proprietary systems, and they're not telling exactly how they achieve "parity without striping".

Yes, it will rebuild the drive from the parity drive. I've done it on a small scale as a test. I believe, but am not absolutely sure, that the array is available during rebuild, but the drive that is being rebuilt is not available. Could be wrong on that however.
post #44 of 164
Good replies BomberJim
post #45 of 164
Quote:
Originally Posted by Dark_Slayer View Post

Magic

lol
post #46 of 164
Quote:
Originally Posted by Dark_Slayer View Post

I've wondered the same thing for a while. I thought hard drives could save two levels of deleted information under the top level allowing a lot of deleted stuff to be recovered, but the amount of data that can be recovered is too high for my mind to wrap around it
I've read several stories of people losing a drive and rebuilding it from their parity, it's better than nothing

Parity is a pretty cool concept.
post #47 of 164
post #48 of 164
Thread Starter 
post #49 of 164
Quote:
Originally Posted by blueiedgod View Post

I am too looking to expand the storage on the server. Bear with me, I am running Windows 2000 Server on a Pentium II 233 MHz, running RAID5.
Perhaps my mind is stuck in a traditional RAID where redundancy is achieved by writing in 2 locaitons.
If, for the simplicity of mathematics, I have 5x 3Tb drives, and I dedicate 4 of them to the pool, and 1 of them to the parity, then my pool should be 12 Tb, correct?
How is the parity achieved on the single 3 Tb drive? I just can't get over the notion that 12 Tb of data can squeezed into 3 Tb of space...
Should 1 of the 4 pooled drives fail, and I replace it with a blank, will FlexRAID rebuild the data in the pool using the 3 Tb parity drive?
Will the array be accessible during the rebuild (like RAID 5)?
Sorry for all the questions.
I would like to utilize my older IDE drives, which are all sub 500 Gb, which will push the drive count over 20, and UnRaid requires the "pro" license for the high drive count.

I will try at the above...

But I need to break it down so my ADD mind works properly.
Quote:
Originally Posted by blueiedgod View Post

I am too looking to expand the storage on the server. Bear with me, I am running Windows 2000 Server on a Pentium II 233 MHz, running RAID5.
Perhaps my mind is stuck in a traditional RAID where redundancy is achieved by writing in 2 locaitons.

It's definitely a different concept that you can back up 15TB of data with 3TB of space.
Quote:
Originally Posted by blueiedgod View Post

for the simplicity of mathematics, I have 5x 3Tb drives, and I dedicate 4 of them to the pool, and 1 of them to the parity, then my pool should be 12 Tb, correct?

(5) 3TB drives with 4 as DATA drives (called DRU) and the remaining 3TB drive as a PPU (Parity drive) [back up]

So 4 data, one parity. Then you would have a pool size of 12TB. Yes. 4x3=12. 3TB Parity would provide backup if one of those drives went down. You could actually remove one of those drives and replace it with a blank 3TB drive and re-write the data back to it. Pretty cool if you ask me.
Quote:
Originally Posted by blueiedgod View Post

How is the parity achieved on the single 3 Tb drive? I just can't get over the notion that 12 Tb of data can squeezed into 3 Tb of space...
Should 1 of the 4 pooled drives fail, and I replace it with a blank, will FlexRAID rebuild the data in the pool using the 3 Tb parity drive?
Will the array be accessible during the rebuild (like RAID 5)?
Sorry for all the questions.
I would like to utilize my older IDE drives, which are all sub 500 Gb, which will push the drive count over 20, and UnRaid requires the "pro" license for the high drive count.

Parity keeps track of the total - it's math.

remember that all digital information is stored as either a 0 or a 1. it's never anything different.

so basically it reads each unit across all the drives and records the sum on parity drive.

Look at this:



That is a good example.


It records the totals across your drives and stores the answer on the parity drive. That is why the parity drive must be as large as your largest drive so it can hold that many parity totals.

If you subtracted one of those data drives you could figure out what is missing or what digit the missing drive had. Just look at the digits you know... and the answer and you can figure out what it was. That is basically how flexraid works.
You can back up more than 5 drives. I use it on 8. But as you increase drive counts you increase the chance of simultaneous drive failures. That's a problem.

Last-

Yes.

Your "pool" is always available. Flexraid is real time. Once you set it up- your drive is always available unless your doing a rebuild. You can get an email each day on the status if you want to know it's ok. It can email you your parity is up to date and correct if you need to know or want to be sure. I don't bother.
also,
Flexraid will allow you to use your older drives. You can then replace them with larger drives in future. I am doing this. replacing a 500GB drive with a 3TB gains you 2.5TB of space without needing another SATA port or SATA power. It's the way to do it when you find $$$ and good deals on large drives. Plus- it's good to replace older drives before they fail. it's just easier.
post #50 of 164
Yep, it's math, and the first linked video is another good explanation. You're not backing up (or protecting) files, you're playing with digital data - 1's and 0's. Where the magic comes in is when you add a second parity drive to guard against two data drive failures. The calculations are now not so straight forward. Flexraid allows an unlimited number or parity drives.
post #51 of 164
Quote:
Originally Posted by bomberjim View Post

Yep, it's math, and the first linked video is another good explanation. You're not backing up (or protecting) files, you're playing with digital data - 1's and 0's. Where the magic comes in is when you add a second parity drive to guard against two data drive failures. The calculations are now not so straight forward. Flexraid allows an unlimited number or parity drives.

I am thinking of adding a second Parity drive.

Do you do this?

What is the recommendation ?
post #52 of 164
Quote:
Originally Posted by bomberjim View Post

That's what makes UnRaid and Flexraid proprietary systems, and they're not telling exactly how they achieve "parity without striping".
Yes, it will rebuild the drive from the parity drive. I've done it on a small scale as a test. I believe, but am not absolutely sure, that the array is available during rebuild, but the drive that is being rebuilt is not available. Could be wrong on that however.

unRAID parity is pretty well explained. It's only proprietary because to achieve what they did, they had to create their own solution that was on a kernel level.

FlexRAID drives will not be available during a rebuild for realtime pooled arrays (snapshot arrays it might be - haven't tried). unRAID can simulate failed drives the same as a hardware RAID controller does. I pulled a drive out of my array and was still able to read the contents that were on that drive from the server. The drive had many read/write errors and so accessing it while it was still in the server would occasionally hit points that slowed it down to a crawl or stall the server. Taking the drive out allowed the drive to be read at the speed of parity creation. Once a new drive was placed in, the contents were re-created. Accesssing the array is possible during this time, but slows down the rebuild process so is not really recommended as more time spent building increases the chances of a second failure.
Quote:
Originally Posted by Mfusick View Post

I am thinking of adding a second Parity drive.
Do you do this?
What is the recommendation ?

Are you using realtime parity protection? Last I remember, you can't do dual parity in that mode.
post #53 of 164
Apologies for going off on a bit of an tangent here but I'm deliberating what OS for Flexraid. I had assumed Win 7 is all I need but hear some are using WHS. Can anyone tell me what is the main reasons for using WHS over plain old Win 7?

Thanks
post #54 of 164
It's cheaper than Win 7, it's a server OS, and it's built in backup of client computers (and itself) is very nice as well. If your designing a multi-purpose machine Win 7 is a good choice, if it's just a server then WHS 2011 should at least be considered.
post #55 of 164
Quote:
Originally Posted by Dark_Slayer View Post

Thanks, I'm thinking the same thing, but I'll still eventually grab the free trial and take a look
I didn't realize I'd be the only one to want something done this way, but I've got pretty high hopes since flexraid is just a software jbod with parity
I mostly just want to keep all of my 1TB stuff on that drive, but I guess it doesn't really matter. I suppose I can still backup from the pool

I am in the exact same situation as you and wondered about this exact situation. I would rather not have music and pictures which are small files in comparison to a full bitrate MKV spanning accross multiple drives in the pool. Like you said, if for some reason I wanted to break up the pool, I'd rather not deal with hunting through how ever many drives you have to find all the music or picture files.

i was thinking and I'm hoping Flexraid can do this: Keep your pics and music on a larger unpooled drive but still have the parity drive "see" this drive as well when it calculates the parity? Can Flexraid do this?

In other words... Can Flexraid parity the pooled drives AND additional un-pooled drives with a single parity drive?
post #56 of 164
Whs is much cheaper and better suited for a pure server.
post #57 of 164
You guys are over concerned with small files spanning drives.

It's a total non issue.

Lets say you have a music folder and you add a few mp3 files.. They go into that folder on that drive.

It's only when that drive fills up that it would get stored to another drive.

And not all drives. One drive. Inside a folder also named music.

If you broke down your array you would see you have a music folder on two different drives. Only reason it's on two drives is cause the first one was full.

I think everyone is over reacting to the whole "spanned across multiple drives small files" thing. It's not an issue.

When you break up your pool you don't have random small files all over the place.

The solution is of coarse in your pool have a small amount of top level drives with everything inside that.

Example: folder for video. All movies and TV shoes inside it. I have ten folders or categories in mine. HD not HD Bluray DVD TV shows Disney etc.. Those are my libraries I point my mediabrowser to. You have to set it up this way anyway if you want it to work properly.

Then on my pool I have folder called music: anything has to do with music inside. In folders. Organized.

Then I have folder called programs: anything has to do with programs inside

Then I have folder called documents: anything appropriate inside.

Then I have folder called misc.

That's it.

So of I break my array up then everything is still organized in just a few folders.

It's not like my music is all over the place.

It's all inside a music folder. Perhaps on two drives but that's only because my music drive filled up. Otherwise Flexraid would keep filling the original drives this is the default.
post #58 of 164
Quote:
Originally Posted by Mfusick View Post

(5) 3TB drives with 4 as DATA drives (called DRU) and the remaining 3TB drive as a PPU (Parity drive) [back up]
So 4 data, one parity. Then you would have a pool size of 12TB. Yes. 4x3=12. 3TB Parity would provide backup if one of those drives went down. You could actually remove one of those drives and replace it with a blank 3TB drive and re-write the data back to it. Pretty cool if you ask me.
.

If I had 6 DRU's, and 1 PPU, and I had two data drives fail, or 3, what would happen? Would I loose one of the drives, or could that PPU rebuild both?
If I had two PPU's, does that protect the data in case one of the PPU's fails during a rebuild?




Thanks!

Thanks ncarty97 & bomberjim too
post #59 of 164
Quote:
Originally Posted by Guinness77 View Post

If I had 6 DRU's, and 1 PPU, and I had two data drives fail, or 3, what would happen? Would I loose one of the drives, or could that PPU rebuild both?
If I had two PPU's, does that protect the data in case one of the PPU's fails during a rebuild?
Thanks!
Thanks ncarty97 & bomberjim too

Depends.

You can use more than one parity drive which can provide additional security for a much more unlikely event. Flexraid can guard you against two drives failing- if that is what your asking.

I just think it's unreasonable to think your going to have more than 2 drives fail at the exact same time. Unless of a house fire or flood or EMP terrorist attack it seems unlikely a normal level event would happen to cause that.

If your worried about that- CRASHPLAN offers unlimited backup with remote cloud for a very affordable price.

You could just back up you entire server offline. That's a better idea for someone who is ultra data loss and data security concerned.

I am cool with just simple double parirty. I plan to use only two parity drives all the way up to 30TB pool. I think it's reasonable to go about 8 drives per parity drive before you need to add more parity. You can of coarse go double parity on 6 drives in a pool for extra security on simultaneous drive failure.
post #60 of 164
So one parity drive can protect against 2 or more drives failing at once?
Having multiple parity drives just protects the data in case the parity drive fails?
How many drives can you have on 1 parity?
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers
AVS › AVS Forum › Video Components › Home Theater Computers › Flexraid help deciding