In case you're wondering this is yet another drive pool thread, but this one has a little twist. I think? So please keep reading.
I'll try to keep this as brief as I can. About 6 years ago I've built a 7TB (8TB total, 1TB lost to parity) hardware RAID5 array on Adaptec 31605 card. I have since outgrew the array capacity and simply started putting my files on a bunch of drives on yet another file server. As you can imagine it was kind of a nightmare, not only half of my data was essentially JBOD with no redundancy whatsoever, but my data was also spread across 2 different computers across multiple hard drives.
In an effort to reorganize this whole mess and make sure it's all redundant I'm currently in the process of upgrading my fileserver. I have decided that another hardware RAID array would not be a good solution for me, so I opted for a JBOD configuration with SnapRAID for parity protection. I have purchased 5 4TB seagates, copied all of my data to the new drives, and dedicated 1 drive to parity. I have downloaded SnapRAID and it's currently calculating parity. I still want to get a second drive for dual parity, and one or two more drives for data, but as of right now the new system is live. Wooooooo!
Just about the only thing left to do was to find a good drive pooling solution. I have been reading a lot in the past week about drivepool and drivebender, liquesce, etc. I have finally settled on DrivePool, so I installed it yesterday and gave it a whirl. And this is where I ran into a problem... Apparently I totally misunderstood how DrivePool would work. What I wanted to do is this. Because my data is mostly static I wanted to make the drive pool "read only" and manually manage my data on my hard drives. I wanted to manually copy the data to the data drives on my file server and I wanted the drive pool to simply "pool" all the data from my hard drives and represent it as one big hard drive I could access anywhere from the network inside my place. I did not want the drive pool to manage my data, I did not want it to duplicate data, I did not want it to rebalance it. I wanted to balance the data across the hard drive manually myself, and I wanted to have SnapRAID provide redundancy. The automatic drive pool rebalancing would just increase the chances of something failing because SnapRAID does not calculate parity in real time. Basically I wanted to copy data to the individual hard drives and have the drive pool pick up the changes, pool all of the drives together, and immediately show newly copied files in one big pool. And this is not how DrivePool works. As I have learned, I cannot do that, I cannot just copy data to the individual hard drives and have that data show up in the pool. In order for any data to be available in the Pool, I actually have to copy data into the Pool and have the pool manage the data across hard drives. This is not what I wanted originally because it means it would be up to the pool software to manage my data across the hard drives and rebalance it if necessary.
I've been looking more closely into DriveBender and it appears it works in the same manner, i.e. you have to copy files onto the pool instead of individual hard drives, but I cannot tell for sure without giving it a try. No idea on Liquesce, that one has even less documentation than the other two.
So can anyone suggest a solution to my problem? I want to:
- Manually manage data on the individual hard drives
- I want my pool software to simply pool all the individual hard drives into one big read only pool with NO balancing/redundancy management whatsoever
- I would prefer the changes to the data on individual hard drives reflected immediately in the pool
Is there anything that would work the way I want to? Or am I stuck with using junctions/symlinks/hardlinks? BTW I'm still not sure on the differences between the three, so if anyone could chime on using those, and which one would be the best solution for me, I would appreciate that too.
It seems that the thread conversation is starting to veer off in all the different directions that are tangent to what I want to do, so I'd like to summarize what I need in a bunch of small bullet points. I want:
- Fault tolerance with two parity drives and I do not want my data striped, essentially I want software RAID4 with two parity drives which can be achieved by either RAID-F(formelly FlexRAID) or SnapRAID
- I must retain access to the individual hard drives so that I can write data to them
- I must be the only one to have write privileges to the raw hard drives
- Rest of the people in the house need to be given write access to specific folders only to limit the damage they can do
- I must be able to access all of the data on my hard drives from a single place as if all of this data was "merged" into a single folder, i.e. if I have four drives
I need to be able to access all of the folders from one place, that is that place should look like this:
- Last bullet point, that place/pool/whatever should be read only to minimize accidental damage
Edited by JazzJackRabbit - 10/25/13 at 12:07pm