AVS › AVS Forum › Video Components › Home Theater Computers › Limetech un-Raid Media Storage Server support thread
New Posts  All Forums:Forum Nav:

Limetech un-Raid Media Storage Server support thread - Page 8

post #211 of 2123
Thread Starter 
Quote:
Originally Posted by Birdman71 View Post

Another software question:

Is it possible to exclude any of the drives from the array and run 1 or 2 drives as stand alone?

For media storage I can imagine a need for high-speed writes (HD recording across network) and it may be beneficial to allow at least 1 drive and maybe 2 to operate without the parity calculation and write. I may be way off base here but it seems that there could be a significant performance advantage, especially if multiple writes are being performed to different volumes simultaneously.

Ultimate flexibility would be on a per drive assignment, either free-standing or as part of the parity cluster.

Birdman

The Birdman. A most excellant suggestion which has just been moved close to the top of the Future Feature list!
post #212 of 2123
Thread Starter 
Quote:
Originally Posted by madpoet View Post

Tom, you know far more about the subject than I do but I will have to disagree. I've had 2 different RAID5 solutions where, when the Master (or IDE0, or whatever) went bad it caused the Slave (or IDE1) to ALSO be marked bad. This isn't misinformation. The Slave wasn't bad obviously, and had to be reinitialized. But it can cause a real headache. This was with both Software RAID5 in Windows 2000 and with a competitor product. And both times the support people I talked to knew immediately what had happened and how to fix it. So... I'm not sure this is an issue with your solution or not. But it definately does happen, and is a concern.

Concern noted. My point was simply that there is no interdependence between the two devices. In the case you cite here, had it been the Slave which "went bad" probably the master would have also been marked bad (by an obviously flawed driver stack).

Yes, this is an issue with multiple device on a single channel or bus. I've seen SCSI systems where a bad device caused the whole bus to hang. But in this case, the driver was smart enough to not fail all the devices.

In the UnRaid driver we NEVER disable more than one device. For example, if a drive fails in such a way as to hang the channel, then what will happen is the low-level IDE driver will fail all the queued I/O to it. But when UnRaid sees the first write error, it will fail that drive (that is, disable it). Thereafter, any other error results in just failing the original host request.

In this case what would happen is your windows application using the share would see an i/o error. The UnRAID management console will show a disable disk and possibly a non-zero error count on other disks. I would guess that 99% of the time the correct disk would be marked as failed. You would then remove this disk and all should be ok again.

This is how it should work by design - but we haven't encountered a disk which failed like this, and we haven't tried to simulate it [add another item to the todo list ]
post #213 of 2123
Thread Starter 
Quote:
Originally Posted by Thomas J. Coyle View Post

limetech,

I checked the Intel website for the indicated motherboard part number. Intel has the following generic part number descriptions on their website:

D865GLC: micro ATX

D865GBF: full size ATX

I assume that adding the additional "L" to the part number adds the optional LAN capability. Intel indicates that the LAN option can be either 10/100 or 1000. Does the "K" term indicate the LAN speed as they do not indicate anything in the board description?
Regards,
TCIII

I haven't been able to figure out Intel's model numbering scheme, but I believe that's correct. You definitely want to get the one with the "LK" at the end of the model number - that indicates it has the PRO/1000 GigEthernet chip.

BTW - I really really really like Intel motherboards for these reasons:
1. Their quality and price is as good as anyone else
2. Their documentation is outstanding. This is reason enough to exclusively use them.
3. Their tech support is actually helpful. Early in this project we noticed a flaw in the bios where, after setting it for USB boot, if you added or removed a hard drive the bios would "revert" to hard drive boot. I sent them an email detailing this "bug" and got a reply that they would "look into it". Well, in a later bios release guess what? They actually fixed it! Don't know if was because of my email, perhaps they knew about this already, but that is true support.
post #214 of 2123
Thread Starter 
Quote:
Originally Posted by Birdman71 View Post

Madpoet,

It's that master/slave failure that people have talked about in the past that has prevented me from rolling out an array in the past. After reading Tom's post I wonder if this issue isn't a problem in regards to their software. Maybe (pure speculation) RAID 5 is the key ingredient to the master/slave problem? If Raid 5 has to write data to every drive in the array for every write event, maybe a failed drive (master or slave) on a channel prevents the other same channel drive from writing (due to write caching). If the array controller senses that both drives had write failures it may flag them both as failed? If this is the scenario than the potential still exists in the unraid softare, but only on the shared parity drive channel. Since the main data drives are treated independently the system would never try to write to a maste/slave config for the other ten data drives.

Again, pure speculation on my part. Hopefully someone with a better understanding of this will chime in.....inquiring minds want to know.

Very good analysis.

Quote:
I'm pretty excited about this application. I've been following the NAS threads under the media server forum and like the expandability options here with 12 drive slots v. 4.

Any discussion of UnRAID over there?

Quote:
Tom,
What's the upgrade path from PATA to SATA for people that purchase now?

Do you mean for those who purchase a complete server? If so, you could take it apart and replace some/all of the disk trays with SATA versions along with replacing the PCI disk controller cards.

If you mean for the software-only product, then SATA support will be a free upgrade.

Quote:
Also looking for more information regarding the mixing of PATA and SATA in a system once you have the SATA version finished. How many SATA ports are on the MBs you are recommending and are you planning on supporting using both PATA and SATA MB channels at the same time?

Birdman (still trying to drum up support for a powerbuy)

There's no fundamental reason why you couldn't build a system with a mix of PATA and SATA drives. You would have to decide how many of each type you want and their slots. It's a bit complicated creating the correct "configuration" file that tells the UnRAID software which devices correspond to which slots - but definitely do-able.

I don't anticipate offering any preconfigured products which mix PATA and SATA, but you could certainly roll your own.
post #215 of 2123
I must be a lot less curious than the average AVSForumer. You guys are keeping Tom way busy with questions . Being the impatient type I just went ahead and bought one. I installed 3 500gb drives when I first got it. Installation went fine and I had a terabyte of space to fill. I started ripping movies and realized that I needed more space (I have 600 movies), had some "extra" paypal money and ordered 2 more 500gb disks. When I went to install them, I couldn't see the utility to stop the array (I could , however, see the files on the 2 drives). I looked on the website for help for my problem and didn't find any. It was Saturday afternoon, and my 11 year old daughter was expecting to watch a movie that night, so I panicked and shut off the power supply. When I powered back up, I couldn't see the utility or the files on the drives. Now I was in trouble. I sent Tom and email, and an hour later he called me and quickly fixed my problem. Great service, especially on a Saturday. The only downside was that it took nearly 3 hours before the array was back up. I remember the website saying something about 400gb in 45 minutes....

For my first performance test, I tried playing a movie and ripping one at the same time, and the playback stutters. However, playing 2 movies from the same disk at the same time worked fine.

p.s. Compusa had 500gb on sale for $309, with a $40 rebate so I bought 2 more... it's a disease
post #216 of 2123
I have 4 PATA and 8 SATA drives right now. I didn't realize I could put them both in the same box / system. Would loading all 12 drives in a single system be hot? Would one PATA controller card and one SATA controller card cover me?
post #217 of 2123
Quote:
Originally Posted by 3-way View Post

I have 4 PATA and 8 SATA drives right now. I didn't realize I could put them both in the same box / system. Would loading all 12 drives in a single system be hot? Would one PATA controller card and one SATA controller card cover me?

I'm thinking you'd go 0 PATA controller card and 2 SATA controller card. For the PATA, you'd use the on-board connections, and each of the SATA cards would give you 4 connections. (Assuming no on-board SATA).

Then again, I'm guessing ...and trying to get my number of posts up.
post #218 of 2123
ohlwiler,
Have you received your motherboard and processor from zipzoomfly.com yet?
Regards,
TCIII
post #219 of 2123
I received my motherboard today. The motherboard came with the back plane cutout, floppy cable, ide cable and driver CD. Testing will have to wait until my case arrives next week. It was my first time ordering from Zipzoomfly. I usually buy from Newegg. Everything went smooth.

It looks like the software from Lime may be the last item to arrive.

Scott
post #220 of 2123
Tom,

Would something like this work in the IDE version of Un-RAID?

http://www.firewire-1394.com/ide-sat...ter-bridge.htm

This could probably allow people to run SATA and IDE drives mixed together.

-Noel

*Update*

OOPS, it says Windows OSes only. I guess it needs drivers. Nevermind.
post #221 of 2123
Thread Starter 
Quote:
Originally Posted by justmikie View Post

I must be a lot less curious than the average AVSForumer. You guys are keeping Tom way busy with questions . Being the impatient type I just went ahead and bought one. I installed 3 500gb drives when I first got it. Installation went fine and I had a terabyte of space to fill. I started ripping movies and realized that I needed more space (I have 600 movies), had some "extra" paypal money and ordered 2 more 500gb disks. When I went to install them, I couldn't see the utility to stop the array (I could , however, see the files on the 2 drives).

One thing we need to work on in the s/w is better feedback about what's happening during an operation. Some operations take "a long time" and it can appear that the system is locked up when in fact it's still crunching away. For example, to "mount" a reiserfs file system it takes close to 45 seconds once the disk capacity is over around 200GB. Now when there's multiple drives to mount we try to do them in parallel, but it still takes almost 45 sec times the number of drives to complete (seems to be highly CPU-bound). Anyway, my point is that sometimes, especially when adding drives, it's possible to get into a state where it appears things have ground to a halt. I'm not saying this happend in your case, but other people have, shall I say, pointed this out to us as a problem to be addressed.

Quote:


I looked on the website for help for my problem and didn't find any. It was Saturday afternoon, and my 11 year old daughter was expecting to watch a movie that night, so I panicked and shut off the power supply. When I powered back up, I couldn't see the utility or the files on the drives. Now I was in trouble. I sent Tom and email, and an hour later he called me and quickly fixed my problem. Great service, especially on a Saturday.

I also have a 9 year old daughter who expects to watch her movies when she's been told she can, and is very upset when that can't happen...........

Quote:


The only downside was that it took nearly 3 hours before the array was back up. I remember the website saying something about 400gb in 45 minutes....

The system was clearing your new drives before adding them to the array. The current code executes this "clearing" operation before starting the array because it was simplest and safest to do it this way - but we'll fix this in another release. Another way to add new drives is to first "initialize" the array configuration data (via Tools menu), in effect making it think all the drives are "new". Then when started, the shares become available much faster and a backgroun operation is started to regenerate parity. A problem doing it this way is there's no recovery if a drive fails during this operaiton.

Quote:


For my first performance test, I tried playing a movie and ripping one at the same time, and the playback stutters. However, playing 2 movies from the same disk at the same time worked fine.

I'd be interested in knowing some more details about your first test:
a) was ripping done on same PC as viewing?
b) was ripping writing to same disk as the one being read for viewing?
c) what's your network configuration and speed?
d) anything else happening on your network?

Quote:


p.s. Compusa had 500gb on sale for $309, with a $40 rebate so I bought 2 more... it's a disease

That's a great price, but man, I hate dealing with rebates....
post #222 of 2123
Thread Starter 
Quote:
Originally Posted by 3-way View Post

I have 4 PATA and 8 SATA drives right now. I didn't realize I could put them both in the same box / system.

We haven't actually tried this, but it should work Note that our preconfigured server is only all-PATA or all-SATA. You would have to build your own system and use our s/w.

Quote:


Would loading all 12 drives in a single system be hot?

You definitely need proper cooling; however, the system will generally run cooler than you might expect because most of the time most of the drives can be spun down.

Quote:


Would one PATA controller card and one SATA controller card cover me?

You can use the 2 on-board PATA controllers for 4 PATA drives and either a single 8-port SATA controller or two 4-port SATA controllers. We're currently testing a couple 4-port controllers.
post #223 of 2123
Thread Starter 
Quote:
Originally Posted by musicmann View Post

...
Then again, I'm guessing ...and trying to get my number of posts up.

Me too
post #224 of 2123
Thread Starter 
Quote:
Originally Posted by ohlwiler View Post

...
It looks like the software from Lime may be the last item to arrive.

cott

All the Flash orders are being shipped tomorrow (Friday). You will get a tracking number and email. I apologize for the delay. When we only offered complete servers it was sufficient and desirable to have Friday be "shipping day" - and that is when the FedEx pick up occurs for us. This week, all the other orders will also get shipped tomorrow but next week we'll start shipping within a day of receiving the order.
post #225 of 2123
Tom,

What is your s/w licensing policy going to be? Will we have to buy a license for each machine that the s/w is on or will you allow more than one up to a reasonable amount? Maybe a discount for additional licenses?
Regards,
TCIII
post #226 of 2123
ohlwiler,

Thanks for the update. I noticed that the D865GLCKL was back in stock at newegg.com, but the price is substantially more than zipzoomfly.com so I ordered from them. Should have the parts on hand by Tuesday of next week. I plan to run the s/w on both my Asus A7N8X-E Deluxe and the Intel motherboard as I have two DVD servers at the moment. I have asked Lime Technology what their licensing policy is.
Regards,
TCIII
post #227 of 2123
Tell me if I have these scnearios correct.

Time goes by and all drives have spun down.
I read a file from one drive - only the drive I want to read from spins up and the file is read.

Time goes by and all drives have spun down.
I write a file to one drive - ALL drives spin up to calculate parity, the file is written to the target drive, and the parity info is written to the parity drive.
post #228 of 2123
Hi,
Given that most of my storage is on external USB 2.0 and Firewire devices, is it possible that your software would work with these external drives. All the data is 'static' i.e it never changes once it has been put on an external drive. Can I run the S/W in offline mode. By this I mean, can I just calculate parity once and be done with it until I need to add an additional drive to the array? This way the parity drive can be off unless needed to either restore data or recalculate parity.
post #229 of 2123
I think I can answer these
Quote:
Originally Posted by kchad View Post

Tell me if I have these scnearios correct.

Time goes by and all drives have spun down.
I read a file from one drive - only the drive I want to read from spins up and the file is read.

Correct.

Quote:
Originally Posted by kchad View Post

Time goes by and all drives have spun down.
I write a file to one drive - ALL drives spin up to calculate parity, the file is written to the target drive, and the parity info is written to the parity drive.

Not correct.

No need to spin up all the drives since it is easy to recalculate the required parity by reading the existing contents of the blocks on the parity drive and the drive being written to and comparing their old values to the new data being written. We do need to read the old contents of the parity drive and the data drive for those blocks being written to get their old values first for the comparison.

Therefore, only the parity drive and the one data drive need to spin up.

At a single bit level, if the bit on the parity drive was a "1" and the old data bit was a "0" and the new data bit value also a "0" then the value of the parity bit does not need to change since if the number of "1s" at this bit postion across all the drives did not change. It would also not need to change if the old and new data bit values were both a "1".

If however, the old data bit was different than the new data bit (0 to 1, or 1 to 0) then the value of the parity bit would also have to change (from whatever it was to the opposite value) to keep an even number of "1s" across that bit position in the unRaid array. In this example it was originally a "1" so it would have to change to a "0".

Joe L.
post #230 of 2123
kchad,
Having to first read any data block before writing does lead to an interesting way to improve performance (at least sometimes)

If I'm editing a large MS-Word document and only change one word to correct a sentence and then re-save it to disk the odds are that most of the data blocks it will occupy on the disk are unchanged.

Therefore, when we perform the read of each old data block and then compare it to the new we don't need to access the parity drive at all if the old and new data were identical. We would not need to write anything to either as no changes would need to be written to either disk. In fact, we would not need to even go through the process of calculating parity for that block since it too would not change if the data was not changing.

It would not speed up storage of media (movies/music) much since they are usually not re-saved after editing, but would speed up some tasks if you use the unRaid server as I expect to for storage of MS Word documents where continued editing sessions and periodic saves to disk occur.

I wonder if the unRaid uses this technique to possibly speed up writes?

In other words, if there is no need to write a block of data, don't write it.
(seems obvious to me)

Joe L.
post #231 of 2123
Anyone got a good source for the drive trays, or a better version?
post #232 of 2123
I have to ask, is there a specific feature of this motherboard that makes it better than others for this task? I've got a couple other Intel-based motherboards around, and wouldn't mind saving $100 or so. GigE support and everything
post #233 of 2123
Quote:
Originally Posted by madpoet View Post

I have to ask, is there a specific feature of this motherboard that makes it better than others for this task? I've got a couple other Intel-based motherboards around, and wouldn't mind saving $100 or so. GigE support and everything

Back on page 1 of this thread Tom said:
Quote:


We use an Intel motherboard which is based on the 865G chipset. One thing nice about this chipset is that there is a "Dedicated Network Bus" on the northbridge chip which provides a direct data path for the gigE chip straight to memory, bypassing the PCI bus (where all the disk controllers sit). So network activity does not contend with disk activity.

Sounds like Tom used it as it supported better performance than other designs where disk-io had to share the PCI-bus with network traffic from an ethernet card. (and, of course, it supported booting from the USB drive)
post #234 of 2123
maybe other 865 chipset boards would work.... Asus makes the P4P800 series which I believe has the same set up...

post #235 of 2123
I would tend to agree with Jim, than boards based on the 865G chipset, or others, such as the 875P or E7210 chipsets would have this feature.
http://www.intel.com/products/chipsets/865g/
http://www.intel.com/products/chipsets/875p/
http://www.intel.com/products/chipsets/e7210/

I mention the E7210, because I have been looking for a mobo with 64-bit PCI to support the wider bandwidth needs of PCI SATA cards with a higher number of ports. This is a server-class board, but still supports 478 pin CPUs.
post #236 of 2123
madpoet,

Newegg.com sells the Cremax mobile docks for $15.99, but they want $5.99 to ship each one.
I personally like the Kingwin KF-101=IPF in beige or black. Zipzoomfly has the Kingwin beige version for around $22.50 and the black version for $23.99 with free shipping for either.
Also, CompUsa has a version of the Kingwin KF-101 that they sell under their own brand. However, instead of having a 60mm bottom fan like the Kingwin, it has a 40mm front and 40mm rear fan and goes for $19.99.
I only buy mobile docks that have metal trays and metal docks as they transfer heat better than the plastic kind. As we all know, heat is a hard drives worst enemy.
You might want to search on eBay and see if you can make a bulk buy of mobile racks.
Regards,
TCIII
post #237 of 2123
I really like un-RAID concept and I'm getting pretty close to pulling the trigger on a DIY box based on the lime's OS. (A power buy would probably push me over the fence. ) Anyway, while researching components, I ran across the following page:

CM Stacker

What's the difference between the various Coolermaster Stacker variants (STC-T01, RC-810, and RC-820)? I guess the obvious difference is that the "new" 810 supports ATX boards and the (also "new") 820 supports BTX boards... but since the original STC-T01 supports both ATX and BTX, why would anyone want one of the "new" cases?

Comparison
post #238 of 2123
xact,

I have two of the regular CM Stackers and must say that they are quality cases and they better be for the price that they get for them. Unless you plan to run a BTX motherboard sometime in the future, the regular Stacker should suit your needs. I have installed the cross-flow fan (used an 80mm temp control fan) and have the 4 into 3 chassis and its accompanying fan mounted in the front in the empty spaces where I do not have mobile racks at the moment. CM provides everything that you need except for the cross- flow fan.
Regards,
TCIII
post #239 of 2123
Quote:


Anyone got a good source for the drive trays, or a better version?

I looked around for the best price on the Cremax MB123AK drive trays. The best I found was from Census PC. For quantity 12 the price was $22.47 each with shipping. In the end I decided not to go that route primarily because of the cost. Instead I decided to order extra drive cages for the Cooler Master case. Changing out a drive will be a little more difficult, but still doable. One other thing that pushed me away from using a removeable drive tray was the claims I read on usenet about removeable drive trays not conforming to ATA-100 standards. It isn't too often that the cheaper solution gives possibly superior results. If I was using SATA drives I would probably use drive trays.

Scott
post #240 of 2123
I just noticed that the Newegg shipping on the Cremax MB123AK drive trays isn't quite as bad as I thought. To ship 12 trays they only want $54.32. Average cost each comes out to $20.52.

Scott
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers
AVS › AVS Forum › Video Components › Home Theater Computers › Limetech un-Raid Media Storage Server support thread