Guide To Building A Media Storage Server - Page 193

Originally Posted by kapone

Nah...my stack is better... and I have two of these.....

Originally Posted by mitgib

Couple of quick questions. Took a couple days to get caught up on this thread, but I got a few ideas, and before jumping to any conclusion, a little counter thought might prove useful.

My current media server is WHS with an OS drive on the motherboard controller, then a 3ware 9690SA to a Chenbro CK12803 with 15 1tb drives in RAID5, I'm exporting in 2TB slices to the OS since WHS is fairly retarded about large arrays.

I'm down to 4tb of free space and at the rate I purchase Blu-Rays, expansion is not far off. I caught up on this thread to hopefully get an idea about who is selling drives cheap, well that didn't happen for me with 1tb drives, but I did get an idea about 2tb drives seem to be quickly dropping in price and per byte costs are now well under 1tb drives, so while I would need to expand to a 2nd chassis, populate it with 2tb drives, then remove the slices from the original RAID5 array, WHS would migrate the data to the new array using the 2tb drives, and rebuild/expand the original chassis with more 2tb drives and sell off the 1tb drives for whatever I can get.

Now this will keep my data intact while migrating it, but this also needs to hopefully solve an issue I've had ever since I built this box, which is that dang balancing act WHS does every hour. If it happens when I am watching a movie, the stream stutters for a short period then back to normal, if I move the data to the local node playing it, no issue, so it is deffinatly WHS. My thought was could/should I build this new array of 16 2tb drives as RAID6/10 (or 2 8 drive arrays for that matter). I'm looking for a smoother stream, and obviously more space.

I just recieved a Chenbro UEK and before I order anything else, just want to explore my options.

Originally Posted by pclausen

So I've been trying to get my 1680x controller to see my E1 for the last couple of weeks, and I just don't think it's going to happen.

I've had both my TQ and E1 on the bench for some time now. The 1170 and all its 24 drives have been pulled and I have tried the 1680x in all my PCIE slots by itself, and no go as far as the E1 being picked up.

I have tried with drives in the E1 and without. I have tried booting the E1 before the TQ and visa versa, no go.

I've pulled the 1680x and left it out for several days with all power disconnected to both the TQ and E1.

I'm currently using a 1 meter 3ware SFF-8088 - SFF-8088 cable and have a 2 meter 3ware cable on order. However, I'm not at all optimistic that will fix me.

Also, the 1680x is clearly communication with the E1 as after the E1 is powered on, all the drives in the enclosure become active (the blue activity LEDs are constantly blinking), then when the TQ is powered up, the 1680x will "ping" each drive and turn off the blue LED, and then the 1680x BIOS lets the OS boot.

So I'm ready to move to Plan B, which entails picking up a pair of SFF-8088 to SAS breakout cables. This way I can install my E1 drives in my TQ enlosure and bypass that ^&*% SAS expander in the E1 enclosure all together. My hope is that this will allow me to bring my 6x2TB RAID6 array back online.

If that works, my next step would be to pick up a 3ware 9690SA-4IAE controller, and a bunch of Hitachi 2TB drives, and hope the 3ware will play nice with the E1. If so, once the new raid has been created, copy over all data from the old E1 drives mounted in the TQ via breakout cables.

If the 3ware won't work with the E1 either, I guess I'll need to re-evaluate my options, and talk to SuperMicro about what is actually supposed to work with the SAS expanders.

The really frustrating part is that I had no issues for a long time until I decided to power down my system...

Originally Posted by analogueaddict


The PCIe X4 socket has a theoretical maximum throughput of 1GB/s with PCIe 1.0 spec or 2GB/s at PCI 2.0 spec, the X8 is double that assuming the number of lanes to the socket on the motherboard is in keeping with the physical size of the socket (some boards supply X8 sockets but only run them at X1 or X4 speeds). The X8 socket design of the PERC 6/i board should accomodate 2GB/s throughput as it is only a PCI 1.0 spec board. I cannot find reference as to whether the Supermicro controller is PCI 1.0 or 2.0 so I would suggest conservatively it's a 1GB/s throughput card, so the PERC card has higher throughput. The question is whether you'll ever need that kind of performance?

I hope this helps with your decision, but suffice to say they are both very good controllers and I wish they were available when I was designing my system, I am sure the design would be very different to what I have now.

Best wishes,

Dave

Originally Posted by Sciúrus Arcânus

Hi, I really appreciate your input. My mainboard is a Gigabyte GA-H55M-S2H and the main PCIe slot is X16 and 2.0 while the other is X4 electrically but X16 physically and presumably 1.x spec (since Gigabyte does not specify as they do with the true X16).

So, based upon your details that seems to mean that either card will work in the X4 slot but the PERC's performance potential would be halved while still effectively the same as the MV8 (alternatively, the PERC could be used to its full potential in the true X16 slot)?

A note in the Gigabyte manual seems to state that the true X16 slot operates at up to X4 in CrossfireX mode (i.e. when the X4 is also populated with a GPU). If that is so, would it affect non-GPU cards -i.e. if a GPU is installed in the X16 slot and a storage controller in the X4 slot then will the GPU performance suffer?

My goal is to cost effectively and compactly run more media drives while just maintaining drive-to-drive performance similar to the onboard controller rather than reducing it with port multiplier or PCI cards.

Originally Posted by stevetoney

another drastic step would be to try the MB from the TQ in the E1

or get a non raid SAS SM card and run some 8087 cable out to the e1 just to see if you can see the E1.

another test would be to pull your OS drive - set aside and use another drive for a clean OS install just to check, but as you noted recognized vol's show up at boot before OS

it is rare but possible that something on the HBA port, cable path, or expander port backplane simply went out so it is not fully working

Originally Posted by MiBz

There's a much better way, without having to break things up into 2TB LUNs, and you'll also avoid anything to do with WHS's storage pool for Video storage.

Build your new array using 2TB drives.
Remote desktop into WHS, right click on my computer and select manage.
Select Disk Management and you'll see the new array volume which isn't initialized and shows as unallocated.

Right click on the grey area of the volume, intialize as GPT volume.
Format it NTFS |64k clusters | Assign a drive letter above D (ex: R)
Give it a name (ex: RaidStorage)
Exit from Disk Management

Go to My Computer and the R drive now appears as a drive letter.
Open it
Create a new folder, call it Movies or RaidStorage (or whatever you like)

Open the link to WHS's folder shares and right click on the Video share.
Note the share security settings for each group.
Administrators
Creator Owner
RO_5(WHS\\RO_5) - this is the group containing users with read only access
RW_5(WHS\\RW_5) - this is the group containing users with read write access.
System

Now go back to the folder you created on the array drive
Right click, select Share and set the share security to the same as Video noted above.

Now when you access WHS from any network client you will see a shared called Movies (or whatever you called it) that can be accessed same as all other shares.

The additional step to assign the same share rights and groups as WHS's Video folder means that any users you add or changes to those users you make in the WHS console that you give access to Video will also apply to this folder so you'll never need to do it manually.

Last step in remote desktop
Open the WHS shared folder's link and open the Video folder.
In my computer open the drive letter you assigned to the raid array.
Now drag/copy the movies from the WHS Video share to the raid array.

You can also temporarily do the same for all other data that's on the WHS drive pool.

Once the drive pool is empty, you can start removing each 1TB LUN from WHS's disk management, but before you do this make sure you have at least 1 1TB single drive in the drive pool that WHS can move data like client backups to when it prepares to eject the raid LUNs.

At that point you have all your video files safely on the raid array outside the WHS pool and you can still use 2,3,4 x 1TB drives (with duplication on) for your pics, music, documents and whatever else you want.

Setting up the LUN, mapping it as a disk, and putting your file shares on it is great, but there are a few configuration steps you need to take if you experience your share settings disappear every time you reboot (even thought the files remain).

First, ensure that the Server service is dependent on the Microsoft iSCSI Initiator Service. To do this, go into the Services MMC, open the Server service properties, and check under the Dependencies tab. No Microsoft iSCSI Initiator? Open Regedit (Run - regedit.exe) and navigate to HKEY_LOCAL_MACHINE\\System\\CurrentControlSet\\Services\\lan manserver. Find the key "DependOnService" and set it's value to MSiSCSI. The Server service is responsible for creating the shares; if this service starts up before the iSCSI LUNs are ready on the server, then the shares will not appear.

Second, make sure you set up the Microsoft iSCSI Initiator to automatically restore the connection and drive letters. Under the Targets tab, when you highlighted the target and clicked "Log On", did you check "Automatically restore this connection when the system boots"? If not, remove the connection and log it back on, this time selecting the correct option.

Originally Posted by kapone


I'm thinking low power, probably atom based motherboards (still need to find one with 6 SATA ports), or cheap mATX boards with low power CPUs with something like OpenFiler as the iSCSI target node software, hooked into a dedciated subnet on the switch, so that all iSCSI trafiic is isolated, and use WHS, modify it's services configuration, and then use iSCSI targets as disks in DE.

lol..I know, lots of moving parts, but hey isn't that the fun part? Watcha guys think?

Originally Posted by MiBz

I'm not so convinced an iSCSI backstore is a good idea for WHS. I see tremendous potential in file corruption and broken tombstones, but this is just my intuition based on experiences with iSCSI and knowing how WHS works. I always enjoy your experiments, so I look forward to it.

btw SuperMicro has some new Atom boards out with 6 SATA ports !

Originally Posted by MiBz

btw SuperMicro has some new Atom boards out with 6 SATA ports !

Originally Posted by kapone

Well, if the iSCSI implementation itself is stress tested and solid (and it is/should be) the "potential" for corruption is the same as that for local hard drives, as far as WHS is concerned. Now, DOES WHS have corruption issues with local HDs? Well, that's probably up for debate. It "seems" like with PP2 (and the upcoming PP3) they fixed some of the corruption issues (it doesn't even write to a buffer area anymore, straight to disk).

Yes, I saw that Supermicro board...still too expensive . We need something in the $70 range to be cost effective WITH 6 SATA ports (or more).

Originally Posted by kapone

I had started creating this schematic back in Sep' 08 but never finished working on it. I still think this is feasible..... with "enough" performance that it can easily saturate a gigabit link and provide redundancy at the node level. No expensive raid cards, no SAS adapters or backplanes. Simple ITX (or maybe mATX) motherboards in each node. Actually two of them in each node to provide 12 SATA connections for a 2U node box.

Originally Posted by the_beast666

I would still avoid Atom for this though. Pineview is better than the crappy 945GC, but still uses way to much power to be paired with an 8-11W cpu.

A decent AMD 785G board (such as this, with its 6 SATA ports uses little more power at idle but provides the expansion options required to make each node run well (add in an 8-port Supermicro HBA and a decent NIC - Pro/1000 PT or a hardware iSCSI card like a Broadcom NetXtremeII if Linux support is available yet for iSCSI offload). This gives you lots of storage drives and enough cpu to run software RAID6 on each node, all for similar money to a mid-range Atom board.

Originally Posted by kapone

The point of using gigabit as a transport is multiple SMALL nodes, not a few large ones. I'm focussing on "6" as the magic number for the number of SATA ports since that's about the ideal number for a RAID-5 array. Now, RAID-5 vs RAID-6 is a whole another debate, and I'll leave that out. But with multiple small arrays of 6 disks, yes, you take a hit on HDD costs, to have redundancy, but you gain better fault tolerance than a monolithic card and/or one big array.

Originally Posted by kapone

No no no.... That kills the whole "cheap" objective. That board itself is $85. You'll still need a CPU (since you need memory in either case, I'm leaving that out). And no addin cards. That's the whole point of having "low powered" boards with multiple SATA ports so that they ARE the equivalent of an add in card.

The point of using gigabit as a transport is multiple SMALL nodes, not a few large ones. I'm focussing on "6" as the magic number for the number of SATA ports since that's about the ideal number for a RAID-5 array. Now, RAID-5 vs RAID-6 is a whole another debate, and I'll leave that out. But with multiple small arrays of 6 disks, yes, you take a hit on HDD costs, to have redundancy, but you gain better fault tolerance than a monolithic card and/or one big array.

Originally Posted by pclausen

Appreciate the feedback Steve. Keep in mind that my 1680x has external connectors only. That being the case, I think I could achieve the same goal by getting a longish SFF-8088 to SFF-8087 cable and run it from one of the 1680x external ports, directly to the SFF-8087 connector on the E1 backplane.


Yes, I will likely try that as well in an effort to pin-point what is not working.


I did have plans to move from W7 x64 to 2008R2. I've been using R2 at work on my new VMs and really like it.


True, but it sure is proved a bear to troubleshoot. And at the drop of a hat, things might begin working again, with me/us being no closer to knowing why there were issues in the first place.

At work I'm playing with a stack of HP LeftHand P4500 SAN nodes, each communicating with a pair of switches via dual GigE connections. The whole stack is combined into a single cluster, giving me the ability to create a volume that spans all the nodes. Each node is comprised of 12 450GB 15K SAS drives, set up as a pair of 6 member raid 5 arrays. All these 6x450GB raid 5 arrays are then combined using network raid across all the nodes, and presented to the OS'es as an iSCSI target.

Physically, each node is a 2U HP DL180 running some heavily customized flavor of lunix and a SAS raid controller and proprietary LeftHand software.

I'd love to create a similar setup at home, using 2TB SATA drives and open source software. This way, I can loose an entire node and it won't impact my data.

Originally Posted by mitgib

I think this is a great idea MiBz, totally avoids the hourly balancing WHS does. I'm not very good with Windows, how would I create a storage area which comprises multiple partitions/arrays? Similar to LVM, I am a linux geek, why did I choose Windows for this? Or is this something I should avoid and just tell MyMovies different shares as I go along and fill arrays should I choose to do 8 drive RAID6 arrays?

Originally Posted by the_beast666

I would still avoid Atom for this though. Pineview is better than the crappy 945GC, but still uses way to much power to be paired with an 8-11W cpu.

Originally Posted by ohpleaseno

I know this is an old bump, but I just did a Norco 4220 and save for the RAM, I used exactly the same parts as you, but I never saw your post.

So far, the WD Scorpios in Raid 1 are working great. Data migration from the old WHS to the new one is a time-comsuming PITA though.

Originally Posted by FrankieFiero

How well do those Norco cables (C-SFF8087-D) work with the 4220 case? I am using the ones that came with my Areca card and they don't sit very well in 3 of the 5 backplanes, the fold of the sheet metal interferes with the latch on the cable. I just don't feel very comfortable unless I get that satisfying 'click' of the cable into the socket. A light tug is all it takes for the cable to come out.

Originally Posted by carpenike

Here's a potential alternative to WHS that's free/open source.:

http://www.amahi.org

Originally Posted by scientest

Interesting, have you tried it out?

Originally Posted by ilovejedd

Hmm, Amahi didn't really catch my interest but GlusterFS sure did. Gah, I see a lot of reading ahead. >.<