or Connect
AVS › AVS Forum › Video Components › Home Theater Computers › Guide To Building A Media Storage Server
New Posts  All Forums:Forum Nav:

Guide To Building A Media Storage Server - Page 17

post #481 of 7891
Quote:
Originally Posted by HappyFunBoater View Post

Very interesting. I think I've got a legal copy of this that I could throw onto Virtual PC and play around a little. Thanks!

One more quick question: Can the storage be USB? I like to take my backups offsite, so it would be nice if the client backup app would pause until the USB drive was present.

Drives can be added to WHS by SATA, eSATA, and USB it doesn't really care.
You can then decide if you want that specific drive to be added to the storage pool or not.

I'm not sure if you can specify that only the client images be kept on a USB drive or not, but I do know that there's a utility to backup WHS on an external eSATA or USB connected drive, so I would imagine yes.

BTW you can download a full functioning WHS install DVD from Microsoft that's good for 6 months here;
Just click on Test, then join connect (It's free). It provides you the download and a 6 month install key
post #482 of 7891
I am currently reusing an nForce4 motherboard (Abit AN8-SLI, 1.8GHz AMD 64+, 1GB RAM) for my Windows Server 2008. Even on a gigabit network, this server only managed to achieve a write speed of about 40 MB/s and read speed of about 20 to 30 MB/s. Hard disk is a Western Digtal SE16 500GB, so it should be able to handle more than 40MB/s.

To achieve read/write sustained speed of above 50 MB/s, do I need a server class motherboard like the Intel 3200 series? Or can I just slap in an Intel Pro1000 PCI network card?

I am using an unmanaged SMC gigabit switch. And my client PCs are AMD 780G and Intel P965-based chipsets, so they are fairly new technology.
post #483 of 7891
Quote:
Originally Posted by garycase2001 View Post

... Do you simply share that folder (e.g. "DVDs") for other PC's on the network to access??

You setup shares using the WHS console (very simple) and tell WHS what users you want to have access to that share and at what level (ie read only, full access etc). WHS does the rest.

Quote:


... and is it correct that you can simply add more disks to the storage pool without any user intervention needed in terms of assigning the disk to specific folders? (i.e. WHS just uses what it needs)

You toss drives into the system, whatever model, type, capacity etc.
Then you go to the WHS console and just click on the drive you added and tell it to add that capacity to the storage pool. WHS does the rest. There's no drives letters to worry about.


Quote:


And if duplication is NOT enabled for a folder, what happens when one disk fails? Does it maintain replicated directory info so it can show you which files were lost, etc.
This is looking like a very good option

It keeps track of tombstones of where files are stored, but if duplication is off, the data is not redundant. But you should always have backup of your data on a seperate system anyways. Nothing can replace that. Not even hardware Raid 5, 6 etc.

Quote:


If duplication IS enabled and a disk fails, how does WHS let you know WHICH disk failed?

Yes if you look at the pic I posted, you'll see the status column that indicates HEALTHY. This changes if there's an issue with a drive, which in most cases gets reported by SMART to the OS. Drives are pretty good about that now. Remember there's no risk of a drive falling out of an array to cause data loss or a parity based data loss, because there is no raid going on. You don't have any of those risks here. Just a if the drive itself is failing and most of the time this happens over time and SMART notifes the system. For this to work you to set AHCI in the bios and install WHS with AHCI drivers.

I installed 2 addins for disk management. One is called Disk Management which shows you these details;


The other addin is called Remote Notification, which emails you to let you know something is wrong. like a pending drive fault/failure that looks like this.
I sent a test notifcation the email went to my iphone in a few seconds.

Btw this pic is not from my system, it's from the we got served addin's page


Again, nothing is perfect and you should always have an offline backup of critical data you can't afford to lose.
post #484 of 7891
Quote:
Originally Posted by MiBz View Post

... I rip everything to it's native VOB (TS folders) using DVDFabPlatinum saving only the movie and 5.1 audio which is very easy, just point and click with DVDFab.

Agree DVD Fab makes it very simple. I've done that for a few hundred DVDs; but before I load the rest of my collection (~ 3000 DVDs) I'd like to decide for SURE how I'm going to store/index/access them. Right now I just open PowerDVD and then open the movie I want to watch -- but the wife would like a more point & click interface [Double-click on the movie's icon].

My question is more about how you browse & play the movies => Do you use "My Movies" (which apparently limits the interface to Media Center) ... just open a player (PowerDVD etc.) and then open the files ... use some other movie browser ...etc. I'm looking for the best WAF ... a nice icon-based browser that you could just double-click the icon to play the movie would be very nice --> just curious what others use.
post #485 of 7891
Quote:
Originally Posted by garycase2001 View Post

I'm looking for the best WAF ... a nice icon-based browser that you could just double-click the icon to play the movie would be very nice --> just curious what others use.

Umm..If you are using Vista...I don't think there's anything better than the....built in DVD Library.. Real slick. Been using it since Vista came out, never had an issue. Even my 6 year old nephews and nieces (and of course the wife) can use it without any issues.
post #486 of 7891
Quote:
Originally Posted by kapone View Post

Umm..If you are using Vista...I don't think there's anything better than the....built in DVD Library.. Real slick. Been using it since Vista came out, never had an issue. Even my 6 year old nephews and nieces (and of course the wife) can use it without any issues.

I'll play around with it a bit ==> can you import the library across multiple PC's so the same interface works on every PC in the house without having to recreate the library?

... and are you referring to the Media Player library or is it a Media Center function? [My HTPC's currently use Beyond TV due to its better sharing features than Media Center (Live TV streaming; easy access to all recordings anywhere in the house; etc.)]. However, I'll gladly switch to whatever works best. I plan to "play" a bit with both Media Center and Sage in the next few weeks. Meanwhile, I'm loading DVD's onto a few 1.5TB drives I just bought => I'll just copy them all to whatever I build as the main repository.

Although I'll probably build a 2nd system to hold the backups (simply for convenience), I'm also tempted to just use a stack of bare 1.5TB drives connected via a USB adapter whenever needed. For static content [a drive full of DVDs] that's an easy way to store the backups ... so I figure loading all my DVD's to those drives now isn't wasting any time !!
post #487 of 7891
Quote:
Originally Posted by garycase2001 View Post

I'll play around with it a bit ==> can you import the library across multiple PC's so the same interface works on every PC in the house without having to recreate the library?

... and are you referring to the Media Player library or is it a Media Center function? [My HTPC's currently use Beyond TV due to its better sharing features than Media Center (Live TV streaming; easy access to all recordings anywhere in the house; etc.)]. However, I'll gladly switch to whatever works best.

Assuming you have your DVDs ripped to a central location as VIDEO_TS folders, all it takes is a couple of registry changes, and boom, the same library, all nicely formatted and icon based, is available on any HTPC that is configured for it. You have to make the registry changes on each HTPC to enable the DVD Library and tell WMC where your files are.

Sharing Recorded TV is even easier. One registry change on the HTPCs and a couple of permissions on the server.

Sharing Music and Photos is..no registry change Just go into MC setup and tell it where they are.

What you CAN'T have is a central guide, unless you move to a WMC host PC and extender model. With just a bunch of Media Centers thrown around the house, that's what you can't do. I can live with it. The extenders while nice, are too restrictive.
post #488 of 7891
Quote:
Originally Posted by garycase2001 View Post

I'll play around with it a bit ==> can you import the library across multiple PC's so the same interface works on every PC in the house without having to recreate the library?

If you have a WHS box, the new mymovies addin for WHS can be used to populate xml data for the DVD library if you don't want to use mymovies media center plugin for a movie browser.
post #489 of 7891
Quote:
Originally Posted by kapone View Post

Assuming you have your DVDs ripped to a central location as VIDEO_TS folders, all it takes is a couple of registry changes ...

... Sharing Recorded TV is even easier. One registry change on the HTPCs and a couple of permissions on the server.

Could you provide details on these changes? (or a link)

Another thought/question: Since you can set monitored folders, does this mean the library can be scattered on multiple disks/locations?? i.e. could it simply be scattered around on a bunch of 1.5TB disks and still be seamless from the user's perspective??
post #490 of 7891
Not trying to get out of subject here but seems like there is a lot of knowledgeable people here so i figure it would be ok to ask if not, im sorry. I just finished putting together a Nas server using an old P4 that i decided to put out of service a couple of months ago. Its a P4 1.8Ghz with 512mb , a Promise Sata300 made by promise and so far one 1TB Seagate Hard drive. Btw im running freenas on it. So far is been working great, no problems at all but my question is about RAID. If i want to add a second hard drive (same size) but put it on a raid configuration that means i would have to backup the first drive first somewhere else and then create the array? also if in the future i want to add a third hard drive to the array how do you do that? I'm sorta new to the whole media server deal so you guys help would be really apreciate it.
post #491 of 7891
Quote:


Could you provide details on these changes? (or a link)

The changes are scattered all over the web, I'll try and post a consolidated list of links.

Quote:


Another thought/question: Since you can set monitored folders, does this mean the library can be scattered on multiple disks/locations?? i.e. could it simply be scattered around on a bunch of 1.5TB disks and still be seamless from the user's perspective??

Absolutely. Doesn't have to be a single disk/volume at all. Can be as many as you like, scattered on as many computers on your network (or local for that matter) as you want.
post #492 of 7891
Quote:
Originally Posted by garycase2001 View Post

Although I'll probably build a 2nd system to hold the backups (simply for convenience), I'm also tempted to just use a stack of bare 1.5TB drives connected via a USB adapter whenever needed. For static content [a drive full of DVDs] that's an easy way to store the backups ... so I figure loading all my DVD's to those drives now isn't wasting any time !!

This unit is great for quick -n- swap with USB or eSATA

iStarUSA vAge120-SAU Tray-Less USB 2.0/ eSATA Combo-to-SATA External Storage System
http://www.newegg.com/Product/Produc...82E16816215055
post #493 of 7891
i would disagree.

the My Movies interface is magnitudes better imo.

and with all of the things you can do in the collection management program, and the ease of finding and starting a movie, i would say the WAF is pretty high.

I set it up for my parents, and their approval rating is that it is perfect.
post #494 of 7891
If you are willing to use a system that is still in development and can have some quirks to it, XBMC is the slickest looking system around. It's currently in beta, but is pretty stable at this point. You can run it on an xbox, linux, os x / apple tv, and windows. It can take a little bit to get it up and going intially, but once you do it is a pleasure to use. It really makes VMC look like worthless crap. VMC is super easy to use and setup though.
post #495 of 7891
where are you guys getting your copies of windows server 2008 standard? I cant justify £500 on an operating system for home use when opensuse or fedora will do the same job.

@ Kapone have you tested the ciprico cards on *nix machine? I have 4 of these cards that i got for a steal £100 for all four, should have my server setup this week to test these cards, just need to find the time. My server looks an awful lot like yours kapone I pretty much picked all my hardware before i read your build log :P

Spermicro 4U Chassis CSE-846TQ-R900B
Supermicro X7DWE Motherboard
2 X Intel Quad Core Xeon E5405
4GB Ram
4 X Ciprico 8 port pci express cards
24 Samsung 1TB F1 Hard Drives
post #496 of 7891
Everyone keeps saying that backups on a separate machine are the way to go, but what are people here really doing?

I see that the OP is backing up critical data to a smaller array, but not movies, but for those who've backed up 3000+ DVDs and soon BDs, what options are there for onsite and offsite backup? I understand tape is basically dead because it can't keep up with HD size, so is the only solution to build two of your storage server?

I guess I'm trying to reconcile what I should do with what people are actually doing.

Thanks,

Phil/TW
post #497 of 7891
Quote:
Originally Posted by Torquewrench View Post

Everyone keeps saying that backups on a separate machine are the way to go, but what are people here really doing?

I see that the OP is backing up critical data to a smaller array, but not movies, but for those who've backed up 3000+ DVDs and soon BDs, what options are there for onsite and offsite backup? I understand tape is basically dead because it can't keep up with HD size, so is the only solution to build two of your storage server?

I guess I'm trying to reconcile what I should do with what people are actually doing.

Thanks,

Phil/TW

My strategy is to get your "primary" storage server, RIGHT, and then stop worrying about it.

My strategy is to have atleast one mirrored array in the server and the rest is one (or potentially more) huge RAID array (RAID 5/6 whatever).

My "critical" stuff is on the mirrored array (Pictures, Music, Documents, Software code etc) and all of the non critical things (like Videos, Recorded TV, DVD rips etc) are on the RAID volume(s).

All of the Video stuff is non-critical in the sense, that I can always recreate/re-rip it if need be, however painful that might be. And to not have to do that if a single disk fails (and they WILL fail), is why they are on a RAID volume.

First level of protection - RAID
Second level of protection/backup - the disks themselves

That's way more than enough. There's no offsite backup in existence that can handle this much data. It'd take me years to backup my storage to an offsite system. Unless Comcast implements Gigabit to the house and the backend pipes are something resembling monster sewer sized fiber optic pipes, offsite backup just aint happening.

We buy our DVDs/BDs etc, they get ripped straight away, and are then stored in the closet, pretty much in pristine condition. The discs themselves see very little to none usage after the initial rip. As far as Recorded TV is concerned, even if we lose all of it, it can pretty much be re-recorded when the reruns come on, or watched online from the various stations.

Getting a stable storage setup is critical. Once you have all the disk monitoring, alerts etc in place, you're in good shape. That being said, understand that when you're talking this level of data storage, you're way above even what small companies maintain. So, your hardware/software better be robust, because that IS a huge amount of data.
post #498 of 7891
Quote:
Originally Posted by kapone View Post

My strategy is to have atleast one mirrored array in the server and the rest is one (or potentially more) huge RAID array (RAID 5/6 whatever).

My "critical" stuff is on the mirrored array (Pictures, Music, Documents, Software code etc) and all of the non critical things (like Videos, Recorded TV, DVD rips etc) are on the RAID volume(s).

All of the Video stuff is non-critical in the sense, that I can always recreate/re-rip it if need be, however painful that might be. And to not have to do that if a single disk fails (and they WILL fail), is why they are on a RAID volume.

First level of protection - RAID
Second level of protection/backup - the disks themselves

This ia a great strategy. For those who are extra paranoid or require access to critical data during downtime or maintenance, sync only the critical data to another smaller machine.

This is what I do with my critical data.
I.E. rsync certain volumes to another smaller ITX unRAID server.

Other thoughts:

What I like about the unRAID Protected JBOD strategy is data being compartmentalized at a disk/filesystem level even though there is a single shared view.

With this I'm able to stop the array, remove and replace a disk, then rebuild the removed disk on the replacement disk thereby having a chunk of my data available for mobility, storage or offsite backup.

These disks/filesystems use a standard linux reiserfs per unit allowing mounting and use on other machines capable of reading them.

By using the trayless removable sata units, You can almost swap hard drives like floppies.

Of course there is also the eSATA or USB option for critical offsite data.
See my prior post of an example external trayless sata removable.
post #499 of 7891
Quote:
Originally Posted by TheRonin View Post

where are you guys getting your copies of windows server 2008 standard? I cant justify £500 on an operating system for home use when opensuse or fedora will do the same job.

Mine's free from the University (both Server 2003 & Server 2008 Standard).

Quote:
Originally Posted by TheRonin View Post

@ Kapone have you tested the ciprico cards on *nix machine? I have 4 of these cards that i got for a steal £100 for all four

That a very good price you got it for.
post #500 of 7891
Has anybody ever used one of these?

http://www.sansdigital.com/towerraid/tr8mb.html


I've been looking over the 8 bay device, and while it is somewhat limited compared to a full on server with RAID control card, it appears to me to have many of the basics covered.

RAID levels:
0, 1, 10, 5, JBOD, Spanning

Multiple RAID volumes with a maximum of 5 disks per volume (boo)

It is software RAID, however that also means it has the advantages and disadvantages of software RAID which are user dependent for qualifying.

its an easy connections setup to add to an existing computer at good throughput speeds. (much better than the crappy ass "NAS" devices sold by many companies that are either USB or severely gimped ethernet connections).

as an easy to setup, basic storage system, i'm wondering if it doesn't make a lot of sense for those that want a low end basic system versus building something computer based, that requires a somewhat expensive case and motherboard.

the 8 bay unit retails for $399, and i've found it as low as $359.

might be something for some to consider.
post #501 of 7891
i should add that as with many software raid setups, on the fly expansion is not supported that i've found, however it does support multiple RAID volumes, so a user could create second and third volumes as discs are added.

it also has auto rebuild, supports individual drive SMART reporting, allows for global or dedicated spares to be setup (including multiple on one physical drive), orphan reporting and deletion, ability to easily break an array that is mirrored and replace individual drives than rebuild, and notification of array status.
i havent played with one, just been leafing through the user manual online.

appears to be a pretty good easy to use GUI (proprietary software to setup volumes), and the user manual is full of actual ENGLISH sentences and has a ton of screen shots as well.

Supports win2k, WinXP(32&64bit), WinSrvr2k3(32&64bit) and WinVista(32&64bit(, also Mac through OS10.5.1 and linux as far as i can tell.

not bad.

beats the hell out of a bunch of usb drives.

not sure it would replace WHS, and definitely not a dedicated server, but for those that want decent but inexpensive, it might be a nice option.
post #502 of 7891
Quote:
Originally Posted by TheRonin View Post

where are you guys getting your copies of windows server 2008 standard? I cant justify £500 on an operating system for home use when opensuse or fedora will do the same job.

i have several that i received from microsoft events that i attend.

same as i have several copies of vista ultimate.

i also have a technet subscription for my business i maintain.

the technet subscription is the best way if you want to try out the microsoft products, and my home system is my test system as well, so i'm okay with their EULA in my opinion (and dont care about anybody else's).

i maintain a small server farm (mostly virtual) to do my testing for work at my house and mostly run the technet stuff there, but adding a copy of 2k8 toi my personal server is just another test to me.
post #503 of 7891
Just wanted to respond to the posts about mymovies and WHS. I used to install the server on WHS and the client on my HTPC. Then I just put with the client and the server on my HTPC and just pointed it to the files on the WHS and it works great.
Whenever I copy a movie over to the appropriate share mymovies will pop up on my HTPC asking if I want to add this new movie it found and then it downloads the cover art, actors, director, etc. It's great. And WHS keeps my HTPC backed up. I actually used that feature to restore it once. Saved me hours of re installation and configuration.

For sharing what I did was setup a HTPC account on WHS with read only access to the shares so no one accidentally deletes any files.
Then on my HTPC I set the account to the same username/pass for the account I just made on WHS. Then I just tell VMC and Mymovies to monitor those shares.

I don't get anywhere near the transfer rates as posted in the screen shots. Pre PP1 I was getting anywhere form 10MB/s to 20MB/s. After PP1 I get 50MB/s to 60MB/s. I also installed a Intel Pro 1000GT on the server. It's an old board, I don't know if that makes any difference(Asus P4C800-E deluxe). My router is a D-Link DGL-4300, connection is wired.
Maybe it's the drive speeds? My drives range in size from 250gb to 750gb. I really need to add more but can't afford it right now.
post #504 of 7891
Quote:
Originally Posted by archer75 View Post

Just wanted to respond to the posts about mymovies and WHS. I used to install the server on WHS and the client on my HTPC. Then I just put with the client and the server on my HTPC and just pointed it to the files on the WHS and it works great.
Whenever I copy a movie over to the appropriate share mymovies will pop up on my HTPC asking if I want to add this new movie it found and then it downloads the cover art, actors, director, etc. It's great. And WHS keeps my HTPC backed up. I actually used that feature to restore it once. Saved me hours of re installation and configuration.

For sharing what I did was setup a HTPC account on WHS with read only access to the shares so no one accidentally deletes any files.
Then on my HTPC I set the account to the same username/pass for the account I just made on WHS. Then I just tell VMC and Mymovies to monitor those shares.

I don't get anywhere near the transfer rates as posted in the screen shots. Pre PP1 I was getting anywhere form 10MB/s to 20MB/s. After PP1 I get 50MB/s to 60MB/s. I also installed a Intel Pro 1000GT on the server. It's an old board, I don't know if that makes any difference(Asus P4C800-E deluxe). My router is a D-Link DGL-4300, connection is wired.
Maybe it's the drive speeds? My drives range in size from 250gb to 750gb. I really need to add more but can't afford it right now.

if you are using IDE drives, then 50-60MB/s is about all you are going to get out of one.

i can max my gig eth connection from one of my four drive hdd arrays, but cannot get more than 56MB/s from a single IDE to a single IDE.

since WHS isnt a real raid, but instead more of a spanning solution where files are written to a single drive, i would suspect that you aren't going to get much more than that.
post #505 of 7891
Quote:
Originally Posted by R1Budha View Post

if you are using IDE drives, then 50-60MB/s is about all you are going to get out of one.

i can max my gig eth connection from one of my four drive hdd arrays, but cannot get more than 56MB/s from a single IDE to a single IDE.

since WHS isnt a real raid, but instead more of a spanning solution where files are written to a single drive, i would suspect that you aren't going to get much more than that.

They are all SATA drives.

However as MiBz posted he's getting over 100MB/s consistently.

Not that it matters. I'm just copying movies over. It doesn't have to be fast.
post #506 of 7891
Quote:
Originally Posted by archer75 View Post

However as MiBz posted he's getting over 100MB/s consistently.

I noticed MiBz was using Seagate 1.5TB drives. Those are really fast. Performance is close to a VelociRaptor in terms of sequential read/write.
post #507 of 7891
I would agree, since WHS is relying on single platter speeds your drive speeds are very important. As a side note I noticed that it seems to be directly placing all the incomming files on the 2nd drive and keeping the 1st drive clear.

I ran some tests today for the client backup utils. Very neat stuff. I have a Vista virtualized on Hyper-V for testing purposes so I thought I'd install the client connector on it and try out the backup feature.

Here's what happens.

You navigate to a share called \\\\WHS\\Software\\ClientConnector.exe.
Double click and it installs the connector on the Vista client. It also asks you if the client should wake during hibernate/sleep to backup or only backup when on.

Once installed it places a connector icon in your system tray. Double click it and it prompts you for the admin password and then pulls up the WHS Command Console.

From there you have full control to create a new backup, set an automated backup schedule and decide what local drives to backup using check boxes.

The test Vista client has a 40GB virtual drive of which 17GB is used. I ran a full manual image backup that completed in 9 minutes and change. Not bad.

Once the backup was completed the console looked like this.
If you're backing up more than one client they will all be listed here.


Select the client icon and click on View Backups this window pops up providing a list of backups for that client.


Select the specific backup and click on details;


Now here's something useful. If you click Open in the Restore and View Files section, it opens the backed up C:\\ drive archive in a windows explorer window where you can browse files and folders and drag and drop a file out of it that you may have lost, deleted or corrupted. Nice feature.

I suppose the best part is with the Admin password you control all the client backups from one place, the WHS command console. If you have multiple laptops, desktops and htpc's this is a very cool feature.


From the console you set things like Start and End times you want backups to run (while everyone's asleep - including the machines to be backed up)

Nbr of months to keep monthly backups, weeks for weekly and days for daily backups.

Still feeling my way around WHS. It's not the fastest compared to a hardware raid 5 array with 8-12 or more blazing drives running windows server 2008, but for home media serving and a whole bunch of other neat features, it's seems to be just fine so far.

Hope you guys don't mind the updates.
post #508 of 7891
Just ordered the NORCO RPC-4020 from MWave.com on sale for $265 with $22.90 S/H to my New Mexico address. Sale price is good till end of the month.
I haven't found a better deal yet.
Too bad their server was down for last 2 days, I wanted to build this on Wed., now looks like Friday or Sat at the earliest.

Mario
post #509 of 7891
Quote:
Originally Posted by mariomp View Post

Just ordered the NORCO RPC-4020 from MWave.com on sale for $265 with $22.90 S/H to my New Mexico address. Sale price is good till end of the month.
I haven't found a better deal yet.
Too bad their server was down for last 2 days, I wanted to build this on Wed., now looks like Friday or Sat at the earliest.

Mario

I wish there was a company that would ship to Europe for a reasonable price
post #510 of 7891
Quote:
Originally Posted by MiBz View Post

I Hope you guys don't mind the updates.

No MiBz, not at all, keep them coming.

Do you know if WHS can act like a client OS drive image (?), or whatever Norton Ghost thing is?
I need to re-blast my HTPC and always wanted to have an image (Ghost) but never had any programs to do it (never tried it, yet) nor did I have means of backing up ghost image. Now that I'm getting my server build I would like to learn how to do OS image backup and how to restore it over the network when a client PC goes kaboom
Also, if WHS doesn’t do that, what programs do you guys use, free is always preferred…

Mario
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers
AVS › AVS Forum › Video Components › Home Theater Computers › Guide To Building A Media Storage Server