AVS › AVS Forum › Video Components › Home Theater Computers › Guide To Building A Media Storage Server
New Posts  All Forums:Forum Nav:

Guide To Building A Media Storage Server - Page 237

post #7081 of 7716
3/3/12 at 5:21pm
Quote:
Originally Posted by assassin View Post

Figured out how to manually cleanup the backups I don't want. Will report back. Pre-cleanup had about 1.6TB of backups.

Backup folder down to about 350GB after running cleanup. Much better now.

Edit: Just checked again. Down to 263GB. Much more like it!
Reply
Reply

AVS Top Picks

post #7082 of 7716
3/4/12 at 10:30am
Hello All, First time posting here, This is an awesome forum and i think will be a nice resource as i build my home automation/media platform for my house. I've read about 30% of this thread (the last 20 pages and prob like 80 form the beginning) but have see its migrated to a lot of things)

My media server started a little over a year ago when I upgraded to an i7 mobo and all, which replaced my e6420 P5K deluxe board w/ a GTX8800 that was now sitting on a shelf, My roomate suggested we build a media server, and so it started... I used the old mobo and parts and added a few JBod to the mobo which had 6 sata and my roomate had pci expansion slot that added two. Fast forward a bit, I'm leaving to head back to NJ and we split the server up ( he had some drives in it and i'm now diving into server stuff with Raid) I plan on using the server for Ripping dvd's etc, recording live tv, and running all the home automation stuff. The board i have is more than enough to handle, but has old pcie gen 1 slots and pci slots. Most controllers i have been looking at are pcie gen 2. It was running WHS 2011, But i've put 2008 R2 enterprise on mine (overkill but It was free)

Sooo with the above, I'm looking for some advice as to where to go, I've Decided on running a raid 5 (early) and upgrade to Raid 6 later as i get a few more drives(would like to let pricing settle) or just do this off the bat. I currently have about 6tb of data to store. I'm looking for advice if i should upgrade my motherboard to a more modern board to accept the newer raid cards? (i think the system idled at 170 watts)

with that I have 4 WD green 3tb drives, and 5 1tb drives, 500gb system drive and 2 Tb drive thats running solo (probably backups or to become my mainstream array)


So kinda a mishmosh of drives and i'm looking for advice, I need a raid controller with OCE as i plan to expand the array latter on. but i want a good base to start with. As far as backing up i'm not too worried as i have about just under 1tb of data (pictures, music, important files) the rest is tvshows and dvds that would just be a real pain to rerip.
Reply
Reply
post #7083 of 7716
3/4/12 at 2:10pm
Quote:
Originally Posted by blankstar85 View Post

Hello All, First time posting here, This is an awesome forum and i think will be a nice resource as i build my home automation/media platform for my house. I've read about 30% of this thread (the last 20 pages and prob like 80 form the beginning) but have see its migrated to a lot of things)

My media server started a little over a year ago when I upgraded to an i7 mobo and all, which replaced my e6420 P5K deluxe board w/ a GTX8800 that was now sitting on a shelf, My roomate suggested we build a media server, and so it started... I used the old mobo and parts and added a few JBod to the mobo which had 6 sata and my roomate had pci expansion slot that added two. Fast forward a bit, I'm leaving to head back to NJ and we split the server up ( he had some drives in it and i'm now diving into server stuff with Raid) I plan on using the server for Ripping dvd's etc, recording live tv, and running all the home automation stuff. The board i have is more than enough to handle, but has old pcie gen 1 slots and pci slots. Most controllers i have been looking at are pcie gen 2. It was running WHS 2011, But i've put 2008 R2 enterprise on mine (overkill but It was free)

Sooo with the above, I'm looking for some advice as to where to go, I've Decided on running a raid 5 (early) and upgrade to Raid 6 later as i get a few more drives(would like to let pricing settle) or just do this off the bat. I currently have about 6tb of data to store. I'm looking for advice if i should upgrade my motherboard to a more modern board to accept the newer raid cards? (i think the system idled at 170 watts)

with that I have 4 WD green 3tb drives, and 5 1tb drives, 500gb system drive and 2 Tb drive thats running solo (probably backups or to become my mainstream array)


So kinda a mishmosh of drives and i'm looking for advice, I need a raid controller with OCE as i plan to expand the array latter on. but i want a good base to start with. As far as backing up i'm not too worried as i have about just under 1tb of data (pictures, music, important files) the rest is tvshows and dvds that would just be a real pain to rerip.

If you are going to run a dedicated raid controller card, you certainly wouldn't need to upgrade your motherboard as well.
You only need one additional disk for Raid 6 over Raid 5. With your mismash of drives, a standard raid isn't going to suit your needs if you want to utilize them all.
Reply
Reply
post #7084 of 7716
3/4/12 at 7:04pm
yeah its a bad mix but I figure ill go with two hardware raid 6 drivesone with the 1tb drives and the other with the 3tb and use the 2tb as a backup drive. should I use two seperate raid cards or can one handle both? and what pcie gen 1 card would you recommend that's reasonable ? I would like to have oce and hot spare just from what I've read it makes sense
Reply
Reply
post #7085 of 7716
3/4/12 at 7:07pm
Enlighten me. For HTPC what's the point of running a RAID? Seems like an awful big hassle.

Is there something wrong with my approach of just making an actual backup of my media files?
Reply
Reply
post #7086 of 7716
3/4/12 at 7:18pm
Quote:
Originally Posted by assassin View Post

Enlighten me. For HTPC what's the point of running a RAID? Seems like an awful big hassle.

Is there something wrong with my approach of just making an actual backup of my media files?

I'm sure you've heard it before, but RAID isn't backup. However, it can rescue your data if a drive fails.

In my case I have about 6TB of storage with right over 4TB used. I backup only about 850GB using CrashPlan for an offsite backup to a 1.5TB external drive connected to my parents' out-of-state PC. I also have that data backed up locally as well for a total of three copies (original, local, and off-site). This data is irreplaceable such as photos and home videos and documents.

The rest is media. That leaves me with up to 5TB not backed up. With RAID I can restore that data with only an investment of 2TB for my parity drive. To create a true backup I'd need another 5TB.
Reply
Reply
post #7087 of 7716
3/4/12 at 7:32pm
Gotcha.

I think I will take my chances. Its easy to backup and the likelihood of 2 different drives of different ages failing at the same time are odds that I can accept.
Reply
Reply
post #7088 of 7716
3/4/12 at 7:39pm
I'm looking for the read speeds of the raid I plan to have quite a few PC's playing content my important irreplaceable data is well under 2tb so the two arrays will cover mainstream data and the 2tb as backup solely for important files I'm going to look into crashbplan. the rest of the data would just be a pain to replace but don't care if it goes, but ill build some fault tolerance with raid to hopefully allow me to save data on a failure. just Ned help picking the card
Reply
Reply
post #7089 of 7716
3/4/12 at 8:10pm
Just a little food for thought on backup plans... I originally ripped my CD collection in MP3 format a while back and recently started going through again ripping to FLAC. As I've been going through my CD's, I've been coming across a non-trivial number of disks that have 1 or more tracks that are unreadable. In some cases where it's not a favorite album I'm just going to live with the MP3's that I have, but in other cases, I've been hunting down replacement albums. I've been finding that some pretty mainstream stuff is relatively hard to find and/or pricey (Genesis Three Sides Live, for example). I've got multiple copies of my original MP3 rips (1 offsite on a drive at my office) and intend to do the same thing again when I finish re-ripping in FLAC format.

I'm somewhat less concerned about movies as it's less common for me to watch a movie multiple times than listen to the same CD multiple times.

But definitely don't assume that media ripped from original disks is recoverable if you lose your hard drive.
Reply
Reply
post #7090 of 7716
3/4/12 at 8:23pm
Quote:
Originally Posted by assassin View Post

Gotcha.

I think I will take my chances. Its easy to backup and the likelihood of 2 different drives of different ages failing at the same time are odds that I can accept.

That depends on the size of your library. 20GB of movies is easy to backup. 8 TB isn't. Consumer drives have a pretty bad failure rate so protecting yourself against a single or dual simultaneous disk loss can sometimes be preferable.
Or to put it another way, how much would you pay to for insurance against the loss of all your data and how much would you pay to save the hassle of reacquiring it all.

100$ or so for a single 2TB drive for parity is well worth it for me.
Reply
Reply
post #7091 of 7716
3/4/12 at 8:24pm
Quote:
Originally Posted by blankstar85 View Post

I'm looking for the read speeds of the raid I plan to have quite a few PC's playing content my important irreplaceable data is well under 2tb so the two arrays will cover mainstream data and the 2tb as backup solely for important files I'm going to look into crashbplan. the rest of the data would just be a pain to replace but don't care if it goes, but ill build some fault tolerance with raid to hopefully allow me to save data on a failure. just Ned help picking the card

Most read speeds will well exceed the network that is delivering the files in a HTPC environment.

Card choice is highly subjective. It really depends on what your goal and budget is.

The general rule of thumb is that for low end consumer cards, the recommendation is to stick with a software solution like unRaid, FlexRaid, or the raid software in Windows. Most modern enthusiast motherboards can do Raid 5 and so can WHS 2011. Most cheap (sub-100$) consumer cards are less reliable then software and you have the added headache of having to find the exact controller if the card goes bad. You don't have that issue in a software solution.
Reply
Reply
post #7092 of 7716
3/4/12 at 8:30pm
Quote:
Originally Posted by Lars99 View Post

That depends on the size of your library. 20GB of movies is easy to backup. 8 TB isn't. Consumer drives have a pretty bad failure rate so protecting yourself against a single or dual simultaneous disk loss can sometimes be preferable.
Or to put it another way, how much would you pay to for insurance against the loss of all your data and how much would you pay to save the hassle of reacquiring it all.

100$ or so for a single 2TB drive for parity is well worth it for me.

I own six 2TB drives. I can fit my media on 2 of them right now. So I have two drives with the media and then another 2 drives with backups of that media. The other two drives have pictures (also backed up on an external x2), files, OS backups, etc.

I am pretty happy with this and don't really see what RAID could offer me personally. But I completely understand why you are using it.
Reply
Reply
post #7093 of 7716
3/4/12 at 9:19pm
Quote:
Originally Posted by assassin View Post

I own six 2TB drives. I can fit my media on 2 of them right now. So I have two drives with the media and then another 2 drives with backups of that media. The other two drives have pictures (also backed up on an external x2), files, OS backups, etc.

I am pretty happy with this and don't really see what RAID could offer me personally. But I completely understand why you are using it.

That's the exact situation you would want a raid for.
Reply
Reply
post #7094 of 7716
3/5/12 at 5:37am
Quote:
Originally Posted by assassin View Post

I own six 2TB drives. I can fit my media on 2 of them right now. So I have two drives with the media and then another 2 drives with backups of that media.

That's all fine when your collection is relatively small. But it gets impractical quickly as it grows.
Reply
Reply
post #7095 of 7716
3/5/12 at 6:22am
Quote:
Originally Posted by Darin View Post

That's all fine when your collection is relatively small. But it gets impractical quickly as it grows.

Its pretty easy. I am not adding new things as much as I did in the past.
Reply
Reply
post #7096 of 7716
3/5/12 at 8:16am
Quote:
Originally Posted by assassin View Post

Its pretty easy. I am not adding new things as much as I did in the past.

You can always try FlexRAID for free without harming your files. It keeps everything in NTFS format and creates a new virtual drive containing the drive pool. You can then add in SnapshotRAID using a spare drive. If you don't like it then you simply uninstall it. The entire time you can still access the files normally. However, I think you are supposed to write to the virtual drive for proper RAID management to work.
Reply
Reply
post #7097 of 7716
3/5/12 at 10:32am
Yeah, FlexRAID is what I use (though since I'm on WHS v1, I'm only using the parity portion of it, since WHS has pooling built in). If assassin only has two drives worth of data, and doesn't add new data very often, then perhaps simple backups are fine for him. But if you have a lot of data, a parity based solution is the most practical approach. I have over 30TB spread across 20 drives, and I'm adding new data all the time. Doubling that up to 40 drives, and having to add two drives for every one drive of data, would be a nightmare. Two extra parity drives protects everything up to two simultaneous drive failures.

Personally, I think for a home media server, a snapshot RAID-4ish solution is the best answer. You get quasi-backup functionality (in case you accidentally delete or trash some data), it doesn't affect your write performance, the data doesn't depend on the RAID, only one drive needs to be spun up to watch a movie, and you can throw any combination of disk sizes, interfaces, or even locations at it. It's surprising to me that so many people still prefer a conventional RAID5/6 solution. The only significant advantage to it I see is better read speeds, which is lost behind a gigabit network, and moot for blu-ray data rates anyway.
Reply
Reply
post #7098 of 7716
3/6/12 at 8:20am
Yeah I'm just worried about the read speeds if a few devices are reading the same drive at the same time, I wanted to be able to cope with it.

can flex raid handle this? I suppose it might be mute as the possibility of the movies being on the same drives might be small?


As far as the size i guess with my mismosh of drives it would be better to go with 3tb dru and pru to keep it even and let me get more disk space outside the parity drives. since they only need to be 3TB?
Reply
Reply
post #7099 of 7716
3/6/12 at 9:17am
What do you mean by "worried about the read speed"?

You can easily stream 4+ HD movies from even a green 5400RPM drive.
Reply
Reply
post #7100 of 7716
3/6/12 at 2:37pm
@assassin, sorry for mixing up terms, yes stream movies from, I want to be able to stream hd content without issues.

@Darin Thanks for the advice, and info, I'll give Flexraid a try, its a free solution that i can try now and see how it works, itll give me the back up and use all my drives (without upgrading my hardware)

I guess i should use 3tb drus, with 2 seperate ppus? Is it better to have the ppu as one drive or is it ok if they are a few?
Reply
Reply
post #7101 of 7716
3/6/12 at 2:55pm
Quote:
Originally Posted by blankstar85 View Post

As far as the size i guess with my mismosh of drives it would be better to go with 3tb dru and pru to keep it even and let me get more disk space outside the parity drives. since they only need to be 3TB?

With your collection of drives, you could use a single 3TB drive for parity (the PPU), and you would have 4 DRUS (each of the remaining 3TB drives would be a DRU, and the 1TB and 2TB drives could be spanned together into a single DRU). This would give you fault tolerance of 1 drive failure (though in this setup, you could lose both the 1TB and 2TB drives and still recover, since they'd only be one DRU), with 12TB of storage. You can add drives as needed (as long as they are 3TB or less), and you can add additional PPUs if/when you want more fault tolerance. If you ever got to the point of wanting to use drives larger than 3TB, you'd have to re-write your parity into PPUs that are at least as large as your largest DRUs.

As assassin pointed out, read speed shouldn't be a concern for any typical size family. Even less so as your data gets spread out over more drives. What are the chances you'd have 5-6 people wanting to watch a movie, all at the same time, and the movies they wanted to watch all happened to be on the same drive? By the time you factor in the diversity of data spread across multiple drives, your network bandwidth will become a concern faster than the drives.
Reply
Reply
post #7102 of 7716
3/6/12 at 6:51pm
@Darin,

Thanks for the Help, I'm going to get this setup
Reply
Reply
post #7103 of 7716
3/7/12 at 4:58am
Quote:
Originally Posted by Darin View Post

With your collection of drives, you could use a single 3TB drive for parity (the PPU), and you would have 4 DRUS (each of the remaining 3TB drives would be a DRU, and the 1TB and 2TB drives could be spanned together into a single DRU). This would give you fault tolerance of 1 drive failure (though in this setup, you could lose both the 1TB and 2TB drives and still recover, since they'd only be one DRU), with 12TB of storage. You can add drives as needed (as long as they are 3TB or less), and you can add additional PPUs if/when you want more fault tolerance. If you ever got to the point of wanting to use drives larger than 3TB, you'd have to re-write your parity into PPUs that are at least as large as your largest DRUs.

As assassin pointed out, read speed shouldn't be a concern for any typical size family. Even less so as your data gets spread out over more drives. What are the chances you'd have 5-6 people wanting to watch a movie, all at the same time, and the movies they wanted to watch all happened to be on the same drive? By the time you factor in the diversity of data spread across multiple drives, your network bandwidth will become a concern faster than the drives.

Out of curiosity, why did you choose FlexRaid over UnRaid? If you had it to do over again, would you pick FlexRaid? The one nice thing about FlexRaid over UnRaid is you can have more than one parity drive, so if two drives crashed at same time you would not loose any data.
Reply
Reply
post #7104 of 7716
3/7/12 at 5:24am
Quote:
Originally Posted by stepmback View Post

Out of curiosity, why did you choose FlexRaid over UnRaid?

The primary reason is because it can be installed on various operating systems. I'm using WHS, so my server is also handling my client backups, router logging, it's running my MyMovies backend server and ripping, utorrent/sabnzbd, etc. I can put just about anything on it for any other process I may want to have running constantly, vs. something on one of my clients. While unRAID is a good proven solution if all you want is storage for your media with some redundancy, it's not really intended to do anything other than that.

Quote:


If you had it to do over again, would you pick FlexRaid? The one nice thing about FlexRaid over UnRaid is you can have more than one parity drive, so if two drives crashed at same time you would not loose any data.

Yes, and the ability to have a practically unlimited number of parity units is unique to FlexRAID. I'm currently running two parity drives, but I have 21 data drives. I'm pretty much at my limit of comfort for fault tolerance for this number of drives. Once drive prices drop back down, I will likely be adding more parity.

One thing that hasn't been mentioned though: FlexRAID is about to go commercial. The most common complaints about FlexRAID has been that it's always in beta, and the support was inconsistent. It's coming out of beta soon, and the support level has been promised to improve as a commercial product, but it will no longer be free. I'm sure some people will view this as a good thing, and some as bad. Personally, I see it as a good thing, though I think his price target may be a tad high.
Reply
Reply
post #7105 of 7716
3/12/12 at 3:21pm
I notice every once and a while my WHS is a little stuttery. I restart it and its just fine. Any idea how to reduce or prevent this?
Reply
Reply
post #7106 of 7716
3/12/12 at 4:12pm
Quote:
Originally Posted by assassin View Post

I notice every once and a while my WHS is a little stuttery. I restart it and its just fine. Any idea how to reduce or prevent this?

How long after a reboot before it begins to stutter? If reasonable, you can schedule a reboot every week or so during an off use time.
Reply
Reply
post #7107 of 7716
3/12/12 at 5:20pm
Quote:
Originally Posted by alexsquared View Post

How long after a reboot before it begins to stutter? If reasonable, you can schedule a reboot every week or so during an off use time.

Its just every few days. Definitely not a routine issue. I wonder if its the Realtek NIC.
Reply
Reply
post #7108 of 7716
3/12/12 at 6:41pm
Quote:
Originally Posted by assassin View Post


Its just every few days. Definitely not a routine issue. I wonder if its the Realtek NIC.

I went to an intel mic (previous was Realtek) and haven't had stutters since.
Reply
Reply
post #7109 of 7716
3/12/12 at 6:45pm
Quote:
Originally Posted by alexsquared View Post

I went to an intel mic (previous was Realtek) and haven't had stutters since.

Yeah, that's what I read when I started this project but I wanted to give the Realtek a shot.

Which Intel NIC do you recommend? I am willing to try one out to see if there is any difference.
Reply
Reply
post #7110 of 7716
3/12/12 at 6:50pm
The Intel Pro 1000 is the best. Period.
Reply
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Home Theater Computers

AVS Top Picks

AVS › AVS Forum › Video Components › Home Theater Computers › Guide To Building A Media Storage Server