Question for NAS/Server owners about silent HDD failure - AVS | Home Theater Discussions And Reviews
Baselworld is only a few weeks away. Getting the latest news is easy, Click Here for info on how to join the Watchuseek.com newsletter list. Follow our team for updates featuring event coverage, new product unveilings, watch industry news & more!


Forum Jump: 
Thread Tools
post #1 of 26 Old 02-22-2015, 11:16 AM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
Question Question for NAS/Server owners about silent HDD failure

Guys,

I'm at a point where I need a media server. I'm only using it to stream .iso and music files (no tanscoding and speed is not terribly important). The easiest solution is to buy a NAS and set it up w/ RAID 5 or 6 (or similar). But the concern I have is possible HDD "silent" failure. Let's say I went with RAID 5, had a "hard" failure, replaced the disk, and started a RAID rebuild. Then that failed because of another disk had bad tracks/sectors/blocks (a silent failure) and the rebuild failed. Then my data is lost -- all of it.

How common are the "silent" failures? Has anyone ever experienced one? I've been using hard disks since my first 20MB (yes, MB) is the mid-80's and the only silent failure I've seen is from a laptop that was dropped.

Another possibility is to build my own server and use FlexRAID (or something similar). The nice thing about that is it stores your files in a native OS format and your only data loss is the bad disk or any data directly impacted by the bad tracks/sectors/blocks. If you're storing large media files, the situation I described above might mean you only lose an .iso file or two.

So the question is: How common are "silent" failures where your RAID array can't be rebuilt?

Thanks,
Scott
ScottS is offline  
Sponsored Links
Advertisement
 
post #2 of 26 Old 02-22-2015, 11:31 AM
AVS Addicted Member
 
Charles R's Avatar
 
Join Date: Nov 2000
Location: Indianapolis
Posts: 10,723
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 246
Quote:
Originally Posted by ScottS View Post
So the question is:
I think the answer is backup. RAID isn't a substitute. Also it's very easy to accidentally delete data (on purpose) thinking you are deleting other data. Sure some systems offer recovery such as Snapshots but you might not notice until it's too late. Another exposure often overlooked is firmware updates taking the system down... all too common.
Charles R is offline  
post #3 of 26 Old 02-22-2015, 12:23 PM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
I can't disagree with needing a backup, but the question everyone asks is: How do you backup 10TB? I could definitely backup my most critical data (prolly only a few hundred GBs) and just re-create the lost .iso files. That's my only feasible choice.
ScottS is offline  
post #4 of 26 Old 02-22-2015, 12:31 PM
AVS Addicted Member
 
Kal Rubinson's Avatar
 
Join Date: Mar 2004
Location: NYC + Connecticut
Posts: 28,639
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 210 Post(s)
Liked: 211
Quote:
Originally Posted by ScottS View Post
I can't disagree with needing a backup, but the question everyone asks is: How do you backup 10TB? I could definitely backup my most critical data (prolly only a few hundred GBs) and just re-create the lost .iso files. That's my only feasible choice.
You back it up with another 10Tb, either as a NAS or drivestack. Also, you should be running maintenance tests on your RAID drives on a regular basis to detect incipient failures.

Kal Rubinson

"Music in the Round"
Senior Contributing Editor, Stereophile
http://www.stereophile.com/category/music-round

Kal Rubinson is offline  
post #5 of 26 Old 02-22-2015, 12:40 PM
AVS Addicted Member
 
Charles R's Avatar
 
Join Date: Nov 2000
Location: Indianapolis
Posts: 10,723
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 246
Quote:
Originally Posted by ScottS View Post
I can't disagree with needing a backup, but the question everyone asks is: How do you backup 10TB? I could definitely backup my most critical data (prolly only a few hundred GBs) and just re-create the lost .iso files.
Again, it's more how can I not backup what I don't want to lose. Oh I understand the dilemma and luckily what I'm not willing to lose is very small... I backup to rotating USB flash drives and or hard drvies. My NAS storage (5TB or so) is simply for convenience. If you read the NAS vendor's forum it's scary... of course those without an issue never post.

To a large extent I don't consider it a backup unless it's offline. Too many times I see where someone is backing up to another device and the automated process "screws up" or whatnot. I prefer knowing I can plug something in when I need it and it will be there...
Charles R is offline  
post #6 of 26 Old 02-22-2015, 01:12 PM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
Yup. Good points for both.


I was considering writing a script that ran once per week or so that copied every file to a scratch location (and then immediately deleted it). The purpose being just to verify that each file can be read without error. If at any point the script fails, then its time to dig deeper into the health of the raid array.
ScottS is offline  
post #7 of 26 Old 02-22-2015, 10:20 PM
Advanced Member
 
funhouse69's Avatar
 
Join Date: Dec 2013
Posts: 749
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 297 Post(s)
Liked: 92
Quote:
Originally Posted by ScottS View Post
Guys,

I'm at a point where I need a media server. I'm only using it to stream .iso and music files (no tanscoding and speed is not terribly important). The easiest solution is to buy a NAS and set it up w/ RAID 5 or 6 (or similar). But the concern I have is possible HDD "silent" failure. Let's say I went with RAID 5, had a "hard" failure, replaced the disk, and started a RAID rebuild. Then that failed because of another disk had bad tracks/sectors/blocks (a silent failure) and the rebuild failed. Then my data is lost -- all of it.

How common are the "silent" failures? Has anyone ever experienced one? I've been using hard disks since my first 20MB (yes, MB) is the mid-80's and the only silent failure I've seen is from a laptop that was dropped.

Another possibility is to build my own server and use FlexRAID (or something similar). The nice thing about that is it stores your files in a native OS format and your only data loss is the bad disk or any data directly impacted by the bad tracks/sectors/blocks. If you're storing large media files, the situation I described above might mean you only lose an .iso file or two.

So the question is: How common are "silent" failures where your RAID array can't be rebuilt?

Thanks,
Scott
Once 3TB Drives were introduced it is strongly recommended that you go with RAID 6 / 2 Parity Drives just for the reason(s) you mentioned. The time it takes to regenerate / rebuild a RAID can be days depending on many different factors so yes it is very possible that you could have a second drive fail within that time frame.

I am right there with you, I have a Synology NAS with 9 4TB Drives that is pretty much loaded. How do you back up 20+ TB of data off site? I can't afford another NAS as a backup but have most of my data available on my original discs so it would just take a ton of time to restore it.

Denon AVR-X7200 - Klipsch RF-82 II Fronts, RC-62 II Center, RS-52 II Surrounds & R-112SW & Four CDT-5800-C II Ceilings for an Atmos 5.1.4 Setup.
funhouse69 is offline  
post #8 of 26 Old 02-23-2015, 04:36 AM
One-Man Content Creator
 
wmcclain's Avatar
 
Join Date: May 2006
Posts: 18,280
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 939 Post(s)
Liked: 470
A storage solution that keeps a checksum database for the files saves you the work of doing extra scripting to verify file integrity.

I use SnapRAID (http://snapraid.sourceforge.net/) and use its "scrub" function to check 12% of the array every week, meaning the whole server is checked every two months. This takes only about 30 minutes with my current volume, so I could increase the percentage or frequency without difficutly.

The SnapRAID pages provide these interesting links:

-Bill

Review older films here: 1979 and earlier | 1980s | Combined reviews: Strange Picture Scroll
Unofficial OPPO FAQS: BDP-103 | BDP-93 | BDP-83 | BDP-80    
wmcclain is offline  
post #9 of 26 Old 02-23-2015, 07:22 AM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
Quote:
Originally Posted by wmcclain View Post
I use SnapRAID (http://snapraid.sourceforge.net/) and use its "scrub" function to check 12% of the array every week, meaning the whole server is checked every two months. This takes only about 30 minutes with my current volume, so I could increase the percentage or frequency without difficutly.

The SnapRAID pages provide these interesting links:
-Bill
Thanks Bill, those are interesting reads. It's interesting though that no one has reported having a silent failure.

If they are truly very rare, then having a RAID 6 setup with two parity drives should protect your data exceeding well as long as you perform data validation like Bill on a regular basis.
ScottS is offline  
post #10 of 26 Old 02-23-2015, 10:18 AM
Advanced Member
 
smitbret's Avatar
 
Join Date: Nov 2005
Location: East Idaho - Pocatello
Posts: 621
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 248 Post(s)
Liked: 73
I have had my FlexRAID for about 2 years and haven't had a failure, yet.... <crosses fingers>. If your Storage is legit, you should have most of your media already backed up in the form of the original DVD/BD/CD. That's how I look at mine. One reason I chose FlexRAID is because if I have a catastrophic failure of multiple drives, I only lose the data on the drives that failed. I can still access all of the data on the good drives from Windows. SnapRAID and unRAID work in the same fashion. Hardware RAID, FreeNAS, NAS4Free and other RAID arrays that stripe won't give you this luxury.

Luckily, the huge majority of my stuff is HD video and if I lose some of the rips I can always just re rip from the original. Because of that, I can backup everything but my video files and it fits on an external 3TB HDD.

I do a weekly Validation that does both change detection along with datarot (silent data corruption) detection through data checksum validation.

I also do a monthly Verification that does bit for bit verification of the RAID.

I can do either in about 10 hours so I just schedule it for an overnight after it updates.

I also use ECC memory because for an extra $30, why not?

Last edited by smitbret; 02-23-2015 at 10:23 AM.
smitbret is offline  
post #11 of 26 Old 02-24-2015, 11:50 AM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
Bump.


Anyone experienced a silent failure in their RAID array?
ScottS is offline  
post #12 of 26 Old 02-28-2015, 08:16 PM
AVS Special Member
 
DotJun's Avatar
 
Join Date: Dec 2012
Posts: 1,213
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 292 Post(s)
Liked: 76
Quote:
Originally Posted by ScottS View Post
Bump.


Anyone experienced a silent failure in their RAID array?

Can I ask why you keep referring hdd failure as a silent failure or are you actually meaning to say bit rot?

If it's the former, well hdd's will eventually fail. I don't quite understand what you mean about it being silent though.

As for the latter, well really the only way to really combat that is to do checksums. Though I will add that people experience crc errors on file copy far far more than they will bit rot, but you don't see people running out and picking up ecc memory.
DotJun is online now  
post #13 of 26 Old 02-28-2015, 08:35 PM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
I didn't mean to be confusing, But yes, you can also call it bit rot. That's certainly a fair description of what is likely the most common silent failure. But there are others, such as a head crash damaging the disk surface. By silent, I mean that you aren't immediately aware of the problem. It is often call silent data corruption. If a disk fails to spin up (motor problem) or it can't be recognized by the OS (often an electronics problem), or the head actuator breaks (often you hear odd sounds when the disk is trying to seek), you are aware of it immediately -- i.e. a "hard" failure.


But the important part is that you typically aren't aware of the problem -- silent. If you only have 1 disk recovery capability, as in RAID5, and then you suffer a "hard" failure in another disk, then all your data will be lost. Which is why one poster pointed out the importance of scrubbing/validating the data on the disks on a regular schedule.


The reason I'm asking this question is: How safe is RAID 5? If you have silent data corruption and then a hard failure, then you're data is gone! But given the lack of responses from AVSers, it appears that silent data corruption is relatively rare (with modern disks).
ScottS is offline  
post #14 of 26 Old 02-28-2015, 09:41 PM
AVS Special Member
 
blackssr's Avatar
 
Join Date: Feb 2007
Location: South Florida
Posts: 2,946
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 294 Post(s)
Liked: 92
Quote:
Originally Posted by ScottS View Post
Guys,

I'm at a point where I need a media server. I'm only using it to stream .iso and music files (no tanscoding and speed is not terribly important). The easiest solution is to buy a NAS and set it up w/ RAID 5 or 6 (or similar). But the concern I have is possible HDD "silent" failure. Let's say I went with RAID 5, had a "hard" failure, replaced the disk, and started a RAID rebuild. Then that failed because of another disk had bad tracks/sectors/blocks (a silent failure) and the rebuild failed. Then my data is lost -- all of it.

How common are the "silent" failures? Has anyone ever experienced one? I've been using hard disks since my first 20MB (yes, MB) is the mid-80's and the only silent failure I've seen is from a laptop that was dropped.

Another possibility is to build my own server and use FlexRAID (or something similar). The nice thing about that is it stores your files in a native OS format and your only data loss is the bad disk or any data directly impacted by the bad tracks/sectors/blocks. If you're storing large media files, the situation I described above might mean you only lose an .iso file or two.

So the question is: How common are "silent" failures where your RAID array can't be rebuilt?

Thanks,
Scott
Silent failure rates/Smart Failure Rates/etc are not easy to predict. Active management, Pro active raid configuration and a good UPS all help to prevent unwanted failures.
I have a 64 TB Windows Server 2012 R2 Server. It is Raid 6 with 2 global hot spares. I can lose up to 2 drives at once and data will still be safe. The Areca Raid card I use (1883 Series)has a full email client that alerts me of any issues within seconds of trouble. I have almost 35 TB filled up so cloud backup is not really an option for me. I do have 10 4TB Drives that have all my movies backed up should the array completely fail. This is unlikely, but for the cost of a several hdds I can avoid having to re rip all movies again. If you do go raid, I suggest Raid 6 and have 2 global spares in place. Any of the mid level Areca cards are a great place to start. http://www.newegg.com/Product/Produc...-149-_-Product
They are a bit pricey but should last several years before needing replacement. I usually build a new media server every 4-5 years.
blackssr is offline  
post #15 of 26 Old 03-01-2015, 01:51 AM
AVS Special Member
 
DotJun's Avatar
 
Join Date: Dec 2012
Posts: 1,213
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 292 Post(s)
Liked: 76
Quote:
Originally Posted by ScottS View Post
I didn't mean to be confusing, But yes, you can also call it bit rot. That's certainly a fair description of what is likely the most common silent failure. But there are others, such as a head crash damaging the disk surface. By silent, I mean that you aren't immediately aware of the problem. It is often call silent data corruption. If a disk fails to spin up (motor problem) or it can't be recognized by the OS (often an electronics problem), or the head actuator breaks (often you hear odd sounds when the disk is trying to seek), you are aware of it immediately -- i.e. a "hard" failure.


But the important part is that you typically aren't aware of the problem -- silent. If you only have 1 disk recovery capability, as in RAID5, and then you suffer a "hard" failure in another disk, then all your data will be lost. Which is why one poster pointed out the importance of scrubbing/validating the data on the disks on a regular schedule.


The reason I'm asking this question is: How safe is RAID 5? If you have silent data corruption and then a hard failure, then you're data is gone! But given the lack of responses from AVSers, it appears that silent data corruption is relatively rare (with modern disks).

Let me simplify things here. Silent data failure is normally attributed to bit rot. Everything else is just plain old hardware failure. That ticking sound you hear from your drive is only NOT silent if you are around to hear it. So yes, you are only aware of it immediately if you are within the vicinity of the ticking time bomb.

Scrubbing/validating every X amount of time only pertains to non-realtime solutions as realtime does it in realtime

I think you are looking at it the wrong way with your last paragraph. Raid in all its forms isn't meant to keep your data safe from loss, that is what backups and ecc memory and checksums do. The job of raid is to have minimal downtime of your data should you suffer from hdd failure, except of course raid 0 which is only done for speed.

Remember that even backups could suffer from bit rot. Not in a physical sense, but if the data you are backing up onto a disc has bit rot then so will what goes onto the backup medium. If you are really worried about your data then you should look into ecc memory and some form hash checks.
DotJun is online now  
post #16 of 26 Old 03-01-2015, 09:42 AM - Thread Starter
Advanced Member
 
ScottS's Avatar
 
Join Date: Oct 1999
Location: On the East Coast
Posts: 528
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 13 Post(s)
Liked: 12
Thanks for the feedback guys.


Since I have slowly changing data, I'm leaning toward a software, snapshot RAID system like FlexRAID or more likely SnapRAID. UnRAID definitely has advantages with a fault tolerant file system, but it has a built-in OS, and I'd like to stick to Windows.


I'll prolly go with 2 parity disks just for piece of mind. As I mentioned before, I backup all of my critical data (which is really quite small), but I'd have to re-rip my movies in the event of a catastrophic loss. So that's only a loss of time...


If my data ever approaches 10TB, I'll switch to a more sophisticated system like blackssr uses and bite the bullet to buy lot of drives to backup the data as well. Of course, Seagate has promised 20TB shingled drives in the next few years so who knows what the future holds.


Thanks,
Scott
ScottS is offline  
post #17 of 26 Old 03-26-2015, 12:21 PM
AVS Special Member
 
Jim HTPC's Avatar
 
Join Date: Jun 2003
Location: South Florida
Posts: 1,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 5 Post(s)
Liked: 11
I- like BlackSSR, just built a 36TB Raid 6 File Server so silent it can go into the rack of my theater. I've tried all the budget methods, off brands, software raid, and then you realize when it's too late that you wasted all your money, possibly lost your data; when if you had saved up for the real deal, you would have been better off. If you can't swing $4-5k then start with box and add drives as you go.

I don't do global spares (personally) for my own home. Business client's--- yes. I prefer to have a fresh HD on my shelf ready to plug in when I get emailed on my iPhone that there is a disk failure. This way I still need to have 3 disks go bad... to lose data due to hd failure. Also I don't lose 6TB of my array by using a hot spare inside the enclosure by using my method. It's just personal opinion/preference.
Jim HTPC is online now  
post #18 of 26 Old 03-26-2015, 12:57 PM
Advanced Member
 
smitbret's Avatar
 
Join Date: Nov 2005
Location: East Idaho - Pocatello
Posts: 621
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 248 Post(s)
Liked: 73
Quote:
Originally Posted by Jim HTPC View Post
I- like BlackSSR, just built a 36TB Raid 6 File Server so silent it can go into the rack of my theater. I've tried all the budget methods, off brands, software raid, and then you realize when it's too late that you wasted all your money, possibly lost your data; when if you had saved up for the real deal, you would have been better off. If you can't swing $4-5k then start with box and add drives as you go.

I don't do global spares (personally) for my own home. Business client's--- yes. I prefer to have a fresh HD on my shelf ready to plug in when I get emailed on my iPhone that there is a disk failure. This way I still need to have 3 disks go bad... to lose data due to hd failure. Also I don't lose 6TB of my array by using a hot spare inside the enclosure by using my method. It's just personal opinion/preference.
I literally, don't understand what advantage a RAID card/hardware RAID would have over an ECC setup with FlexRAID, SnapRAID or especially a ZFS RAIDz setup for home use. Can someone please explain?
smitbret is offline  
post #19 of 26 Old 03-26-2015, 04:44 PM
AVS Special Member
 
Jim HTPC's Avatar
 
Join Date: Jun 2003
Location: South Florida
Posts: 1,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 5 Post(s)
Liked: 11
Quote:
Originally Posted by smitbret View Post
I literally, don't understand what advantage a RAID card/hardware RAID would have over an ECC setup with FlexRAID, SnapRAID or especially a ZFS RAIDz setup for home use. Can someone please explain?
In my opinion there is no such thing as home or business. To me it all comes down to how much money, and how much time will I waste if I need to redo 36TB of data, and how much time do I need to fuss with build/installation of device. For example if you are trying to store your movies, all over again, how long will it take to redo when your array was full? Time is important to me.

One significance with using Windows is not dealing with EXT3, EXT4, ZFS formats. Those are formats supported by Linux. Which distro of Linux do you use? I recommend using ECC ram regardless. Some argue that hardware specs have improved so much that there is less need for it. RAIDz depends on parity of blocks instead of stripes like RAID 6. Also there are configuration considerations with RAIDz. Performance IOPS vs. space/size of array. You have to select the proper amount of Parity. If you don't know what you're doing or have someone to ask then it can be frustrating.

I've tossed some Thecus's in the trash after some time. I have some 1U4500's, N8800, N7710, and 8900 that I've worked with and/or owned. Between the noise, and fear of firmware upgrade ruining your NAS is a pain. Raid initialization failing due to power management kicking on and putting it to sleep is just insane.

The noise is a complaint in a lot of these NAS boxes too. Right now my Fileserver is quieter than my gaming rig. All my NAS's sound like leaf blowers. Probably THE MOST IMPORTANT thing is that the Windows file server could also double as your HTPC. I have been fortunate to already have a dedicated HTPC before building my new file server so I don't need to do that. I trust Microsoft Server OS for work, so why not for home? Server is designed for being a file server, virtual machines for multiple users in a corporate environment, and more. It has plenty of power to be both HTPC and Fileserver with dedicated Raid card. I run Samsung 850 Pros for the OS. Just make certain you get fast enough CPU(s) and memory.

Also if you don't have a dedicated RAID card, then maintenance checks, and repairs could take longer. Whereas the RAID card I'm using has 12Gb/sec throughput for each port. The card is plugged into a PCIe 3 slot. So it offloads the raid work from the CPU. You can add a battery to the raid card as well. Once my RAID is initialized and ready to go, I do a complete backup of the OS drive. So if there is ever any software update regardless that messes with the system, I can restore it in 10 minutes back to how I first installed the OS. All with hotswap push and pull out drives including OS. You could even do 2 SSDs in Mirror 1 for the OS if you wanted.
Jim HTPC is online now  
post #20 of 26 Old 03-27-2015, 09:44 AM
Advanced Member
 
smitbret's Avatar
 
Join Date: Nov 2005
Location: East Idaho - Pocatello
Posts: 621
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 248 Post(s)
Liked: 73
Quote:
Originally Posted by Jim HTPC View Post
In my opinion there is no such thing as home or business. To me it all comes down to how much money, and how much time will I waste if I need to redo 36TB of data, and how much time do I need to fuss with build/installation of device. For example if you are trying to store your movies, all over again, how long will it take to redo when your array was full? Time is important to me.

One significance with using Windows is not dealing with EXT3, EXT4, ZFS formats. Those are formats supported by Linux. Which distro of Linux do you use? I recommend using ECC ram regardless. Some argue that hardware specs have improved so much that there is less need for it. RAIDz depends on parity of blocks instead of stripes like RAID 6. Also there are configuration considerations with RAIDz. Performance IOPS vs. space/size of array. You have to select the proper amount of Parity. If you don't know what you're doing or have someone to ask then it can be frustrating.

I've tossed some Thecus's in the trash after some time. I have some 1U4500's, N8800, N7710, and 8900 that I've worked with and/or owned. Between the noise, and fear of firmware upgrade ruining your NAS is a pain. Raid initialization failing due to power management kicking on and putting it to sleep is just insane.

The noise is a complaint in a lot of these NAS boxes too. Right now my Fileserver is quieter than my gaming rig. All my NAS's sound like leaf blowers. Probably THE MOST IMPORTANT thing is that the Windows file server could also double as your HTPC. I have been fortunate to already have a dedicated HTPC before building my new file server so I don't need to do that. I trust Microsoft Server OS for work, so why not for home? Server is designed for being a file server, virtual machines for multiple users in a corporate environment, and more. It has plenty of power to be both HTPC and Fileserver with dedicated Raid card. I run Samsung 850 Pros for the OS. Just make certain you get fast enough CPU(s) and memory.

Also if you don't have a dedicated RAID card, then maintenance checks, and repairs could take longer. Whereas the RAID card I'm using has 12Gb/sec throughput for each port. The card is plugged into a PCIe 3 slot. So it offloads the raid work from the CPU. You can add a battery to the raid card as well. Once my RAID is initialized and ready to go, I do a complete backup of the OS drive. So if there is ever any software update regardless that messes with the system, I can restore it in 10 minutes back to how I first installed the OS. All with hotswap push and pull out drives including OS. You could even do 2 SSDs in Mirror 1 for the OS if you wanted.
But with FlexRAID or SnapRAID under a Windows environment, there is still no real-life benefit other than pure speed.
smitbret is offline  
post #21 of 26 Old 03-31-2015, 11:49 AM
AVS Special Member
 
DotJun's Avatar
 
Join Date: Dec 2012
Posts: 1,213
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 292 Post(s)
Liked: 76
Quote:
Originally Posted by smitbret View Post
But with FlexRAID or SnapRAID under a Windows environment, there is still no real-life benefit other than pure speed.

One of the best features added to raid cards was the advent of the hot spare. I'd much rather lose out on a drive slot and know that my array starts to rebuild as soon as it degrades vs not. It will be your luck that your array degrades when you are on vacation or something and that shiny drive in your closet is just sitting there looking pretty.

I don't know what hard card you are running but I don't know any that can handle more than two drive failures.

The biggest reason to run software raid, imo at least, is the ability to run numerous parity drives. If you are willing to run smaller capacity arrays then yes, raid 6 is totally fine, but when rebuild times take a week or more it doesn't as good anymore right?
DotJun is online now  
post #22 of 26 Old 03-31-2015, 05:48 PM
Advanced Member
 
smitbret's Avatar
 
Join Date: Nov 2005
Location: East Idaho - Pocatello
Posts: 621
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 248 Post(s)
Liked: 73
Quote:
Originally Posted by DotJun View Post
One of the best features added to raid cards was the advent of the hot spare. I'd much rather lose out on a drive slot and know that my array starts to rebuild as soon as it degrades vs not. It will be your luck that your array degrades when you are on vacation or something and that shiny drive in your closet is just sitting there looking pretty.

I don't know what hard card you are running but I don't know any that can handle more than two drive failures.

The biggest reason to run software raid, imo at least, is the ability to run numerous parity drives. If you are willing to run smaller capacity arrays then yes, raid 6 is totally fine, but when rebuild times take a week or more it doesn't as good anymore right?
I can't tell if your post is pro-Software RAID or con. I haven't run hardware RAID in a few years.

The idea of a hot spare is nice, but I agree with the following blogger that it's probably not necessarily the best process:

http://blog.open-e.com/why-a-hot-spa...is-a-bad-idea/

I would much rather have my array shut down and stay that way until I can get in to the system and verify everything before doing a data rebuild and risk further data loss.
smitbret is offline  
post #23 of 26 Old 04-01-2015, 04:42 PM
AVS Special Member
 
DotJun's Avatar
 
Join Date: Dec 2012
Posts: 1,213
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 292 Post(s)
Liked: 76
Quote:
Originally Posted by smitbret View Post
I can't tell if your post is pro-Software RAID or con. I haven't run hardware RAID in a few years.

The idea of a hot spare is nice, but I agree with the following blogger that it's probably not necessarily the best process:

http://blog.open-e.com/why-a-hot-spa...is-a-bad-idea/

I would much rather have my array shut down and stay that way until I can get in to the system and verify everything before doing a data rebuild and risk further data loss.

I'm pro software raid due to being able to use multiple parity drives.

Most people run raid for minimal downtime. If downtime is not a concern of yours, shutting down your degraded server, then why not just run jbod and use a backup to disc instead? It's probably more affordable and a lot less complex to boot.
DotJun is online now  
post #24 of 26 Old 04-02-2015, 09:59 AM
Advanced Member
 
smitbret's Avatar
 
Join Date: Nov 2005
Location: East Idaho - Pocatello
Posts: 621
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 248 Post(s)
Liked: 73
Quote:
Originally Posted by DotJun View Post
I'm pro software raid due to being able to use multiple parity drives.

Most people run raid for minimal downtime. If downtime is not a concern of yours, shutting down your degraded server, then why not just run jbod and use a backup to disc instead? It's probably more affordable and a lot less complex to boot.
Yeah, on a certain level I agree with that, although FlexRAID is no more trouble on boot up than just booting into Windows once the setup is completed and the only time I really reboot is for hardware swaps and if a Windows update requires it. In the 2+ years I have been running FlexRAID I haven't had a disc fail (knock on wood) but I like that if one or two fail then my kids can still watch their Disney movies until I get home from work. I just don't want the automatic rebuild from the hot spare until I have had a chance to check things out.

If I had completely redundant backup I would probably run JBOD instead of FlexRAID. I just don't want to backup the TBs of data from DVD and BD rips when I have the discs sitting in my garage that can be re-ripped if necessary. Running FlexRAID with 2 parity discs just means I don't have to invest in another NAS for backup and that I will most likely never have to re-rip all the discs again. I've thought of building an Atom based or even RaspPi based backup NAS but just haven't seen enough need to go that far and invest in the HDDs for it. If 6TB drives got cheap enough then I might consider it. But yeah, up till now, JBOD with backup would have served me just as well.

I backup everything else (except for the DVD/BD rips) to a 3TB External HDD because anything that cannot be replaced needs to be backed up.
smitbret is offline  
post #25 of 26 Old 04-05-2015, 12:20 AM
AVS Special Member
 
robnix's Avatar
 
Join Date: Dec 2003
Posts: 1,989
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 260 Post(s)
Liked: 342
Quote:
Originally Posted by smitbret View Post
I can't tell if your post is pro-Software RAID or con. I haven't run hardware RAID in a few years.

The idea of a hot spare is nice, but I agree with the following blogger that it's probably not necessarily the best process:

http://blog.open-e.com/why-a-hot-spa...is-a-bad-idea/

I would much rather have my array shut down and stay that way until I can get in to the system and verify everything before doing a data rebuild and risk further data loss.
Sorry, but this guy isn't an independent blogger in any way, he writes for a company that sells a ZFS based storage solution. It used to be a SCST with XFS solution. His logic for not having hot spares is not just flawed but dangerous.

Quote:
  1. Run a full data backup. In one form or another you should already have this. Why would I run a fresh full backup against a degraded array when I should already be within a few hours RPO? What comprises a backup is another discussion, but between BR discs you should have on hand, and a real time cloud solution for your soft media, you should be able to restore anything you care about from media that's not on your primary file server.
  2. Verify the backed-up data for consistency, and verify whether the data restore mechanism works. This should already be tested as well. No reason to add to the time that you're dealing with new backups while your array is degraded.
  3. Identify the problem source, i.e. find the erroneous hard disk. If possible, shut down the server, and make sure the serial number of the hard disk matches that reported by the RAID controller. You should already have the disk identified.
  4. Replace the hard disk identified as bad with a new, unused one. If the replacement hard drive had already been used within another RAID array, make sure that any residual RAID metadata on it has been deleted via the original RAID controller. So now he's saying it's OK to replace a failed drive with a USED drive...
  5. Start the rebuild of the RAID. By the time he's done with steps 1-4, my array is already rebuilt by a hot spare.
And then....


Quote:
Being aware of Murphy’s Law, no one would risk an immediate rebuild after a drive failure – but by using a Hot-Spare this is exactly what will happen.

So his logic is not to use a hot spare because you'll have a failure during the rebuild, but you won't have that same failure during steps 1-4...?


Your backups should already meet your RPO. This means you need to decide what, where, why, and how recent.

Hardware and software RAID both have data scrubbing mechanisms to ensure data consistency. There should be zero reason to verify your data before a rebuild.

What hot spares do is buy you time. Considering you may have purchased your drives in a batch, time is crucial.

Looky here!
robnix is offline  
post #26 of 26 Unread Today, 11:41 PM
AVS Special Member
 
DotJun's Avatar
 
Join Date: Dec 2012
Posts: 1,213
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 292 Post(s)
Liked: 76
Quote:
Originally Posted by robnix View Post
Sorry, but this guy isn't an independent blogger in any way, he writes for a company that sells a ZFS based storage solution. It used to be a SCST with XFS solution. His logic for not having hot spares is not just flawed but dangerous.

[/LEFT][/LIST]And then....





So his logic is not to use a hot spare because you'll have a failure during the rebuild, but you won't have that same failure during steps 1-4...?


Your backups should already meet your RPO. This means you need to decide what, where, why, and how recent.

Hardware and software RAID both have data scrubbing mechanisms to ensure data consistency. There should be zero reason to verify your data before a rebuild.

What hot spares do is buy you time. Considering you may have purchased your drives in a batch, time is crucial.

And that's why I didn't even comment on that blog
DotJun is online now  
Sponsored Links
Advertisement
 
Reply Networking, Media Servers & Content Streaming

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off