Originally Posted by hdkhang
I'm not intimately familiar with ZFS, but am curious to know how it knows that something is bad automatically? My understanding is that this can only happen if you actually access the file in some way, otherwise the system will be completely wasteful if it is continually checking all files for silent corruption.
Why is that "wasteful"? Your server sits idle most of the day. It should be doing something useful for you.
My opinion on the silent corruption is similar to your dismissal of using native format drives. If the file sits there in a corrupted state but I am able to repair that file, then the difference to me is inconsequential.
Who says you will be able to repair that file under FlexRAID at all? There are lots of horror stories on the FlexRAID forums of people encountering bugs, or the parity data becoming corrupt, etc.
As hard drives become larger and larger with the bits packed tighter and tighter, the stability of these systems is called into question. I'd rather have a filesystem that actively works to prevent a problem before you have the problem. Have you ever had to run chkdsk on a 3TB drive? I have... it takes a very long time. And even chkdsk only scrubs the metadata, not the actual data itself. Its estimated that firmware bugs in hard drives cause 5-10% of all silent corruption errors. Why wouldn't you want a file system that checksums the block of data, the metadata, and even the pointer to the metadata? Most filesystems store the metadata with the block of data. If the block is corrupt, then the metadata is useless. You need an independant source of metadata, and a way to check it.
Again, most of the discussion here is on predominantly static data, most of which is infrequently accessed. Even in a ZFS environment, the chances of picking up on an automatic healing of silent corruption is very low.
I don't think you get it. Static data has a higher chance of bit-rot. The more data is accessed, the more chance the hard drives' built in error correcting routines have the chance to fix it before it reaches the PC. If it can't do that this would be like UPS saying, "We guarantee that your package wasn't damaged when we picked it up." Not quite the guarantee you were looking for.
Which do you think is a bigger plus... having some chance of recovering lost data in the event of a catastrophic failure OR ensuring that one or two instances of silent corruption are corrected on the fly?
I've already stated my opinion on this matter.
Again, for home server use, having all the bells and whistles doesn't mean much if there are aspects of which you would rather not have to deal with. e.g. ZFS inability to add data on the fly to the primary array without having to make new vdevs or whatnot, all drives spinning up to read a single file and the associated wear and tear that has on a typical home usage scenario, not to mention power draw and there are many others.
Ugh... the power draw argument. We are talking about a few dollars a year in most areas of the the US.
And I'm not sure what you are referring to when you say " ZFS inability to add data on the fly to the primary array without having to make new vdevs or whatnot"
1. Not a point of comparison, anything ZFS can use, FlexRAID/SnapRAID/unRAID can use. Actually, being that FlexRAID/SnapRAID can reside in the windows space, it has a good chance of driver support as well.
Not really a pro for ZFS, this was more of an anti-con.
2. SnapRAID/FlexRAID do daily parity updates, during these updates, all the checking that impacts the data is done, this would be similar to your whole ZFS auto repair silent corruption thingo upon access of files.
Nope. Snapshot-RAID can not compensate for hard drive errors or guarantee the integrity of the data when the data is read... only when validating or restoring. If your parity data is corrupt... you have no chance of recovery. if you are "updating" invalid data, then your parity becomes just as corrupt, and you can only recover the corrupt data.
3. ZFS cannot add already filled drives. FlexRAID/SnapRAID/unRAID can add empty drives instantly as well. So this point is lost by ZFS.
I can't argue this point. Its a nice feature. But once again you are trusting of the native filesystem already having your data intact.
4. unRAID pools remain online while you are repairing. SnapRAID doesn't have pooling but depending on the solution you employ to pool your drives, the non impacted data is still available. It is only the lost data that is unavailable. In the case of unRAID it can also do emulate failed drives the way ZFS and hardware RAID does. I have tested this functionality quite a bit.
The point still stands in comparison of ZFS and FlexRAID.
6. For home use, who cares about this point? Why would I care to mix and match RAID levels in the same pool? In fact, with pooling software, I can do just that. I can have hardware RAID mixed with software RAID mixed with whatever else, it would be a mess and not worth doing however.
This is certainly an advanced use, and probably not recommended for home use, I'll admit. But the fact that you aren't using hardware RAID with ZFS keeps you from having to make/break RAID sets at the hardware level to do this, which adds an extra layer of complexity when using this with a user-space snapshot-RAID.
7. Again, for home use in storing of media files, how often does do your movies/music change? Most people know what these systems are designed for and what they are not good at, it is a personal decision. Many keep documents and photos separately or have them duplicated.
Surely, you jest.
8. Expand further on why this is important in a home environment? I have UPS on every one of my computers. How often are you pulling power to drives while the machine is running?
No, but you can. Obviously, this is the extreme scenario, but a cable can come loose, vibrations in the case can cause heads to go crazy, you never know. Not everyone has the foresight to use a UPS. What about hardware faults? A bad power supply?
9. Compression is useless for compressed data (music, movies and photos) so this point is useless for home media servers. For documents and the like, most people don't have terabytes upon terabytes of the stuff to worry about the small file savings.
If you only use your fileserver to store media, then you are limited in your vision.
10. Not relevant today for home server usage
What was the average hard drive size 10 years ago? What will be the size 10 years from now? Data accumulation is not getting stagnant or smaller. It's estimated that ZFS will last us for 30 years before it will be outdated. This is not just an enterprise problem... the very fact that you need a file-server in the first place should show you that this is relevant.
11. WHS handles that dedup stuff
Only for backups. Not for user-data. And I don't think this feature survived into WHS2011. ZFS can dedupe at the block level all
12. Plenty of backup software available, if using Crashplan for instance, then backup is automatic and so I don't think this is an important point of distinction for the inteded usage.
The point is... this is baked into ZFS. No need to seek out and validate a 3rd party tool
13. NTFS volumes can encrypt data
I never said it didn't. ZFS is a filesystem as well as a pooling/RAID system. This is also an anti-con for ZFS, and is very easy to setup.
14. Can't argue with Free... but you do realise that SnapRAID on Linux is also free.
Sure, but SnapRAID doesn't do pooling, so you have to use LVM or Greyhole. ZFS takes all of these features and presents them to the user as one unified toolset, rather than having to piece it all together.
Despite this, many people are happy to pay to have a single point of support. Not everyone is willing or able to put the time and effort in to self troubleshoot. Also, it is only free in so far as you don't require functionality from Windows for instance. If you need to virtualize windows for functionality (which many people still do) then it is not free anymore.
I don't get your point here. Who says you need to run Windows to run a fileserver? If you need a virtualization platform, then VMWare ESXi is free... Citrix XEN is free... KVM through Linux (ProxMox is awesome, btw) is free... even the core-hyperv windows 2008 (no GUI) OS is free.
And as for support... anyone can read or ask questions in the forums of their favorite product. As a one-man show, Brahim is very eager to use that community for end-user support... and I don't blame him.