Originally Posted by ajhieb
So is the latency on a USB3 drive, such that it will double the time needed to write new parity? That's what I'm trying to find out... how big of a difference is it actually going to make?
Also, if you write/modify as extensively as you do, why the choice of snapshot RAID? Wouldn't more traditional RAID be more appropriate?
It's not one factor but many.
First- USB drives read and write pretty slow. Although the theoretical spec for USB3.0 is pretty good- the reality is not. USB must get routed through your CPU too, (one of the reasons it's slower) so the performance can be effected by such a process and speed is variable. SATA3 is much faster in reality. So if you are reading and writing at twice the speed for the same amount of data it's going to take half as long, or twice as long depending on what side you are on.
Not really a super big deal. But if you double capacity from 2TB to 4TB now you need to write and read twice as much data with 4TB drives instead of 2TB drives. That's again making it worse.
So in a scenario where you have a USB drive reading and writing at an average of say 50MB sec, and a SATA drive doing twice that at 100MB/sec - and you have 4TB of data per drive rather than 2TB per drive it will literally take you 4 times as long.
That means for someone like Assassin with 2TB drive and 5 hour parity times (when data is added or moved) now becomes 20 hours to do the same thing. 5 hours isn't a problem. That's why he does not get the issue. 20 hours is a problem. And for someone who uses their server often having it running for 20 hours doing parity just sucks. That is why I feel how I do. I think if you are building a flexraid server you should spec it and design with the intention of having worst case scenario parity still able to be completed while you are sleeping and done in the morning so it doesn't effect you.
I can't run MCM on my TV shows and rename them and mess around with all the metadata while parity is running. It will throw off the integrity of the parity process. And running it a second time again when I am done it a pain in the ass too. I'd rather just never have that become a problem because my server is never busy doing it when I might want to use it.
Assassin can feel as he does. He's not wrong. He's just not seeing it how I see it. I'm not making this stuff up either. This is stuff I learned by living it.