Mfusick's How to build an affordable 30TB Flexraid media server: Information Requested.! - Page 145 - AVS | Home Theater Discussions And Reviews
Forum Jump: 
Reply
 
Thread Tools
post #4321 of 4328 Old Today, 12:36 AM
AVS Special Member
 
DotJun's Avatar
 
Join Date: Dec 2012
Posts: 1,124
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 239 Post(s)
Liked: 68
Quote:
Originally Posted by Ken.F View Post
Verify is a bit for bit verification of every file in the pool. I have 10 data drives with 30 TB on them so I guess 14 hours isn't too bad. I only do those once a month so it's not that inconvenient. An update will check for changes in the pool and will calculate parity just for the files that changed since the last update. That doesn't take very long unless you move a lot of data between updates. I have updates scheduled every night so it's usually pretty fast.

Ahhh ok. I'm trying out snapraid right now and it's pretty dang fast as far as the parity calcs go but the sync command that I run might be the same as the update command that you run. I'm not sure if snapraid has a verify/validate command so I guess we can't really compare them.
DotJun is offline  
Sponsored Links
Advertisement
 
post #4322 of 4328 Old Today, 12:56 AM
Member
 
Quaraxkad's Avatar
 
Join Date: May 2004
Location: Port Orange, FL
Posts: 68
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 40
Quote:
Originally Posted by DotJun View Post
Ahhh ok. I'm trying out snapraid right now and it's pretty dang fast as far as the parity calcs go but the sync command that I run might be the same as the update command that you run. I'm not sure if snapraid has a verify/validate command so I guess we can't really compare them.
Snapraid sync is the same as FlexRAID update. Snapraid scrub -p 100 is the same as FlexRAID verify (both products can also do partial array verifies). Snapraid diff is the same as FlexRAID quick-validate. Snapraid has no analog to FlexRAID validate. Also note that the speed reports are not comparable, you may get a reading of 20MBps in FlexRAID and 500MBps in snapraid, when in reality they are both processing exactly the same data and will finish at the same time. FlexRAID essentially reports the weakest link, while Snapraid sums up the totals across all drives being read/written. The FlexRAID method is a little more "sane" as far as a relevant number goes because it's equally valid no matter how many drives are in the array. The snapraid number is artificially inflated by simply adding new drives, even when the resulting process ends up taking longer than it did with fewer drives.
ajhieb likes this.

Last edited by Quaraxkad; Today at 12:59 AM.
Quaraxkad is offline  
post #4323 of 4328 Old Today, 08:48 AM
AVS Special Member
 
Ken.F's Avatar
 
Join Date: Jul 2010
Location: West Rockhill, PA
Posts: 2,244
Mentioned: 4 Post(s)
Tagged: 0 Thread(s)
Quoted: 344 Post(s)
Liked: 351
Big fat fail on today's validate. With the settings of Processes 8, I/O Buffer 50MB and Threads 8 it ran really smooth during verify yesterday, but during validate today all processor cores were pegged at 100%. Memory use was good during both verify and validate. I noticed the validate beginning to slow down at 54%. About an hour later it was only up to 56%. I hit pause it and let it sit for about a half hour but the processor use remained at 100% and the little spinning icon in the FlexRAID configuration never stopped. I hit abort and waited another half hour and still no change, processor still at 100%. Resource Monitor showed no disk activity whatsoever on any of the pool volumes.

Until I figure this out I'm going to cancel my validate schedule and run verify weekly. Time to go reboot the server.
Ken.F is offline  
post #4324 of 4328 Old Today, 09:08 AM
Member
 
Quaraxkad's Avatar
 
Join Date: May 2004
Location: Port Orange, FL
Posts: 68
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 40
Quote:
Originally Posted by Ken.F View Post
Big fat fail on today's validate. With the settings of Processes 8, I/O Buffer 50MB and Threads 8 it ran really smooth during verify yesterday, but during validate today all processor cores were pegged at 100%. Memory use was good during both verify and validate. I noticed the validate beginning to slow down at 54%. About an hour later it was only up to 56%. I hit pause it and let it sit for about a half hour but the processor use remained at 100% and the little spinning icon in the FlexRAID configuration never stopped. I hit abort and waited another half hour and still no change, processor still at 100%. Resource Monitor showed no disk activity whatsoever on any of the pool volumes.
This sounds like what FlexRAID does when it encounters an unreadable file caused by bad sector. It's not graceful at all when this happens and can result in BSODs, freezing, or just never being able to end the process. Check Event Viewer for drive or controller related errors. Are you running StableBit, HDSentinel, or anything like that?
Quaraxkad is offline  
post #4325 of 4328 Old Today, 10:07 AM
AVS Special Member
 
Ken.F's Avatar
 
Join Date: Jul 2010
Location: West Rockhill, PA
Posts: 2,244
Mentioned: 4 Post(s)
Tagged: 0 Thread(s)
Quoted: 344 Post(s)
Liked: 351
Quote:
Originally Posted by Quaraxkad View Post
This sounds like what FlexRAID does when it encounters an unreadable file caused by bad sector. It's not graceful at all when this happens and can result in BSODs, freezing, or just never being able to end the process. Check Event Viewer for drive or controller related errors. Are you running StableBit, HDSentinel, or anything like that?
No, not running StableBit or anything like it. I find it odd that yesterday's verify passed with "all bits match" and today a validate decided to hang.

Checked the Event Viewer and didn't see any drive or controller errors. I did find it spit out a low memory warning at 5:27AM. I was still asleep at that time so I never saw the warning. When I checked Resource Monitor this morning the memory use was below 6GB and the validate appeared to be running OK other than all of the processor cores were at 100%.

5:27:01 - "Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: svchost.exe (736) consumed 3473154048 bytes, ehrecvr.exe (2712) consumed 259993600 bytes, and ehshell.exe (10004) consumed 229203968 bytes."
That svchost.exe PID 736 is the same process that has been giving me memory trouble all along.

5:27:07 - "Application popup: Windows - Out of Virtual Memory : Your system is low on virtual memory. To ensure that Windows runs properly, increase the size of your virtual memory paging file. For more information, see Help."

My page file is manually set for 24072 MB (1.5x 16GB RAM) on a HDD not included in the pool. Should I let the system manage it? I'm afraid if I do that it will just keep expanding the page file until the disk fills up.

Between 10:42 and 10:52 last night I got 5 of these errors:
"The server was unable to allocate from the system nonpaged pool because the server reached the configured limit for nonpaged pool allocations."
FlexRAID wasn't even doing anything at that time.
Ken.F is offline  
post #4326 of 4328 Old Today, 01:42 PM
Member
 
Quaraxkad's Avatar
 
Join Date: May 2004
Location: Port Orange, FL
Posts: 68
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 40
I would temporarily Disable all non-essential services that are running under that svchost process. Shown in your screenshot here: http://www.avsforum.com/forum/attach...5&d=1425084056. Most of those are optional and you'll never need them so they can't be permanently disabled anyway. You also have a *ton* of processes running that could likely be disabled on startup, I highly doubt you need half of what's running.
Quaraxkad is offline  
post #4327 of 4328 Old Today, 03:12 PM - Thread Starter
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 26,004
Mentioned: 9 Post(s)
Tagged: 0 Thread(s)
Quoted: 297 Post(s)
Liked: 1290
I feel for you Ken. This is about the point where I format the drive clean and start over. Thinking it will fix, then wait to see what I install that brings it back.

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #4328 of 4328 Old Today, 05:25 PM
AVS Special Member
 
Ken.F's Avatar
 
Join Date: Jul 2010
Location: West Rockhill, PA
Posts: 2,244
Mentioned: 4 Post(s)
Tagged: 0 Thread(s)
Quoted: 344 Post(s)
Liked: 351
Quote:
Originally Posted by Quaraxkad View Post
I would temporarily Disable all non-essential services that are running under that svchost process. Shown in your screenshot here: http://www.avsforum.com/forum/attach...5&d=1425084056. Most of those are optional and you'll never need them so they can't be permanently disabled anyway. You also have a *ton* of processes running that could likely be disabled on startup, I highly doubt you need half of what's running.
Thank you for taking the time to look over all of the information and I appreciate the advice. The system is 100% rock-solid stable with plenty of resources to spare until FlexRAID starts running an operation. Do you really think FlexRAID running through 40GB of memory is caused by some extra services running in the background? Maybe it could make a difference but I just don't suspect that being the root of the problem.
BTW, the svchost.exe 736 doesn't even show up in Process Explorer when FlexRAID is idle.

Quote:
Originally Posted by Mfusick View Post
I feel for you Ken. This is about the point where I format the drive clean and start over. Thinking it will fix, then wait to see what I install that brings it back.
No, I'm not going to format it and start over. But If I did, I know exactly which program to install that will bring the problem back. FlexRAID.

Thanks again to everyone for your help. I'm getting frustrated so I'm going to put this on the back burner for a while. I have update and verify running without any drama so I don't even need to run validate.

One last screenshot.
Attached Thumbnails
Click image for larger version

Name:	a much needed break.jpg
Views:	4
Size:	913.4 KB
ID:	578929  
Ken.F is offline  
Reply Home Theater Computers

Tags
Flexraid , Intel Core I5 4670k 3 4ghz Lga 1150 Quad Core Desktop Processor , Intel Core I3 4130 3 4 3 Fclga 1150 Processor Bx80646i34130 , Asrock , Asrock Z87m Extreme4 Lga1150 Intel Z87 Chipset Ddr3 Quad Crossfirex Quad Sli Sata3 Usb3 0 Microatx M , Asus , Asus Us , Asus Computer , Asus Components , Norco , Asus Computer International Direct , Flexraid Raid F , Seagate , Seagate Hard Drives , Seagate Freeagent Theater 1080p Hd Media Player Stcea201 Rk , Hitachi , Dell , Windows 7 Vista Xp Media Center Mce Pc Remote Control And Infrared Receiver , Hp Oem Window Media Center Mce Pc Remote Control And Infrared Receiver For Windows7 Vista Xp Home Pr , Intel , Intel Core I5 3570 3 4 Ghz Processor , Intel Pentium G2020 2 9ghz Lga 1155 Dual Core Desktop Processor , Intel Pentium G3220 3 0ghz Lga 1150 Dual Core Desktop Processor , Amd , Amd A10 6700 Richland 4 2ghz Socket Fm2 65w Quad Core Desktop Processor Amd Radeon Hd Ad6700okhlbox , Amd A6 5400k 3 6ghz Socket Fm2 Dual Core Desktop Processor , Amd A8 5600k 3 6ghz Socket Fm2 Quad Core Desk
Gear in this thread - 1080p by PriceGrabber.com



Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off