Need alternative to FlexRAID w/ Pooling and snapshot - AVS Forum
Forum Jump: 
 
Thread Tools
post #1 of 19 Old 09-25-2012, 08:50 PM - Thread Starter
Senior Member
 
Diverge's Avatar
 
Join Date: Jan 2007
Location: Long Island, NY
Posts: 294
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Requirements:
  • Must do drive pooling and snapshot
  • Must be installable to Windows 7
  • Windows Media Center must be able to use pooled drive for recordings and live TV


I can't use FlexRAID because it has permission problems at the driver level that prevent files from being written and read at the same time - specifically Windows Media Center recording of live TV, and it's live TV buffer. Brahim is aware of the problems, and is supposed to fix them in the next driver update. But I was told 3 weeks ago it should be done in 3 weeks, now I'm being told in the next month. I'm not holding my breath anymore.

I know snapraid offers similar options as FlexRAID, but does anyone have firsthand experience that it will work for my requirements?

Anyone have any recommendations?
Diverge is offline  
Sponsored Links
Advertisement
 
post #2 of 19 Old 09-26-2012, 05:54 AM
Advanced Member
 
ncarty97's Avatar
 
Join Date: Mar 2006
Posts: 745
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 16
Why do you need such a large drive for the liveTV buffer? Whether you use snapraid, flexraid or something else, wouldn't it just be easier to have one drive as your OS/liveTV buffer drive and then archive your recordings nightly to the pooled drive? That's what I do and I have no issues at all.
ncarty97 is offline  
post #3 of 19 Old 09-26-2012, 10:35 AM - Thread Starter
Senior Member
 
Diverge's Avatar
 
Join Date: Jan 2007
Location: Long Island, NY
Posts: 294
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by ncarty97 View Post

Why do you need such a large drive for the liveTV buffer? Whether you use snapraid, flexraid or something else, wouldn't it just be easier to have one drive as your OS/liveTV buffer drive and then archive your recordings nightly to the pooled drive? That's what I do and I have no issues at all.

The large drive isn't just for media center, or live buffer. It hold all my movies and tv shows, as well as other stuff. My systems is basically a HTPC & NAS. The OS drive is a msata SSD, and there are only 4 sata ports for the 4 mechanical drives that were in a pool w/ flexraid. The systems is based around an Intel DQ77KB motherboard.in an Lian Li PC-Q16 case.

https://picasaweb.google.com/104979206991404158659/HTPCNASLianLiPCQ16 the 2 error screencaps at the end are the errors media center gives when you try to play a channel being recorded w/ FlexRAID, because it tries to read the recording as it's writing (recording). So you can't watch a channel and record at the same time, and live tv is a crap shoot if it works because it's basically the same thing. recording to buffer and then reading off it.
Diverge is offline  
post #4 of 19 Old 09-26-2012, 11:40 AM
Advanced Member
 
ncarty97's Avatar
 
Join Date: Mar 2006
Posts: 745
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 16
Quote:
Originally Posted by Diverge View Post

The large drive isn't just for media center, or live buffer. It hold all my movies and tv shows, as well as other stuff. My systems is basically a HTPC & NAS. The OS drive is a msata SSD, and there are only 4 sata ports for the 4 mechanical drives that were in a pool w/ flexraid. The systems is based around an Intel DQ77KB motherboard.in an Lian Li PC-Q16 case.
https://picasaweb.google.com/104979206991404158659/HTPCNASLianLiPCQ16 the 2 error screencaps at the end are the errors media center gives when you try to play a channel being recorded w/ FlexRAID, because it tries to read the recording as it's writing (recording). So you can't watch a channel and record at the same time, and live tv is a crap shoot if it works because it's basically the same thing. recording to buffer and then reading off it.

How big is your SSD drive? If it's not big enough to hand the live buffer and a a few recorded shows before they get archived to the pooled drive, it would probably be easier to buy a bigger OS drive than to try to find some pooling solution that you can use. I have a 160GB on my HTPC and it more than meets our needs. Windows 8 Storage spaces might suit you, but obviously the whole Win8 issue comes into hand.

Would DriveBender work? Not sure if that would produce the same problems or not, but I think it works in Windows 7.
ncarty97 is offline  
post #5 of 19 Old 09-26-2012, 01:22 PM - Thread Starter
Senior Member
 
Diverge's Avatar
 
Join Date: Jan 2007
Location: Long Island, NY
Posts: 294
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by ncarty97 View Post

How big is your SSD drive? If it's not big enough to hand the live buffer and a a few recorded shows before they get archived to the pooled drive, it would probably be easier to buy a bigger OS drive than to try to find some pooling solution that you can use. I have a 160GB on my HTPC and it more than meets our needs. Windows 8 Storage spaces might suit you, but obviously the whole Win8 issue comes into hand.
Would DriveBender work? Not sure if that would produce the same problems or not, but I think it works in Windows 7.

the SSD is only 64GB. While 2.5" SSD's are coming down in price, msata SSD's aren't following yet. I thought about buying a bigger one, but would rather not. Can't really put much on the 64GB, especially HD NFL Football games, they take up a lot. You kinda gave me an idea. I could just use 3 drives in the pool (2 data, 1 parity) and leave 1 outside the pool for mediacenter. At least for the time being, until the bug is fixed in FlexRAID. I'm still curious what else is out there, and if anyone has a similar system with snapraid and know if it works.

I was actually looking at drivebender, but didn't pull the trigger for a couple reasons.
1) it doesn't have snapshot, and I'd rather not use 2 different programs to pull it off. It only supports making duplicates of files, so it would sorta be equivalent to raid 1 (mirrored).
2) Not sure if I would run into same permissions problems - but I guess I could always try a trial. But reason #1 still exists.
Diverge is offline  
post #6 of 19 Old 09-26-2012, 06:20 PM
Member
 
wraslor's Avatar
 
Join Date: Feb 2009
Posts: 140
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 15
A combo of Snap raid and liquesce might be what you're looking for.
wraslor is offline  
post #7 of 19 Old 10-14-2012, 11:57 AM
Senior Member
 
Wilky13's Avatar
 
Join Date: Apr 2011
Posts: 376
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 17
Quote:
Originally Posted by Diverge View Post

the SSD is only 64GB. While 2.5" SSD's are coming down in price, msata SSD's aren't following yet. I thought about buying a bigger one, but would rather not. Can't really put much on the 64GB, especially HD NFL Football games, they take up a lot. You kinda gave me an idea. I could just use 3 drives in the pool (2 data, 1 parity) and leave 1 outside the pool for mediacenter. At least for the time being, until the bug is fixed in FlexRAID. I'm still curious what else is out there, and if anyone has a similar system with snapraid and know if it works.
I was actually looking at drivebender, but didn't pull the trigger for a couple reasons.
1) it doesn't have snapshot, and I'd rather not use 2 different programs to pull it off. It only supports making duplicates of files, so it would sorta be equivalent to raid 1 (mirrored).
2) Not sure if I would run into same permissions problems - but I guess I could always try a trial. But reason #1 still exists.

i had the same problem with Flexraid and WMC. I eventually had to remove one of my drives from the Flexraid pool and use it as the WMC liveTV/recordedTV location.

Sucks, because now it isn't covered by the parity drive anymore if it fails.

I have to return some video tapes....
Wilky13 is offline  
post #8 of 19 Old 10-18-2012, 12:39 PM
Newbie
 
lloydct's Avatar
 
Join Date: Mar 2007
Posts: 4
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by ncarty97 View Post

Why do you need such a large drive for the liveTV buffer? Whether you use snapraid, flexraid or something else, wouldn't it just be easier to have one drive as your OS/liveTV buffer drive and then archive your recordings nightly to the pooled drive? That's what I do and I have no issues at all.

How do you handle the nightly archiving to your pool?
lloydct is offline  
post #9 of 19 Old 10-18-2012, 01:25 PM
AVS Special Member
 
Dark_Slayer's Avatar
 
Join Date: May 2012
Posts: 2,285
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 50 Post(s)
Liked: 225
I'm late to the commenting party on this, but I believe you could add most of the drive back to your storage pool by partitioning it.

There are powershell scripts that you can task schedule and move day old recordings to the pooled partition after they are a day old (or just move them manually).

I don't think you'll find a solution for real time parity protection
Dark_Slayer is offline  
post #10 of 19 Old 10-18-2012, 02:01 PM
AVS Special Member
 
jrwalte's Avatar
 
Join Date: Jun 2007
Posts: 2,537
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
You just need to create an expert mode FlexRAID config instead of cruise control.

With expert mode, you can set your 7MC to record directly to the OS drive path instead of using the storage pool and then point your storage pool to the same folder. However, you couldn't use hard drive space past the 1 HD except to manually move them to another drive in the pool (and define the DVR HD as restricted so the storage pool doesn't add files to it). I would discourage this route in any setup, though. You do not want your Live TV Buffer on a snapshot parity. It will basically make your parity worthless because the buffer is constantly editing and deleting which is corrupting your snapshot.

You'd be surprised how quickly the live buffer can corrupt data. I accidentally deleted about 800GB of recordings due to moving the DVR drive to a new HTPC and pointing 7MC to the record folder but not telling the DVR to use the entire drive (it defaults to 200GB or so). So when the next recording came up, 7MC saw that it was over it's defined allowed space and deleted all the oldest recordings to make room. In the meantime, live TV was running for about 2 hours before I noticed this. When I went to use a deleted file recovery software, I could only get about 200GB of data back. That's how quickly the live TV buffer wrote all over the drive and corrupted the deleted files. This exact same scenario would play out on your snapshot parity if a drive failed and you were watching live TV before performing a snapshot update.

Only realtime RAID would work in what you are seeking, and that won't work because it won't be fast enough in writing throughput to do multiple recordings. The only setup that would be capable is hardware RAID5/6, and it isn't worth the investment.

Instead, you should look to set your DVR/Live Buffer on a drive/folder outside of your snapshot parity. If you want to parity backup your recordings, then you should use a script/program to move the files from the Recorded TV folder to your storage pool with snapshot parity. Another option is in expert mode, you can define folders for the snapshot parity to ignore (so you could set the live buffer folder to be skipped) and it won't corrupt your parity. I haven't been able to figure out how to get it to work.
jrwalte is offline  
post #11 of 19 Old 10-18-2012, 02:48 PM
AVS Special Member
 
Dark_Slayer's Avatar
 
Join Date: May 2012
Posts: 2,285
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 50 Post(s)
Liked: 225
Quote:
Originally Posted by jrwalte View Post

I haven't been able to figure out how to get it to work.

Sounds like an okay option. It comes down to preference of ignoring parity protection on a folder or completely isolating the live tv / recorded tv from the parity. There are lots of scripts supposedly, but it's not impossible to go once/week to your recorded tv folder manually hit Ctrl+A, Ctrl+X change to a storage folder and hit Ctrl+V. If the partition method is used, the on disk cut paste time is ridiculously fast. In any case you could just set it and come back later.

jrwalte, why do you mention you haven't been able to figure it out at the end of a description of how you figured it out?

I think parity protection on a live TV buffer falls into the "law of diminishing returns" When you do the math of hard drive failure probability coupled with absolute necessity of protecting the live tv buffer vs the cost and complexity it seems wise to ignore this rathole. Most live tv is repeated later or can be found again from streaming services, etc.
Dark_Slayer is offline  
post #12 of 19 Old 10-20-2012, 01:53 PM
AVS Special Member
 
jrwalte's Avatar
 
Join Date: Jun 2007
Posts: 2,537
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
My point was you can include the live TV buffer in the snapshot, but define a clause in the advanced settings to ignore files and or folders. I haven't been able to figure out the proper code to place in that advanced setting to get it to ignore folders. It isn't as simple as just adding "[drive]:\[folder to ignore]".
jrwalte is offline  
post #13 of 19 Old 10-22-2012, 06:09 AM - Thread Starter
Senior Member
 
Diverge's Avatar
 
Join Date: Jan 2007
Location: Long Island, NY
Posts: 294
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by jrwalte View Post

You just need to create an expert mode FlexRAID config instead of cruise control.
With expert mode, you can set your 7MC to record directly to the OS drive path instead of using the storage pool and then point your storage pool to the same folder...

I'm pretty sure this wouldn't work in my case. My OS drive is a 60GB msata SSD, which isn't big enough, and also not ideal for constant writes going to it all the time. I don't even have my Recorded TV folder set for parity. I just need to use the mechanical HD, which is part of the pool, for my recording drive.

This past week Brahim remotely connected to my system and took some logs using process monitor to see what is going on with the issue. He said it's in flexraid's driver, at the kernel level, that it is reporting the wrong error code or something like that. Anyway, he hopes to fix it in the next driver update.

Here's my bug entry in the tracker if anyone is interested http://bug.flexraid.com/view.php?id=247
Diverge is offline  
post #14 of 19 Old 12-13-2012, 09:20 PM
Advanced Member
 
etc6849's Avatar
 
Join Date: Dec 2006
Location: Irmo, SC
Posts: 618
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 17 Post(s)
Liked: 63
You CAN record directly to Flexraid (including the live tv buffer), by changing it from a removable volume (of course, you still have to have flexraid installed on the same PC as WMC).

Try these steps:
Go to your raid-f/pool in the web client.
Go to Preferences and Settings.
Go to Run-Time Properties
Click the arrow by Advanced Properties.
Scroll down and change the Removable field from true to false.
Restart Flexraid.

I just downloaded Flexraid last week, so maybe my version is newer and that's why it works, but it seems to be working after changing the volume to a fixed drive. Are there any drawbacks to doing this?

Premise, a FREE home automation program. Open-source Z-Wave Premise Module found here.
etc6849 is online now  
post #15 of 19 Old 12-14-2012, 04:32 AM
AVS Special Member
 
Nevcairiel's Avatar
 
Join Date: Mar 2010
Location: Germany
Posts: 1,004
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 107
I have mine set to Fixed drive as well, removable had some permission issues for me. Another thing was that i changed it from "FlexRAIDFS" to "NTFS" in one of the same advanced properties. After that, it behaves just like any other drive for me.
Nevcairiel is offline  
post #16 of 19 Old 01-28-2013, 08:42 AM
Newbie
 
tenshou1's Avatar
 
Join Date: Jan 2013
Posts: 5
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by wraslor View Post

A combo of Snap raid and liquesce might be what you're looking for.

Besides having SnapRaid and Liquesce, are there any other alternatives?

Liquesce is very buggy at the moment.
tenshou1 is offline  
post #17 of 19 Old 01-28-2013, 08:59 AM
Advanced Member
 
ncarty97's Avatar
 
Join Date: Mar 2006
Posts: 745
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 16
Quote:
Originally Posted by lloydct View Post

How do you handle the nightly archiving to your pool?

Sorry missed this. I use the built in TV Archive that comes with WHS, but you can easily setup a task to do it as well if you don't use WHS.
ncarty97 is offline  
post #18 of 19 Old 01-28-2013, 10:36 AM
Member
 
wraslor's Avatar
 
Join Date: Feb 2009
Posts: 140
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 15
Also it should be noted that in the latest version Liquesce has disabled the ability to share the drive over the network.
wraslor is offline  
post #19 of 19 Old 01-29-2013, 09:28 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 21,394
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 649
Quote:
Originally Posted by Diverge View Post

Requirements:
  • Must do drive pooling and snapshot
  • Must be installable to Windows 7
  • Windows Media Center must be able to use pooled drive for recordings and live TV


I can't use FlexRAID because it has permission problems at the driver level that prevent files from being written and read at the same time - specifically Windows Media Center recording of live TV, and it's live TV buffer. Brahim is aware of the problems, and is supposed to fix them in the next driver update. But I was told 3 weeks ago it should be done in 3 weeks, now I'm being told in the next month. I'm not holding my breath anymore.

I know snapraid offers similar options as FlexRAID, but does anyone have firsthand experience that it will work for my requirements?

Anyone have any recommendations?


Local installed scratch disc ???

Move over to server after recording completed ????

-

"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
Reply Home Theater Computers

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off