[SOLVED] - Community Feedback: Finding a good balance between size, quality and conversion time. - AVS Forum | Home Theater Discussions And Reviews
Baselworld is only a few weeks away. Getting the latest news is easy, Click Here for info on how to join the Watchuseek.com newsletter list. Follow our team for updates featuring event coverage, new product unveilings, watch industry news & more!


Forum Jump: 
 
Thread Tools
post #1 of 9 Old 04-29-2014, 09:51 PM - Thread Starter
Newbie
 
Phylum's Avatar
 
Join Date: Apr 2014
Posts: 5
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10

I hope this is the right place to post a question like this.  My knee-jerk instinct was to drop this in HTPC but I feel like this may be more appropriate.  If not, please correct me.

 

I'm super new to the world of HTPC and all that it encompasses, so please use that filter when reading this & formulating a response.

 

I stood up a basic HTPC quite a while ago now that met our immediate needs and so far am quite pleased with it. Its a Windows 8.1 machine with a Ceton InfiniTV card and we rely on Windows Media Center with storage on a Synology NAS via iSCSI LUN. (Although that will change soon to a UNC path directly to the NAS vs UNC path to a share of an iSCSI LUN on a machine.) While doing some maintenance recently, I discovered that a typical hour-long show in WTV format is roughly 7GB. As a result, I have something like 3TB+ of media, which is about 449 WTV files (e.g.: 181 some odd episodes of Seinfeld, a couple dozen episodes of several shows, a handful of moves - like 6). 

 

I found a product called MCEBuddy that helps automate the conversion process with what looks like loads of advanced features.  (Way above my head anyway.)  MCEBuddy seems to leverage ffmpeg, handbrake and some other utilities for removing commercials and converting the videos into some other format.  In my limited use, about a week now at most, it seems to get the job done with relative ease.  Because I'm sort of 'in a rush' to accomplish a specific task revolving the home network, I'm using a profile that converts it into an 'unprocessed' MKV file.  The 'conversion' process is super quick, but I'm still left with a massive file that's just maybe 130MB smaller than the original. Sure, I could remove commercials - and I want to once I get around the subtitle timing shift issue - but I doubt that's going to bring this down substantially. (Meaning, I don't expect the removal of commercials to result in a 3-4GB file.) 

 

So, I need a little help, but not specifically for MCEBuddy.  This is more about the general process seasoned AV/HTPC folks use to get their videos into 'acceptable' formats, which I realize is a very broad and generic statement.

 

Now, I'm not some super AV expert, but I do know that I like, prefer & want good quality videos. I'd like the keep the same level of quality as it was broadcast in because I think the picture quality is darn good. (e.g.: If I start with a 1080p WTV file, I'd like to have a 1080p file at the end, not 720) It would be great to get those file sizes down, but I don't want to do so at the expense of quality.  However, if going down a smidgen meant a barely noticeable difference in quality and a significantly smaller file (e.g.: 1-2GB) then I'm all ears!  

 

For starters, I think I need a little help re-adjusting my expectations:

  1. I'm hoping to cut the file sizes in half, maybe more like the 1-2GB range, without sacrificing [too much] quality. I just don't want to go from ultra crystal clear 4K quality to 1993 VHS quality.   (Obvious exaggeration) 
  2. I'm looking for a process that won't take half a day to convert a file.  I did a test run once [with MCEBuddy] and found it took somewhere between 4 & 6.5 hours to convert a single file.  That to me was ages too long, especially when I have so many files files to convert.  (That's 93-94 days of converting non-stop at ~5 hours per video!)

Remember, I'm speaking with zero understanding about video files or video container formats.  Its a little difficult for me to understand (conceptually) why it takes long to convert, and why files are so huge.

 

 

My objective here is to get some feedback / advice from the community on conversion processes that

  • maximizes quality
  • retains captions / subtitles (requirement for our home)
  • retains multi-channel audio (e.g.: if it was broadcast in 5.1 even though I only have 2.1)
  • produces [significantly] smaller file sizes
  • converts as quickly as possible

 

 

I'm hoping for responses that include things like

  • What application(s) you're using to convert your WTV files
    • MC-TV Converter, AVS Video Converter, some Slysoft product etc.
  • What your conversion process looks like
    • feed WTV file to ffmpeg, then resultling output to handbrake, then to some other utility or a 2 pass conversion process etc.
  • What settings you find work best

 

 

Lastly, some information about the systems available for conversion:

  • HP ML150G6 Windows 8.1, 1xE5504, 10GB RAM, GT 610
  • Dell XPS 720 Windows 7, Q6700, 8GB RAM, dual GTX 560Ti 448 core edition
  • Dell OptiPlex 960 Windows 7, Q9400, generic video card (but I could pull one of the GTX 560 from the above system if having SLI doesn't greatly benefit the conversion process)

 

Bonus Questions:

  1. Is there a hardware configuration that greatly helps to reduce conversion times?
    1. e.g.: X core processor, X video card(s), X video driver, X RAM, SSD etc.
    2. e.g.: or specific hardware:
      1. Intel vs AMD processor
      2. Desktop grade processor versus server grade processor
      3. Intel versus ATI versus nVidia
      4. Intel quicksync drivers over some other driver
      5. 20GB+ RAM etc.
  2. Is there a 'formula' for gauging roughly how much time conversions take based on CPU and or Core count? (Big Assumptions here of course: We're assuming a typical HD WTV formatted file, and we're assuming we'd be using the same conversion process & video file for each test.)
    1. e.g.: 4xcores = 4 hours; 8xcores = 2 hours
    2. e.g.: 1CPU with 4xcore = 4 hours; 2CPU's with 4xcores each = 1 hour

 

 

Looking for as much advice & constructive criticism as possible!

Phylum is offline  
Sponsored Links
Advertisement
 
post #2 of 9 Old 04-29-2014, 11:25 PM
AVS Special Member
 
Matt L's Avatar
 
Join Date: Aug 2000
Location: Flushing, MI
Posts: 5,033
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 105 Post(s)
Liked: 115
I'm a firm believer in leaving quality alone, cut the commercials and so on but only you can be the judge of what quality is acceptable. Are these archival programs? Are you going to go back and watch them in 4 or 5 years? If you and you move to a 4K display how will they look? My thought is hard drives are cheap, keep the quality the best you can and add more space as needed.
Phylum likes this.
Matt L is offline  
post #3 of 9 Old 04-30-2014, 07:13 AM
AVS Addicted Member
 
Kelson's Avatar
 
Join Date: Jul 2004
Location: Delaware - The First State (USA)
Posts: 11,325
Mentioned: 6 Post(s)
Tagged: 0 Thread(s)
Quoted: 1085 Post(s)
Liked: 668
I collect a lot of HDTV on my server -- been doing it for years. I have used WMC as my DVR (I'm not familiar with MCE buddy) but primarily use a TiVo. I use Video ReDo to process all my HDTV captures. My first recommendation to anyone doing this is to go out and buy Video ReDo H.264 (a.k.a. VRD) -- it is made for this and makes life very easy.

The average network show has ~20 min of commercials per hour. Commercials are encoded and broadcast with the same bitrate as the show so eliminating commercials will decrease the file size by about 1/3. That is a major savings right off the top. VRD has auto commercial detect and a batch mode. It is not perfect -- none are -- because many shows will have segments where there is no black frame(s) between the last commercial and the continuation of the show. No black frame = no way to detect the break point to cut. If you are not picky about having every bit of commercial removed and you have a lot of videos to process you can set up a batch mode to detect and delete and walk away. Otherwise, I generally review the cuts and adjust the endpoints as needed -- takes me about 5 min per 1 hr show.

VRD can save the edited file in the most popular formats. That includes saving as .wtv with all the meta-data so you can use it with WMC as a native recording. Unfortunately, if the cable co has copy-protected the recorded show, WMC's DRM will not let you do anything with the file except play it through WMC or an extender -- you can't edit or re-code the recording.

If you want a smaller file and you are willing to live with the picture degradation, VRD can re-code the original MPEG-2 to H.264. The VRD default is 2-pass encoder with 30% reduction in file size going from MPEG-2 to H.264. This does a pretty good job with minimal degradation -- can't tell any difference with brightly lit fast action scenes, but the deep blacks get a little crushed and shades of black become splotchy. I don't like what I see on a 65" plasma so I never re-code anything. Depending on your display size and quality and personal taste, you may be able to tolerate it and even more compression to reduce the file to 50% or less. You will have to experiment to determine how far you can go. When it comes to re-coding, the fastest processor wins. VRD and Handbrake are both multi-threaded so they use all the cores of an i7 processor. For a 41 min show (1 hr show with all the commercials removed), it takes VRD about 2 hr on an i7 to do a 2-pass re-code with 30% size reduction. You can set up a VRD batch and just let it run overnight.
Phylum likes this.

- kelson h

The bitterness of poor quality lasts long after the sweetness of the low price is forgotten . . . life is too short to drink bad wine

Kelson is offline  
post #4 of 9 Old 05-01-2014, 04:49 PM - Thread Starter
Newbie
 
Phylum's Avatar
 
Join Date: Apr 2014
Posts: 5
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by Matt L View Post

I'm a firm believer in leaving quality alone, cut the commercials and so on but only you can be the judge of what quality is acceptable. Are these archival programs? Are you going to go back and watch them in 4 or 5 years? If you and you move to a 4K display how will they look? My thought is hard drives are cheap, keep the quality the best you can and add more space as needed.

 

You bring up some really good points!  I do quite a bit of hosting/entertaining so I try to keep 'community faves' or just plain awesome movies. But when it comes to TV shows, its mostly just those in the house, and the casual stopper-byer, that binge watches a season.  But, for what its worth, I re-watch all of the Office periodically, done it 3 times in the past 5 years.  Tickles me every time.  Seinfeld is relatively new to the collection so I've only watched it all the way through once, but I hunt-n-peck episodes as the mood strikes.

 

Storage is relatively cheap, but its hard to justify to the wife 4x4TB drives at $130/each, let alone another $500+ NAS + drives.  She's like "just dump the data".

 

Overall though, there probably should be a process in place to purge shows/movies that see little to no activity every so often.

Phylum is offline  
post #5 of 9 Old 05-01-2014, 05:33 PM
AVS Addicted Member
 
Kelson's Avatar
 
Join Date: Jul 2004
Location: Delaware - The First State (USA)
Posts: 11,325
Mentioned: 6 Post(s)
Tagged: 0 Thread(s)
Quoted: 1085 Post(s)
Liked: 668
Quote:
Originally Posted by Phylum View Post

Overall though, there probably should be a process in place to purge shows/movies that see little to no activity every so often.
I find an effective way to do that is to lock your storage capacity. I have a Media-PC and server with 16 TB of storage where 12TB is for BD rips. I can generally store 35 BD/TB. When things fill up I make some decisions as to what we will probably never watch again and delete it. It's not too hard to cull from a 420 disk collection. I'm determined to not add any more storage to the server -- that forces things.

- kelson h

The bitterness of poor quality lasts long after the sweetness of the low price is forgotten . . . life is too short to drink bad wine

Kelson is offline  
post #6 of 9 Old 05-01-2014, 08:42 PM
AVS Special Member
 
Matt L's Avatar
 
Join Date: Aug 2000
Location: Flushing, MI
Posts: 5,033
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 105 Post(s)
Liked: 115
Yeah, I'm getting there myself. I have nowhere near the capacity you have, only 4T, and it's filling up. I'll have to cull the heard...
Matt L is offline  
post #7 of 9 Old 05-02-2014, 01:42 PM - Thread Starter
Newbie
 
Phylum's Avatar
 
Join Date: Apr 2014
Posts: 5
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by Kelson View Post

I collect a lot of HDTV on my server -- been doing it for years. I have used WMC as my DVR (I'm not familiar with MCE buddy) but primarily use a TiVo. I use Video ReDo to process all my HDTV captures. My first recommendation to anyone doing this is to go out and buy Video ReDo H.264 (a.k.a. VRD) -- it is made for this and makes life very easy.

The average network show has ~20 min of commercials per hour. Commercials are encoded and broadcast with the same bitrate as the show so eliminating commercials will decrease the file size by about 1/3. That is a major savings right off the top. VRD has auto commercial detect and a batch mode. It is not perfect -- none are -- because many shows will have segments where there is no black frame(s) between the last commercial and the continuation of the show. No black frame = no way to detect the break point to cut. If you are not picky about having every bit of commercial removed and you have a lot of videos to process you can set up a batch mode to detect and delete and walk away. Otherwise, I generally review the cuts and adjust the endpoints as needed -- takes me about 5 min per 1 hr show.

VRD can save the edited file in the most popular formats. That includes saving as .wtv with all the meta-data so you can use it with WMC as a native recording. Unfortunately, if the cable co has copy-protected the recorded show, WMC's DRM will not let you do anything with the file except play it through WMC or an extender -- you can't edit or re-code the recording.

If you want a smaller file and you are willing to live with the picture degradation, VRD can re-code the original MPEG-2 to H.264. The VRD default is 2-pass encoder with 30% reduction in file size going from MPEG-2 to H.264. This does a pretty good job with minimal degradation -- can't tell any difference with brightly lit fast action scenes, but the deep blacks get a little crushed and shades of black become splotchy. I don't like what I see on a 65" plasma so I never re-code anything. Depending on your display size and quality and personal taste, you may be able to tolerate it and even more compression to reduce the file to 50% or less. You will have to experiment to determine how far you can go. When it comes to re-coding, the fastest processor wins. VRD and Handbrake are both multi-threaded so they use all the cores of an i7 processor. For a 41 min show (1 hr show with all the commercials removed), it takes VRD about 2 hr on an i7 to do a 2-pass re-code with 30% size reduction. You can set up a VRD batch and just let it run overnight.

 

Sorry for the delay in responding.  I drafted this after the other comment but got busy at home our new little one that's starting to crawl.

 

Heck of a response - I really appreciate your insight!

I know nothing of TiVo's and how they work.  I wanted one years ago but I didn't have cable then.  I was under the impression they moved from to a subscription model, is that right?  If so, I'm not interested in spending another $X/month (or year) just for a TiVo.  Then again, I have no idea of their benefits so I'm foolishly quick to pass judgement.  If it revolutionizes the current home theater setup, it could be worth it.  The biggest gripe right now is that I don't have a remote for the PC requiring a mouse at least.  (I did buy a PlayStation remote and used something like BluMote to get it working but it seems to get disconnected frequently which is frustrating.  Can't tell if its the remote, the iogear Bluetooth dongle, or the PC.  I'll troubleshoot that further later)

 

Do you have any idea how VRD compares to comskip or ShowAnalyzer?  I like the fact that, if I'm understanding you correctly, you can manually see where its going to make the cut and approve it before actually committing.  That's excellent!  In my brief testing with comskip, it seemed to work well and the donator version is supposedly faster & better.

 

I suppose 30% of 3GB, 7GB or 15GB is better than nothing.  Its a pretty decent dent and I don't have to sacrifice quality.  And as tastes come & go, or as space runs low, we'll purge.

I don't have any i3+ systems in the house yet.  I'm trying to understand a bit more about Intel QuickSync, which processors have it, which chipsets support it and whether there may be a lethal combination to maximize effectiveness.

Phylum is offline  
post #8 of 9 Old 05-02-2014, 01:43 PM - Thread Starter
Newbie
 
Phylum's Avatar
 
Join Date: Apr 2014
Posts: 5
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10

I really appreciate your comments here - thanks so much for helping to change & shape my perspective.

 

Happy HTPC'ing!

Phylum is offline  
post #9 of 9 Old 07-26-2014, 03:48 PM
Member
 
smcinaz's Avatar
 
Join Date: Nov 2012
Posts: 16
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
It is weird to me that you are using WMC in Windows 7 (or 8) and yur file sizes are so big, I am using it on my machine and an average 1 hour .wtv hdtv file is just a hair over a gb and is a bit smaller after converting to mkv's.
smcinaz is offline  
Sponsored Links
Advertisement
 
Reply Networking, Media Servers & Content Streaming

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off