Handbrake doesn't seem to be fully compressing Bluray rips - AVS Forum
Forum Jump: 
 
Thread Tools
post #1 of 14 Old 05-16-2014, 09:02 AM - Thread Starter
Newbie
 
AudioBeast's Avatar
 
Join Date: Mar 2014
Posts: 8
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10

Hello folks,

 

Condensed version: When I try to compress my largest movies the output file is only 1GB smaller. Why is this and how do I fix it?

 

First of all, this forum has been extremely helpful when I started my Home Theater and HTPC journey last year. Once again I am in need of your assistance! I have over 560 HD movies on my HTPC and more than 450 are uncompressed Blurays in MKV format. As you can imagine it is taking up considerable amount of space on my HDDs (I have a total of 16TB for storage alone). I was impressed with how well Handbrake is able to compress films without any noticeable artifacts and flaws while watching it on my 106" screen and Epson 5020. I compressed two films using the following settings (got these from AVS Member Someone65478):

 

Anamorphic: Strict
Filter: Off, Default, Off, Off, Deblock on the left
Video Codec: H264 MKV
Framerate: Same as source

Quality: Constant Quality set at 16 (I know most people use 18 but I wanted to preserve the most I could)
 
Use Advanced Tab
Reference Frames: 4
Maximum B-Frames: 4
Cabac, 8x8 Transform, Weighted P-Frames all check
Pyramidal B-Frames: Normal (Default)
Adaptive B-Frames: Optimal
Adaptive Direct Mode: Automatic
Motion Est Method: Uneven Multi-Hexagon
Subpixel Motion Est: 10:QPRD in all frames
Motion Est Range: 16
Partition Type: All
Trellis: Always
 
"ref=4:bframes=4:b-adapt=2:direct=auto:me=umh:subme=10:analyse=all:trellis=2"

 

The films were the first Chronicles of Narnia and The Dilemma and their sizes where pretty much halved! I muxed them with their original HD audio and it looks and sounds perfect. I compared the uncompressed films with it's compressed counterpart and could not find any flaws! Now I moved on to compressing the rest of my movies (starting from largest file size to smallest) but the films I tried to compress using these settings (The Godfather III and The Hunger Games) only compressed them by 1 GB! The GF III only went from 38GB to 37GB after 20 hours (i5-4670K CPU)!

 

My question is why wasn't this compressed like the other films or at least to a greater degree? And how can I remedy this?

 

Thanks for bearing with my long post! You guys rock!

AudioBeast is offline  
Sponsored Links
Advertisement
 
post #2 of 14 Old 05-16-2014, 03:28 PM
AVS Special Member
 
Aleron Ives's Avatar
 
Join Date: Oct 2009
Posts: 2,832
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 322 Post(s)
Liked: 374
You need to use a higher CRF. The compression ratio will always vary wildly between movies in CRF mode depending on how compressible each one is. Movies with lots of action and visual effects will be larger than dramas where people just stand around talking the entire time, and movies recorded on actual film will be larger than movies recorded digitally, because the film grain counts as "detail" that requires extra bitrate to represent. An old, grainy action movie will be huge at CRF 16. You can reduce the size a little bit by increasing deblocking to 1:1 or 2:2 (by detecting the grain as blocking and thus removing it), but that will decress the sharpness of the image and probably won't reduce the size that much. CRF 16 may be appropriate for animation, but it's overkill for live action. The only other way to reduce the size is to downscale to 720p.
ilovejedd likes this.
Aleron Ives is offline  
post #3 of 14 Old 05-16-2014, 03:33 PM - Thread Starter
Newbie
 
AudioBeast's Avatar
 
Join Date: Mar 2014
Posts: 8
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by Aleron Ives View Post

You need to use a higher CRF. The compression ratio will always vary wildly between movies in CRF mode depending on how compressible each one is. Movies with lots of action and visual effects will be larger than dramas where people just stand around talking the entire time, and movies recorded on actual film will be larger than movies recorded digitally, because the film grain counts as "detail" that requires extra bitrate to represent. An old, grainy action movie will be huge at CRF 16. You can reduce the size a little bit by increasing deblocking to 1:1 or 2:2 (by detecting the grain as blocking and thus removing it), but that will decress the sharpness of the image and probably won't reduce the size that much. CRF 16 may be appropriate for animation, but it's overkill for live action. The only other way to reduce the size is to downscale to 720p.

Thanks for the response! Yeah, all this is still new to me. Your explanation makes sense, however, I prefer to not go below 1080p. When a film is more dialogue driven, by how much should I increase the CRF by? Thanks!

AudioBeast is offline  
post #4 of 14 Old 05-16-2014, 03:51 PM
AVS Special Member
 
Aleron Ives's Avatar
 
Join Date: Oct 2009
Posts: 2,832
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 322 Post(s)
Liked: 374
You should ideally use the same ratefactor for every movie. The easiest way to tell what your ideal RF is is to take a few clips from a few movies (e.g. 30 seconds of action), encode those scenes at various ratefactors, and then pick the highest RF where you don't notice any artifacts. If you're totally new to detecting video artifacts, you can start at something like CRF 30 (which should make the artifacts pretty obvious) and reduce the RF until you find the highest one where you can't see the artifacts anymore. You'll then know to use that RF for all of your movies.

It's worth noting that you shouldn't scrutinise the videos too much, because what matters is real viewing conditions. You can make any screenshot look bad by enlarging it and staring at it with your face 3" away from the screen, but that's (hopefully) not how you watch movies, so such a comparison isn't really valid. If you can't see artifacts when playing the movie at full speed while sitting at your normal viewing distance, then it doesn't really matter if you can see artifacts in individual screenshots.

CRF 18 is generally considered to be the point where the encode will be transparent to the source, but everyone has different tolerance levels for artifacts. There's no point in using an absurdly small RF if it doesn't look any better to your eyes (except perhaps for the warm squishy feeling you get inside from wasting space).
ilovejedd and AudioBeast like this.
Aleron Ives is offline  
post #5 of 14 Old 05-16-2014, 03:54 PM - Thread Starter
Newbie
 
AudioBeast's Avatar
 
Join Date: Mar 2014
Posts: 8
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10

I really appreciate the feedback. I'll definitely try that. Are the settings I am currently using (other than CRF) ok or are they too much?

AudioBeast is offline  
post #6 of 14 Old 05-16-2014, 04:04 PM
AVS Special Member
 
Aleron Ives's Avatar
 
Join Date: Oct 2009
Posts: 2,832
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 322 Post(s)
Liked: 374
One setting that might help a little would be to increase the maximum consecutive B-frame limit from 4 to 5-6. You could also allow more than 4 reference frames for PC playback, but then your 1080p files might not be compatible with hardware decoders, such as those in Blu-ray players and DVRs. More than 5 reference frames may not yield much benefit for live action content, anyway.
AudioBeast likes this.
Aleron Ives is offline  
post #7 of 14 Old 05-16-2014, 04:44 PM
Advanced Member
 
ElJimador's Avatar
 
Join Date: Mar 2011
Posts: 532
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 37 Post(s)
Liked: 86
I Handbrake all my movies too and use CQ 21 for blurays and 19 for DVDs. Granted, I'm watching on a 40" TV and if you're watching on a 106" screen then I could definitely see why you'd want a lower CQ setting for your movies, however 16 is extremely low for blurays. You may find some where it still compresses by half but on the whole your compression at that setting is going to be so little that it won't even be worth the time and effort to Handbrake them at all. If I were you I'd try maybe CQ 19 and work from there. Also keep in mind that the file sizes you're going to get can vary wildly based on the content. For example I have a lot of old movies and movies with a lot of grain typically don't compress as much. So even at CQ 21, Wizard of Oz after I ran it through Handbrake still wound up being over 70% of the full rip file size. Same for Giant (which was a 10+ hour encode btw on my i7 laptop). So if you still have some that don't compress all that much it's not that you're doing anything wrong. At some point I checked a sample of 30-40 movies I had compressed at CQ 21 and on average the resulting file sized were 35-40% of the original rips but the range varied everywhere from under 10% to over 70%. (That's w/the non-Handbrake supported HD audio tracks muxed back in + all English commentary and other audio tracks kept also btw). So look at what kind of compression you get on average to decide if it's worthwhile for you and don't get thrown too much by the ones that wind up nearly the same size.
pittsoccer33 and AudioBeast like this.
ElJimador is offline  
post #8 of 14 Old 05-16-2014, 07:28 PM - Thread Starter
Newbie
 
AudioBeast's Avatar
 
Join Date: Mar 2014
Posts: 8
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10

Got it! Thanks Aleron Ives and ElJimador!

AudioBeast is offline  
post #9 of 14 Old 05-17-2014, 04:30 AM
Advanced Member
 
tman247's Avatar
 
Join Date: Sep 2007
Location: UK
Posts: 875
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 9 Post(s)
Liked: 31
First, a small recommendation. Try 'VidCoder', which is a great (simpler) front-end to Handbrake. I've switch across recently with some great results. Handbrake is a brilliant tool, but all the advanced settings can be daunting. Next, why are you re-encoding from 1080P to 1080P? Other than taking a ridiculous amount of time, you won't really see any benefit in file size. A good quality 720P rip (with Variable Frame rate and a CQ setting of 14-16) will produce a file a 1/3 of the size of a full 1080P rip, but with little loss of quality at normal viewing distances. I keep my best backups at 1080P w/FLAC, and other ones at 720P w/AAC. Saves a lot of space in the process as well. Just something to think about.
tman247 is online now  
post #10 of 14 Old 05-17-2014, 04:44 AM
AVS Special Member
 
ajhieb's Avatar
 
Join Date: Jul 2009
Posts: 1,485
Mentioned: 7 Post(s)
Tagged: 0 Thread(s)
Quoted: 362 Post(s)
Liked: 406
Quote:
Originally Posted by tman247 View Post

First, a small recommendation. Try 'VidCoder', which is a great (simpler) front-end to Handbrake. I've switch across recently with some great results. Handbrake is a brilliant tool, but all the advanced settings can be daunting. Next, why are you re-encoding from 1080P to 1080P? Other than taking a ridiculous amount of time, you won't really see any benefit in file size. A good quality 720P rip (with Variable Frame rate and a CQ setting of 14-16) will produce a file a 1/3 of the size of a full 1080P rip, but with little loss of quality at normal viewing distances. I keep my best backups at 1080P w/FLAC, and other ones at 720P w/AAC. Saves a lot of space in the process as well. Just something to think about.

Just out of curiosity, why would you re-encode to variable frame rate? It's still going to be a 24fps stream when you're done, right?

RAID protection is only for failed drives. That's it. It's no replacement for a proper backup.
ajhieb is online now  
post #11 of 14 Old 05-17-2014, 10:02 AM
Member
 
BillBragg's Avatar
 
Join Date: Dec 2004
Location: Sanford, FL.
Posts: 108
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
You might want to consider installing the Handbrake nightly build ... currently at svn6196. With your CPU and the HD4600 graphics those conversions will drop down to about 10 minutes when you select the H.264 QSV mode. That mode uses your onboard video hardware for decode, de-interlace, and re-encode.

At 10 minutes, you could do a lot more experimenting with the settings to see what suits you compression wise.
BillBragg is offline  
post #12 of 14 Old 05-17-2014, 10:09 AM
Advanced Member
 
ElJimador's Avatar
 
Join Date: Mar 2011
Posts: 532
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 37 Post(s)
Liked: 86
Quote:
Originally Posted by BillBragg View Post

You might want to consider installing the Handbrake nightly build ... currently at svn6196. With your CPU and the HD4600 graphics those conversions will drop down to about 10 minutes when you select the H.264 QSV mode. That mode uses your onboard video hardware for decode, de-interlace, and re-encode.

At 10 minutes, you could do a lot more experimenting with the settings to see what suits you compression wise.

I would be careful about this. There was a whole thread here recently about the quality of the intel quick sync encoding and while it's apparently gotten pretty good the consensus was that old, slow CPU encoding is still going to give you better quality and I doubt that someone who is watching on a 106" screen and has been using CQ 16 for blurays until now is going to be interested in that. Wouldn't hurt for him to test it of course but doesn't seem to me like what the OP is aiming for.
ElJimador is offline  
post #13 of 14 Old 05-17-2014, 03:29 PM
AVS Special Member
 
Aleron Ives's Avatar
 
Join Date: Oct 2009
Posts: 2,832
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 322 Post(s)
Liked: 374
Quote:
Originally Posted by ajhieb View Post

Just out of curiosity, why would you re-encode to variable frame rate?

You wouldn't, unless the source had mixed 24/30 fps content, which wouldn't happen in a Blu-ray movie. If it was a TV show old enough to have both film and video components, then VFR might be appropriate.
ajhieb likes this.
Aleron Ives is offline  
post #14 of 14 Old 05-17-2014, 04:05 PM
AVS Special Member
 
ajhieb's Avatar
 
Join Date: Jul 2009
Posts: 1,485
Mentioned: 7 Post(s)
Tagged: 0 Thread(s)
Quoted: 362 Post(s)
Liked: 406
Quote:
Originally Posted by Aleron Ives View Post

You wouldn't, unless the source had mixed 24/30 fps content, which wouldn't happen in a Blu-ray movie. If it was a TV show old enough to have both film and video components, then VFR might be appropriate.
.Thank you. As always, you are a fountain of information.

RAID protection is only for failed drives. That's it. It's no replacement for a proper backup.
ajhieb is online now  
Reply Home Theater Computers

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off