x264 Encoding Options for Hardware Compatibility & DXVA

[ExDeus]

This thread is a spin-off of some posts regarding how to get MKV files encoded with x264 to work with hardware acceleration (a/k/a DirectX Video Acceleration, or DXVA) on ATI HD2000, HD3000, HD4000 and NVIDIA 8000 and 9000 video cards.

The following settings will ensure hardware compatibility with players like the PS3, and also allow smooth DXVA playback on ATI and NVIDIA graphics cards when using the Cyberlink H.264 decoder included with PowerDVD, or the free MPC Video Decoder (32-bit, 64-bit), available as a standalone DirectShow filter, or as part of Media Player Classic HomeCinema. While the settings detailed below ensure compatibility with either the Cyberlink or the MPC decoder, it is really the Cyberlink decoder that demands the stream both comply with L4.1 and declare L4.1 in the stream.

The settings have been tested to work with PowerDVD, ZoomPlayer, Media Player Classic HomeCinema, using the Haali Splitter or the Nero Digital Parser, and using the Overlay Mixer, VMR7, and VMR9 Windowless/Renderless video renderers.


The Bottom Line
Everybody should be encoding HD content (1080p, 720p) to Profile High @ Level 4.1. For smooth playback, "--level 4.1" should be used to mark the file as compatible when encoding. The best MeGUI profile for x264 is the DXVA-HD-HQ profile.

Edit: Avoid B-pyramids --- they still have an outstanding fix required.
Note: Older Nvidia driver versions exhibited decoding errors with B-pyramid enabled. This is no longer an issue since Forceware 169.21.

The MeGUI DXVA-HD-HQ profile options:

--level 4.1 --ref 4 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --filter -1:-1 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxrate 50000 --me umh


Everybody should be encoding SD content (576p, 480p, or less) to Profile High @ Level 3.1 For smooth playback, "--level 3.1" should be used to mark the file as compatible when encoding. The best MeGUI profile for x264 is the DXVA-SD-HQ profile.

The MeGUI DXVA-SD-HQ profile options:

-level 3.1 --ref 8 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --filter -1:-1 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 14000 --vbv-maxrate 17500 --me umh


The Details
The associations responsible for the HD DVD and Blu-ray Disc standards approved the H.264 profile/level High@L4.1 for HD content, and High@L3.1 for SD content. As such, the Cyberlink decoder enforces compliance with the levels.

That has resulted in two implications:

1. H.264 video must comply with the L4.1 and L3.1 limitations for the Decoded Picture Buffer (DPB) size, which limits the video resolution and number of reference frames.

2. H.264 streams must declare L4.1 or L3.1 in the "level_idc" flag (set by --level with x264) in order to achieve smooth DXVA playback.


The Settings
Keep in mind that, no matter the resolution, it is still generally recommended to only use between 1-5 reference frames, usually centering on 3.

A patch was added in x264 revision 721 that changes the way --ref works, allowing it to specify the total Decoded Picture Buffer (DPB) size. Now, B-frames can be used without the DPB size exceeding the --ref value.

The DPB size is the max at the given resolution, so if your resolution is between the resolutions given, use the lower DPB size (i.e., 1920x816 is between 1920x720 and 1920x864, so you can have a max DPB of 5).

Resolution: Max DPB Size (x264 revision 721+ encoder settings)
L4.1
1920x1088: 4 (--ref 4)
1920x864: 5 (--ref 5)
1920x720: 6 (--ref 6)

L4.1
1280x720: 9 (--ref 9)
1280x648: 10 (--ref 10)
1280x588: 11 (--ref 11)
1280x540: 12 (--ref 12)


1280x498: 13 (--ref 13)
1280x462: 14 (--ref 14)
1280x432: 15 (--ref 15)
1280x405: 16 (--ref 16)

L3.1
720x(any): 8 (--ref 8)
720x576: 11 (--ref 11)
720x480: 13 (--ref 13)

Resolution: Max DPB Size (x264 revision < 721 encoder settings, bframes+b-pyramid count towards DPB size)
L4.1
1920x1088: 4 (--ref 3 --bframes 3 --no-b-pyramid)
1920x864: 5 (--ref 4 --bframes 3 --no-b-pyramid)
1920x720: 6 (--ref 5 --bframes 3 --no-b-pyramid)

L4.1
1280x720: 9 (--ref 8 --bframes 3 --no-b-pyramid)
1280x648: 10 (--ref 9 --bframes 3 --no-b-pyramid)
1280x588: 11 (--ref 10 --bframes 3 --no-b-pyramid)
1280x540: 12 (--ref 11 --bframes 3 --no-b-pyramid)
1280x498: 13 (--ref 12 --bframes 3 --no-b-pyramid)
1280x462: 14 (--ref 13 --bframes 3 --no-b-pyramid)
1280x432: 15 (--ref 14 --bframes 3 --no-b-pyramid)
1280x405: 16 (--ref 15 --bframes 3 --no-b-pyramid)

L3.1
720x(any): 8 (--ref 10 --bframes 3 --no-b-pyramid)
720x576: 11 (--ref 10 --bframes 3 --no-b-pyramid)
720x480: 13 (--ref 12 --bframes 3 --no-b-pyramid)


Retrofitting MKVs or other H.264 videos
Existing MKV, MP4, TS or AVI files that work with DXVA, but do not play back smoothly, can have their level_idc value changed, if the H.264 video stream actually complies with Profile High@L4.1 (HD) or L3.1 (SD).


Courtesy of DJBlu, you can use one of his IDC Changer utilities to automatically change the level_idc flag for MKV or AVI files. Note that they are intended for use on 720p files, because they only need the level_idc flag changed, while 1080p files usually exceed L4.1 specs. If you have an L4.1 1080p file, IDC Changer will work for it, too, but your decoder or hardware may choke (errors or video corruption) when you come across video that exceeds the L4.1 specs.

IDC Changer comes in three varieties:

IDC Changer GUI v0.3 - GUI version for changing single files
IDC Multi-Changer GUI v0.1 - GUI version for changing multiple files at once
IDC Changer Command-Line v0.1 - CLI version for batch scripts, etc.


To manually change your files with much more work, continue on.

This process is for MKV files and has been adapted from a process originally described by DJBlu.

Download the following programs:

MKVtoolnix
MKVExtractGUI
H264info
Optional: AVInaptic

Extract MKVtoolnix into a folder.
Extract MKVExtractGUI into the same folder.
Extract H264info into another folder (can be the same folder).
Optional: Extract AVInaptic into another folder (can be the same folder).

Open your MKV with MKVExtractGUI. Check Options menu > Tabs > TimeCode. Check the boxes for all the video, audio, and subtitle tracks you want to keep and click Extract.

Once all are extracted, you will have separate files for your video, audio and subtitle tracks, plus TimeCode_TrackX.txt files for each track.

Open H264info.

Set Input = Extracted .h264 file
Set Output = new_file_name.h264

In the "Level (1 - 5.1)" box, enter 4.1 for an HD stream, or 3.1 for an SD stream.

Click Start and wait for the stream to be fixed.

Open the MKVtoolnix mkvmerge GUI (mmg.exe).

Click add and select the new_file_name.h264 file created by H264info.

Click add and select a file for each audio and subtitle track you need to add.

In the Tracks box, click on the "MPEG-4 part 10 ES" video track.

At the far right of the Timecodes box at the bottom of the window, click Browse and select the TimeCode_TrackX.txt file for your video track (probably Track1).

Click the "Format specific options" tab.

In the FPS box, enter the frames per second of your original MKV file. If you don't know the framerate, you can use the AVInaptic tool to open the original MKV file to get all manner of information, including the FPS. If your video is FILM content, the likely choice is "24000/1001". If your video is progressive NTSC VIDEO, the likely choice is "60000/1001". If your video is interlaced NTSC VIDEO, the likely choice is "30000/1001". If your video is PAL VIDEO, the likely choice is "25".

Click on your audio track in the Tracks box.

Click on the "General track options" tab.

At the far right of the Timecodes box at the bottom of the window, click Browse and select the TimeCode_TrackX.txt file for your audio track (probably Track2). Do the same for any subsequent tracks.

Create an "Output filename" in the box at the far bottom of the window and click "Start muxing".

Wait for the new MKV to be remuxed and you should have an MKV file with smooth DXVA playback.

[Herc]

needs a sticky and ExDeus deserves mucho kudos.

[arfster]

*sounds of applause*

*supports stickification*


Just to add a simple summary for those confused by the legion of details above: you can fix most (almost all?) of your current 720p x264 files to play with acceleration, but not most 1080p ones (some will work, but most break the 4.1 DPB buffer limit - with 720p files it needs a crazy number of ref frames to do this so it's a much rarer problem).

Hopefully as encodes become more aware of the problems of too many ref frames on hardware compatibility they will begin to stick to 4.1 standards, and most new 1080p encodes will then work fine also.

[rebkell]

Originally Posted by arfster

*sounds of applause*

*supports stickification*

Echoes the sounds of applause, any encoding hints and tips are much appreciated by me in a big way, doom9 is a great place, but can be a bit intimidating, I don't mind putting in the time to learn things, but really appreciate some specific guidelines like in this thread.

[AaronBerquist]

I also support a sticky of this thread!

[brew99]

Thanks for the concise guide......just a note with MKVExtractGUI to get the timecode files to produce, you need to "check" that option in the OPTIONS - TAB - Timecode. This was defaulted to unchecked when i installed.

Thanks again

[mickwall]

Great work Ex.

[bonker99]

Well after a week of banging my head up against the wall of 12yrs old noise that HD forums seem to be at the mo and basically gaining nothing, this has moved me on a whole bunch.

Nice to see some *facts* for a change!

Thanks very much.

[Valnar]

ExDeus and some other folks both here and Doom9 did a great job figuring this all out. I use AutoMKV to create transparent encodes of my DVD's. Same quality - smaller size - DXVA support. I am generally not worried about fitting them to removable media because there would be no point. It'll easily be smaller than a DVD-5 and if media compatibility is important, I recommend DVD Rebuilder Pro and keep them in DVD format.

You'd have to create or edit an existing .XML profile to get this, but here is a quality oriented "transparent" (CRF - basically 1 pass) profile that I use.

 --pass 1 --crf 18 --bframes 3 --no-psnr --mixed-refs --trellis 1 --ref 3 --filter -2,-1 --subme 6 --direct auto --vbv-maxrate 14000 --me umh --level 3.1 --merange 16 --weightb --b-rdo --bime --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct

For HD and L4.1, I use this:

 --pass 1 --crf 18 --bframes 3 --no-psnr --mixed-refs --trellis 1 --ref 3 --filter -2,-1 --subme 6 --direct auto --vbv-maxrate 25000 --me umh --level 4.1 --merange 16 --weightb --b-rdo --bime --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct

Some other parameters are x264 defaults and they get inserted by AutoMKV, but you get the idea. The current beta of AutoMKV can be found here.

-Robert

[FLorp]

Outstanding work in finding a solution for this problem, guys. Well done!

Couple of questions:

1) Is the 5.1 value stored in a header of the h.264 stream, or does it affect values throughout? In other words, is it something that could be found and changed in a hex-editor?

2) Is there a command-line version of H264info? Normally I love GUIs, but I have quite a few MKVs to convert, and would love to batch it.

Thanks again,

Florp

[tetsuo55]

Originally Posted by FLorp

Outstanding work in finding a solution for this problem, guys. Well done!

Couple of questions:

1) Is the 5.1 value stored in a header of the h.264 stream, or does it affect values throughout? In other words, is it something that could be found and changed in a hex-editor?

2) Is there a command-line version of H264info? Normally I love GUIs, but I have quite a few MKVs to convert, and would love to batch it.

Thanks again,

Florp

1) The header is stored at every start of every packet, so that means there are thousands of them in a stream, a PS3 however can be fooled by hex-editing only the first packet

2) i have no idea sorry

[akmofo]

How long does the total procedure take? (I know this will vary based on processor type, movie length, etc.) Is this a pretty quick procedure? ie < 1 hour? Or like a 5 hour/movie procedure?

[arfster]

Originally Posted by akmofo

How long does the total procedure take? (I know this will vary based on processor type, movie length, etc.) Is this a pretty quick procedure? ie < 1 hour?

Something like that. I can't remember exactly, but it didn't take that long.

At some point someone will write a little batch script to automate it, you might want to wait till then.

[anthony.s]

I don't think the process is CPU dependent but as it writes out a new stream the time will be limited by our HD. On my laptop with one HD a 4Gb file takes <5 mins to demux, process and remux each which I'm sure on a multi-drive system would be reduced.

[elec999]

You can play x264 on PS3. question.
Thanks

[mickwall]

Chaps,

Bit of a question please.

I've got a 720p x264 encode that plays choppy through powerdvd codec, so i ran it through the process and it worked fine, great job.

But i've got a 1080p encode that just plays black screen, i ran it through the process and its still black screen.

Is there something i've missed? Is there something else i need to do to the file? What are the parameters i should be looking out for?

Cheers.

[Valnar]

Originally Posted by mickwall

Chaps,

Bit of a question please.

I've got a 720p x264 encode that plays choppy through powerdvd codec, so i ran it through the process and it worked fine, great job.

But i've got a 1080p encode that just plays black screen, i ran it through the process and its still black screen.

Is there something i've missed? Is there something else i need to do to the file? What are the parameters i should be looking out for?

Cheers.

The options it was encoded with may still be out of L4.1 spec. Download avinaptic and post the output.

-Robert

[mickwall]

Robert,

Here you go, this the version that i "fixed".
If you need any further info, please let me know.

[ Generic infos ]

Play duration: 02:23:27 (8607.264 s)
Container type: matroska
Creation time: 12/01/2008 07:01:49 UTC
Number of streams: 2
Type of stream nr. 1: video (V_MPEG4/ISO/AVC)
Type of stream nr. 2: audio (A_DTS)
Audio streams: 1
Muxing Application: libebml v0.7.7 + libmatroska v0.8.1
Writing Application: mkvmerge v2.1.0 ('Another Place To Fall') built on Aug 19 2007 13:40:07

[ Relevant data ]

Resolution: 1920 x 800
Width: multiple of 32
Height: multiple of 32

[ Video track ]

Codec ID: V_MPEG4/ISO/AVC
Resolution: 1920 x 800
Frame aspect ratio: 12:5 = 2.4
Pixel aspect ratio: 1:1 = 1
Display aspect ratio: 12:5 = 2.4
Framerate: 23.809524 fps

[ Audio track ]

Codec ID: A_DTS
Channels (container): 6
Sample rate: 48000 Hz

[ About H.264 encoding ]

SPS id: 0
  Profile: High@L4.1
  Num ref frames: 8
  Chroma format idc: YUV 4:2:0
PPS id: 0 (SPS: 0)
  Entropy coding type: CABAC
  Weighted prediction: No
  Weighted bipred idc: B slices - implicit weighted prediction
  8x8dct: Yes
  Custom intra4X4 luma: 
      6   7  10  16
      7   7  11  17
     10  11  12  20
     12  13  20  16
  Custom intra4X4 chromau: 
     16  16  16  16
     16  16  16  16
     16  16  16  16
     16  16  16  16
  Custom inter4X4 luma: 
     10  13  28  41
     13  14  32  84
     28  32  41 111
     41  46 111  16
  Custom inter4X4 chromau: 
     16  16  16  16
     16  16  16  16
     16  16  16  16
     16  16  16  16
  Custom intra8X8 luma: 
      9   9  10  10  11  16  21  29
      9   9  10  10  11  16  21  29
     10  10  10  10  11  16  22  31
     10  10  10  11  11  17  23  33
     11  11  11  11  11  19  25  36
     12  12  13  13  14  20  27  40
     15  15  16  16  21  27  31  45
     20  20  21  22  36  40  45  16
  Custom inter8X8 luma: 
     12  13  15  18  20  70 163 255
     13  13  16  18  20  72 170 255
     15  16  17  19  21  81 190 255
     18  18  19  21  23  96 228 255
     20  20  21  23  25 120 255 255
     33  34  36  39  45  32 255 255
     64  66  71  80 164 255 255 255
    151 155 169 192 255 255 255  10

[ Profile compliancy ]

Profile to check: MTK PAL 6000
Resolution: 1920 x 800 > 720 x 576
Framerate: 23.809524 <> 25

Is it the dreaded num_ref_frames? If so, how can that be changed?

Cheers, Mick

[arfster]

Originally Posted by mickwall

Is it the dreaded num_ref_frames? If so, how can that be changed?

Only be re-encoding -(

Basically the "fix" in this thread is bypassing a decoder problem, the files themselves are fine. However, most 1080p encodes break the 4.1 ref_frame limits, and there's no way round that.

[Valnar]

Originally Posted by mickwall

Is it the dreaded num_ref_frames? If so, how can that be changed?

Yep. Only can fix by reencoding.

After worrying about DXVA for quite some time now, I've come to the personal conclusion that for PC playback, nothing beats a fast Core2Duo. At least then you know you can play any file. But even so, for the purpose of "posterity", it's still wise to encode so that DXVA support is maintained. When more hardware players become available, you'll be happy you did.

Robert

[DJBlu]

IDC Changer.zip

Here you go guys. Just use this file to locate your 720p or lower resolution files. And select the new IDC. Click Change and bobs your uncle.

Enjoy.

You don't need to demux or remux. Just find the MKV/AVI and it will change it.

[FLorp]

Originally Posted by mickwall

Robert,

Here you go, this the version that i "fixed".
If you need any further info, please let me know.

Framerate: 23.809524 fps

Cheers, Mick

At a guess, I think the new framerate is your problem. I ran into the same thing when I used the 24000/1001 preset in mmg. It should result in 23.976blahblah, but for some reason it often results in 23.809524. If you type in the exact frame rate from your source mkv, it should fix it.

Florp

[FLorp]

Originally Posted by DJBlu

Attachment 98809

Here you go guys. Just use this file to locate your 720p or lower resolution files. And select the new IDC. Click Change and bobs your uncle.

Enjoy.

You don't need to demux or remux. Just find the MKV/AVI and it will change it.

Holy crap, DJBlue, you RULE! That works!!

[DJBlu]

Originally Posted by FLorp

Holy crap, DJBlue, you RULE! That works!!

No worries m8

Let me know of any that it doesn't work on

The program will only pick up 4 profiles at the moment I am still finding the others.

[DJBlu]

IDC Changer.zip

Updated Version.

[TheShadowRunner]

DJBlu, would it be possible for you to make an IDC Changer build that doesn't rely on .NET?
Thanks,

TSR

[7o9]

Originally Posted by DJBlu

Updated Version.

Thanks for the update DJBlu, now it works fine for me.

I have 'fixed' some High@L5.1 MKV files, and it certainly had an effect on the Cyberlink decoder. I'm still doing something wrong as the CPU usage didn't go down, but I'm quite sure that is me.

Any chance you can add High@L5.1 to your program as selectable option (perhaps with a warning that it won't be accelerated), so I can revert my MKV's to their original state for more testing?

Great job.

[ChrisII]

I'm a HUGE x264 fan, and this was a pretty good guide w/ respect to the settings.

If you're looking for a GUI, meGUI is pretty good and can be downloaded @ www.x264.nl (bottom of page).

Make sure you have Microsoft Framework 3.0+ if you're using with Vista.

As for the question about encoding times, I've only done 720x400 copies myself, and it really depends on the filter chain that you use. If you dont' do any type of pre-processing, and are using the resolution above (720x400), you'd manage about 50fps (or 1.66 realtime for NTSC video).

I know that the Xbox 360 supports High @ 4.1, but apparently it's a little hit-or-miss. I do Main@4.1 instead, you don't really loose out much IMO

[brew99]

Originally Posted by DJBlu

Attachment 98882

Updated Version.

many thanks DJBlu, this is so easy to use, and I now havy accelerated 720p mkv files!

Just a note for anyone using MPC with Cyberlink h262 decoder filter in Vista. I found that I need to have MPC output (in the options) set to "EVR (Vista/.Net)" as apposed to "EVR Custom". This is the only way I could get acceleration.

[brew99]

Originally Posted by 7o9

Thanks for the update DJBlu, now it works fine for me.

I have 'fixed' some High@L5.1 MKV files, and it certainly had an effect on the Cyberlink decoder. I'm still doing something wrong as the CPU usage didn't go down, but I'm quite sure that is me.

Any chance you can add High@L5.1 to your program as selectable option (perhaps with a warning that it won't be accelerated), so I can revert my MKV's to their original state for more testing?

Great job.

DJBlu.....as 7o9 requested, I would also like to see an option to revert back to HL@5.1 if possible. It seems as though much more testing of mkv's will be in the near future, in order to get HA and stutter free playing.