MythTV works great except for playing my HD mkv x.264 files. My set up:

Mythbuntu 8.04 with the current nvidia driver, 169.12

AMD Athlon 64 X2 6400+ Windsor 3.2GHz Socket AM2 125W Dual-Core
2 GB RAM
nVidia GeForce 7050 PV

I've tried mplayer with various options and niceness, best luck is having a movie play for like 5 mins, then the screen freezes

For example:
mplayer -benchmark -ac hwdts -vfm ffmpeg -lavdopts fast=1:skiploopfilter=all:threads=8

I've tried to increase the niceness too. No luck.

Hmmm...That CPU should cut it I would think. I had no problems playing back 720P mkv with my old lowly xp2800 and certainly no probs with my current core2 at 3ghz with no skiploop requirements. Are you running the video from a remote file server? When the screen freezes, is the machine hung altogether? I'm wondering if the machine is overheating and crashing or something? Also, does it happen on any matroska container file or do you have only one that you've tried? Try without hwdts and also I'd certainly drop the number of threads down though since you have a dualcore. That could be hurting you.
-Trouble

Hmmm...That CPU should cut it I would think. I had no problems playing back 720P mkv with my old lowly xp2800 and certainly no probs with my current core2 at 3ghz with no skiploop requirements. Are you running the video from a remote file server? When the screen freezes, is the machine hung altogether? I'm wondering if the machine is overheating and crashing or something? Also, does it happen on any matroska container file or do you have only one that you've tried? Try without hwdts and also I'd certainly drop the number of threads down though since you have a dualcore. That could be hurting you.
-Trouble
I'm running all on one system, a la front-end/back-end.

Yes when the screen freezes the PC is pretty much hung, I can ssh in, but its slow.

Pretty much any matroska file, I've been testing with just one mainly, but others crash too. I got the files from a friend and they work for him.

I've tried with and without hardware DTS and without threads at all. Same issue. But shouldn't hardware DTS lessen the load on the PC? Anyway I want my Denon to process the DTS.

Also CPU was at 27 C after a crash. Got it from cat /proc/acpi/thermal_zone/THRM/temperature.

Critical is 90 C

cat /proc/acpi/thermal_zone/THRM/trip_points
critical (S5): 90 C
passive: 80 C: tc1=4 tc2=3 tsp=60 devices=CPU0
active[0]: 80 C: devices= FAN

You might want to be sure you are using a recent-ish build of mplayer

http://sourceforge.net/project/showfiles.php?group_id=185512&package_id=260588

Easy point and click 32 bit and 64 bit debs there

Try using threads=2 - using 8 on a 2 core processor doesn't make make sense. I play h.264 files all day long with 2 threads and slightly less processing speed and I don't have any problems. I also use one of the newer builds of mplayer.
My $0.02
I also use x64 builds of mythbuntu and they are frontend only.

Try using threads=2 - using 8 on a 2 core processor doesn't make make sense. I play h.264 files all day long with 2 threads and slightly less processing speed and I don't have any problems. I also use one of the newer builds of mplayer.
My $0.02
I also use x64 builds of mythbuntu and they are frontend only.
I'm running MPlayer 1.0rc2-4.2.3.

I'll try 2 threads.

Scared to declare victory, but I've got an HD moving up and running now for the longest ever so far. Didn't upgrade mplayer, but used 2 threads. I found this mplayer command options for HD x.264 content on the MythTV Wiki:
http://www.mythtv.org/wiki/index.php/HD_Playback_Reports

mplayer -fs -zoom -quiet -vo xv -monitoraspect 16:9 -lavdopts threads=2:fast:skiploopfilter=all -sws 0

I added -ac hwdts so my Denon processes the surround:

mplayer -fs -zoom -quiet -vo xv -monitoraspect 16:9 -lavdopts threads=2:fast:skiploopfilter=all -sws 0 -ac hwdts

Movie is playing along for 12 minutes now, before mplayer would hang about 1-3 minutes into playing a file.