Sounds like something else is interfering with your video playback. I have a C2D 4300 with an Intel 965 video card (mobo integrated), 1gb RAM, and it plays back fine on a 47" LCD screen.
FFDShow: No problem playing back 720p, h264, AC3 files (slight stutter on 1080p)
CoreAVC: No problem playing back 1080p, h264, AC3 files
Zoomplayer as the player, though Media Player Classic works too
Since my computer has less impressive specs than yours, I'm suspecting a software issue.
Here's what I did to get my software the way I wanted it:
--(Uninstall everything related in your case. Use Insurgent from http://www.cccp-project.net/
to remove extra stuff)
--Install your DVD player (PowerDVD in my case)
--Install the Combined Community Codec Pack. (My codec pack of choice. Very light.)
--Use Radlight Filter Manager to change PowerDVD h264 to less preferred, FFDshow for normal preferred for h264, and CoreAVC to preferred for h264