This is something I've been looking into but, unfortunately it appears that the only way to reduce the reference frames is to re-encode.
However, I know that some of the files I am having trouble with have an aspect ratio of 2.35:1 (which would be 1920x816 instead of 1920x1080), so the additional 264 pixels in the height is "wasted" in showing the black bars at the top and the bottom. When I get home I am going to try and resize the videos to 1920x816 without re-encoding and see if they'll play right on the Oppo. If that works, I won't have to worry about reducing the number of reference frames (unless it has more than 5). This wouldn't work though for videos with an aspect of 16:9 since you'd be chopping off the image at the top and the bottom.