Are you sure is not the source?
I'm just asking cause I'm experiencing more or less the same issue as you but it's usually with my Dish box. In order to listen to TV content at a "good" listening level, I need to turn up the volume quite high actually (especially for the HD channels!!). If it is the standard SD channels I'm watching, a good comfortable (i.e. NOT loud) level is like from -15 to -25... When an HD channel is selected, I actually have to go from -15/-25 to -3 or more...
This doesn't happen with the PS3 though.
Listening to MP3's only, it gets really loud from -5 to -13, and at comfortable levels from -15 to -30... (same with Blu Ray movies).
Also, what is the problem with increasing the level/volume?
I don't get why you would like to watch a movie at -20 if it doesn't sound loud enough, why not drive it to whatever number/value makes you comfortable?!? (it's not a smart-ass question, I really want to know!
P.S. If it matters, I have Cambridge S30 as fronts, S50 as center, Emotiva Ultra 10 as sub.