Let me tell you how I handled a similar situation. I'm hard of hearing and prefer cans (headphones) for TV and music where I want it kind of loud, concerts, etc, and others are around. My wife's hearing is perfect, and she doesn't find the 5.1 (with just 3.1 hooked up) any advantage.
Onkyo HTIB, 6+ years old
Charter HD box
TV optical to Onkyo
BD copper to Onkyo
Cans, when used, to Onkyo
"better" speakers to Onkyo
Regular TV, I wear my aids, we're both comfortable
With both listening to a HD movie on TV or BD, good sound, she sets TV speakers to suit her, I use the cans and suit me ... no aids
Just me, TV sound to 0 (not muted, MUTED shows on screen), cans unplugged, 95dB from the speakers.
Rarely she wants the room-filling sound but not loudly enough for me ... I have a 100 ohm dual pot to speakers B on the Onkyo and a headphone jack on its output.
This has worked quite well. She needn't understand my schemes to listen to CDs, watch DVDs, or watch TV, with sound through the TV. If she/we want the better speakers used, I have to do it and "clean up" afterwards.