Depends on how you are getting things around.My video comes from a central location/source. At the destination the user should be able to decide if the audio is played through the TV (headphone) or through the in-ceiling speakers. Is there a way to accommodate this?
I use wireless headphones plugged right into the back of the cable box. I turn down the sound from my main speakers when I don't want to listen to them (after kids are in bed) and I turn on my wireless headphones.
Your listening scenario is a bit vague, so while there may very well be a solution, I'm not sure beyond what I described you are trying to achieve.
Depending on your distributed system, you could make the headphones their own source and run them through a headphone amp or use wireless headphones.
Most headphones don't accept HDMI, but you often run your sources direct to a audio switching using analog audio.