I have a four in, 2 out HDMI matrix switch box that could possibly help you out (if I understand the issue correctly). The two outputs are independent--If source 1 is going out of output 1, then output 2 cannot also use source 1--it can use any of the other three. And so on.
You could connect your sources to the switchbox (it has a remote), connect one output to your AVR and one to your headphone source box. You could then connect one AVR output to your TV and your headphone source box output to your TV on another input.
This would allow you to to select the source you want and have it sent to either the receiver or the headphones, as you wish. From there, the picture should go to the TV from either the AVR or the headphone box, while the audio should go to the speakers or the phones.
So, four sources in HDMI >> switchbox.
Switchbox output 1 >> AVR>> TV
Switchbox output 2 >> headphone box>>TV
No need to swap cables every time you want to use a source (unless you have more than four). Mine works with HDMI cables up to 35 feet long (feeding a projector that is far from the box as well as fed by a PS3 from another room that does double duty for a total of 70 feet of HDMI--with the box at centre).
My box is not 3-D compatible (if that matters) but I'm sure there are some that are. Mine cost me about 120$ four years ago. Don't know if it is still available. There is a similar device at Monoprice
for less than half what I paid (but I live in Canada and had to pay shipping and import duties, the actual device cost about 75$ back then).
This is an interesting device (the headphone thing). How much was it, if it's not too indiscreet to ask?