First off, I don't own an Apple TV, so I am not expert there, but I do own a Yamaha RX-A AVR. I'm assuming your Apple TV is connected to your AVR. Therefore, I am taking a guess that it is most likely a setting in the AVR.
I have the RX-A800 and I can make changes to the audio output for each input.
I'm running 7.1 (with front presence speakers), and I like everything to come through that way, even basic TV. So I adjusted the settings in all my inputs to output in 7-channel stereo vs. just stereo or one of the other modes that Yamaha has. I'm guessing maybe you need to do the same for whatever input your Apple TV is connected to.
If you check the settings for the input of your XBox, you might see they are different from the input of your Apple TV.