I was never pleased with the sound quality of 'stereo' mode- it always sounded a bit loose and lacking in precision. The following is quoted directly from John B. and related to your question. If you seek further info or have issue with it, please contact John B. All I can say is his suggestion worked wonders for me, YMMV etc.
I prefer to play two channel/ stereo listening in the Matrix mode with all channels turned off except for the front left and right channels. In Matrix mode the front left and right channels are as close to Analog Direct mode as possible in a CB3HD when the speakers are set to Full. The Stereo mode was designed to feed a multi channel signal to just the front left and right speakers, the thinking was based on a 1995 world when some people were playing all multichannel formats via a two speaker system.