I have solved this issue with one of two different tricks:
1) Using a receiver where you can set the crossover frequency with each speaker (which includes most of them these days), set the crossover to a lower frequency for the center speaker than with the left and right speakers, and set the speakers to small. For example, if the center crossover is 50hz, but the LR crossover is 80hz, more bass from the center channel will be sent to the speaker, rather than the sub, which solves the problem.
2) Set the LCR crossover to the same frequency, but just tune the EQ so this doesn't happen, with special attention to the frequencies that are bugging you. You could either tune the EQ for all LCR so this doesn't happen, or you could tune the bass frequencies down in the center speaker, but up higher in LR speakers, if that's what you prefer. It depends on what you like better, and on the capabilities of your receiver. If you tune all LCR the same, you get a more consistent soundstage for movies. You might like to tune it for more bass boost in LR than in C if you like that for music. Aside from the frequency that is bugging you, you could leave the rest the same, so the soundstage is still the same for most frequencies.
Incidentally, once I started using a measurement microphone and Dirac, which has very precise EQ control, this problem disappeared for me, without any tricks, despite using the same EQ for LCR and a fair amount of bass boost in my "house curve." I just made a curve where the bass boost tapered off before it affected male voices.