I have been asking this question for years. I'm not entirely sure I've received a satisfactory answer and certainly have never found a link that properly explains things. The link above is one of the better ones, but doesn't explain, at all, why a impedance matching speaker selector is needed.
Generally, I think of the impedance matching volume controls as calculating the number of pairs of speakers connected the volume control to present an 8-ohm load to the speaker selector. So, one pair of speakers: 1x, 2 pairs of speakers, 2x, etc. Then you set all your volume controls to the speakers connected to them, not the entire system. Let the impedance matching speaker selector handle the rest. It will auto configure depending on how many speakers are currently active on the selector itself.
Not sure if that's right, and I'd love to speak to someone directly at some point about it.
I do get the concept of a bunch of volume controls being set to the entire system, but it seems to me that it would continuously attenuate volume across the system and lower audio quality.
It also confounds me that when you go to non-impedance matching volume controls that they suggest that you don't use a speaker selector.
*sigh*
This is what I get for using one amplifier channel per speaker for all of these years.