An HDMI switcher is not a passive device. A lot more than switching is going on inside, even though the final outcome is that it selects amongst different inputs. It's impact on the video quality can be huge. I've seen tremendous differences in video performance from one pre/pro to another. Yes, all they do is switch video in some cases, up or cross-convert in others. In switching alone, a good switcher can keep noise levels down to substantially.
If you believe a $100 switcher is the same, then that's what you should buy, but I'm telling you that the difference can be huge. Granted, I don't know what a $2k switcher can really do, but if you are running a $250,000 video/HT system, you DON'T put a $100 switcher. That's a given.