Yeah once you get past around 12 ins and outs you're taking big equipment and potentially big money. Many years ago I did an impulse buy of an Extron Matrix 3200 off of eBay for super cheap (relative to its MSRP, at least). I think it's configured for 32x32 RGBHV and stereo, can handle HD resolutions, has an LCD front panel controller, RS232 computer interface, and so on.
It also occupies 4-5 feet of rack space, so just finding somewhere to put it can be a bear. This stuff is designed to be modular, so for example there's one unit that handles just the "R" lines, another for "G", another for sync signals, and each individual piece is 5-10U of rack space. There are special cables that pass the control signals between them so that they can act in concert.
In these commercial switches the video connectors are usually BNC, so you also need a huge pile of adapters or custom cables if you're going to hook up consumer gear -- plus some special tools that can attach/detach BNC connectors in high-density configurations, because once the lines are attached you can't fit your fingers in there to manipulate them.