That's a good point. I don't think ARC was intended to allow HT enthusiasts to use the display as the "hub". It's limited to spdif at this point and can't pass lossless audio (DTS-MA, Dolby TrueHD). It's good for providing spdif audio from the TV for internet apps without adding an optical cable and that's about it.
Most who want a full HD audio/video system will need to continue to use the display primarily as a monitor and use either an AVR or VP as your HDMI switcher.