If you have multiple USB sockets on the computer, just connect the devices to them. That is all you have to do. The USB uses time-sharing multiplexing between any number of connected devices, so there is no such thing as a "splitter'. Many computers have USB sockets both on the rear panel and the front panel (or left and right on laptops etc.), so look carefully to see how many you have.
If you need additional USB connections, just buy an inexpensive USB HUB, which has one input to connect to the computer and multiple USB jacks for additional devices. It just expands the number of physical USB connections available.
It does not matter if a device is on or off. When you send any signal on the USB, it is available to all USB devices that are connected, but devices for which the data is inappropriate (such as printers) will simply not recognize or use it.
Many USB devices only become "active" when they receive a dedicated set of "handshaking" signals that address and activate them. Until this happens they are semi-dormant.
If one device is off it is the same as if were not connected at all, and has no effect on other devices that are on and connected.
If you need a hub, I recommend the D-LINK 7-port USB hub, which you can get for about $25 at Wal-Mart and Amazon.
Sometimes a device works better connected directly to one of the computer USB jacks than through the hub, so you might want to use a direct connection to the computer where available for the audio devices and then connect other devices through the hub.