Perhaps the remote extender only works with one input at a time. It may not be designed to work with both inputs used at the same time. Do you have a second set of extenders that you could test with? Use one set for VGA and one set for serial?
Can you list what extender you are using so we can look up the specifications on it?
Can you list what extender you are using so we can look up the specifications on it?