i have 2 sets of speakers and a sub with high level input. can i hook the a output from my amp to the front speakers AND the sub, and hook b to the rear set of speakers?

The A and B channels are the same channels and overloading the outputs like that will damage the speaker outputs for sure,
If your avr has preamp outs then get a small amp to drive the second set of speakers but what you described will only bring you problems.

You might want to mention the equipment you are using. It sounds like you might be trying to get 5.1 sound but it also sounds like your receiver or preamp is not a 5.1 box. Tell us what your goal is and what boxes you have.

..Doyle