Are these MKV files?
A little scripting with mkvmerge would make new versions with only specific tracks retained. It would be helpful if your files had consistent numbering of the audio tracks.
I haven't tried it, but I recall MKV allows tracks to be tagged as disabled, perhaps made invisible? That would be faster than making a new copy. The batch tool would be mkvpropedit, which is part of the mkvmerge package.
Does this happen with all your files? I've had no issues with either AC3 passthru or AAC conversion on Handbrake. I use High profile for DVD and Normal for Blu-ray.