Is HDMI upscaling a requirement to get all video to the set over the single hdmi cable going to it?
No, scaling is not a requirement, but A-D conversion is. You'll need to check specifics on each AVR to see if it can convert A-D. Worst case you'll need to check the manuals for the video pathway charts.
That said, the A-D conversion quality on entry and mid-level AVRs is marginal at best. It may be more cost-effective to look at replacing the source devices that rely on analog video output.