Yes, amps by themselves are quite neutral if properly working and within "limits".
But the processors (AVR, pre-pro) may differ in how they process the sound. Different processors may do things differently with Direct & Pure Direct mode + Bass management. For example, with Denon & Marantz, Pure Direct & Direct actually allows subwoofer output with certain settings, which may be very basic, but they allow for 2.1, 2.2 mode even in Pure Direct. Onkyo and Yamaha do not allow this. Other processors (Anthem, Integra, NAD, Arcam, Lexicon, ect) may not allow this as well.
I recently find that Direct & Pure Direct 2.2 mode improved the SQ for me and makes the sound more enjoyable for me. Bass and midrange seems to have improved clarity, articulation, and authority. I absolutely love it, but I can't speak for everyone.
If you use room correction and EQ, they will also differ with different processors.
So perhaps you could look into different processors and trying to use different settings. For example, setting speakers to Small vs Large, LFE THX vs LFE + Main, Ultra2 THX Sub On vs Off, XO 40Hz vs 80Hz, and other settings. Room Correction vs NO Room Correction.
I prefer Denon & Marantz, but others will have to find what's best for their systems.