I think whats at work here is phasing.
In the above link (table 2-9), you can see that when two signals combine, the resulting amplitude differs with respect to phase. 0 degrees = +6db, 90 degrees = +3db, 120 degrees = +0 db, 180 degrees = cancellation.
If the distance from the sub to the listening position is different from the mains to listening position, then their will be phase issues. And they will be different for different frequencies on the basis that their corresponding wavelengths are different. So in this case, I think the best approach is to measure with mains AND sub operating, and making small changes in location in the sub and remeasuring. Getting each to look good driven separately is a good starting point, but if the cumulative FR curve due to phase is bumpy, then you must tackle the problem measuring both working together.
Jim, when you say making small changes in location in the sub, are you referring to actually moving the subs, or are you referring to adjusting the sub timing delays in the AVR menu (what we have been calling the "sub distance tweak")? Or try both?