Originally Posted by RUR
Not only was Lyngdorf not (able to be?) included, but they "tested" an old, old, old, featureless version of Trinnov's Optimizer as used in the Sherwood AVR. That version bore scant resemblance to the one used in full-featured Trinnov products at the time, which are even more feature-rich today.
And, the test protocols left much to be desired, as I believe I mentioned at the time. I'd be careful not to draw any hard and fast conclusions.
Test protocols left much to be desired?
I’ll absolutely take offense to that.
We documented the setup procedures quite well with pictures and text. We sought setup advice for months before.
We used an elaborate drum-stand setup to have the exact same mic positions for each AVR autoeq process to make it fair. Speakers were unmoved for each setup process, subs unmoved, mic locations unmoved. All calibrations done in advance of the blind testing, the day before in a low stress, long careful day of setup. Even the guests present during the setup the day before were required to sit in the same place in the back of the room for each run, way out of the range of the listening position - but just to make sure we had a fair run.
MLV volumes were level matched using a 800-1000hz test tone on the Disney calibration disk since the autoeq processes left us with such disparate main volumes we couldn’t otherwise reasonably compare the Auto EQ results.
The procedure was fair, equitable, and quite repeatable. The results each auto EQ produced was of its own volition. The audience audition was blind. It couldn’t have been any fairer, because there wasn’t much human interaction or subjective meddling in the project at all. In fact I’ve never read of a fairer comparison on these forums by unpaid enthusiasts. Any of the —“you should have done this" type feedback were based on subjective preferences (i.e. "Well, I do this to make it sound right") that would have messed with the auto EQ processes intended result that the manufacturer designed into their unique flavor of auto EQ.
The only pass I give, as the organizer of said meet, was Dirac. And it was because it was brand new. Nobody knew how to operate it at the time, the owner,
shipped it to us without even having a chance to mess with it first, and it showed up pretty last minute and we didn’t get it right - I said this clearly in the write up.
Now as to applicability. I clearly stated the takeaways were NOT universally applicable to the reading audience. They were applicable to my room, with my speakers, with my sub positions. Because different speakers in different rooms with different mic positions (and even different individual vendor mics off the same assembly line) would generate different results.
The clearest take away from the Auto EQ testing, the point of made beyond conclusive, is that at that time (and almost certainly now too) that auto EQ calibrations do NOT make for a equitable reference level experience vendor to vendor.
Thus we cannot assume my reference settings, as calibrated with my AVRs auto EQ, is even close to your reference settings, as calibrated with your auto EQ.