Originally Posted by John Schuermann
I understand, and one of the main reasons we did our big double blind speaker shootout was to illustrate how speaker comparisons SHOULD be done, and to prove it is not impossible to do even in a generic home environment. I realize that is a practical challenge, but the idea was to get people to think critically about how to compare speakers, and about why going in to listen to various speakers in differing locales with differing audition material is a highly flawed method for doing so.
It was a (noble, I think) attempt to raise the bar on this particular topic, to get people to at least think about how they might start controlling the variables so that a listening comparison has greater validity. Even matching volume levels between speakers and blindfolding the listener would go a long way toward improving the situation, and hopefully the approaches outlined here and in my shootout thread get people thinking critically about such things. Who knows - the effort to raise this bar might encourage other dealers to try the same, or influence consumers to request these kinds of listening tests. I used to level match speaker comparisons back in the early 90s when I worked retail, so it can be done (!).
FWIW, we had over 40 people show up for the listening tests, spread out over two days, with several traveling great distances to attend
The resistance to blind testing always amazes me, at least on one level.
I mean, if you really care about the truth and getting to the bottom of things, it just makes sense to try to try to reduce confounding variables that we know exist, like sighted bias. I thoroughly enjoy blind testing (well, except for the pain-in-the-ass part, practically speaking). I actually find it really cool when my own assumptions or perceptions are shown to be wrong. It's part of learning. I took classes in psychology and to me the most energizing aspect of those classes were learning about all our biases and how our intuitions and modes of thinking can lead us astray when we aren't careful.
On the other hand, though blind testing is intellectually appealing to me, I can also conceive of why they seem so distasteful to many if not most people. Our subjectivity, our personal experience, is essentially the way we get through life. You WANT to have some confidence in the inferences you make from your experiences, so it can be de-stabilizing to have that rug pulled out from under you by any method that undermines that confidence. That's why I encounter in audiophile circles over and over the refrain "Look, I'll trust MY ears thank you; they've got me this far in life and since YOU weren't there testing a cable with me who are YOU to cast skepticism on MY experience."
Basically it's the old break-down between the "objectivist" and "subjectivist" school of epistemology.
Personally I have, in one sense, a foot in both camps. Epistemologically I'm an objectivist. The scientific method, recognizing our capacity for error and taking it seriously in to account in it's method, is the most reliable way of investigating empirical claims. I have a foot in the "subjectivist" camp insofar as I really love the subjective aspect of music and audio and I love talking about it, describing it in ways that for the pure scientist, isn't usefully reliable. I visit various forums from the most "subjectivist" to the "only the facts, please" objectivist. The measurements-or-bust crowd satisfy my desire for no B.S. and really getting a grasp on what's going on. But there can be such disdain for even subjective descriptors "sorry, if you aren't supplying measurements your subjective descriptions are useless to us" that I go running back to some subjectivist audiophile forums to just enjoy discussing the actual subjective impression of the sound.
I navigate this by, hopefully, constraining my actual confidence level and claims to be in line with that which is, or can be verified, more "objectively." So....for instance...I really love owning my old Conrad Johnson tube amps. They appeal to me aesthetically, and conceptually: the sound is produced by those glowing tubes? Cool! I also *perceive* the sound of my speakers driven by those amps to be slightly different than when I use a solid state amp (e.g Bryson, Harmon Kardon), in a way that I prefer.
Now, there is as I understand it some level of plausibility there, given the way tube amps can react in measurably/audibly different ways with different speakers. But it's also possible that in a blind test with a solid state amp, I would have a hard time telling any difference. I don't know. But given how the CJ amps satisfy on various other levels, and given I perceive (have the impression) of more pleasing sound with those amps, I'll happily keep using them while also happily admitting "I couldn't tell you with confidence I'd pick them over a solid state amp in blinded conditions." So I feel free to do whatever pleases me in my own purchases, but I will constrain any actual claims to the level of evidence I actually have in favor of that claim.
At least that's my approach.
How do you feel about the intersection of our subjectivity and the objectivity we can apply in the audio world, John?