Originally Posted by pjp
Honestly, and I don't mean this in a critical way, but I really literally don't understand a word of almost your entire post. It's just gobbledegook that nothing anybody ever experiences can be valid. I've put together at least 100 sound systems and, although I have done my own blind tests, I could not possibly care less about your DBT methodology and I can't think of anything less relevant to a real-world audio system design. It is very, very easy to hear differences in systems you are intimate with and it is very common that my listening tests reveal conclusions that are the polar opposite of what my biases would dictate.
Third party DBT is listening to two variations of a foreign system -- that has nothing to do with the real world where you are testing on a system that you have listened to for months or years and are intimate with. Not only do I not need to blind test, I don't need to even go back. When I added new speakers, replacing 15-year-old speakers, I expected them to sound better, and I desperately wanted them to sound better, and they didn't. I don't need to hook up the old speakers to know the new ones aren't as good. If you say speakers are too easy, I had the exact same experience with a new amp (listening results were directly opposite to my biases, preferences, and presumptions of which would sound better).
Alan measures performance objectively (which normally you would demand, but somehow it isn't valid when he does it) and he listens. Every impression he forms is backed up by data. He has no basis for any bias -- he's not selling or buying anything, he couldn't care less if one capacitor is better than another, or one cable is better than another, or one connector type is better than another. There is no basis for bias when you are building something because you just don't care which approach or component sounds best, you're just trying to figure out what is best. You try everything, listen to everything, measure everything and you learn a lot in the process. Experience means everything. Spouting testing methodology without experiencing anything is just a massive waste of time to me (and to the forum because all of that DBT stuff has already been discussed a gazillion times without an ounce of meaningful content or information in any of those posts).
AND, even if I did a true DBT 100% accordingly, they will say you did not publish the finding, so it's not valid or say you make it up.
I read some of the article and videos linked to me here, you really have to be careful and read and watch through it. I found so many problems, like testing the best of the best speakers with a SINGLE speaker, testing the best of the best power amp(I am talking about really high end amps) with a $1995 pair of studio monitor.
The one that really takes the cake is for ABX on amp, they record the signal at the output of two amps, then play it back on ANOTHER system for people to compare the two amps. Can you imagine this? recording the output signal, the kicker is play it back on ANOTHER system to judge the amps!!! they call this is scientific.
Yes, I do spend the time to read. I sat through over an hour of the video that talk about testing with single speaker and went to teach Consumer Reports the right way to test speaker to get better rating.
Yes, we are not professional tester, I live with my system day in and day out, I can tell when it's different. AND I absolutely agree just because I design with something in mind, the result is more surprising than going as expected. Like I said I expect the 7 months of hard work will produce big improvement, theory said so.......BUT NOT!!! I'd be the first one to admit this.