Originally Posted by arnyk
So what? As I have detailed in other posts, those weren't the only controls that we used.
There were none documented in the article Arny. Why? Seems like you believe in controls yet there is not one word about them in the article. The one and only test that has ever been documented to include you is missing this critical ingrediant.
Besides, the above post is again demonizing ABX for problems that it shares with many other forms of subjective testing.
I said nothing about ABX being a bad thing. I pointed out serious issues with the people
using it. I showed specific examples of flaws in testing that your proposed controls would not have caught. So beside the problems, we lack awareness of what a control really is in this context.
ABX doesn't have to be perfect, it merely needs to be appreciably better. Since the most common equipment evaluation technique to this day is the not level matched, not time-synched, sighted evaluation, its hard to imagine why anybody keeps picking nits with ABX.
No test is perfect so we are good on that
. Not having said a word about other comparisons, not sure how they became the metric of comparison. I have not advocated such tests as a substitute. But rather, stating that you are protocol errors in your testing and the rules that you stipulate did not include a control. Fact that other people do bad tests doesn't excuse you and I from practicing the same.
And there are countless tests being run everyday that are proper but not ABX such as codecs which routinely have controls. Here is a random comparison that came up in my first search results:
Look on the very right. The worse scoring one (3.5 KHz LPF) is the control. It is a low pass filtered version of the original song that should present clear degradation. If that track had garnered higher praise, we would know something needed investigating.
Now look at your article. There is no such control. We don't know if any mistakes were made.
I don't know about you, but I have long known that nothing is totally idiot-proof. Often the idiot is me! ;-)
. For this reason, I would rely on people who are far more careful than me to construct serious tests. When I published the results of my speaker wire testing, I tested and retested until I built up high confidence in not having made mistakes.