Originally Posted by arnyk
It is true that DBTs and a SBTs are both blind tests, but that does not mean that they are any way equivalent even though they may look that way to a casual viewer of the test.
A DBT experiment is an experiment in which any information about the test that might lead to bias in the results is concealed from both the tester(s), and the subject(s). There is nobody ho knows the correct identity of the unknowns that has any visibility at all to anybody who can possibly affect the outcome of the test.
A SBT experiment is an experiment in which any information about the test that might lead to bias in the results is concealed from the subject(s) but not the tester(s) or other persons who are present during the test. There is now someone who knows the correct identity of the unknowns who has some kind of visibility to somebody who can possibly affect the outcome of the test.
For many years SBTs were considered to be adequate until the well-known case of "Clever Hans the Talking Horse" in the early 1800s. Clever Hans was a horse who amazingly enough tapped out correct answers to difficult questions with his hooves. The tests were SBTs in that someone who knew the correct answers was visible to the horse and communicated with the horse whether intentionally or unintentionally via his body language. When the tests were upgraded to DBTs by removing anybody who knew the correct answers from the presence of the horse, the horse suddenly lost his question-answering abilities.
A listening test can't be done with the listener acting totally in a vacuum. The listener needs to know which trial is the current trial, for example. The easiest way to do a double blind test is to prepare a script of the trials and correct identities of the unknowns and give it to someone who controls the technical side of the test, and for example does the switching. This person is completely concealed from everybody else who is involved. A second person calls out the trial numbers, which the concealed person uses to do the switching, and the listeners use to record their results.
The first innovation of ABX was to build a machine that was a mechanical test coordinator that controlled the unknowns and kept the listeners updated as to trial numbers, etc. The second innovation was the ABX comparison method itself which allows sighted evaluations for the purpose of learning what to listen for, and blind tests for the purpose of determining the outcome of the test.
So, I've had another viewing of the interview, and the thing that stuck out the most to me was Scott's laugh!
Outside of this, I found that the interview dealt minimally with DBT's or SBT's, but spoke more to the recorded source qualities and common challenges surrounding the development and retention of such, and the reproduction qualities of electronic devices themselves, more than the test methodology, which for me, was why I posted the link initially. - The conversations in these regards, are very informative, and worth revisiting from time-to-time. Jon's, analogies were easy to relate to and follow. I personally found him to be self effacing, and honest. He said nothing that bordered on heresy (I understand that you may perceive otherwise), and in fact, his statement predominately echoed, the majority beliefs of this forum.
With regards to SBT's and DBT's, Id say that while the title leads one to anticipate that the debate would heavily include a discuss about the goodness's and usefulness of such test, it frankly does not. What was discussed / shared was reasonable, non-combative, and in agreement with my personal sensitivities. I never felt at any point that I was in need of filtering out much, if any of what I was hearing, from entering my mainstream of thinking.
On the whole, the central theme of the interview is about porting a greater understanding of all of the variables, which contribute to one discernment of subjective audio qualities, not just under a SDT or DBT, but in general everyday enjoyment of ones system. For me, the weighting was put in the right places, far away from a focus on the qualities of SDT's and DBT's because frankly, we all have stereos, but less then 1% of us have access to such test, outside of format testing, via Foo Bar. But again, for me, anything outside of Red Book, is of insufficient fidelity, for my current level of proficiency as an audiophile. Hardware differences are what I'm personally paying the most attention to, these days, and if I was to endeavor down the path of the aforementioned test format, I'd personally use both variants, along side of my everyday listens, as a means to filter down to my final choices. But ultimately, as I have historically, I would simply use my home as the environment, and my ear-brain combination, to determine what I personal like the most, given the combination of options available to me. I have not, nor would I ever accept another's persons beliefs about the sonic qualities of any recording, or playback device; at best I would put the items mentioned on my list to personally audition, in my own home. If a retailer will not permit me to fully pay for the items, and take them home, with the understanding that I will be making many back and fourth exchanges, for a period of no less than 30-days, then they will not get my business. There is no other way, to determine, what's best for you - to you!
I haven't purchased new components in almost 10-years. The last time I did, I subjectively auditioned all of the components on my objective wish list, in every combination available; and I came to a final combination, after several weeks of auditions, in my home. My investment wasn't lofty, but it wasn't paltry either. I didn't utilize either of the controlled tests cited within this thread, I did however, use a basic A/B relay switch to audition each component. It wasn't a controlled setting outside of the levels always being match to at least .5dB. I always knew what I was listening to and never went 'blind' if you will. I have suffered no cognitive dissidence with regards to these purchases. I should also add that I spend very little time listing to the system in a critical manner anymore, as I am spending to much time in here (and with family of course)!
I am due for a DAC upgrade - for me, I am able to quickly discern sonic differences between DAC's, which makes it requisite for me to bring them home for audition. I agree with Jon, that the analog filters produce the most discernible, audible differences; as I also mentioned last month, in my thread summarizing the AVS debate on Jitter, between Amir vs. everyone else, if you will. At that time I stated that I would put my money on the filters, Jon's comments, in this regard, mirror my own. As a predominantly back ground music system, these days, I don't really need to upgrade my DAC, which is why I probably won't.
Personal auditioning, listening, enjoying of a system in ones home, as they intend to use it, day in, and day out, year after year, is the best test method of all. There is currently no universal test, subjective, objective or of hybrid, that can surpass the aforementioned.
I'm not sure that I paid attentions to your warnings or not...