Validty of blind testing - Page 7 - AVS Forum
Forum Jump: 
 
Thread Tools
post #181 of 355 Old 02-05-2014, 11:04 AM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,093
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 71 Post(s)
Liked: 293
Quote:
It is very funny! Or tragic
Or stupid.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
Sponsored Links
Advertisement
 
post #182 of 355 Old 02-05-2014, 11:10 AM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by mcnarus View Post

Now your'e just playing word games. Validity actually means something—and it's not synonymous with "perfect."
But sighted comparisons are completely invalid for other reasons entirely. You're trying to resurrect them by nitpicking about some other test. Won't work, bub.

If you like "shades of accuracy" better, then I mean the same in both cases.

Your later comments, wrt sighted tests, rests on the assumption personal bias cant be removed from the equation. Like I said in an earlier post, I have often expected one thing and perceived another. I can provide dozens of examples of this. So I don't buy that expectation invalidates objectivity entirely. On this point, I assume the best we can do is agree to disagree.

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #183 of 355 Old 02-05-2014, 11:24 AM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,093
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 71 Post(s)
Liked: 293
Quote:
If you like "shades of accuracy" better, than I mean the same in both cases.
No, I don't. It's just the same pseudoscientific rationalization you've been using all along.
Quote:
Your later comments, wrt sighted tests, rests on the assumption personal bias cant be removed from the equation. Like I said in an earlier post, I have often expected one thing and perceived another. So I don't buy that expectation invalidates objectivity entirely.
Then you don't understand what expectation bias is. It has nothing to do with conscious thought. The fact that your conclusion doesn't match your initial conscious expectation proves nothing. It may only mean that some bias you were not consciously aware of proved more powerful. You don't know. And it is the fact that you don't know that makes sighted comparisons invalid.

If you want to conclude that A causes B, then you have to run a test in which A is the only possible cause of B. The trouble with sighted comparisons is that there are two possible explanations for the difference you heard:

1) There really was a difference to be heard.

2) Something in your brain made you think there was a difference when there really wasn't.

If your test can't eliminate #2 as a possibility, then it's invalid. Period.
Jack D Ripper likes this.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #184 of 355 Old 02-05-2014, 11:26 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,301
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 690 Post(s)
Liked: 1144
Quote:
Originally Posted by jim19611961 View Post


My argument wouldn't be whether ABX reduces memory reliance or bias. I think I agree it does. The question is, do these steps make the test valid?

There are standard criteria for judging the validity of a test, and ABX does well with them:

http://en.wikipedia.org/wiki/Test_validity
"
Evidence to support (or question) the validity of an interpretation can be categorized into one of five categories:
Evidence based on test content
Evidence based on response processes
Evidence based on internal structure
Evidence based on relations to other variables
Evidence based on consequences of testing
"
I submit that ABX does very well when judged by these criteria. Furthermore the traditional audiophile sighted evaluation falls flat on its face. I could lay out all the details, but that would be one very long post!

For example ABX produces the same results for things like the threshold of hearing as have all previous generally accepted tests. If you look at Fletcher Munson's original ca.1930 results, and the latest versions of the same thing, you find pretty much the same results. It turns out that at the most sensitive frequency, the threshold of human hearing is about the same as the acoustic noise created by Brownian Motion of air molecules. If our hearing was more sensitive, all we'd hear that we don't hear now is hiss. It all fits together. One universe. One human animal.
arnyk is offline  
post #185 of 355 Old 02-05-2014, 11:34 AM
AVS Special Member
 
David James's Avatar
 
Join Date: Dec 2000
Location: Longmont, CO
Posts: 1,926
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 12 Post(s)
Liked: 62
Quote:
Originally Posted by jim19611961 View Post

If you like "shades of accuracy" better, then I mean the same in both cases.

Your later comments, wrt sighted tests, rests on the assumption personal bias cant be removed from the equation. Like I said in an earlier post, I have often expected one thing and perceived another. I can provide dozens of examples of this. So I don't buy that expectation invalidates objectivity entirely. On this point, I assume the best we can do is agree to disagree.
You don't have to buy it, it's free as the result of science and testing. Your anecdotes don't trump the science.
David James is offline  
post #186 of 355 Old 02-05-2014, 11:44 AM
FMW
AVS Special Member
 
FMW's Avatar
 
Join Date: Jun 2007
Posts: 4,775
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 246 Post(s)
Liked: 700
Quote:
Originally Posted by jim19611961 View Post

Probably. But I would pose that validity is not a yes or no question as much as it is shades of validity. That is to say that given two flawed test procedures, one being a bit more valid than the other, doesn't make the more valid one completely valid nor does it make the less valid one completely invalid.

A bit more valid. After actually having done the tests I can tell you that the bias controlled tests are valid and the sighted tests are invalid. That's why get some situations in which something is not audible without bias but is always audible with bias. The bias controlled tests prove that hearing bias exists. How could a test that doesn't control bias be anywhere near as valid as one that does? It just doesn't make sense.
FMW is online now  
post #187 of 355 Old 02-05-2014, 11:44 AM
AVS Addicted Member
 
amirm's Avatar
 
Join Date: Jan 2002
Location: Washington State
Posts: 18,028
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 700 Post(s)
Liked: 398
Quote:
Originally Posted by David James View Post

You don't have to buy it, it's free as the result of science and testing. Your anecdotes don't trump the science.
His "anecdotes" tell me he has real life experience with audio testing with external data that instructs him as to whether the conclusions are likely right or wrong.

I love to hear other people's anecdotes to see if any that indicates they have a real feel for this versus repeating talking points of the camp.

Amir
Founder, Madrona Digital
"Insist on Quality Engineering"

amirm is offline  
post #188 of 355 Old 02-05-2014, 11:53 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,301
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 690 Post(s)
Liked: 1144
Quote:
Originally Posted by jim19611961 View Post

Your later comments, wrt sighted tests, rests on the assumption personal bias cant be removed from the equation. Like I said in an earlier post, I have often expected one thing and perceived another. I can provide dozens of examples of this. So I don't buy that expectation invalidates objectivity entirely. On this point, I assume the best we can do is agree to disagree.

The "I got results I didn't expect" is actually one of the oldest anti-DBT anecdotes around - right after "My wife heard the difference in the kitchen". It's usually false if for no other reason that you didn't know what the true answer was by some more reliable independent means.

Trying to control your personal bias is inherently a kind of desensitization. Nothing bettter than sitting there with the ABX push buttons on your hand thinking "I can reach whatever conclusion I want based on anything without fear of my biases affecting the outcome".

arnyk is offline  
post #189 of 355 Old 02-05-2014, 11:54 AM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by mcnarus View Post



2) Something in your brain made you think there was a difference when there really wasn't.

So our perceptional mechanisms are undermined by undetermined brain activity?

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #190 of 355 Old 02-05-2014, 11:57 AM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by arnyk View Post


Trying to control your personal bias is inherently a kind of desensitization. Nothing better than sitting there with the ABX push buttons on your hand thinking "I can reach whatever conclusion I want based on anything without fear of my biases affecting the outcome".


That sounds very similar to the argument Atheists use against believers.

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #191 of 355 Old 02-05-2014, 11:58 AM
FMW
AVS Special Member
 
FMW's Avatar
 
Join Date: Jun 2007
Posts: 4,775
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 246 Post(s)
Liked: 700
Quote:
Originally Posted by jim19611961 View Post

So our perceptional mechanisms are undermined by undetermined brain activity?

No. They are affected by them.
FMW is online now  
post #192 of 355 Old 02-05-2014, 12:02 PM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by FMW View Post

No. They are affected by them.

Then you must also believe that other perceptions are affected by subconcious brain activity, yes?

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #193 of 355 Old 02-05-2014, 12:06 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,093
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 71 Post(s)
Liked: 293
Quote:
Then you must also believe that other perceptions are affected by subconcious brain activity, yes?
Sure, ever heard of an optical illusion?

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #194 of 355 Old 02-05-2014, 12:09 PM
FMW
AVS Special Member
 
FMW's Avatar
 
Join Date: Jun 2007
Posts: 4,775
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 246 Post(s)
Liked: 700
Quote:
Originally Posted by jim19611961 View Post

Then you must also believe that other perceptions are affected by subconcious brain activity, yes?

Of course. I recommend you tune in to an interesting TV program called Brain Games. It is all about perception. Highly recommended.
FMW is online now  
post #195 of 355 Old 02-05-2014, 12:10 PM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,301
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 690 Post(s)
Liked: 1144
Quote:
Originally Posted by jim19611961 View Post

Quote:
Originally Posted by arnyk View Post


Trying to control your personal bias is inherently a kind of desensitization. Nothing better than sitting there with the ABX push buttons on your hand thinking "I can reach whatever conclusion I want based on anything without fear of my biases affecting the outcome".


That sounds very similar to the argument Atheists use against believers.

Umm maybe to you.

BTW I'm somewhat into Apologetics. I can't make any connection between what I just wrote and arguing about faith. Please share what you mean.

I've probably participated and organized in more DBTs than anybody you'll ever meet except possibly Dr. James Johnson. I just report what I've seen and realized over the past 40 or so years..

DBTs are all about removing faith from audio belief. That may mean that only true believers in golden earism argue against it? ;-).
arnyk is offline  
post #196 of 355 Old 02-05-2014, 12:11 PM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by mcnarus View Post

Sure, ever heard of an optical illusion?

But in the realm of you visual data, your confident what is an illusion, and what is not? Are you sure?

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #197 of 355 Old 02-05-2014, 12:16 PM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,301
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 690 Post(s)
Liked: 1144
Quote:
Originally Posted by jim19611961 View Post

Quote:
Originally Posted by FMW View Post

No. They are affected by them.

Then you must also believe that other perceptions are affected by subconcious brain activity, yes?

Of course.

I think that the last few decades of brain studies have shown conclusively that the brain is one organ and if functions as one unit. I don't think you can't have one thing happening in this corner over here and say it has no effect at all on what's happening over there.

Have you been watching the Brain Series on Charlie Rose?

Type in "Brain Series" here: http://www.charlierose.com/search

Also have you read "This is your Brain On Music" by Letivin? Its only about $12 at Amazon.
arnyk is offline  
post #198 of 355 Old 02-05-2014, 12:17 PM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by arnyk View Post



BTW I'm somewhat into Apologetics. I can't make any connection between what I just wrote and arguing about faith. Please share what you mean.

The argument centers around the idea that someone can believe in something (God) for which their is no compelling evidence for. Furthermore, many claim to hear, feel or in some other manner scientifically unprovable as well contact with said entity (God).

If you believe in science, then by the same logic, it would be impossible to believe in God. Perhaps all of you making the argument against me are Atheists. If so, at least your consistent. But if not, then the argument your presenting to me, which basically is, some subconscious bias makes me detect something that isnt there, holds true for believing in God despite not one scientific fact that validates it.

In the case of someone who believes in science AND believes in God, you must consider that some part of your brain makes you believe God is there when in fact he isnt (according to science).

So one is left with a conundrum. Either parts of my brain undermine my ability to trust my senses, or they do not. If the argument is that the subconscious is powerful enough to make one believe something that is not there, than that is one definition of delusional. Since this argument pervades all sighted audio tests, one must say that either all these folks are prone to delusion, or the premise is flawed.

And so, your comment "I can reach whatever conclusion I want based on anything without fear of my biases affecting the outcome" is basically the reasoning believers use to continue to believe in God despite the complete lack of evidence. Many claim to hear, feel or even envision God without any reservation or consideration that their desire to believe could effect their perceptions that he exists.

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #199 of 355 Old 02-05-2014, 12:18 PM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,301
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 690 Post(s)
Liked: 1144
Quote:
Originally Posted by jim19611961 View Post

Quote:
Originally Posted by mcnarus View Post

Sure, ever heard of an optical illusion?

But in the realm of you visual data, your confident what is an illusion, and what is not? Are you sure?

It all depends. But as I've said on this thread before and you seem to please yourself by ignoring, any listening test that does not give a natural treatment to audible illusions is doomed because listening to music itself, and particularly stereo and multichannel, are illusions.
arnyk is offline  
post #200 of 355 Old 02-05-2014, 12:19 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,093
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 71 Post(s)
Liked: 293
Quote:
But in the realm of you visual data, your confident what is an illusion, and what is not? Are you sure?
I'm fairly confident that we have the means to answer this question in almost all cases. Objectively, it's either an oasis or a mirage.

But you're still missing the point. The point is not that we know it's a mirage. The point is that you do not know that it is not a mirage. Whereas, if we get a positive result in a DBT, we know that it is not a mirage. You, with your sighted comparison, are the guy who sees water in the distance. I, with my DBT, am the guy who's actually sticking his toe in the pond.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #201 of 355 Old 02-05-2014, 12:28 PM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,301
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 690 Post(s)
Liked: 1144
Quote:
Originally Posted by jim19611961 View Post

Quote:
Originally Posted by arnyk View Post



BTW I'm somewhat into Apologetics. I can't make any connection between what I just wrote and arguing about faith. Please share what you mean.

The argument centers around the idea that someone can believe in something (God) for which their is no compelling evidence for. Furthermore, many claim to hear, feel or in some other manner scientifically unprovable as well contact with said entity (God).

Actually there are many things in Science for which no adequate proof exists. There are many things we have little for but a name. Some things we have no clue about until next week or later.

Furthermore, all findings of Science are provisional until we obtain a better finding. I've seen lots of better findings be discovered over my lifetime. I can remember when the Big Bang Theory was a joke.

You think that everything we believe in now is completely true? LOL!
Quote:
If you believe in science, then by the same logic, it would be impossible to believe in God.

I think you are wrong. I'm a theist even a Christian, and as you may have suspected I also believe very strongly in Science.

It appears to me that you are imposing your obviously very narrow and limited belief system on a universe full of people including me, without any thought that you could be wrong about a few details.

For the record I believe that all reasonable interpretations of Science agree with all reasonable interpretations of Christianity. Prove me wrong!
arnyk is offline  
post #202 of 355 Old 02-05-2014, 12:38 PM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by mcnarus View Post

I'm fairly confident that we have the means to answer this question in almost all cases. Objectively, it's either an oasis or a mirage.

But you're still missing the point. The point is not that we know it's a mirage. The point is that you do not know that it is not a mirage. Whereas, if we get a positive result in a DBT, we know that it is not a mirage. You, with your sighted comparison, are the guy who sees water in the distance. I, with my DBT, am the guy who's actually sticking his toe in the pond.

What i think I hear you saying is that if blindfolded test subjects disagree whether their toe is in a pond or not, then the test is invalid. But if they all said their toe wasn't in a pond, then the test is valid. Or if they all agree it is, then its valid.

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #203 of 355 Old 02-05-2014, 12:42 PM
AVS Addicted Member
 
amirm's Avatar
 
Join Date: Jan 2002
Location: Washington State
Posts: 18,028
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 700 Post(s)
Liked: 398
Quote:
Originally Posted by arnyk View Post

I've probably participated and organized in more DBTs than anybody you'll ever meet except possibly Dr. James Johnson. I just report what I've seen and realized over the past 40 or so years..
Do you have them documented anywhere online? I don't recall you ever describing them in this forum. As you see, I have shared a number in this thread already. For that reason, I doubt that you have done that many double blind tests but seeing them documented elsewhere will add some credibility to your position.
Quote:
DBTs are all about removing faith from audio belief.
That is a misconception. Many corrupt DBTs have been created by people who believe in one side of the story and once they got that result, which happens to always be a negative outcome, they thought they were done. The only thing worse than audiophile bias, is DBT experimenter bias! We can dismiss the sighted test easily but put the stamp of "DBT" on a test and folks line up to believe!

Amir
Founder, Madrona Digital
"Insist on Quality Engineering"

amirm is offline  
post #204 of 355 Old 02-05-2014, 12:46 PM
AVS Special Member
 
jim19611961's Avatar
 
Join Date: Mar 2011
Location: Texas
Posts: 1,343
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 90 Post(s)
Liked: 80
Quote:
Originally Posted by arnyk View Post

Actually there are many things in Science for which no adequate proof exists. There are many things we have little for but a name. Some things we have no clue about until next week or later.

Furthermore, all findings of Science are provisional until we obtain a better finding. I've seen lots of better findings be discovered over my lifetime. I can remember when the Big Bang Theory was a joke.

You think that everything we believe in now is completely true? LOL!
I think you are wrong. I'm a theist even a Christian, and as you may have suspected I also believe very strongly in Science.

It appears to me that you are imposing your obviously very narrow and limited belief system on a universe full of people including me, without any thought that you could be wrong about a few details.

For the record I believe that all reasonable interpretations of Science agree with all reasonable interpretations of Christianity. Prove me wrong!

You asked what the argument was. I provided a massive oversimplification of it.

I certainly don't believe everything is figured out by science. To the contrary, I think science is in its infancy. And for the record, I believe in God also.

I am not trying to impose anything upon you. I am am merely citing the argument, or least a very narrow part of a much larger narrative.

I wouldn't begin to try to prove you wrong because I believe you are right (in that God exists).

But the fact remains that people that DO believe in God dont/cant have their beliefs grounded in science.

My Room
http://www.gearslutz.com/board/studio-building-acoustics/817205-my-listening-room.html

My Music
http://rateyourmusic.com/~jim1961

My Equipment

Rega - Apollo
Rega - DAC
Goldpoint Passive
(2) Classe CA-100 bridged power amps (350w)
Jenzen Next ( http://www.troelsgravesen.dk/Jenzen-NEXT.htm )
...
jim19611961 is offline  
post #205 of 355 Old 02-05-2014, 12:47 PM
AVS Addicted Member
 
amirm's Avatar
 
Join Date: Jan 2002
Location: Washington State
Posts: 18,028
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 700 Post(s)
Liked: 398
Quote:
Originally Posted by mcnarus View Post

I'm fairly confident that we have the means to answer this question in almost all cases. Objectively, it's either an oasis or a mirage.

But you're still missing the point. The point is not that we know it's a mirage. The point is that you do not know that it is not a mirage. Whereas, if we get a positive result in a DBT, we know that it is not a mirage. You, with your sighted comparison, are the guy who sees water in the distance. I, with my DBT, am the guy who's actually sticking his toe in the pond.
There is no assurance at all that you know "it is not a mirage." It is only on paper with idealistic assumptions that would be true. Recall my example of 64 kbps audio matching the CD. Clearly that was a mirage, not an oasis.

And where would I read positive DBTs that you all have discussed? Earlier an amplifier test was posted that was negative. Was Krab and all those people in the other thread wrong to point out that test as having value?

Amir
Founder, Madrona Digital
"Insist on Quality Engineering"

amirm is offline  
post #206 of 355 Old 02-05-2014, 12:49 PM
AVS Addicted Member
 
amirm's Avatar
 
Join Date: Jan 2002
Location: Washington State
Posts: 18,028
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 700 Post(s)
Liked: 398
Quote:
Originally Posted by Jack D Ripper View Post

Yes, it is funny how it works out that way.  All of the problems that exist blind exist sighted.  But with sighted, additional problems are added.  And yet people argue that adding problems is better!  It is very funny!  Or tragic, depending on one's point of view.
What is even funnier is that it often does NOT work that way! Many of our double blind tests violate the Heisenberg uncertainty principle. We insert instrumentation/controls into the experiment which immediately make them something different than what we wanted to observe. In quantum mechanics' famous double slit experience, the light moves like a wave when we are not trying to instrument how it enters the slits. As soon as we put sensors to detect what goes through, it acts like a particle and not a wave!

Iin audio the problem is not that severe but can be. Imagine wanting to test the hypothesis that a high-end cable sounds better than garden variety. If we want instant switching, we run them through an AB box. As soon as we do that, we have violated the Heisenberg principle. If someone's theory is that a fancier cable sounds better, surely routing signals through some kind of box violates the very thing that we wanted to test. The standard answer is, "we can measure or perform double blind tests of the box to show that it makes no difference." The problem with that argument is that it assumes its conclusion as the assumption! We cannot convince someone they are wrong by assuming up front that they are.

How about this: you want to test two amps against each other. How do you level match them? Add a box in front of them that has gain control? Well, that is an amplifier by itself. Who says it didn't mask what you are trying to find? And haven't we already deviated from the actual scenario and hence added "problems" that we now have to deal with?

Clearly it is the voice of inexperience to say that we don't add problems when we try to run blind tests. In many cases we are disturbing the experience. To know whether that impacts the results again takes significant experience and knowhow. And adds significantly to the burden of testing. Take two sources over HDMI and try to switch between them. You will likely hear a glitch for a couple of seconds. Wouldn't that make the job of telling the samples apart harder and hence, adding a "problem" to the test?
jim19611961 likes this.

Amir
Founder, Madrona Digital
"Insist on Quality Engineering"

amirm is offline  
post #207 of 355 Old 02-05-2014, 12:49 PM
Advanced Member
 
andyc56's Avatar
 
Join Date: Feb 2013
Posts: 576
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 17 Post(s)
Liked: 182
Any concept having an implementation can be implemented poorly. A poor implementation of a concept is not by itself a valid reason to dismiss the concept as invalid, but it's an argument that's commonly made.
CruelInventions likes this.
andyc56 is offline  
post #208 of 355 Old 02-05-2014, 12:50 PM
Member
 
dbrown3611's Avatar
 
Join Date: Sep 2013
Location: The Great Plains of Nebraska, USA
Posts: 42
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 24
I spent over 20 years working for a Clinical Research Organization that performed FDA trials for pharmaceutical companies. All most all of these were double blind studies. To be clear, my work was not in conducting these trials (I maintained analytical chemistry instrumentation amongst other tasks) but one becomes familiar with how they are conducted. A number of key points in relation to these trials:

- The methodology was thoroughly vetted by an independent Institutional Review Board before any trial was undertaken
- Results were reviewed by teams of statisticians (many with PhD's) before any conclusions were drawn.

Once a report was finalized and released to the client one could be confident of the reported results. Importantly, others could duplicate the methodology and receive statistically identical results.

Missing from this discussion of those arguing against the validity of double blind tests is repeatability. If others can not perform your test and receive similar results, perhaps you have an invalid test.

It is my understanding this site is about science. If you are introducing subjectivity into your test methodology or into your results interpretation you have anecdotal evidence at best. Certainly one can not draw conclusions from that.
dbrown3611 is offline  
post #209 of 355 Old 02-05-2014, 12:53 PM
AVS Special Member
 
RobertR's Avatar
 
Join Date: Dec 2000
Location: California
Posts: 6,116
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 142 Post(s)
Liked: 386
Quote:
Originally Posted by jim19611961 View Post

I think partly whats at work here is the difference between a clear and concise memory and an impression. Personally, I readily admit that music content with say five instruments playing together is not going to make a concise memory whereby I can recall what each is doing exactly even a few seconds later. What I will have a few seconds or minutes or months later is an impression. Less precise than memory, but longer lasting.
Your characterization of differences as mere "impressions" is very much at variance with what the typical High End reviewer and "Golden Eared" audiophile says. They do NOT say it's "mere impression". They make declarative statements about one component having a clearly audible "wider soundstage", "blacker background", "greater focus", etc etc etc. This is sometimes accompanied by statements such as "the difference was like horses roaring out of the speakers". Nothing "impressionistic" about it. I'm sure you're aware of this.

What fascinates me is how these "roaring differences" suddenly become ever so delicate and frail, ready to crumble at a moment's notice when subject to all the overwhelming "stress" and "memory difficulties" imposed by a DBT.
RobertR is offline  
post #210 of 355 Old 02-05-2014, 12:59 PM
AVS Addicted Member
 
amirm's Avatar
 
Join Date: Jan 2002
Location: Washington State
Posts: 18,028
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 700 Post(s)
Liked: 398
Quote:
Originally Posted by andyc56 View Post

Any concept having an implementation can be implemented poorly. A poor implementation of a concept is not by itself a valid reason to dismiss the concept as invalid, but it's an argument that's commonly made.
Not trying to dismiss the concept if you are referring to me. As I have repeatedly said, I believe in double blind testing. Maybe what is not understood is that many of the tests people swear by are done "poorly." I always say that all audio tests suck, the good ones suck less! smile.gif To average Joe and double blind believer, they are all models of perfection. See my last two replies. Folks even think they have found an oasis just because they conducted the test blind! Why is it that it is always the other camp that doesn't get these concept?

Amir
Founder, Madrona Digital
"Insist on Quality Engineering"

amirm is offline  
Closed Thread Audio theory, Setup and Chat

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off