Observations of a controlled Cable Test - Page 6

I find this thread really amazing. A subjectivist thinks he will be able to easily discern the difference between two cables in a blind test since he is able to do so in a sighted test ---- but, by his own admission, is not able to do so when blind. But some other subjectivists refuse to believe this test has signficant meaning on a more global basis.

Here are the facts as I see them:
1. Mike L. has, if nothing else, proven he is one of the good guys and is not whining that things were not fair.

2. One person who was sure he could tell the difference in a blind test, could not

3. As far as I know, no one else has ever been able to tell the difference in a statistically meaningful way in a blind test either.

4. I have done some sighted and blind tests as well and "thought" I could tell the difference when sited and NEVER got it correct when blind

And here is MY opinion. If 5000 other individuals try to do what Mike did and also fail, there will continue to be many others who will refuse to agree that no SIGNIFICANT diference exists.

This subject should be grouped with two other topics that are usually fruitless discussions: (a) Religion (b) Politics

James Randi has been taunting Machine Dynamica lately on the JREF site. Maybe some audiophile wants to step up to the plate....?

Come on! Straw man. No test could ever prove that....one reason being, it's patently untrue. One can contrive two cables that certainly will sound differently. Nor was that even remotely what Mike's test was 'trying to prove'.


And in terms of challenging standard audiophile rhetoric, that's a key result. We can say that the reality of differences Mike thought he heard, quite confidently, wasn't supported by the results.

Wrong.. Mike claimed to confidently hear differences on his own. He claimed to confidently hear differences under test conditions, too. Both he and the testers used manual cable switching. The results indicate he probably didn't hear differences in the first place, in his 'usual' sighted protocol involving manual switching. It doesn't rule out that under more sensitive conditions -- that HE had NEVER yet used -- he MIGHT score positive on a DBT for these cables.

It's quite possible you will not hear a difference between an mp3 and its source wav, even in a DBT, if you make the mp3 right.

'Make of it what you will'. Yeah, go tell it to the experimental psychology community, not to mention the medical testing community, not to mention the lossy codec development community, you self-described 'dumb audiophile'. Obviously they need a new testing paradigm...DBT just isn't working!

One and only one thing was learned.

Mike used to claim that these differences were night and day... as do many other audiophiles.

They are not.

As has been stated, with instantaneous ABX, it is theoretically possible (highly unlikely i feel) that a positive result could be obtained.

To state any other result that above is to draw far too much out of a simple test with some decent controls in place.

The machina dynamica stuff scares me. that is truly snake oil shite.

I THINK THAT THIS TRULY EXPLAINS HOW ARE MIND WORKS AND TAKES SENSORY PERCEPTIONS OF THE BRAIN TO A WHOLE NEW LEVEL!!!

http://www.reuters.com/article/domes...33509820071121

jOEY
One of my better finds, if I don't say so myself

So possibly just by seeing the cables in place will increase endorphin output and then increase the sensory pleasure. So technically they do sound better. Sort of like a kissing booth the more beautiful girls kiss is always going to seem better, but blindfolded we may not be able to tell a difference or even enjoy the kiss of a chimpanzee.

Joey

An open book test is easier than a closed book test

If you can use an ABX box and accept its transparency, then you can take as long as you want, weeks, months. Keep everything on, answers saved in memory. You could do one trial a day

Yes, that time interval certainly didn't help and it hindered memory. But that point alone is never questioned when sighted listening is used I guess then memory is not hindered one bit

Actually, the sound is much better if a VERY special screw made out of Titanium/Beryllium composite is used to mount this puppy on the outlet. (Unfortunately it is only available to audiophiles who have spent over $ 20K in Ultra Hi-End HT Gear.)

I've heard some people ponder if there can be a research without subjectivity. From what I remember, the simple act of defining terms entails necessary subjectivity. It might be that subjectivity also enters when the tests are selected. Could it be that in some respects, the definitions determine what the outcomes will be--how something is defined determines how it can be measured? Musings centered upon objective outcomes based upon subjective selection?

I mention this only to point out that what may appear as objective analysis may contain subjectivity.

"One and only one thing was learned.
Mike used to claim that these differences were night and day... as do many other audiophiles.
They are not."

It might be that some research is launched simply to discover why current testing/theories don't jibe with experience--"What did we miss?" "What don't we know?"

What comes to my mind is the continuing research into nature vs nurture. It seems as if some research results can be interpreted to support obesity in grand children being caused by the grandmother experiencing famine while in the last trimester. It seems as if meth. tags, that "turn a gene off" can be passed from the first generation to the third. It also seems as if meth. tags can be created by stress, and that they are accumulated--indication of environment effecting how given genes interact, function.

Off hand, there seem to be many worth while avenues to further peruse?

Aural memory might be worth while to peruse?

Removing visual cues could be accomplished by placing a tunnel over the cables so that one cannot know what cable is being used? I'm not suggesting this be done; just pointing out that more testing can be done, if one likes to play around designing and implementing tests.

I like to listen to music.

Bob, a novice with not only electronics, but also and research; and, easily confused by facts.

I don't like closed book tests. They are too stressful and stop me from knowing answers I know with an open book test . Sadly, our joking nicely sums up the cable debate. It's very easy to pass a test or think you know the answers when the book is "open".

Actually, I still say there was no useful conclusion drawn from the test; all it did was confirm what was known already and didn't clarify anything that was in doubt. The test highlighted sighted bias yes (well, duh) and even showed that this can be carried over into blinded conditions, but that doesn't mean that the cables couldn't sound different (it's those network boxes that are bugging me) and be differentiated under proper conditions, e.g. instantaneous switches.

I'll say again, with those long cable switches, even if there were differences no one would have been able to reliably differentiate between the two sets of cables.

Of course, in all probability the cables do sound the same but in no way does this test confirm or deny that fact.

So, you can say "wrong" as much as you like but it doesn't change the facts dude.

This argument the test couldn't be reliable because of the long time between switches is absurd. For one, this is almost always how audiophiles 'test' cables--they manually install the new cables and then listen to music and 'hear' the incredible differences--not with instantaneous switching (has any audiophile used an ABX switch to test a new cable against his current ones?).

Secondly, remember Mike thought he could hear differences and was picking the correct cable every time--it just turns out he was not. Also, the 'long' switch times did not seem to hinder his ability when he was not blind...

I agree that this test only proves a very narrow conclusion. But also remember, this test is not a singularity and its conclusion follows every other similar test before it. Given enough of these 'narrow' tests showing the same result we can start making bit broader conclusions...

Really, that's not the point I was making. Now think about it for a minute.


Also irrelevant.

Am I the only one that gets this?

Sure, anyone with common sense knew that Mike's hearing was being influenced by sighted bias; that is something that's been shown so many times, in so many studies and tests that it's almost become a cliché. Yes, they don't sound as different as he claimed but that opinion was always going to be coloured by his expectation bias anyway, so again, duh!

However, we still don't really know if those Opus cables with the weird boxes on them sound any different from Monster because test was not performed in a way which could possibly have highlighted any small differences.

I, personally, don't care if audiophiles use incorrect testing methodologies to come up with incorrect conclusions as objectivists usually spot this and raise a red flag. But people here seemed to be drawing certain conclusions from this flawed test that should not be drawn; that the Opus and Monster cables sound identical.

Objectivists, myself included, have time and again pointed out to audiophiles doing this sort of test that aural memory is rated in single digit seconds and yet most people here seem to be now ignoring that fact.

Really, I'd appreciate it that when you take sound bites, you take enough to show what I was saying, in context, so that I'm not misrepresented. I'm not sure if you did that deliberately but from your post it looks as if I'm against your stated position when it's patently obvious that I'm not.

Here's the full text from that section you quoted;


As you've seen over on AA your "key result" is, in the general scheme of things audophile, worth little at best. Even the most extreme audiophiles these days seem to accept that sighted bias exists but a lot have put a positive twist on this and massaged it into their religion.

Now, actually proving beyond a doubt that those cables really do sound identical would have been a result.

The key result isn't for audiophiles...they're often immune to reason. That was addressed to the gentle reader following along.



Let's assume you really mean proving beyond a reasonable doubt (science doesn't truck in absolute proofs of nonexistence): as a reader of AA do you seriously believe that such a thing is even *possible* with that crowd?

No scientific test would prove that they COULDN'T sound different, i.e., under any circumstances to anyone. All the tests can do is point to likelihoods that a difference was heard.



'No one'? How sure you seem. But switching interval isn;t the only parameter in effect. The magnitude of the difference, and the sensitivity of the listener also count. A large enough difference, and a good enough listener, conceivably could overcome the deficit introduced by the switching interval.


"In no way". How sure you seem. Actually, this test goes a little way towards confirming that no real difference was heard by Mike.




You sure do seem to like putting things in absolute terms, dude.

Of course I meant reasonable; well, it was early but besides that sometimes (okay, it's getting to be always) it is very tedious to have to fully qualify every trivial statement in order to avoid pedantic exchanges.

And no, I don't think anything will convince the true believers but that's besides the point; the data from a valid test would be interesting to the rest of us.

yeah, I use the highest Bit Rates which are said to be CD quality. Its great that 500GB drives are under $100 and 1TB drives are sometimes under $200 these days. Considering HD movies are 30-40 GBs ripped and CD quality mp3s are 10+Meg a song

They're ignoring it because the results of this particular test leans towards their belief. I'm in the crowd that believes this test doesn't definitively prove anything, and that it was flawed from the beginning. I'd even be willing to bet that in spite of this test, Mike has no intentions of keeping the Monsters over the Opus. As others have pointed out, if this is to continue, then the next step should be a straight ABX in order to ascertain any differences instantaneously. I also think that there should be more than one person subjected to the test.

Then you've got to find out just where these differences exist assuming they're of the incredibly subtle short duration type that only register for short term hearing.

I quote:

"We stopped at 8 because it was quite clear that it was just random. I believe our requirement for a positive result was either 10 out of 10, or 17 out of 20, we intended to do 20 ABs. Less than that and it could have been criticized as just good luck."

I have an unvariable requirement of all tests I do: we agree in advance how many there will be, and we do ALL of them, regardless of how the results are turning out as we go along. In fact , we usually do the FULL RUN of tests - that would have been 20, in the aborted set described here - and we DO NOT REVEAL ANY RESULTS UNTIL THE FULL RUN IS COMPLETED. That way, we avoid any possible "discouragement" factor being invoked.

The 8 trials done were not at all significant; they were too few. In any case, we have only a sketchy idea of the complete protocol, so we cannot judge whether it was proper.

Much more importantly - the statement "Less than that, and it could have been criticized as just good luck" could equally well be re-written as, ""More than that, and it could have been criticized as just bad luck." This set of tests proved nothing...

Whoa!

Hilariously, it was *objectivists* (like me!) who were pushing Mike to make this an ABX test in the first place, as we know it is designed to maximize the subject's powers of discrimination. He's the one who seemed uncomfortable with that.

I've said nothing about this particular test that overstates its importance. In fact I've downplayed its informational value, including challenging ChrisWiggles' own characterization of the test results. I have encouraged Mike to do new tests that address the shortcomings in the protocol, and to practice.

But let's keep one thing in mind. None of us objectivists were claiming with certainty that Mike was going to hear differences, he did. He went into this test believing that under those conditions he was going to be able to hear differences. And during the test he was confident, too:


So let's be clear, the test didn't prove whether or not there are audible differences between cables. Not between any two pairs, and not between these two pairs in particular. Not for any listener, and not for Mike L. in particular.

What it did prove is that Mike L.'s confidence in his own discerning ability needs adjusting, downward. And though it doesn't prove anything about subjectivists in general, it certainly ought to cause any intellectually honest one to question their own confidence, too.

This is pretty much what I wrote in my first critique of the results -- best practice means you only reveal results during the test, IF you intend to complete ALL the trials agreed upon beforehand. For this test that would have meant finishing at least ten (one of two previously-agreed stopping-points, the other being 16).

Btw, Mr. Randi: welcome!


The significance of 8 trials (7, actually, in this case, as one trial was 'aborted' before any answer was given)-- or any number of trials -- is measurable and expressible as chances of error. On the one hand type I error is 'measured' by p value , the chance of incorrectly concluding a positive result (for 7 trials the lowest possible p value is 0.008, for 7/7 correct) Balanced against this is Type II error, the chance of incorrectly concluding a negative result, which increases as the number of trial decreases. And balanced against that in an audio trial is the occurrence of listener fatigue. 16 trials is commonly considered a good compromise as a minimum number of trials (as per Sensory Evaluation Techniques, 3rd ed., by Meilgaard, Civille, and Carr). So rather than having 'no significance at all', the test is more accurately characterized as 'not significant enough' to meet scientific standards.



True, but that can happen even with 16 trials done properly.