or Connect
AVS › AVS Forum › Display Devices › Ultra Hi-End HT Gear ($20,000+) › Observations of a controlled Cable Test
New Posts  All Forums:Forum Nav:

Observations of a controlled Cable Test - Page 6

post #151 of 384
I find this thread really amazing. A subjectivist thinks he will be able to easily discern the difference between two cables in a blind test since he is able to do so in a sighted test ---- but, by his own admission, is not able to do so when blind. But some other subjectivists refuse to believe this test has signficant meaning on a more global basis.

Here are the facts as I see them:
1. Mike L. has, if nothing else, proven he is one of the good guys and is not whining that things were not fair.

2. One person who was sure he could tell the difference in a blind test, could not

3. As far as I know, no one else has ever been able to tell the difference in a statistically meaningful way in a blind test either.

4. I have done some sighted and blind tests as well and "thought" I could tell the difference when sited and NEVER got it correct when blind

And here is MY opinion. If 5000 other individuals try to do what Mike did and also fail, there will continue to be many others who will refuse to agree that no SIGNIFICANT diference exists.

This subject should be grouped with two other topics that are usually fruitless discussions: (a) Religion (b) Politics
post #152 of 384
Quote:
Originally Posted by Gordon Shumway View Post

Nope..it's real and I bet there's someone on this site who has purchased one or more...


James Randi has been taunting Machine Dynamica lately on the JREF site. Maybe some audiophile wants to step up to the plate....?
post #153 of 384
Quote:
Originally Posted by AdrianMills View Post

Hi Gordon,

being an objectivist myself it would be nice to say that this proved that all cables sound the same

Come on! Straw man. No test could ever prove that....one reason being, it's patently untrue. One can contrive two cables that certainly will sound differently. Nor was that even remotely what Mike's test was 'trying to prove'.

Quote:


It did show that Mike's absolute certainty that he could tell the difference was misplaced.

And in terms of challenging standard audiophile rhetoric, that's a key result. We can say that the reality of differences Mike thought he heard, quite confidently, wasn't supported by the results.
post #154 of 384
Quote:
Originally Posted by AdrianMills View Post

Okay, I'll give you a little and remove the "for what it was trying to show". I still say it was, as a testing methodology, flawed as there was no way any useful conclusion could have come out of it.

Wrong.. Mike claimed to confidently hear differences on his own. He claimed to confidently hear differences under test conditions, too. Both he and the testers used manual cable switching. The results indicate he probably didn't hear differences in the first place, in his 'usual' sighted protocol involving manual switching. It doesn't rule out that under more sensitive conditions -- that HE had NEVER yet used -- he MIGHT score positive on a DBT for these cables.
post #155 of 384
Quote:
Originally Posted by penngray View Post

That is a great post, I think more are like you and wonder what all this expensive stuff really gives anyone.

Heck I enjoy music when I play MP3s as much as when I play a CD (the CDs are ripped and gone now). I dont care much about doing lossless either. I dont get excited about all this stuff in the end and I wouldnt really appreciate MikeL's system at all.

Just give me loud music and girls dancing around pools


NOTE: This doesnt mean I can not hear the difference (everything under 5K is pretty well the same though!), it just means I dont enjoy it more.


It's quite possible you will not hear a difference between an mp3 and its source wav, even in a DBT, if you make the mp3 right.
post #156 of 384
Quote:
Originally Posted by Bulldogger View Post

Stereophile suggest Type 2 errors in small sample testing http://www.stereophile.com/features/141/index2.html Personally I think one should look at the results of DBT in general. They typically do not show a difference on subjective measures REGARDLESS of the expermental variable. Make of that what you will. Taste test usually show no difference that's why you don't see them touted in advertisements. The taste test are never DB that you do see touted.

'Make of it what you will'. Yeah, go tell it to the experimental psychology community, not to mention the medical testing community, not to mention the lossy codec development community, you self-described 'dumb audiophile'. Obviously they need a new testing paradigm...DBT just isn't working!
post #157 of 384
One and only one thing was learned.

Mike used to claim that these differences were night and day... as do many other audiophiles.

They are not.

As has been stated, with instantaneous ABX, it is theoretically possible (highly unlikely i feel) that a positive result could be obtained.

To state any other result that above is to draw far too much out of a simple test with some decent controls in place.

The machina dynamica stuff scares me. that is truly snake oil shite.
post #158 of 384
I THINK THAT THIS TRULY EXPLAINS HOW ARE MIND WORKS AND TAKES SENSORY PERCEPTIONS OF THE BRAIN TO A WHOLE NEW LEVEL!!!

http://www.reuters.com/article/domes...33509820071121

jOEY
One of my better finds, if I don't say so myself
post #159 of 384
So possibly just by seeing the cables in place will increase endorphin output and then increase the sensory pleasure. So technically they do sound better. Sort of like a kissing booth the more beautiful girls kiss is always going to seem better, but blindfolded we may not be able to tell a difference or even enjoy the kiss of a chimpanzee.

Joey
post #160 of 384
Quote:
Originally Posted by krabapple View Post


Mike L. seemed to have no 'stress' problems when comparing cables on his own. He heard those 'obvious' difference just fine.

An open book test is easier than a closed book test
post #161 of 384
Quote:
Originally Posted by mike lavigne View Post


my future efforts will be focused on how to duplicate my performance on test #1 and #7 where my aural memory was most strong. it's also possible i was just 'lucky' on #1 and #7 and i never really heard any differences. time will tell.

If you can use an ABX box and accept its transparency, then you can take as long as you want, weeks, months. Keep everything on, answers saved in memory. You could do one trial a day
post #162 of 384
Quote:
Originally Posted by AdrianMills View Post

That has been done with an ABX box yes, but has it been done with ~1 minute change intervals?

Yes, that time interval certainly didn't help and it hindered memory. But that point alone is never questioned when sighted listening is used I guess then memory is not hindered one bit
post #163 of 384
Quote:
Originally Posted by Dean Roddey View Post

Of course, if you'd been using these:

http://www.machinadynamica.com/machina44.htm

THEN the test would have come out otherwise.

Actually, the sound is much better if a VERY special screw made out of Titanium/Beryllium composite is used to mount this puppy on the outlet. (Unfortunately it is only available to audiophiles who have spent over $ 20K in Ultra Hi-End HT Gear.)
post #164 of 384
I've heard some people ponder if there can be a research without subjectivity. From what I remember, the simple act of defining terms entails necessary subjectivity. It might be that subjectivity also enters when the tests are selected. Could it be that in some respects, the definitions determine what the outcomes will be--how something is defined determines how it can be measured? Musings centered upon objective outcomes based upon subjective selection?

I mention this only to point out that what may appear as objective analysis may contain subjectivity.

"One and only one thing was learned.
Mike used to claim that these differences were night and day... as do many other audiophiles.
They are not."

It might be that some research is launched simply to discover why current testing/theories don't jibe with experience--"What did we miss?" "What don't we know?"

What comes to my mind is the continuing research into nature vs nurture. It seems as if some research results can be interpreted to support obesity in grand children being caused by the grandmother experiencing famine while in the last trimester. It seems as if meth. tags, that "turn a gene off" can be passed from the first generation to the third. It also seems as if meth. tags can be created by stress, and that they are accumulated--indication of environment effecting how given genes interact, function.

Off hand, there seem to be many worth while avenues to further peruse?

Aural memory might be worth while to peruse?

Removing visual cues could be accomplished by placing a tunnel over the cables so that one cannot know what cable is being used? I'm not suggesting this be done; just pointing out that more testing can be done, if one likes to play around designing and implementing tests.

I like to listen to music.

Bob, a novice with not only electronics, but also and research; and, easily confused by facts.
post #165 of 384
Quote:
Originally Posted by CharlesJ View Post

An open book test is easier than a closed book test

I don't like closed book tests. They are too stressful and stop me from knowing answers I know with an open book test . Sadly, our joking nicely sums up the cable debate. It's very easy to pass a test or think you know the answers when the book is "open".
post #166 of 384
Quote:
Originally Posted by krabapple View Post

Wrong.. Mike claimed to confidently hear differences on his own. He claimed to confidently hear differences under test conditions, too. Both he and the testers used manual cable switching. The results indicate he probably didn't hear differences in the first place, in his 'usual' sighted protocol involving manual switching. It doesn't rule out that under more sensitive conditions -- that HE had NEVER yet used -- he MIGHT score positive on a DBT for these cables.

Actually, I still say there was no useful conclusion drawn from the test; all it did was confirm what was known already and didn't clarify anything that was in doubt. The test highlighted sighted bias yes (well, duh) and even showed that this can be carried over into blinded conditions, but that doesn't mean that the cables couldn't sound different (it's those network boxes that are bugging me) and be differentiated under proper conditions, e.g. instantaneous switches.

I'll say again, with those long cable switches, even if there were differences no one would have been able to reliably differentiate between the two sets of cables.

Of course, in all probability the cables do sound the same but in no way does this test confirm or deny that fact.

So, you can say "wrong" as much as you like but it doesn't change the facts dude.
post #167 of 384
Quote:
Originally Posted by AdrianMills View Post

Actually, I still say there was no useful conclusion drawn from the test; all it did was confirm what was known already and didn't clarify anything that was in doubt. The test highlighted sighted bias yes (well, duh) and even showed that this can be carried over into blinded conditions, but that doesn't mean that the cables couldn't sound different (it's those network boxes that are bugging me) and be differentiated under proper conditions, e.g. instantaneous switches.

I'll say again, with those long cable switches, even if there were differences no one would have been able to reliably differentiate between the two sets of cables.

Of course, in all probability the cables do sound the same but in no way does this test confirm or deny that fact.

So, you can say "wrong" as much as you like but it doesn't change the facts dude.

This argument the test couldn't be reliable because of the long time between switches is absurd. For one, this is almost always how audiophiles 'test' cables--they manually install the new cables and then listen to music and 'hear' the incredible differences--not with instantaneous switching (has any audiophile used an ABX switch to test a new cable against his current ones?).

Secondly, remember Mike thought he could hear differences and was picking the correct cable every time--it just turns out he was not. Also, the 'long' switch times did not seem to hinder his ability when he was not blind...

I agree that this test only proves a very narrow conclusion. But also remember, this test is not a singularity and its conclusion follows every other similar test before it. Given enough of these 'narrow' tests showing the same result we can start making bit broader conclusions...
post #168 of 384
Quote:
Originally Posted by JJay View Post

This argument the test couldn't be reliable because of the long time between switches is absurd. For one, this is almost always how audiophiles 'test' cables--they manually install the new cables and then listen to music and 'hear' the incredible differences--not with instantaneous switching (has any audiophile used an ABX switch to test a new cable against his current ones?).

Really, that's not the point I was making. Now think about it for a minute.

Quote:
Originally Posted by JJay View Post

Secondly, remember Mike thought he could hear differences and was picking the correct cable every time--it just turns out he was not. Also, the 'long' switch times did not seem to hinder his ability when he was not blind...

Also irrelevant.

Am I the only one that gets this?

Sure, anyone with common sense knew that Mike's hearing was being influenced by sighted bias; that is something that's been shown so many times, in so many studies and tests that it's almost become a cliché. Yes, they don't sound as different as he claimed but that opinion was always going to be coloured by his expectation bias anyway, so again, duh!

However, we still don't really know if those Opus cables with the weird boxes on them sound any different from Monster because test was not performed in a way which could possibly have highlighted any small differences.

I, personally, don't care if audiophiles use incorrect testing methodologies to come up with incorrect conclusions as objectivists usually spot this and raise a red flag. But people here seemed to be drawing certain conclusions from this flawed test that should not be drawn; that the Opus and Monster cables sound identical.

Objectivists, myself included, have time and again pointed out to audiophiles doing this sort of test that aural memory is rated in single digit seconds and yet most people here seem to be now ignoring that fact.
post #169 of 384
Quote:
Originally Posted by krabapple View Post

Come on! Straw man. No test could ever prove that....one reason being, it's patently untrue. One can contrive two cables that certainly will sound differently. Nor was that even remotely what Mike's test was 'trying to prove'.

Really, I'd appreciate it that when you take sound bites, you take enough to show what I was saying, in context, so that I'm not misrepresented. I'm not sure if you did that deliberately but from your post it looks as if I'm against your stated position when it's patently obvious that I'm not.

Here's the full text from that section you quoted;
Quote:


being an objectivist myself it would be nice to say that this proved that all cables sound the same but unfortunately this test was not only flawed for what it was trying to show but even if it hadn't been, it still wouldn't have shown that there, as you put it, "really are little to no differences between most if not all cables".

Quote:
Originally Posted by krabapple View Post

And in terms of challenging standard audiophile rhetoric, that's a key result. We can say that the reality of differences Mike thought he heard, quite confidently, wasn't supported by the results.

As you've seen over on AA your "key result" is, in the general scheme of things audophile, worth little at best. Even the most extreme audiophiles these days seem to accept that sighted bias exists but a lot have put a positive twist on this and massaged it into their religion.

Now, actually proving beyond a doubt that those cables really do sound identical would have been a result.
post #170 of 384
Quote:
Originally Posted by AdrianMills View Post

As you've seen over on AA your "key result" is, in the general scheme of things audophile, worth little at best.


The key result isn't for audiophiles...they're often immune to reason. That was addressed to the gentle reader following along.


Quote:


Even the most extreme audiophiles these days seem to accept that sighted bias exists but a lot have put a positive twist on this and massaged it into their religion.

Now, actually proving beyond a doubt that those cables really do sound identical would have been a result.

Let's assume you really mean proving beyond a reasonable doubt (science doesn't truck in absolute proofs of nonexistence): as a reader of AA do you seriously believe that such a thing is even *possible* with that crowd?
post #171 of 384
Quote:
Originally Posted by AdrianMills View Post

Actually, I still say there was no useful conclusion drawn from the test; all it did was confirm what was known already and didn't clarify anything that was in doubt. The test highlighted sighted bias yes (well, duh) and even showed that this can be carried over into blinded conditions, but that doesn't mean that the cables couldn't sound different (it's those network boxes that are bugging me) and be differentiated under proper conditions, e.g. instantaneous switches.

No scientific test would prove that they COULDN'T sound different, i.e., under any circumstances to anyone. All the tests can do is point to likelihoods that a difference was heard.


Quote:


I'll say again, with those long cable switches, even if there were differences no one would have been able to reliably differentiate between the two sets of cables.

'No one'? How sure you seem. But switching interval isn;t the only parameter in effect. The magnitude of the difference, and the sensitivity of the listener also count. A large enough difference, and a good enough listener, conceivably could overcome the deficit introduced by the switching interval.

Quote:


Of course, in all probability the cables do sound the same but in no way does this test confirm or deny that fact.

"In no way". How sure you seem. Actually, this test goes a little way towards confirming that no real difference was heard by Mike.


Quote:


So, you can say "wrong" as much as you like but it doesn't change the facts dude.


You sure do seem to like putting things in absolute terms, dude.
post #172 of 384
Quote:
Originally Posted by krabapple View Post

Let's assume you really mean proving beyond a reasonable doubt (science doesn't truck in absolute proofs of nonexistence): as a reader of AA do you seriously believe that such a thing is even *possible* with that crowd?

Of course I meant reasonable; well, it was early but besides that sometimes (okay, it's getting to be always) it is very tedious to have to fully qualify every trivial statement in order to avoid pedantic exchanges.

And no, I don't think anything will convince the true believers but that's besides the point; the data from a valid test would be interesting to the rest of us.
post #173 of 384
Quote:


It's quite possible you will not hear a difference between an mp3 and its source wav, even in a DBT, if you make the mp3 right.


yeah, I use the highest Bit Rates which are said to be CD quality. Its great that 500GB drives are under $100 and 1TB drives are sometimes under $200 these days. Considering HD movies are 30-40 GBs ripped and CD quality mp3s are 10+Meg a song
post #174 of 384
Quote:
Originally Posted by AdrianMills View Post

Objectivists, myself included, have time and again pointed out to audiophiles doing this sort of test that aural memory is rated in single digit seconds and yet most people here seem to be now ignoring that fact.

They're ignoring it because the results of this particular test leans towards their belief. I'm in the crowd that believes this test doesn't definitively prove anything, and that it was flawed from the beginning. I'd even be willing to bet that in spite of this test, Mike has no intentions of keeping the Monsters over the Opus. As others have pointed out, if this is to continue, then the next step should be a straight ABX in order to ascertain any differences instantaneously. I also think that there should be more than one person subjected to the test.
post #175 of 384
Then you've got to find out just where these differences exist assuming they're of the incredibly subtle short duration type that only register for short term hearing.
post #176 of 384
I quote:

"We stopped at 8 because it was quite clear that it was just random. I believe our requirement for a positive result was either 10 out of 10, or 17 out of 20, we intended to do 20 ABs. Less than that and it could have been criticized as just good luck."

I have an unvariable requirement of all tests I do: we agree in advance how many there will be, and we do ALL of them, regardless of how the results are turning out as we go along. In fact , we usually do the FULL RUN of tests - that would have been 20, in the aborted set described here - and we DO NOT REVEAL ANY RESULTS UNTIL THE FULL RUN IS COMPLETED. That way, we avoid any possible "discouragement" factor being invoked.

The 8 trials done were not at all significant; they were too few. In any case, we have only a sketchy idea of the complete protocol, so we cannot judge whether it was proper.

Much more importantly - the statement "Less than that, and it could have been criticized as just good luck" could equally well be re-written as, ""More than that, and it could have been criticized as just bad luck." This set of tests proved nothing...
post #177 of 384
Whoa!
post #178 of 384
Quote:
Originally Posted by Rutgar View Post

They're ignoring it because the results of this particular test leans towards their belief. I'm in the crowd that believes this test doesn't definitively prove anything, and that it was flawed from the beginning. I'd even be willing to bet that in spite of this test, Mike has no intentions of keeping the Monsters over the Opus. As others have pointed out, if this is to continue, then the next step should be a straight ABX in order to ascertain any differences instantaneously. I also think that there should be more than one person subjected to the test.


Hilariously, it was *objectivists* (like me!) who were pushing Mike to make this an ABX test in the first place, as we know it is designed to maximize the subject's powers of discrimination. He's the one who seemed uncomfortable with that.
post #179 of 384
I've said nothing about this particular test that overstates its importance. In fact I've downplayed its informational value, including challenging ChrisWiggles' own characterization of the test results. I have encouraged Mike to do new tests that address the shortcomings in the protocol, and to practice.

But let's keep one thing in mind. None of us objectivists were claiming with certainty that Mike was going to hear differences, he did. He went into this test believing that under those conditions he was going to be able to hear differences. And during the test he was confident, too:
Quote:


until the test was over i felt confident that i could hear real differences and that i had a 'lock' on which characteristic was which cable.

Quote:


after the first 3 tests i was getting a bit tired and the ear protection started causing a slight headache. at no time did i feel that was affecting my listening.

Quote:


when i made my choice known for #8 i was confident that i was 100% for all 7. then my friend Ted said 'that's it.....test over'.

So let's be clear, the test didn't prove whether or not there are audible differences between cables. Not between any two pairs, and not between these two pairs in particular. Not for any listener, and not for Mike L. in particular.

What it did prove is that Mike L.'s confidence in his own discerning ability needs adjusting, downward. And though it doesn't prove anything about subjectivists in general, it certainly ought to cause any intellectually honest one to question their own confidence, too.
post #180 of 384
Quote:
Originally Posted by James Randi View Post

I quote:

"We stopped at 8 because it was quite clear that it was just random. I believe our requirement for a positive result was either 10 out of 10, or 17 out of 20, we intended to do 20 ABs. Less than that and it could have been criticized as just good luck."

I have an unvariable requirement of all tests I do: we agree in advance how many there will be, and we do ALL of them, regardless of how the results are turning out as we go along. In fact , we usually do the FULL RUN of tests - that would have been 20, in the aborted set described here - and we DO NOT REVEAL ANY RESULTS UNTIL THE FULL RUN IS COMPLETED. That way, we avoid any possible "discouragement" factor being invoked.

This is pretty much what I wrote in my first critique of the results -- best practice means you only reveal results during the test, IF you intend to complete ALL the trials agreed upon beforehand. For this test that would have meant finishing at least ten (one of two previously-agreed stopping-points, the other being 16).

Btw, Mr. Randi: welcome!

Quote:


The 8 trials done were not at all significant; they were too few. In any case, we have only a sketchy idea of the complete protocol, so we cannot judge whether it was proper.

The significance of 8 trials (7, actually, in this case, as one trial was 'aborted' before any answer was given)-- or any number of trials -- is measurable and expressible as chances of error. On the one hand type I error is 'measured' by p value , the chance of incorrectly concluding a positive result (for 7 trials the lowest possible p value is 0.008, for 7/7 correct) Balanced against this is Type II error, the chance of incorrectly concluding a negative result, which increases as the number of trial decreases. And balanced against that in an audio trial is the occurrence of listener fatigue. 16 trials is commonly considered a good compromise as a minimum number of trials (as per Sensory Evaluation Techniques, 3rd ed., by Meilgaard, Civille, and Carr). So rather than having 'no significance at all', the test is more accurately characterized as 'not significant enough' to meet scientific standards.


Quote:


Much more importantly - the statement "Less than that, and it could have been criticized as just good luck" could equally well be re-written as, ""More than that, and it could have been criticized as just bad luck." This set of tests proved nothing...

True, but that can happen even with 16 trials done properly.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Ultra Hi-End HT Gear ($20,000+)
AVS › AVS Forum › Display Devices › Ultra Hi-End HT Gear ($20,000+) › Observations of a controlled Cable Test