AVS Forum banner

Observations of a controlled Cable Test

99K views 383 replies 63 participants last post by  JapanDave 
#1 ·
[continued from http://www.avsforum.com/avs-vb/showt...18365&page=59]


Morning!


I got back a little late last night and didn't do a writeup then, I figure Mike will post his impressions at some point too.


First of all, kudos to Mike for being the always-gracious host, and also for his fairness and open mindedness for this trial. It is fair to recognize that he only had something to lose, and nothing really to be gained except perhaps an objective vindication of his position.


And to cut to the chase, Mike could not identify the Monster from the Opus MM with any accuracy (nor the reverse, which also would have been a positive result if he had been consistently wrong) using our testing methodology. We stopped the test a little less than halfway through, I think we got through 8 A/Bs before we gave up.


There were 4 of us present at the test. Mike had performed sighted listening tests on his own, and before I arrived he performed sighted listening tests while wearing a blindfold to acclimate himself to the testing method. His blindfold was a pair of duct-taped safety glasses. He could not see anything except a tiny bit directly below him. Theoretically, he could have strained his neck and been able to see the floor by the speakers where the wire was, but this was not allowed. After all, cheating would only serve to cheat himself. While switching wires, Mike wore earmuffs to minimize any possibility of identifying wires by the noises of switching the wires.


The plan was to do twenty A/B trials, which were chosen by flipping a coin. This was done before I arrived. After I arrived, we decided to check the level-match between wires. We did not have a method to level-match the wire if they were non-matched, however we also felt that the wire should match. Because the Opus uses some kind of network box, it conceivably would be possible that the signal were attenuated or otherwise altered and change the level, making it possible to identify the wire this way, rather than by sound 'quality' per se. The levels matched with a sine wave tone within 1/100th of a volt, the difference between the two sides was several hundredths of a volt with both wires. The difference between the L and R channels was greater than the possible difference between wires, which anyway was insignificant. We decided to check this because while we had no way of matching levels precisely, if there were a level difference we would have added a step to our methodology by turning down the volume all the way and then allowing mike to adjust the volume up, which would eliminate the ability to judge any difference in volume. We didn't need to do this, so Mike could listen without having to change the volume all the time, however he was free to change volume if he wanted.


There was, being generous, about one minute of a wait between changing cables. This was the case for both the sighted preparatory tests, and the blind tests. His amp basically goes straight through, so even with it turned off, it is feeding the speakers for a while. We would wait a little bit after turning off the amp before disconnecting the speaker wire and switching wire. We took care to be quiet while putting the wire down so as not to make any noise which could distinguish the wire. We changed the wire each and every time, regardless of whether switching to the other cable. At each time Mike would wear the earmuffs, even if he was going back to a known test, again to eliminate any possibility of identifying the wire by the noise of changing the wires.


We did at one point stop and go play pool for a few minutes and then come back to the test, doing a sighted (though still blindfolded) test to re-familiarize Mike with the cables.


So our results with Mike as our listener were clear: for this particular methodology, Mike could not accurately identify a difference in the cables. Again, a backwards result of Mike wrongly identifying the cables reliably also would have been a positive result.


I went into this with a fairly open mind. I did not have a previous opinion on the difference in speaker wire. However, I am fairly objective, and I did have two main expectations. First was that speaker wire, even if it did have an audible difference, would be very insignificant in the scope of the overall system. If there was going to be a difference, I was expecting it to be extremly subtle, and small. In other words, being objective about speaker wire tells me that it is fairly straightforward if you have sufficient gauge that the wire should basically get everything to the speaker with a precision that exceeds our hearing ability. Second, if there was going to be a difference, the suggested possibility of the network box in the Opus rolling off the highs may have caused an expectation of slightly less brightness/detail.


I personally went through the first few trials sighted, helping to change the wires. Going into the test I had no set opinion beside the expectations I just described. And while I am quite objective, objectivism also tells me things about vinyl which are quickly disproven in Mike's system. So I am open to these kinds of subjective things as well. As I went through 3 sighted trials, I began to form an opinion which basically told me that there WAS a difference in cables. Subtle, but I can say with confidence that it was present. I thought the Monster cable was a bit harsher, with a little bit less separation between instruments. This may have been a bias from my expectation that the Opus network box could have been attenuating the high frequencies, but I swore it was there. It is interesting to me that while I formed this opinion not having heard Mike or anyone else describe what they thought the difference was, after we stopped the whole test, Mike described what he previously thought the sighted difference sounded like (what he was listening for in the test) was practically identical to what I thought I was hearing sighted.


Then, for my own curiousity, I did the remaining 5 A/Bs before we stopped the test at 8 A/Bs blinded. I didn't wear ear muffs, and I just closed my eyes. I could have cheated if I wanted, but I did not (I guess you just have to take my word). I also, to check the rigor of our methdology, tried to actively listen to the cables being switched and see if I could identify either, to see if there was a way of identifying which was which that might have influenced mike even with earmuffs. There was no way to identify wire this way. I had looked at our test sheet, but I made no effort to try to memorize the sequence or have any idea what I would be listening to. In my 5 trials, I was certain on 3, and relatively uncertain on 2. I only matched 3 our ot 5, which is basically just blind luck, and I only got 2 of the 3 I was certain about correct. In other words, as a second, just self-interested and self-blinded tester, I could not with any reliability identify a difference in the cables.


The results for Mike and also the non rigorous one for myself certainly undid the opinion I was forming during my sighted listening. To characterize the position I've come to, I think it's fair to say that the objectivist arguments have a great deal of merit. It is fair to say that the onus is on subjectivists to discern speaker cable differences reliably. However it is ALSO fair to say that it is very difficult to prove a negative, especially for people with a subjectivist bent. Objectivists must also be honest about what our test results tell us: that with this methodology, and with this methodology ONLY, we could not reliably discern any differences in the cables. This does not prove that there are no differences with the cable, only that there were no differences discerned with this method.


In my personaly opinion, doing a long line of tests like this is generally unpleasant to do, you get tired of listening to the same stuff over and over, and after a little while it all starts to sound the same. Also, Mike only went back to a known A/B once during the test. I still would be interested in a similar test but spread out in time, doing maybe only one or two A/Bs at a time. This keeps fatigue down, and things don't all start to sound the same (again, just for the sake of argument assuming that there is an audible difference).


However, this test does tell us that even assuming for the sake of argument that there is a difference, that such a difference is extremly small, since it could not be percieved reliably with this method. After we finished the test, for curiosity we cut the ends of 16 gauge extension cords, and connected them to the system. We did this sighted. And the system sounded absolutely fantastic. The result I came away with through the whole experience is that people should generally not fret about speaker wire. Using a speaker wire of sufficient gauge for the task gets the job done. Even the unreliable sighted 'impressions' I had of the difference between Monster an Opus, which can not in any way be fairly attributable to actual difference, was extremely subtle. And the 16 gauge sounded basically exactly the same too.


After the test Mike was very gracious, and very fair and honest in accepting the result. I think he is probably curious to do some more testing in a different A/B format, perhaps along the lines I suggested. Or perhaps an ABX type test. It was curious that after the test, he characterized his certainty that during the test, it seemed fairly clear which was which. However, he was not identifying the cables with any consistency (rightly or wrongly) during the test. He did stop though and we left to go play pool to get away from the testing, so it seemed to me as an observer anyway that he was beginning to have difficulty, at least from his perspective, in identifying which was which. However, he was before that time not being at all consistent.


So I think that's basically everyting that either side would want to know. And I'm sure there's plenty of ammunition for both. I can say now that I am of the opinion that speaker wire basically makes no significant difference in sound quality, given sufficient gauge. Even the 16gauge extension cord was fantastic. I am still open to the possibility of audible differences with speaker cable, however such differences must be quite small and very subtle.


After the test, Mike played a handful of records, and I played some CDs as well. And what I came away with perhaps more than that is that Mike has probably the best overall system I've ever heard especially for BIG music. Big orchestral and rock sounded simply unbelievable. Santana, the Gates of Kiev, and Zeppelin among others on vinyl were just absolutely incredible. A completely new experience. I was trying to be extremely picky, and while I've heard a system here and there do small things differently in ways that in my opinion were slightly better, stepping back and considering the whole picture: it's simply an absolutely incredible system. The best I've ever heard big complex music.


So thanks again to Mike for being open and fair through the whole test and setting it all up and putting in the time and effort even though he really had nothing to gain. And for the pizza!



So flame on. If I were Mike, I frankly wouldn't care that much anyway, cable difference or not, it's one hell of a music system.
 
See less See more
1
#130 ·
I don't have much to add except to say that I'm MOST impressed by the way Mike has conducted himself.


I strongly doubt that the cables skeptics (and I include myself in this generalisation) would have been as gracious in accepting the results had they been otherwise. It would have been instant...'you didn't do this correctly, or what about that...'


Thankyou Mike for doing the test, and even more so thankyou for the manner in which you have reported it.
 
#131 ·
Wow what a thread....


So basically in a nutshell and in plain English, w/o all the "what if's" or "if I practice beforehands" etc let me see if I got this straight.


The listener, Mike, who is adament that he CAN tell the sound differences between cables,was sat down, blindfolded, told to listen to cable A....a bit later told to listen to cable B. This was done a few times and most of the times the listener, Mike, had no real clue which one was which, and in the end basically guessed to the best of his ability because ...well...it was basically impossible to tell the audible differences between the two...am I pretty close in giving this Cliff Notes non flowery assesment of what happened?


If so, why all this talk of "I need to prepare better next time" or "I was stressed" etc etc...I mean isn't the GIANT point of this test to plug in one set of cables, play music, switch to the other set, and say "Yeppers, cable A is brand "X"..I can tell because the lows sounded deeper..." or whatever....and in the listeners perfect world, he would be able to without a shadow of a doubt pick the same cable over and over and over again??


I don't understand why the listener needs to have more practice sessions to better understand each cable etc..or a less stressful environment...shouldn't you be able to just plop down blindfolded in a chair...listen to A then B and w/o a doubt pick out the cable you say is so easy to tell apart from the rest???


Please enlighten me as to why the story/test and the after effects are getting spun all around...if the listener couldn't tell a difference, then that should be the end of it...he couldn't tell a differnce, end of story, and he can move on...he can of course buy whatever he like, but now he will know in the back of his mind that there really are little to no differences between most if not all cables and marketers have indeed lined their pockets with cash for many years claiming their cable will INDEED sound better than most everyone elses..and people buy into it with their hard earned money.


Am I completely off base here???? My apologies if I am...it just to ME, seems VERY black and white..no real middle ground.


Oh and again, Mike you indeed were a brave soul to go through with the test and in the end find that your beliefs were indeed based on what could be very good marketing spin on the part of the boutique cable maker(s)...most folks in your boat would never have been willing to be proven "wrong" if you will about your former cable beliefs....cheers for taking the test.
 
#132 ·

Quote:
Originally Posted by Gordon Shumway /forum/post/12284299


Wow what a thread....


So basically in a nutshell and in plain English, w/o all the "what if's" or "if I practice beforehands" etc let me see if I got this straight.


The listener, Mike, who is adament that he CAN tell the sound differences between cables,was sat down, blindfolded, told to listen to cable A....a bit later told to listen to cable B. This was done a few times and most of the times the listener, Mike, had no real clue which one was which, and in the end basically guessed to the best of his ability because ...well...it was basically impossible to tell the audible differences between the two...am I pretty close in giving this Cliff Notes non flowery assesment of what happened?

Not quite right....he thought he was identifying the cables correctly; he didn't think he was 'guessing' (except for the discarded trial).
 
#133 ·
The moral of the story is that, if cables make a difference, but you can't reliably tell in an A/B situation, even with a minute between cable changes, it doesn't improve your *enjoyment* of music, except through the knowledge that it's there.


IOW, I use regular 14 gauge cables, not because I don't think some high-end cables aren't somewhat better, but because it really won't affect how much I enjoy music. I guess I'm kind of the opposite of the "I heard something, I must be enjoying music more now" crowd.
 
#136 ·
Hi Gordon,


being an objectivist myself it would be nice to say that this proved that all cables sound the same but unfortunately this test was not only flawed for what it was trying to show but even if it hadn’t been, it still wouldn't have shown that there, as you put it, "really are little to no differences between most if not all cables".


It did show that Mike's absolute certainty that he could tell the difference was misplaced.


However, there has been enough research out there showing how short our aural memory is to indicate that the time it took to switch the cables in Mike's test was just too long to pick out subtle differences even if they are there. So, what I would like to see is an instant ABX switch box trial or failing that, a similar test to Mike's but with a cable with known, obvious and unsubtle issues (silly network boxes with high end attenuation of whatever) pitted against the monster to see if real differences can be discerned under those conditions.
 
#137 ·

Quote:
Originally Posted by AdrianMills /forum/post/12285182


...but unfortunately this test was not only flawed for what it was trying to show...

I think that is a mildly unfair statement. The test was a genuine attempt to remove sighted bias from the equation. Conducting a "true" scientific experiment is not an easy task. Absent that, people can use some procedures that help them to come close. That is exactly what this test did. The aural memory issue is irrelevant, insofar as it was Mike's "skills" that were being tested, and Mike did not want instant switching.
 
#138 ·
Some might wonder if the test procedures were capable of capturing the test subject's experience?


For me, the most intriguing question is "What test procedures are necessary to capture the experience of "music changing when cables are changed"?


I suspect it will be necessary to rate a cable across criteria.


When I change tubes in my phono amplifier, I use a couple of different albums, one of which I use for changes in detail and depth, height and width of the hall, "speed" and timbre of un-amplified instruments, and the other for enjoyment of a fast paced concert. But, ultimately I'm looking for improving my enjoyment of the music, and fascination with how the instruments play with each other. Being able to identify the "sound" with a specific tube compared to another isn't part of improving my enjoyment.


A few times, I've put a different type of tube in each channel's power supply, not to play "guess which tube is in each channel", but to listen to the change in the music.


Bob
 
#139 ·

Quote:
Originally Posted by QQQ /forum/post/12285403


I think that is a mildly unfair statement. The test was a genuine attempt to remove sighted bias from the equation. Conducting a "true" scientific experiment is not an easy task. Absent that, people can use some procedures that help them to come close. That is exactly what this test did. The aural memory issue is irrelevant, insofar as it was Mike's "skills" that were being tested, and Mike did not want instant switching.

Okay, I'll give you a little and remove the "for what it was trying to show". I still say it was, as a testing methodology, flawed as there was no way any useful conclusion could have come out of it. And I guess that people will chime in that if he had heard something yada yada... Well, given the known limitations on our aural memory that was extremely unlikely in the first place with this testing methodology unless he's some kind of wonder man with a 1/1 billion aural memory. This is why I'd like to see the same test done with obviously different sounding cables - faked somehow if needed - just to determine if it's possible with those slow cable switches to hear the difference. My guess is probably not but it would be interesting to try.
 
#140 ·

Quote:
Of course, if you'd been using these:

http://www.machinadynamica.com/machina44.htm


THEN the test would have come out otherwise.

you have got to be kidding me!!! someone is really trying to embarass audiophiles with this piece of crap.

Quote:
This is why I'd like to see the same test done with obviously different sounding cables - faked somehow if needed - just to determine if it's possible with those slow cable switches to hear the difference. My guess is probably not but it would be interesting to try.


I don't know about aural memory, but when a piece of equipment sounds a lot better it is easy to define the differences and determine which is which! Example: pre amps. The problem is that all these cable companies and their reveiwers always claim sonic differences that are equal to if not greater than a new piece of equipment. That is completely false.


Joey
 
#141 ·

Quote:
Originally Posted by AdrianMills /forum/post/12285722


- just to determine if it's possible with those slow cable switches to hear the difference. My guess is probably not but it would be interesting to try.

does this not get to the crux of the matter though?? If I (and of course this is only my take on things) could not straight away pick a difference after spending twenty thou on an upgrade, no matter how long between switches, then to me it's simply not worth the money.


There's gotta be better bang for the buck elsewhere in the audio chain, hasn't there?
 
#142 ·

Quote:
Originally Posted by terry j /forum/post/12285773


does this not get to the crux of the matter though?? If I (and of course this is only my take on things) could not straight away pick a difference after spending twenty thou on an upgrade, no matter how long between switches, then to me it's simply not worth the money.

Well, it may be that differences, larger even than subtle, can be hidden by the limitations of our aural memory. I guess, at least to a certain point, the time limit we have in which differentiations can be made is very much dependant on how large the differences are.


Also, how big a difference has to be to be worth X amount more is a value judgement that’s very much a subjective and personal call even for us objectivists isn’t it?
 
#143 ·

Quote:
Originally Posted by AdrianMills /forum/post/12285935


Well, it may be that differences, larger even than subtle, can be hidden by the limitations of our aural memory. I guess, at least to a certain point, the time limit we have in which differentiations can be made is very much dependant on how large the differences are.


Also, how big a difference has to be to be worth X amount more is a value judgement that’s very much a subjective and personal call even for us objectivists isn’t it?

Stereophile suggest Type 2 errors in small sample testing http://www.stereophile.com/features/141/index2.html Personally I think one should look at the results of DBT in general. They typically do not show a difference on subjective measures REGARDLESS of the expermental variable. Make of that what you will. Taste test usually show no difference that's why you don't see them touted in advertisements. The taste test are never DB that you do see touted.
 
#144 ·

Quote:
This is why I'd like to see the same test done with obviously different sounding cables - faked somehow if needed - just to determine if it's possible with those slow cable switches to hear the difference. My guess is probably not but it would be interesting to try.

This has been done with different guage wire and yes you can tell the difference (ie 100 feet of 18AWG vs 100 Feet of 10AWG).


Its a good point though to atleast validate the test just to show that you can hear a difference at some point.
 
#145 ·

Quote:
IOW, I use regular 14 gauge cables, not because I don't think some high-end cables aren't somewhat better, but because it really won't affect how much I enjoy music. I guess I'm kind of the opposite of the "I heard something, I must be enjoying music more now" crowd.

That is a great post, I think more are like you and wonder what all this expensive stuff really gives anyone.


Heck I enjoy music when I play MP3s as much as when I play a CD (the CDs are ripped and gone now). I dont care much about doing lossless either. I dont get excited about all this stuff in the end and I wouldnt really appreciate MikeL's system at all.


Just give me loud music and girls dancing around pools




NOTE: This doesnt mean I can not hear the difference (everything under 5K is pretty well the same though!), it just means I dont enjoy it more.
 
#146 ·

Quote:
Originally Posted by penngray /forum/post/12286010


This has been done with different guage wire and yes you can tell the difference (ie 100 feet of 18AWG vs 100 Feet of 10AWG).


Its a good point though to atleast validate the test just to show that you can hear a difference at some point.

That has been done with an ABX box yes, but has it been done with ~1 minute change intervals?
 
#147 ·

Quote:
Originally Posted by Bulldogger /forum/post/12285991


Stereophile suggest Type 2 errors in small sample testing http://www.stereophile.com/features/141/index2.html Personally I think one should look at the results of DBT in general. They typically do not show a difference on subjective measures REGARDLESS of the expermental variable. Make of that what you will. Taste test usually show no difference that's why you don't see them touted in advertisements. The taste test are never DB that you do see touted.

It's pretty much a given that subjectivity makes a difference; but as has been said over and over, that's really not the point, at least it's not the point if you're testing for real world differences that exist outside of someone's head.
 
#148 ·
First, I'll admit I did not read this whole thread.


But I wonder, does Mike, or other testers, usually wear glasses? Maybe the reflection

off the glasses effected what he heard. Wearing something to block your sight

also changes the shape and other acoustical properties of your head. Perhaps

this interferes with your ability to discern subtle differences?

Or maybe you just can't hear the difference if you cannot see what is being played.

One day we might sort this out. Until then, the evidence seems to point toward

no difference between cables (assuming they are adequate, well made, and designed

for the job).
 
#150 ·
Hmmm, I'm thinking I should get into the cable making business. There is money to be made. And if nothing else I could probably write off all my music costs . . . In order to make quality cables, I'd have to go to many live shows, I'd also have to upgrade my equipment. Heck, I might even have to buy a new house
Then I'd have to do research

on the effects of food and drink on my cognitive listening ability. Damn, another tax deductible expense. If anybody runs with this idea, I'll expect a reasonable royalty. 5% or gross income sounds good.

Give thanks we can spend time and money debating such things.
 
#151 ·
I find this thread really amazing. A subjectivist thinks he will be able to easily discern the difference between two cables in a blind test since he is able to do so in a sighted test ---- but, by his own admission, is not able to do so when blind. But some other subjectivists refuse to believe this test has signficant meaning on a more global basis.


Here are the facts as I see them:

1. Mike L. has, if nothing else, proven he is one of the good guys and is not whining that things were not fair.


2. One person who was sure he could tell the difference in a blind test, could not


3. As far as I know, no one else has ever been able to tell the difference in a statistically meaningful way in a blind test either.


4. I have done some sighted and blind tests as well and "thought" I could tell the difference when sited and NEVER got it correct when blind


And here is MY opinion. If 5000 other individuals try to do what Mike did and also fail, there will continue to be many others who will refuse to agree that no SIGNIFICANT diference exists.


This subject should be grouped with two other topics that are usually fruitless discussions: (a) Religion (b) Politics
 
#153 ·

Quote:
Originally Posted by AdrianMills /forum/post/12285182


Hi Gordon,


being an objectivist myself it would be nice to say that this proved that all cables sound the same

Come on! Straw man. No test could ever prove that....one reason being, it's patently untrue. One can contrive two cables that certainly will sound differently. Nor was that even remotely what Mike's test was 'trying to prove'.

Quote:
It did show that Mike's absolute certainty that he could tell the difference was misplaced.

And in terms of challenging standard audiophile rhetoric, that's a key result. We can say that the reality of differences Mike thought he heard, quite confidently, wasn't supported by the results.
 
#154 ·

Quote:
Originally Posted by AdrianMills /forum/post/12285722


Okay, I'll give you a little and remove the "for what it was trying to show". I still say it was, as a testing methodology, flawed as there was no way any useful conclusion could have come out of it.

Wrong.. Mike claimed to confidently hear differences on his own. He claimed to confidently hear differences under test conditions, too. Both he and the testers used manual cable switching. The results indicate he probably didn't hear differences in the first place, in his 'usual' sighted protocol involving manual switching. It doesn't rule out that under more sensitive conditions -- that HE had NEVER yet used -- he MIGHT score positive on a DBT for these cables.
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top