AVS Forum banner

1 - 20 of 384 Posts

·
Banned
Joined
·
20,735 Posts
Discussion Starter #1
[continued from http://www.avsforum.com/avs-vb/showt...18365&page=59]


Morning!


I got back a little late last night and didn't do a writeup then, I figure Mike will post his impressions at some point too.


First of all, kudos to Mike for being the always-gracious host, and also for his fairness and open mindedness for this trial. It is fair to recognize that he only had something to lose, and nothing really to be gained except perhaps an objective vindication of his position.


And to cut to the chase, Mike could not identify the Monster from the Opus MM with any accuracy (nor the reverse, which also would have been a positive result if he had been consistently wrong) using our testing methodology. We stopped the test a little less than halfway through, I think we got through 8 A/Bs before we gave up.


There were 4 of us present at the test. Mike had performed sighted listening tests on his own, and before I arrived he performed sighted listening tests while wearing a blindfold to acclimate himself to the testing method. His blindfold was a pair of duct-taped safety glasses. He could not see anything except a tiny bit directly below him. Theoretically, he could have strained his neck and been able to see the floor by the speakers where the wire was, but this was not allowed. After all, cheating would only serve to cheat himself. While switching wires, Mike wore earmuffs to minimize any possibility of identifying wires by the noises of switching the wires.


The plan was to do twenty A/B trials, which were chosen by flipping a coin. This was done before I arrived. After I arrived, we decided to check the level-match between wires. We did not have a method to level-match the wire if they were non-matched, however we also felt that the wire should match. Because the Opus uses some kind of network box, it conceivably would be possible that the signal were attenuated or otherwise altered and change the level, making it possible to identify the wire this way, rather than by sound 'quality' per se. The levels matched with a sine wave tone within 1/100th of a volt, the difference between the two sides was several hundredths of a volt with both wires. The difference between the L and R channels was greater than the possible difference between wires, which anyway was insignificant. We decided to check this because while we had no way of matching levels precisely, if there were a level difference we would have added a step to our methodology by turning down the volume all the way and then allowing mike to adjust the volume up, which would eliminate the ability to judge any difference in volume. We didn't need to do this, so Mike could listen without having to change the volume all the time, however he was free to change volume if he wanted.


There was, being generous, about one minute of a wait between changing cables. This was the case for both the sighted preparatory tests, and the blind tests. His amp basically goes straight through, so even with it turned off, it is feeding the speakers for a while. We would wait a little bit after turning off the amp before disconnecting the speaker wire and switching wire. We took care to be quiet while putting the wire down so as not to make any noise which could distinguish the wire. We changed the wire each and every time, regardless of whether switching to the other cable. At each time Mike would wear the earmuffs, even if he was going back to a known test, again to eliminate any possibility of identifying the wire by the noise of changing the wires.


We did at one point stop and go play pool for a few minutes and then come back to the test, doing a sighted (though still blindfolded) test to re-familiarize Mike with the cables.


So our results with Mike as our listener were clear: for this particular methodology, Mike could not accurately identify a difference in the cables. Again, a backwards result of Mike wrongly identifying the cables reliably also would have been a positive result.


I went into this with a fairly open mind. I did not have a previous opinion on the difference in speaker wire. However, I am fairly objective, and I did have two main expectations. First was that speaker wire, even if it did have an audible difference, would be very insignificant in the scope of the overall system. If there was going to be a difference, I was expecting it to be extremly subtle, and small. In other words, being objective about speaker wire tells me that it is fairly straightforward if you have sufficient gauge that the wire should basically get everything to the speaker with a precision that exceeds our hearing ability. Second, if there was going to be a difference, the suggested possibility of the network box in the Opus rolling off the highs may have caused an expectation of slightly less brightness/detail.


I personally went through the first few trials sighted, helping to change the wires. Going into the test I had no set opinion beside the expectations I just described. And while I am quite objective, objectivism also tells me things about vinyl which are quickly disproven in Mike's system. So I am open to these kinds of subjective things as well. As I went through 3 sighted trials, I began to form an opinion which basically told me that there WAS a difference in cables. Subtle, but I can say with confidence that it was present. I thought the Monster cable was a bit harsher, with a little bit less separation between instruments. This may have been a bias from my expectation that the Opus network box could have been attenuating the high frequencies, but I swore it was there. It is interesting to me that while I formed this opinion not having heard Mike or anyone else describe what they thought the difference was, after we stopped the whole test, Mike described what he previously thought the sighted difference sounded like (what he was listening for in the test) was practically identical to what I thought I was hearing sighted.


Then, for my own curiousity, I did the remaining 5 A/Bs before we stopped the test at 8 A/Bs blinded. I didn't wear ear muffs, and I just closed my eyes. I could have cheated if I wanted, but I did not (I guess you just have to take my word). I also, to check the rigor of our methdology, tried to actively listen to the cables being switched and see if I could identify either, to see if there was a way of identifying which was which that might have influenced mike even with earmuffs. There was no way to identify wire this way. I had looked at our test sheet, but I made no effort to try to memorize the sequence or have any idea what I would be listening to. In my 5 trials, I was certain on 3, and relatively uncertain on 2. I only matched 3 our ot 5, which is basically just blind luck, and I only got 2 of the 3 I was certain about correct. In other words, as a second, just self-interested and self-blinded tester, I could not with any reliability identify a difference in the cables.


The results for Mike and also the non rigorous one for myself certainly undid the opinion I was forming during my sighted listening. To characterize the position I've come to, I think it's fair to say that the objectivist arguments have a great deal of merit. It is fair to say that the onus is on subjectivists to discern speaker cable differences reliably. However it is ALSO fair to say that it is very difficult to prove a negative, especially for people with a subjectivist bent. Objectivists must also be honest about what our test results tell us: that with this methodology, and with this methodology ONLY, we could not reliably discern any differences in the cables. This does not prove that there are no differences with the cable, only that there were no differences discerned with this method.


In my personaly opinion, doing a long line of tests like this is generally unpleasant to do, you get tired of listening to the same stuff over and over, and after a little while it all starts to sound the same. Also, Mike only went back to a known A/B once during the test. I still would be interested in a similar test but spread out in time, doing maybe only one or two A/Bs at a time. This keeps fatigue down, and things don't all start to sound the same (again, just for the sake of argument assuming that there is an audible difference).


However, this test does tell us that even assuming for the sake of argument that there is a difference, that such a difference is extremly small, since it could not be percieved reliably with this method. After we finished the test, for curiosity we cut the ends of 16 gauge extension cords, and connected them to the system. We did this sighted. And the system sounded absolutely fantastic. The result I came away with through the whole experience is that people should generally not fret about speaker wire. Using a speaker wire of sufficient gauge for the task gets the job done. Even the unreliable sighted 'impressions' I had of the difference between Monster an Opus, which can not in any way be fairly attributable to actual difference, was extremely subtle. And the 16 gauge sounded basically exactly the same too.


After the test Mike was very gracious, and very fair and honest in accepting the result. I think he is probably curious to do some more testing in a different A/B format, perhaps along the lines I suggested. Or perhaps an ABX type test. It was curious that after the test, he characterized his certainty that during the test, it seemed fairly clear which was which. However, he was not identifying the cables with any consistency (rightly or wrongly) during the test. He did stop though and we left to go play pool to get away from the testing, so it seemed to me as an observer anyway that he was beginning to have difficulty, at least from his perspective, in identifying which was which. However, he was before that time not being at all consistent.


So I think that's basically everyting that either side would want to know. And I'm sure there's plenty of ammunition for both. I can say now that I am of the opinion that speaker wire basically makes no significant difference in sound quality, given sufficient gauge. Even the 16gauge extension cord was fantastic. I am still open to the possibility of audible differences with speaker cable, however such differences must be quite small and very subtle.


After the test, Mike played a handful of records, and I played some CDs as well. And what I came away with perhaps more than that is that Mike has probably the best overall system I've ever heard especially for BIG music. Big orchestral and rock sounded simply unbelievable. Santana, the Gates of Kiev, and Zeppelin among others on vinyl were just absolutely incredible. A completely new experience. I was trying to be extremely picky, and while I've heard a system here and there do small things differently in ways that in my opinion were slightly better, stepping back and considering the whole picture: it's simply an absolutely incredible system. The best I've ever heard big complex music.


So thanks again to Mike for being open and fair through the whole test and setting it all up and putting in the time and effort even though he really had nothing to gain. And for the pizza!



So flame on. If I were Mike, I frankly wouldn't care that much anyway, cable difference or not, it's one hell of a music system.
 

·
Registered
Joined
·
18,501 Posts
I haven't kept up on this thread, or I probably would have found an excuse to come down..



An excellent writeup Chris! Your test pretty much covered how I would have done the a/bing, and the outcome was pretty much expected (coming from an objectivist like me! ).


Now I'm REALLY curious as to Mike's comments. Mike, has this test changed your mind about cables, or ??
 

·
Registered
Joined
·
2,365 Posts
well that certainly is a little surprise. Maybe it means that the network boxes are as big a myth as the 10 dollars woth of parts in the PSAudio noise harvester.


I find the extension cord test very telling as well.


And I would really like to hear records again on a good system so count that as a plus for you.
 

·
Registered
Joined
·
251 Posts
Kudos all around to those involved.
 

·
Registered
Joined
·
5,211 Posts
Can all of those who are normally on an insulting and sarcastic bent please try to be adult.


The results here are (to my interpretation) far more about sighted biases than anything else.


And as will undoubtedly happen, they will be stated to be valid only for mikes system. "of course if they used system ABC, then it would have been clearly obvious"



Kudos to mike for stepping up. ANd Like Curt, i am most interested in mikes take on the test.


unless he is sitting in his barn weeping
(joke, just kidding)
 

·
Registered
Joined
·
251 Posts
Given the boxes on those cables I would not have been at all surprised if they were screwing with the signal enough to make an audible difference.


On the other hand I'm not at all surprised either that no difference could be heard but I'd feel more confident about that if an ABX switcher had been used.
 

·
Registered
Joined
·
5,211 Posts
I made comments quite a while back on some other thread about this. When i talked to a friend of mine about this type of testing, he said that for unknown sounds, when comparing for subtle difference (i am REALLY paraphrasing here as this was a 2 hour convo) human auditory memory is about 4 seconds. i am talking a straight passage to passage comparison. Now the more knowledge i have (if i am a professional musician, then i can take those "tones" and put them into the context of what note they are, if the pitch is correct... etc) the longer that gap can be. but in general terms, our ability to tell if they are the same level or slightly different or such things, fades pretty quick after 4 seconds.


Not saying this as an excuse, but as another piece of info for the whole testing methodology thing. This is the reason why an ABX box is preferred, instantaneous switch.
 

·
Banned
Joined
·
1,458 Posts
Well done...basically proved what I and many other friends of mine have known for a long long time...cables are pretty much cables...and it is indeed next to impossible to tell the differences between them when blindfolded. SO it all boils down to placebo effect I guess.


Sad to see so many folks spend a bunch of $$$ on cables thinking they are going to hear an "amazing difference" like the marketing guys who sell them like to spin it all.


Congrats to all who participated in helping dispell the cable myth...at least to some degree.
 

·
Registered
Joined
·
5,399 Posts
Well, I'm also surprised. However, I agree that everyone should try to remain adult about this. Also, we should wait to see what Mike has to say as well.
 

·
Registered
Joined
·
15,247 Posts
Would be fun to run the MIT's through the same gauntlet don't you think, Rutgar?
 

·
Registered
Joined
·
5,399 Posts

Quote:
Originally Posted by Chu Gai /forum/post/12252587


Would be fun to run the MIT's through the same gauntlet don't you think, Rutgar?

I'm not opposed to doing that. But at this point lets wait until Mike chimes in, and let's let the dust settle on this test before everyone start clamoring for another.
 

·
Banned
Joined
·
20,735 Posts
Discussion Starter #15

Quote:
Originally Posted by Gordon Shumway /forum/post/12252363


Well done...basically proved what I and many other friends of mine have known for a long long time...cables are pretty much cables...and it is indeed next to impossible to tell the differences between them when blindfolded. SO it all boils down to placebo effect I guess.

I think it is important to point out that this is not what the test proved affirmatively. While it's buried in my text, it is very important to be clear that this test proves only one thing: that with this particular testing methodology, it was not possible to distinguish any difference between these cables. It does not prove that there are no audible differences between cables or even between these two specific cables. Just that there weren't audible differences the way we tested. It is possible, though in my personal opinion rather unlikely, that a different methodology could yield different results. So for subjectivists who are certain that there are differences in cables, our test does not authoritatively disprove that opinion, however it does shed some doubt as to that position at least on the degree of difference that some subjectivists claim. Clearly, there was not any degree of difference large enough to be discerned with our test, which means that there could be a degree of difference below the perceptual threshold allowed by our test, that the degree of difference is fully below our audible threshold for all possible methodologies, or that there is no difference whatsoever. For objectivists who already were of the opinion that the cables couldn't have an audible effect, this test taken alone bolsters, but does not authoritatively vindicate that position.


It is far more difficult to prove that there isn't a difference. Given the statements by many subjectivists that the difference is easy to discern, it should be easier to prove that there is a difference. I think that all this test shows us is that, even assuming for the sake of argument that there are audible differences, that such differences are much harder to discern than many may claim. But this test taken alone does NOT prove that there are no audible differences in cables. Objectivists who arrive at that position do so based on the merits of other knowledge about both cables, and our hearing capabilities. You cannot arrive at that conclusion based only on this test, which was only capable of determining that under these specific methods and with these specific cables, there were no audible differences.


We just have to remain honest about both the clarity of the result and the limits of applicability of that result. This test alone is not close to universally definitive for these reasons.
 

·
Premium Member
Joined
·
2,247 Posts
Great write-up Chris and thanks to Mike L. taking one if you know what I mean.


The results do tend to lend creedence to the cables sound the same group but I'm not sure a scientist would draw any general sweeping tidy conclusions at this point at least not from this test alone. The results suggest that this Monster cable used can not audibly be differentiated from the Opus during blind-folded testing with delay of about one minute between tests. The next set of tests or research should now concentrate on auditory memory to determine how that factor could have influenced the outcome. Perhaps all this test proves is poor audio memory at one minute test intervals.


Another area of concern is the blind-folded portion of the listening, often when I implement system changes it's the differences in imaging that is most telling, sound changes constantly for me depending on many factors BUT imaging of a particular song stays relatively constant, predictable and repeatable. Ignoring or hobbling this important area of this hobby by blind-folding the participants and thereby losing image location references is akin to blindfolded testing for color differentiation. Perhaps imaging is a placebo effect at play, would be interesting to perform some tests on that aspect of audio.
 

·
Banned
Joined
·
20,735 Posts
Discussion Starter #17

Quote:
Originally Posted by Hughman /forum/post/12252872


Great write-up Chris and thanks to Mike L. taking one if you know what I mean.


The results do tend to lend creedence to the cables sound the same group but I'm not sure a scientist would draw any general sweeping tidy conclusions at this point at least not from this test alone. The results suggest that this Monster cable used can not audibly be differentiated from the Opus during blind-folded testing with delay of about one minute between tests. The next set of tests or research should now concentrate on auditory memory to determine how that factor could have influenced the outcome. Perhaps all this test proves is poor audio memory at one minute test intervals.

That is absolutely correct, and a fair characterization.


However, it did also prove that the difference percieved using a sighted test, with the same test interval, was largely in our heads. That is also a very interesting aspect of the result, as I also was certain that I was hearing a subtle difference between cables, and this is coming from someone who despite my explicit desire to be open, impartial, and non-opinionated, am naturally something of an objectivist. There are certain aspects of my expectations that I cannot simply toss aside, such as my expectation that speaker cables, even assuming an audible difference, would pale in significance to the impact of room, speakers, electronics. And yet I too was quite convinced that there was a subtle difference during the sighted trials. This is an interesting phenomenon. It was what prompted me to try my own hand at seeing how difficult it was to distinguish between the two cables, partly because I thought that given my impressions sighted, I could do it reliably blinded. Part of that motivation was that Mike got the second two wrong, and I thought that perhaps I could do it better, that surely given what I was hearing sighted, I could differentiate between the to. Well, we both failed equally!


Quote:
Another area of concern is the blind-folded portion of the listening, often when I implement system changes it's the differences in imaging that is most telling, sound changes constantly for me depending on many factors BUT imaging of a particular song stays relatively constant, predictable and repeatable. Ignoring or hobbling this important area of this hobby by blind-folding the participants and thereby losing image location references is akin to blindfolded testing for color differentiation. Perhaps imaging is a placebo effect at play, would be interesting to perform some tests on that aspect of audio.

That's possible. I personally am of the opinion that not seeing the speakers improves imaging, so I think that being blindfolded actually helps hear the system better. In addition, eliminating senses is a classic way of focusing and sharpening our other remaining senses. So if anything, I think all of us present at the test were under the assumption that being blindfolded would aid, rather than inhibit the test. Certainly if one wanted to avoid being blindfolded, you could rig up a screen at the speakers so you could not see behind them, if that was your preference.
 

·
Premium Member
Joined
·
2,247 Posts

Quote:
Originally Posted by ChrisWiggles /forum/post/12252969


That's possible. I personally am of the opinion that not seeing the speakers improves imaging, so I think that being blindfolded actually helps hear the system better. In addition, eliminating senses is a classic way of focusing and sharpening our other remaining senses. So if anything, I think all of us present at the test were under the assumption that being blindfolded would aid, rather than inhibit the test. Certainly if one wanted to avoid being blindfolded, you could rig up a screen at the speakers so you could not see behind them, if that was your preference.

I would think it would be very difficult to accurately remember imaging without a sighted reference point ie: that bell eminates exactly 1 foot out from that little spot on the wall. Much of the difficulty of proving or disproving any of this is out memory and the noisefloor this creates possibly obscuring changes. Using a reference point to place an image can pretty much remove this memory factor from the imaging portion of the test. But as you say, blind-folding may very well aid in the auditory portion of the test.


Terrific testing though and what an eye opener it must have been.
 

·
Registered
Joined
·
5,788 Posts

Quote:
Originally Posted by ChrisWiggles /forum/post/12252785


I think it is important to point out that this is not what the test proved affirmatively. While it's buried in my text, it is very important to be clear that this test proves only one thing: that with this particular testing methodology, it was not possible to distinguish any difference between these cables.

It didn't actually prove that either to a high degree of probability. You only did 8 trials! So 'impossibility' was hardly demonstrated.


But it does have two important facets:


1) Mike was pretty confident he heard differences; the stats belie that confidence. Chris W also 'confidently' heard differences sighted, that had no support when 'blinded'.


2) thus pointing up HOW VERY EASY IT IS for people to convince themselves, 'confidently' that differences exist...in sighted comparison. When in fact this confidence may not be warranted in the least. To me this is key -- the psychological fact is, people can and do tend to underestimate how easily their senses can be fooled.



Chris, I also wonder about this statement of yours

Quote:
objectivism also tells me things about vinyl which are quickly disproven in Mike's system.

What did 'objectivism; tell you about vinyl that was quickly disproven in Mike's system? Objectivism doesn't 'tell' us that LP's can't sound great...or even that they MUST sound subjectively worse than a CD of the same material.


As for anyone's objections about switching, etc...everyone needs to keep in mind that Mike did his OWN sighted tests, and also sighted trials during this test, where he was CONFIDENT he heard differences, under 'slow switch', nonABX conditions. *THAT* is the phenomenon that was tested; indeed, that is the phenomenon he kept reporting for his cables, the phenomenon that got him interested in Randi's challenge in the first place. Not some 'possible' difference that might be heard under more discriminative conditions. He was being tested on differences he already claimed to hear.
 

·
Banned
Joined
·
20,735 Posts
Discussion Starter #20

Quote:
Originally Posted by krabapple /forum/post/12253250


It didn't actually prove that either to a high degree of probability. You only did 8 trials! So 'impossibility' was hardly demonstrated.

It most certainly did. The result over those 8 trials, and my 5 was just statistically random guessing. We stopped at 8 because it was quite clear that it was just random. I believe our requirement for a positive result was either 10 out of 10, or 17 out of 20, we intended to do 20 ABs. Less than that and it could have been criticized as just good luck.


Quote:
What did 'objectivism; tell you about vinyl that was quickly disproven in Mike's system? Objectivism doesn't 'tell' us that LP's can't sound great.

Certainly. But there is a lot of data that tells us that there is less detail in LP compared to CD. True or not, what I heard on Mike's system, scratches, high noise floor and all, was simply incredible. It wasn't by any means scientific, I just forwarded that as evidence that I approached the trial with an open mind. I was perfectly open to hearing cable differences, and indeed during the sighted trials I was confident that I was. It would be very easy for me to enter the trial with a great deal of bias if I so chose, dead-set against hearing any differences, convinced it was simply impossible. I tried my very best to go in with a completely open mind, having formed no opinion on the matter. That is never attainable with anything, but objectivity also involves impartiality or its pursuit. I don't think subjectivists can simply dismiss my presence or my own interpretations as being pre-emptively biased, and the same goes for everyone else there. These kinds of things, as always, should always be learning experiences, rather than ideological pursuits. And I think that is part of being objectively fair and open minded as well. If Mike had reliably heard the differences blinded, I would be just as open to that possibility.
 
1 - 20 of 384 Posts
Top