[continued from http://www.avsforum.com/avs-vb/showt...18365&page=59]
Morning!
I got back a little late last night and didn't do a writeup then, I figure Mike will post his impressions at some point too.
First of all, kudos to Mike for being the always-gracious host, and also for his fairness and open mindedness for this trial. It is fair to recognize that he only had something to lose, and nothing really to be gained except perhaps an objective vindication of his position.
And to cut to the chase, Mike could not identify the Monster from the Opus MM with any accuracy (nor the reverse, which also would have been a positive result if he had been consistently wrong) using our testing methodology. We stopped the test a little less than halfway through, I think we got through 8 A/Bs before we gave up.
There were 4 of us present at the test. Mike had performed sighted listening tests on his own, and before I arrived he performed sighted listening tests while wearing a blindfold to acclimate himself to the testing method. His blindfold was a pair of duct-taped safety glasses. He could not see anything except a tiny bit directly below him. Theoretically, he could have strained his neck and been able to see the floor by the speakers where the wire was, but this was not allowed. After all, cheating would only serve to cheat himself. While switching wires, Mike wore earmuffs to minimize any possibility of identifying wires by the noises of switching the wires.
The plan was to do twenty A/B trials, which were chosen by flipping a coin. This was done before I arrived. After I arrived, we decided to check the level-match between wires. We did not have a method to level-match the wire if they were non-matched, however we also felt that the wire should match. Because the Opus uses some kind of network box, it conceivably would be possible that the signal were attenuated or otherwise altered and change the level, making it possible to identify the wire this way, rather than by sound 'quality' per se. The levels matched with a sine wave tone within 1/100th of a volt, the difference between the two sides was several hundredths of a volt with both wires. The difference between the L and R channels was greater than the possible difference between wires, which anyway was insignificant. We decided to check this because while we had no way of matching levels precisely, if there were a level difference we would have added a step to our methodology by turning down the volume all the way and then allowing mike to adjust the volume up, which would eliminate the ability to judge any difference in volume. We didn't need to do this, so Mike could listen without having to change the volume all the time, however he was free to change volume if he wanted.
There was, being generous, about one minute of a wait between changing cables. This was the case for both the sighted preparatory tests, and the blind tests. His amp basically goes straight through, so even with it turned off, it is feeding the speakers for a while. We would wait a little bit after turning off the amp before disconnecting the speaker wire and switching wire. We took care to be quiet while putting the wire down so as not to make any noise which could distinguish the wire. We changed the wire each and every time, regardless of whether switching to the other cable. At each time Mike would wear the earmuffs, even if he was going back to a known test, again to eliminate any possibility of identifying the wire by the noise of changing the wires.
We did at one point stop and go play pool for a few minutes and then come back to the test, doing a sighted (though still blindfolded) test to re-familiarize Mike with the cables.
So our results with Mike as our listener were clear: for this particular methodology, Mike could not accurately identify a difference in the cables. Again, a backwards result of Mike wrongly identifying the cables reliably also would have been a positive result.
I went into this with a fairly open mind. I did not have a previous opinion on the difference in speaker wire. However, I am fairly objective, and I did have two main expectations. First was that speaker wire, even if it did have an audible difference, would be very insignificant in the scope of the overall system. If there was going to be a difference, I was expecting it to be extremly subtle, and small. In other words, being objective about speaker wire tells me that it is fairly straightforward if you have sufficient gauge that the wire should basically get everything to the speaker with a precision that exceeds our hearing ability. Second, if there was going to be a difference, the suggested possibility of the network box in the Opus rolling off the highs may have caused an expectation of slightly less brightness/detail.
I personally went through the first few trials sighted, helping to change the wires. Going into the test I had no set opinion beside the expectations I just described. And while I am quite objective, objectivism also tells me things about vinyl which are quickly disproven in Mike's system. So I am open to these kinds of subjective things as well. As I went through 3 sighted trials, I began to form an opinion which basically told me that there WAS a difference in cables. Subtle, but I can say with confidence that it was present. I thought the Monster cable was a bit harsher, with a little bit less separation between instruments. This may have been a bias from my expectation that the Opus network box could have been attenuating the high frequencies, but I swore it was there. It is interesting to me that while I formed this opinion not having heard Mike or anyone else describe what they thought the difference was, after we stopped the whole test, Mike described what he previously thought the sighted difference sounded like (what he was listening for in the test) was practically identical to what I thought I was hearing sighted.
Then, for my own curiousity, I did the remaining 5 A/Bs before we stopped the test at 8 A/Bs blinded. I didn't wear ear muffs, and I just closed my eyes. I could have cheated if I wanted, but I did not (I guess you just have to take my word). I also, to check the rigor of our methdology, tried to actively listen to the cables being switched and see if I could identify either, to see if there was a way of identifying which was which that might have influenced mike even with earmuffs. There was no way to identify wire this way. I had looked at our test sheet, but I made no effort to try to memorize the sequence or have any idea what I would be listening to. In my 5 trials, I was certain on 3, and relatively uncertain on 2. I only matched 3 our ot 5, which is basically just blind luck, and I only got 2 of the 3 I was certain about correct. In other words, as a second, just self-interested and self-blinded tester, I could not with any reliability identify a difference in the cables.
The results for Mike and also the non rigorous one for myself certainly undid the opinion I was forming during my sighted listening. To characterize the position I've come to, I think it's fair to say that the objectivist arguments have a great deal of merit. It is fair to say that the onus is on subjectivists to discern speaker cable differences reliably. However it is ALSO fair to say that it is very difficult to prove a negative, especially for people with a subjectivist bent. Objectivists must also be honest about what our test results tell us: that with this methodology, and with this methodology ONLY, we could not reliably discern any differences in the cables. This does not prove that there are no differences with the cable, only that there were no differences discerned with this method.
In my personaly opinion, doing a long line of tests like this is generally unpleasant to do, you get tired of listening to the same stuff over and over, and after a little while it all starts to sound the same. Also, Mike only went back to a known A/B once during the test. I still would be interested in a similar test but spread out in time, doing maybe only one or two A/Bs at a time. This keeps fatigue down, and things don't all start to sound the same (again, just for the sake of argument assuming that there is an audible difference).
However, this test does tell us that even assuming for the sake of argument that there is a difference, that such a difference is extremly small, since it could not be percieved reliably with this method. After we finished the test, for curiosity we cut the ends of 16 gauge extension cords, and connected them to the system. We did this sighted. And the system sounded absolutely fantastic. The result I came away with through the whole experience is that people should generally not fret about speaker wire. Using a speaker wire of sufficient gauge for the task gets the job done. Even the unreliable sighted 'impressions' I had of the difference between Monster an Opus, which can not in any way be fairly attributable to actual difference, was extremely subtle. And the 16 gauge sounded basically exactly the same too.
After the test Mike was very gracious, and very fair and honest in accepting the result. I think he is probably curious to do some more testing in a different A/B format, perhaps along the lines I suggested. Or perhaps an ABX type test. It was curious that after the test, he characterized his certainty that during the test, it seemed fairly clear which was which. However, he was not identifying the cables with any consistency (rightly or wrongly) during the test. He did stop though and we left to go play pool to get away from the testing, so it seemed to me as an observer anyway that he was beginning to have difficulty, at least from his perspective, in identifying which was which. However, he was before that time not being at all consistent.
So I think that's basically everyting that either side would want to know. And I'm sure there's plenty of ammunition for both. I can say now that I am of the opinion that speaker wire basically makes no significant difference in sound quality, given sufficient gauge. Even the 16gauge extension cord was fantastic. I am still open to the possibility of audible differences with speaker cable, however such differences must be quite small and very subtle.
After the test, Mike played a handful of records, and I played some CDs as well. And what I came away with perhaps more than that is that Mike has probably the best overall system I've ever heard especially for BIG music. Big orchestral and rock sounded simply unbelievable. Santana, the Gates of Kiev, and Zeppelin among others on vinyl were just absolutely incredible. A completely new experience. I was trying to be extremely picky, and while I've heard a system here and there do small things differently in ways that in my opinion were slightly better, stepping back and considering the whole picture: it's simply an absolutely incredible system. The best I've ever heard big complex music.
So thanks again to Mike for being open and fair through the whole test and setting it all up and putting in the time and effort even though he really had nothing to gain. And for the pizza!
So flame on. If I were Mike, I frankly wouldn't care that much anyway, cable difference or not, it's one hell of a music system.
Morning!
I got back a little late last night and didn't do a writeup then, I figure Mike will post his impressions at some point too.
First of all, kudos to Mike for being the always-gracious host, and also for his fairness and open mindedness for this trial. It is fair to recognize that he only had something to lose, and nothing really to be gained except perhaps an objective vindication of his position.
And to cut to the chase, Mike could not identify the Monster from the Opus MM with any accuracy (nor the reverse, which also would have been a positive result if he had been consistently wrong) using our testing methodology. We stopped the test a little less than halfway through, I think we got through 8 A/Bs before we gave up.
There were 4 of us present at the test. Mike had performed sighted listening tests on his own, and before I arrived he performed sighted listening tests while wearing a blindfold to acclimate himself to the testing method. His blindfold was a pair of duct-taped safety glasses. He could not see anything except a tiny bit directly below him. Theoretically, he could have strained his neck and been able to see the floor by the speakers where the wire was, but this was not allowed. After all, cheating would only serve to cheat himself. While switching wires, Mike wore earmuffs to minimize any possibility of identifying wires by the noises of switching the wires.
The plan was to do twenty A/B trials, which were chosen by flipping a coin. This was done before I arrived. After I arrived, we decided to check the level-match between wires. We did not have a method to level-match the wire if they were non-matched, however we also felt that the wire should match. Because the Opus uses some kind of network box, it conceivably would be possible that the signal were attenuated or otherwise altered and change the level, making it possible to identify the wire this way, rather than by sound 'quality' per se. The levels matched with a sine wave tone within 1/100th of a volt, the difference between the two sides was several hundredths of a volt with both wires. The difference between the L and R channels was greater than the possible difference between wires, which anyway was insignificant. We decided to check this because while we had no way of matching levels precisely, if there were a level difference we would have added a step to our methodology by turning down the volume all the way and then allowing mike to adjust the volume up, which would eliminate the ability to judge any difference in volume. We didn't need to do this, so Mike could listen without having to change the volume all the time, however he was free to change volume if he wanted.
There was, being generous, about one minute of a wait between changing cables. This was the case for both the sighted preparatory tests, and the blind tests. His amp basically goes straight through, so even with it turned off, it is feeding the speakers for a while. We would wait a little bit after turning off the amp before disconnecting the speaker wire and switching wire. We took care to be quiet while putting the wire down so as not to make any noise which could distinguish the wire. We changed the wire each and every time, regardless of whether switching to the other cable. At each time Mike would wear the earmuffs, even if he was going back to a known test, again to eliminate any possibility of identifying the wire by the noise of changing the wires.
We did at one point stop and go play pool for a few minutes and then come back to the test, doing a sighted (though still blindfolded) test to re-familiarize Mike with the cables.
So our results with Mike as our listener were clear: for this particular methodology, Mike could not accurately identify a difference in the cables. Again, a backwards result of Mike wrongly identifying the cables reliably also would have been a positive result.
I went into this with a fairly open mind. I did not have a previous opinion on the difference in speaker wire. However, I am fairly objective, and I did have two main expectations. First was that speaker wire, even if it did have an audible difference, would be very insignificant in the scope of the overall system. If there was going to be a difference, I was expecting it to be extremly subtle, and small. In other words, being objective about speaker wire tells me that it is fairly straightforward if you have sufficient gauge that the wire should basically get everything to the speaker with a precision that exceeds our hearing ability. Second, if there was going to be a difference, the suggested possibility of the network box in the Opus rolling off the highs may have caused an expectation of slightly less brightness/detail.
I personally went through the first few trials sighted, helping to change the wires. Going into the test I had no set opinion beside the expectations I just described. And while I am quite objective, objectivism also tells me things about vinyl which are quickly disproven in Mike's system. So I am open to these kinds of subjective things as well. As I went through 3 sighted trials, I began to form an opinion which basically told me that there WAS a difference in cables. Subtle, but I can say with confidence that it was present. I thought the Monster cable was a bit harsher, with a little bit less separation between instruments. This may have been a bias from my expectation that the Opus network box could have been attenuating the high frequencies, but I swore it was there. It is interesting to me that while I formed this opinion not having heard Mike or anyone else describe what they thought the difference was, after we stopped the whole test, Mike described what he previously thought the sighted difference sounded like (what he was listening for in the test) was practically identical to what I thought I was hearing sighted.
Then, for my own curiousity, I did the remaining 5 A/Bs before we stopped the test at 8 A/Bs blinded. I didn't wear ear muffs, and I just closed my eyes. I could have cheated if I wanted, but I did not (I guess you just have to take my word). I also, to check the rigor of our methdology, tried to actively listen to the cables being switched and see if I could identify either, to see if there was a way of identifying which was which that might have influenced mike even with earmuffs. There was no way to identify wire this way. I had looked at our test sheet, but I made no effort to try to memorize the sequence or have any idea what I would be listening to. In my 5 trials, I was certain on 3, and relatively uncertain on 2. I only matched 3 our ot 5, which is basically just blind luck, and I only got 2 of the 3 I was certain about correct. In other words, as a second, just self-interested and self-blinded tester, I could not with any reliability identify a difference in the cables.
The results for Mike and also the non rigorous one for myself certainly undid the opinion I was forming during my sighted listening. To characterize the position I've come to, I think it's fair to say that the objectivist arguments have a great deal of merit. It is fair to say that the onus is on subjectivists to discern speaker cable differences reliably. However it is ALSO fair to say that it is very difficult to prove a negative, especially for people with a subjectivist bent. Objectivists must also be honest about what our test results tell us: that with this methodology, and with this methodology ONLY, we could not reliably discern any differences in the cables. This does not prove that there are no differences with the cable, only that there were no differences discerned with this method.
In my personaly opinion, doing a long line of tests like this is generally unpleasant to do, you get tired of listening to the same stuff over and over, and after a little while it all starts to sound the same. Also, Mike only went back to a known A/B once during the test. I still would be interested in a similar test but spread out in time, doing maybe only one or two A/Bs at a time. This keeps fatigue down, and things don't all start to sound the same (again, just for the sake of argument assuming that there is an audible difference).
However, this test does tell us that even assuming for the sake of argument that there is a difference, that such a difference is extremly small, since it could not be percieved reliably with this method. After we finished the test, for curiosity we cut the ends of 16 gauge extension cords, and connected them to the system. We did this sighted. And the system sounded absolutely fantastic. The result I came away with through the whole experience is that people should generally not fret about speaker wire. Using a speaker wire of sufficient gauge for the task gets the job done. Even the unreliable sighted 'impressions' I had of the difference between Monster an Opus, which can not in any way be fairly attributable to actual difference, was extremely subtle. And the 16 gauge sounded basically exactly the same too.
After the test Mike was very gracious, and very fair and honest in accepting the result. I think he is probably curious to do some more testing in a different A/B format, perhaps along the lines I suggested. Or perhaps an ABX type test. It was curious that after the test, he characterized his certainty that during the test, it seemed fairly clear which was which. However, he was not identifying the cables with any consistency (rightly or wrongly) during the test. He did stop though and we left to go play pool to get away from the testing, so it seemed to me as an observer anyway that he was beginning to have difficulty, at least from his perspective, in identifying which was which. However, he was before that time not being at all consistent.
So I think that's basically everyting that either side would want to know. And I'm sure there's plenty of ammunition for both. I can say now that I am of the opinion that speaker wire basically makes no significant difference in sound quality, given sufficient gauge. Even the 16gauge extension cord was fantastic. I am still open to the possibility of audible differences with speaker cable, however such differences must be quite small and very subtle.
After the test, Mike played a handful of records, and I played some CDs as well. And what I came away with perhaps more than that is that Mike has probably the best overall system I've ever heard especially for BIG music. Big orchestral and rock sounded simply unbelievable. Santana, the Gates of Kiev, and Zeppelin among others on vinyl were just absolutely incredible. A completely new experience. I was trying to be extremely picky, and while I've heard a system here and there do small things differently in ways that in my opinion were slightly better, stepping back and considering the whole picture: it's simply an absolutely incredible system. The best I've ever heard big complex music.
So thanks again to Mike for being open and fair through the whole test and setting it all up and putting in the time and effort even though he really had nothing to gain. And for the pizza!
So flame on. If I were Mike, I frankly wouldn't care that much anyway, cable difference or not, it's one hell of a music system.