AVS Forum banner

Establishing Differences By The +/- 10% Volume Method

43K views 549 replies 45 participants last post by  markrubin 
#1 ·
Amir has asked that a separate thread be started regarding his approach to having established audible differences. That's fair. Personally, I'd rather not hear about back in the day when CD's came out but more current tests that he's performed. Marketing aside, my understanding is that during their inception, issues existed that were discussed in technical publications.


So I'm interested in the approach, how it was that the test was blind, the DAC's involved, the number of trials with success rates, and possible interpretations as to the reason for the findings. I'm also curious whether the tests were confirmed by more scrupulous level matching using test tones and a VOM at the speaker terminals. If there are links that refer to independent measurements of the DAC's involved, that would be of interest. So, Amir, I turn the thread over to you.
 
#2 ·
Thanks for creating the thread. I must say though, I feel like I am on trial or something
. We are all here for a fun exchange. If what I present doesn't meet some high bar, so be it....


The comparison I performed was using a Mark Levinson No360S against the on-board DACs in five to six DVD-A and SACD players, all playing the same time sync'ed CD. In other words, I would listen to the analog output of the player while its digital output would feed the ML DAC. All front panel lights were turned off in addition to video circuits (yes, all of that made a difference in fidelity).


The two sources were fed to the dual inputs of a Stax "earspeaker" electrostatic headphone amp. If you are not familiar with Stax, you can read learn more about them here: http://www.stax.co.jp/Export/ExportProducts.html . I have three of their units and results are consistent across the board although the highest end unit does make the job a bit easier. Using headphones allowed me to completely eliminate the room and take advantage of the amazing transparency of these headphones to listen for the slightest differences. To latter point, I would often listen to material at levels well above what I would use for listening to music, allowing me to hear detail that would otherwise be lost.


I then picked material that made it easier to detect differences between DACs. I am not going to disclose what constitutes such content. Without such material, the job can range from difficult to impossible. One has to know what could be damaged by a DAC and then use music that has such content. To give you an example, when you compress music, it is the transients that suffer. So something like guitar music is much more revealing than say, violin as the latter is much more harmonic than the sharp impulses of a guitar. Voices play the same role. None of these are useful for testing DACs though so don’t use that as a hint to the question posed
. You can’t test the cornering of a car if you just drive it straight….


The comparison was then conducted without knowing which input is which, sitting in front of the headphone amp and toggling back and forth. When necessary, I would go back and re-listen. Once I found which one sounded worse, I would then repeat the exercise by randomizing the inputs and seeing if I could still identify which one was worse. My success rate was 100% in the second test (i.e. could always verify that the first result was not by chance). This testing was repeated a number of times comparing the different sources against each other and the ML.


I did not level match anything. However, once I found one source was worse than the other, I would then turn up the volume to counter any effect there. Indeed, doing so would close the gap some but it never changed the outcome. Note that the elevated level clearly made that source sound louder than the other. So the advantage was put on the losing side.


The results above were later objectively shown to be backed by some science in Stereophile magazine. In reviews of said players and Mark Levinson, it was shown that the former would only resolve to 14 or 15 bits of audio samples. Turning off the front panel pushed some up to 16 bits or so. The ML on the other hand, was tested to have equiv. of 19.5 bits. This is contrast to all the DACs being rated at "24 bits."


Now this testing is a few years old (probably circa 2001 to 2002). Maybe DACs have improved so much that the $20 part in the player is just as good as my then $8000 Mark Levinson DAC (which was hand tuned). If so, then I like to know who has tested the new ones and details of their methodology.


There you have it. Was it worth the wait?



BTW, I hope we use this thread for all objectivity challenges and not pollute any individual threads with them.
 
#3 ·
Thank you for responding, Amir and no you're not on trial. I'm unclear on a few things.

1. Am I to understand that you placed a CD into either the SACD or DVD-A and then ran an analog connection to the headphone amp followed by a digital connection from either the SACD or DVD-A to the Mark Levinson No360S followed by another analog connection to the headphone amp?


2. You indicate that your test probes were music, yes? Does that mean you ran original CD's or were those mp3's? If the latter what details can you provide on the means of compression.


3. Was the Mark Levinson No360S modified and if so were you able to test a unit that was unmodified?


4. If it was only you doing the connections and then placing a blanket or whatever over everything, how was it that it was blind?
 
#4 ·

Quote:
Originally Posted by Chu Gai /forum/post/16217334


1. Am I to understand that you placed a CD into either the SACD or DVD-A and then ran an analog connection to the headphone amp followed by a digital connection from either the SACD or DVD-A to the Mark Levinson No360S followed by another analog connection to the headphone amp?

Correct. That allowed me to have near zero delay in comparisons, something that is sadly missing from HDMI world today, making any DBL test of this kind invalid in my opinion.

Quote:
2. You indicate that your test probes were music, yes? Does that mean you ran original CD's or were those mp3's? If the latter what details can you provide on the means of compression.
What kind of question is that for heaven's sake. Of course these were music titles. And no, they were not MP3s. I gave the example of music compression as to have people understand that you have to use the right material for the problem you want to investigate. As with audio compression, there is material that is more revealing of DAC issues than others. And the reason for that, just like audio compression, is firmly planted in science and objective evaluation.

Quote:
3. Was the Mark Levinson No360S modified and if so were you able to test a unit that was unmodified?

Modified was the wrong term to use here. No360 came in two flavors. One that came from the assembly line and the other version (the "S" I think) that was hand calibrated at ML for better performance. I seem to recall this option cost me a few thousand dollars more.

Quote:
4. If it was only you doing the connections and then placing a blanket or whatever over everything, how was it that it was blind?

Pretty simple. I took advantage of old age and what it does to memory
. Seriously, I would grab the RCA cables and plug them into the back without paying attention to which input was which. After the test, I would then trace the cable to the source.


In the other thread, I made a point about being honest with yourself in these tests. And that is what is going on here. All of the equipment was free to me so I had no interest to defend one or the other. I would have been just as happy to see these $1000+ sources outperform the ML so that I didn't have to use that box for my testing all the time to rule out equipment differences.


Look at it this way. If I didn't care about a fair outcome, I wouldn't even bother to go through the lengths I did to test the equipment. Raise your hand if you have two copies of multiple identical CDs, DVD-As, and SACDs. Seems like my hand is the only one up!
I even have the same title in SACD and DVD-A (from Chesky).


Now let me ask you this. Have you participated in double blind tests? If so, what tests were they?
 
#5 ·
...I did not level match anything....


That's no good. One of the cardinal rules of A-B comparison is that levels have to be matched to remove that factor as being a bias. Limit of detectability is around 0.5 dB, so typically you want to level match better than that. Also need to check absolute signal polarity when comparing analog equipment to make sure that factor is not entering into the equation either.
 
#6 ·

Quote:
Originally Posted by fastl /forum/post/16218873


...I did not level match anything....


That's no good. One of the cardinal rules of A-B comparison is that levels have to be matched to remove that factor as being a bias. Limit of detectability is around 0.5 dB, so typically you want to level match better than that.

I do believe one of the "cardinal rules" of posting on a forum is reading one's entire argument before responding
. Did you see the part where I said that I compensated for that by increasing the volume higher for the lower performing source? If so, how is it again that with its level higher, it would still underperform?

Quote:
Also need to check absolute signal polarity when comparing analog equipment to make sure that factor is not entering into the equation either.

The testing was done across multiple brands/players. Are you suggesting that the ground and signal wires are often swapped out on RCA connectors? And on $1000+ players or $7000 DACs no less?


But let's go with your assumption. In what way do you think that polarity reversal affects the sound?


And while you are at it, can you please explain your personal experiences you have had with double blind tests?
 
#7 ·

Quote:
Originally Posted by amirm /forum/post/16219022




And while you are at it, can you please explain your personal experiences you have had with double blind tests?

Awaiting some replies here.. The close-to-religious zealotry exhibited by some about DBT, does seem at time from personal first hand experience.. Let's see...
 
#8 ·

Quote:
Originally Posted by FrantzM /forum/post/16220044


Awaiting some replies here.. The close-to-religious zealotry exhibited by some about DBT, does seem at time from personal first hand experience.. Let's see...

It is not "religious zealotry" to ask for evidence. Just the opposite, it's religious zealotry to expect people to believe things on faith without evidence. So Frantz
, please stop muddying the waters with the constant mis-characterizations of those who ask for evidence.


With regard to Amir's experience it was just that, an experience, impossible to quantify or refute. It's seems to me he made a reasonable effort within the confines of what he had to work with to eliminate bias.


As for my own blind tests (NOT double blind, which are more difficult to do) and experiences, I have reported them many times. I would compare a 30K Mark Levinson setup (35 Transport and 30 DAC if I recall the model numbers correctly) versus a $500 Denon player. The result was the same as every audiophile cable test that has ever been performed. People would wax poetic about the "HUGE" differences in sound stage, depth, rhythm and so on. Until they didn't know which one was being played. Without except at that moment their golden ears would suddenly become tin ears and they could not guess with any consistency whether they were listening to the 30K player or the $500 one.


As John Dunleavy and others have done, I would also test by telling people that they were listening to the $500 player when they were actually listening to the 30K player and visa versa to test for placebo effect. That always made for a very entertaining experience since without exception they would wax poetic about how much better the 30K player sounded (which of course was really the $500 one).


In a word, 90% of high-end audio is PURE bullsh*t.
 
#9 ·

Quote:
Originally Posted by QQQ /forum/post/16220273


In a word, 90% of high-end audio is PURE bullsh*t.

But, I might add, very expensive bullsh*t!!
 
#10 ·
Mark Levinson transports are not the most revealing. I have done several tests with Wadia vs. ML and the Wadia was more revealing everytime.


I do believe there is a lot of overbuilt overpriced B.S. because I have heard it. I also believe there are some superior products, because I have heard them as well. Wadia is one of those products.


JC
 
#11 ·

Quote:
Originally Posted by joeycalda /forum/post/16220916


Mark Levinson transports are not the most revealing...

This is another good example of audiophile claims versus real evidence. They can't even agree on what does and does not sound good, all the while arguing they can hear differences that they can't prove they can hear in tests. At the time it came out Stereophile claimed the ML DAC was the most revealing DAC that had ever been created and "set a new standard". They also used the tired hyperbole about it being the first DAC that was as good as analog (of course they kept using they cliche for another decade...).

Quote:
I do believe there is a lot of overbuilt overpriced B.S. because I have heard it. I also believe there are some superior products, because I have heard them as well. Wadia is one of those products.

I have heard it too. But I don't believe because I can't hear it when subjected to blind tests and also understand the scientific process, the placebo effect and the fallibility of auditory memory.
 
#12 ·

Quote:
Originally Posted by QQQ /forum/post/16220273


So Frantz
, please stop muddying the waters with the constant mis-characterizations of those who ask for evidence.

Q, I was put on spot to detail to the N'th degree to explain how I did my testing. I think it is only fair to hear that details of how people have performed their testing and how they arrived at the conclusion that using double-blind methodology there is no difference in such equipment.


I still have not heard the answer to the following:


1. How the DBL was conducted.


2. Who participated in it and their hearing capabilities.


3. What content was used.


4. What were they looking for.


Without the above information, and I don't mean this commented addressed to you, it seems that people are just repeating what they have read but not practiced. So far, I have not learned anything about the opposing position because no specific information is provided.
Quote:
With regard to Amir's experience it was just that, an experience, impossible to quantify or refute. It's seems to me he made a reasonable effort within the confines of what he had to work with to eliminate bias.

Thanks. So let's hear the details of the experiments on the other side to see if their efforts were more or less rigorous than mine. Only then do we see if folks had earn the privilege to stop every thread with a call for DBL test
.
Quote:
As for my own blind tests (NOT double blind, which are more difficult to do) and experiences, I have reported them many times. I would compare a 30K Mark Levinson setup (35 Transport and 30 DAC if I recall the model numbers correctly) versus a $500 Denon player. The result was the same as every audiophile cable test that has ever been performed. People would wax poetic about the "HUGE" differences in sound stage, depth, rhythm and so on. Until they didn't know which one was being played. Without except at that moment their golden ears would suddenly become tin ears and they could not guess with any consistency whether they were listening to the 30K player or the $500 one.

I bet you I can teach you to hear the difference in blind comparisons that are set up correctly. While some people are born with this ability, many people are not. As such, walking out confused from a blind test is normal for just about everyone especially if proper testing is not conducted to find the differences.


For formal audio testing, we usually put people through a three month learning routine. And we go through a special process before the tests start to make sure the person knows what they have to work with. For example, I have 3 minute songs where only a fraction of a second, yes, a fraction of a second, is revealing of compression flaws. Unless you can identify that fraction, then the rest of the test is useless. Most people just listen to the music instead of using it as a test tone to detect real differences in equipment.


Now, you can argue successfully that "HUGE" is the wrong way to describe the differences here. I think the difference can be big to someone, but can't be big for most people. Indeed, many people would believe there is no difference at all let alone big. In my case, the difference is small but significant in the way that enhances the experience. I can't explain why without giving away the answer to questions I have posed
.
Quote:
As John Dunleavy and others have done, I would also enjoy testing by telling people that they were listening to the $500 player when they were actually listening to the 30K player and visa versa to test for placebo effect.

This reminds me of a funny story. I was in Fry's about 8-10 years ago. I see a sign for a home theater demonstration and I walk in. I see a nice and comfortable theater so I sit down. A couple of other people show up and then a sales guy comes in and starts playing a movie with lots of special effects. There were a bunch of large speakers all around but I kept thinking, boy that doesn't sound good for this kind of setup. The audio was harsh and really jarring.


The clip finishes and the sales guy turns on the lights. He then goes and asks the first guy in front if he was blown away by the sound. The guy, who you could tell was not into audio and video said: "the voices were not clear. I couldn't understand what they were saying!" The salesman in shock, asked him three times and the guy like a stuck record kept saying "it wasn't not very clear." The sales guy then proceeds to remove the grills and show tiny Bose Accoustimass speakers. With his illusion shattered, he couldn't find a tap dance shoe that would get him out of the jam!


So the placebo effect does not always work as thought.


In addition, I can easily setup experiments that would nullify any difference. Go and ask average person on the street if 128 MP3 is as good as the CD and they say yes (I know this from very large scale testing we did at Microsoft). Does that make 128 MP3 the same as the CD? No. Heck even 384 Kbps MP3 doesn't sound the same as the CD. Come to think of it that would be a great test of your hearing ability. Can you readily tell 384 Kbps compressed music from the original? If not, don't bother going further.
Quote:
In a word, 90% of high-end audio is PURE bullsh*t.

I hate to say this but 90% of all audio is PURE bullsh*t. I used to repair audio equipment. You wouldn't believe how many times I opened typical consumer gear just to find the same parts as the other. Other times I would open a box and it would be two thirds empty yet the front panel and lights look like a much higher end gear.


Most CE companies these days take reference designs from IC manufacturers and simply repackage them. Yes, I am annoyed by non-sense description of what makes some high-end gear better according to the maker. But then again I am just as annoyed with Panasonic's ad in CEPro magazine claiming that their plasma has 18 bit resolution and 120% of HDTV resulting in better picture. Given that, do we stop considering differences between flat panels? No.


We can't control the marketing. But we can learn about these technologies. That is my goal with participating in this discussion. I think science and subjectivity do mix well. Knowing what Fletcher and Munson curves are, does help you understand what Jitter is bad even though that research was conducted 70 years ago before digital music even existed! At the end of the day it is a fine outcome if no one changes their stance, but walks away more educated. Hence the questions I put forth at the beginning of this thread
.
 
#14 ·

Quote:
Originally Posted by QQQ /forum/post/16220273


It is not "religious zealotry" to ask for evidence. Just the opposite, it's religious zealotry to expect people to believe things on faith without evidence. So Frantz
, please stop muddying the waters with the constant mis-characterizations of those who ask for evidence.


.

QQQ


I do not CONSTANTLY mis-characterizes anyone here. I am repeating a question here: How many have subjected THEMSELVES to DBT? ... So far not much answer on who has.. No personal experiences.. The evidence in that case is second hand as related by admittedly credible studies.. The questions stands and for all the power of DBT it is not absolute, far from it.. So again Have you subjected yourself to DBT, QQQ? Who else has and what were your conclusions?


I characterized the unfortunately too common smack-down of any subjective evaluation by the DBT question.. especially on the Audio side as "religious zealotry", yes.. MOST people and judging by so far the absence of replies to the DBT question do not do DBT.. Many believe in DBT without trying it.. It is then a belief, admittedly grounded in science but not really experienced... How would you describe such a behavior, then?
 
#15 ·

Quote:
Originally Posted by QQQ /forum/post/16220273


As for my own blind tests (NOT double blind, which are more difficult to do) and experiences, I have reported them many times. I would compare a 30K Mark Levinson setup (35 Transport and 30 DAC if I recall the model numbers correctly) versus a $500 Denon player. The result was the same as every audiophile cable test that has ever been performed. People would wax poetic about the "HUGE" differences in sound stage, depth, rhythm and so on. Until they didn't know which one was being played. Without except at that moment their golden ears would suddenly become tin ears and they could not guess with any consistency whether they were listening to the 30K player or the $500 one..



Funny and it have been done many times. I know of another fun test where two Linn LP12 lovers said that CD is nowhere near the quality of LP. So the guy that did the test did it like this. He played, openly, the CD and the Linn guys said it was cold, and sounded bad. Then he "played" the same song on the LP12, and all of a sudden, the sound was MUCH better.


But what they didn't know was that he had recorded some vinyl noise from a LP and mixed it with the CD! So they was still listening to the CD with added vinyl noise.
 
#16 ·

Quote:
Originally Posted by NIN74 /forum/post/16223356


Levelmatch within 0,1 dB is a must, otherwise it can be other things that real differences you hear.

Sigh. Really, how hard is it to read the whole argument, digest it and then comment on it? Who said level match doesn't matter? Not only did I not ignore that factor, I did one better.


Let's skin the cat this way.


How much difference does it make to have the level be higher in evaluating DACs? 1%, 10%, 50%?
 
#17 ·
HI


One more thing. I have no doubt that DBT is a great tool... I understand quite well that our perceptions (and their descriptions) are far from reliable. I do however understand the power of education. People, as Amirm has noted, can be taught to perceive difference.. A simple example is that of the musician tuning his instruments.. For most people the instrument sounds just fine, for the musicians around, not so... I have not read any studies on the matter but I would lke to see the results of a long DBT test one in which the subjects would be asked to describe their perceptions over a long period of time. The same gear would be used.. I have the suspicion that the results could be quite different.. Could anyone point toward an Audio DBT study conducted over a relatively long period of time, say, a week?
 
#18 ·

Quote:
Originally Posted by NIN74 /forum/post/16223416


Funny and it have been done many times. I know of another fun test where two Linn LP12 lovers said that CD is nowhere near the quality of LP. So the guy that did the test did it like this. He played, openly, the CD and the Linn guys said it was cold, and sounded bad. Then he "played" the same song on the LP12, and all of a sudden, the sound was MUCH better.


But what they didn't know was that he had recorded some vinyl noise from a LP and mixed it with the CD! So they was still listening to the CD with added vinyl noise.

That is good and well but we can't discuss that test since the people who ran it and took the test are not here. Maybe the test was something completely different since you were not there and there is an explanation for the results.


On AVS, we had a double-blind test about a year ago in High Def section. It was posted and people responded with the results. I was asked to participate also. I turn in and my results and when the outcome was posted, I was apparently in the wrong. I had said that two clips were identical but it was revealed that one was a compressed clip.


But that wasn't the end of the story. I did a binary compare of the files and wouldn't you know, it showed them to be the same bit for bit! The creator of the test went back and found out that he had made a mistake on which clip he had put up here. If there is interest, I can go dig up the thread. But the key was that it took a lot of trying to convince the poster (Steve) that the test was wrong. He was convinced that we were.


Net is that unless we have the people who have conducted the test, we can't examine whether what you state is really what went on or whether mistakes like the above were made. This is why I want to know what *you* have tested. That we can examine as everyone just did with me.
 
#20 ·

Quote:
Originally Posted by sierraalphahotel /forum/post/16224332


I once did a blind test to see if I could tell the difference between butter and "I Can't Believe It's Not Butter!" but the results were inconclusive.


Sean

Excellent. Now we are getting some place. By any chance, was the polarity correct? I mean, you didn't have the stick of butter backward, did you?

 
#21 ·

Quote:
Originally Posted by FrantzM /forum/post/16223370


QQQ


I do not CONSTANTLY mis-characterizes anyone here. I am repeating a question here: How many have subjected THEMSELVES to DBT? ... So far not much answer on who has.. No personal experiences.. The evidence in that case is second hand as related by admittedly credible studies.. The questions stands and for all the power of DBT it is not absolute, far from it.. So again Have you subjected yourself to DBT, QQQ? Who else has and what were your conclusions?


I characterized the unfortunately too common smack-down of any subjective evaluation by the DBT question.. especially on the Audio side as "religious zealotry", yes.. MOST people and judging by so far the absence of replies to the DBT question do not do DBT.. Many believe in DBT without trying it.. It is then a belief, admittedly grounded in science but not really experienced... How would you describe such a behavior, then?

As enlightened
? Seriously though Frantz, it is certainly fair enough to ask how many have participated in double blind tests for idle curiosities sake, but otherwise it's not particularly relevant to the debate, assuming the point of the debate is to present evidence for:


A. The proper scientific method of determining if audible differences exist.

B. The situations where there is evidence that audible differences exist.


B is quite broad so of course it would have to be defined on a case by case basis. i.e. "what situation are we talking about?" I will address that more when I respond to Amir's post.


Now to answer your question, I have participated in blind tests but not double blind tests. Double blind tests can take quite a bit of effort to carry out. Now if your response is than "aha, gotcha" I'm not sure your point. If you would like to tell me that the earth is flat I will respond that it is round. I will tell you that there is no evidence that the earth is flat but abundant evidence that the earth is round. Will you then challenge me and ask me "have you ever been in a spaceship and actually seen the earth"? "Have you ever traveled the circumference of the earth"? Need I do those things to be on very strong scientific footing when I say the earth is round?


The issue is not whether one has or has not conducted double blind tests, the issues is whether:


A. One believes that double blind test are a good way to prove claims of audible differences between components.


B. Whether one wants evidence before choosing to believe in something. Notice, that if I do not believe in something that does not mean I do not believe in it, it simply means that I withhold belief.


I will make one final comment on A before "believers" mis-characterize it. No, of course I am not saying that everything must be backed up with evidence or double blind tests. If you tell me you heard a difference between speaker A and speaker B I will believe you because there is already an overwhelming body of evidence that there are audible differences between speakers. However, if you tell me something that is doubtful, that for instance you can hear the difference between two power cables, than I will want evidenced to support such a claim.


Now on to the more specific subject of one of the things I think (?
) we are discussing here. I personally am not saying that all CD players sound identical, of course not. It's possible some might sound different if they are poorly designed etc. However I suspect that if we took 10 well designed CD players, with good DAC's, that double blind tests would show that the differences between them are inaudible. The reason is simple - CD players have evolved to the point that whatever differences there are between them have passed the threshold of human audibility.


Now here is the part that the "believers" never get. I'll be willing to change my opinion in a New York second if I see any convincing scientific tests carried out by the likes of Floyd OToole or any other number of distinguished non-crackpot scientists. "Believers" on the other hand do not change their opinions no matter how much evidence exists because there beliefs are a matter of faith, based on flawed personal experiences and unverifiable claims.
 
#23 ·

Quote:
Originally Posted by amirm /forum/post/16222618


So let’s hear the details of the experiments on the other side to see if their efforts were more or less rigorous than mine. Only then do we see if folks had earn the privilege to stop every thread with a call for DBL test
.

See my response to Frantz. A person need not have performed tests themselves to rightly use double blind tests as the standard of determining the audibility of differences between components.


Regarding your tests as I stated it sounds like you did a good job of trying to eliminate bias. But surely you also acknowledge that further "real" research would have to be done to quantify your results? Surely you acknowledge that your very limited test (scientifically speaking) no more proves that current day well designed CD players are audibly different than my very limited tests proved there are not.


In fact, I think this is a perfect example of whether one believes in the scientific process (which I assume you do?). You conducted some tests that suggested audible differences. I conducted some tests that suggested otherwise. So where does that leave us? Well for me, based not only on our individual tests but on the body of evidence I have seen it leaves my leaning strongly towards the conclusion that differences between well designed CD players are probably inaudible, while withholding belief on a definitive opinion. Strangely, withholding belief is a strange concept for some and someone at RC the other day told me it was "lame". For some believers it seems that perhaps withholding belief is the same as not believing but they are completely different. And as I stated to Frantz, I would be willing to change my opinion if the face of new/more evidence.


Now to further elaborate my opinion on the audibility of differences between properly designed cables is much more definitive. I think the evidence is pretty overwhelming there. On CD players I'll leave the door open for now
. Heck, I'd love nothing more than to believe such differences exist. It would allow me to in good conscience advise my Clients to spend 20K on a high-end CD player.

Quote:
I bet you I can teach you to hear the difference in blind comparisons that are set up correctly.

Differences between what? Absent that qualifier it's a not a meaningful statement.


Quote:
While some people are born with this ability, many people are not. As such, walking out confused from a blind test is normal for just about everyone especially if proper testing is not conducted to find the differences.

Agree 100%, no intelligent skeptic would claim otherwise. Heck, I'd be thrilled if a single believer in cable differences could hear a single difference with a test tone between properly designed/built cables.!

Quote:
For formal audio testing, we usually put people through a three month learning routine. And we go through a special process before the tests start to make sure the person knows what they have to work with. For example, I have 3 minute songs where only a fraction of a second, yes, a fraction of a second, is revealing of compression flaws. Unless you can identify that fraction, then the rest of the test is useless. Most people just listen to the music instead of using it as a test tone to detect real differences in equipment.

Again, we are in no disagreement. Nor do I think any knowledgeable person would argue that there cannot be audible different between different compression algorithms. The question as always is where/when some differences become inaudible.

Quote:
This reminds me of a funny story. I was in Fry's about 8-10 years ago. I see a sign for a home theater demonstration and I walk in. I see a nice and comfortable theater so I sit down. A couple of other people show up and then a sales guy comes in and starts playing a movie with lots of special effects. There were a bunch of large speakers all around but I kept thinking, boy that doesn't sound good for this kind of setup. The audio was harsh and really jarring.


The clip finishes and the sales guy turns on the lights. He then goes and asks the first guy in front if he was blown away by the sound. The guy, who you could tell was not into audio and video said: "the voices were not clear. I couldn't understand what they were saying!" The salesman in shock, asked him three times and the guy like a stuck record kept saying "it wasn’t not very clear." The sales guy then proceeds to remove the grills and show tiny Bose Accoustimass speakers. With his illusion shattered, he couldn’t find a tap dance shoe that would get him out of the jam!


So the placebo effect does not always work as thought.

Agreed, of course not.

Quote:
In addition, I can easily setup experiments that would nullify any difference. Go and ask average person on the street if 128 MP3 is as good as the CD and they say yes (I know this from very large scale testing we did at Microsoft). Does that make 128 MP3 the same as the CD? No.

We are again in agreement but again I do not think any knowledgeable person wold argue otherwise.
 
#24 ·

Quote:
Originally Posted by KeithR /forum/post/16226537


QQQQ: have you done a proper DBT on video equipment?


its amazing all the objectionists come out on the audio side, but then can tell if 10k projectors really are that much different.

What's amazing to me is how believers endlessly engage in logical fallacies and red herrings. I believe in the value of the scientific process regardless of what claim it applies to. Audio, video, medicine, or that an herb cures cancer.


No one says that we need to demand evidence for everything. As this outstanding video ( http://www.youtube.com/watch?v=T69TOuqaqXI ) points out, I don't need to demand evidence to believe that you really went to work today. So it's up to me if I want to do a double blind test when buying video gear. That does not in any way negate the value of double blind tests! And yes, I have questioned the value of some of the claims people make on the video side many, many times in fact.


So to answer your question I already said I have not participated in double blind tests. Yes I have participated in blind tests with video, and yes just like with audio, if there are visible differences double blind tests could be used to definitely prove them. Video tests are in in fact much easier because it's possible to see two images side by side so you can eliminate the need to rely on memory (anyone want to tell me what the equivalent of the term auditory memory is? "visible memory"?
).


p.s. It's QQQ, not QQQQ
 
#25 ·
OT


QQQ


fair and honest reply. There is no dout that DBt is a very powerful method to eliminate the ever present human biases. I , however, do think that alterations to the protocol should lead to better results. I believe that our perception can be refined by education, training if you will. Optical illusions forever become visible once learned.. Same with something as basic as drawing in perspective.. It takes learning to perceive it, to interpret it if you will. Most DBT , at the least in the Audio domain rely on short comparisons between the gears. I have posted it somewhere else, it bears to repat here so much this forum is being impoverished by an exagerated refutation of any and all subjective evaluations: Can anyone points me to a medium to long term audio DBT tests? This would account for the traing effect I am talking about. Differences that may not appear in a short burst may, in longer listening sessions. I was trained quite recently to hear the differences between strings on a guitar and have become fairly good at it.. I don't play the guitar but am , now able to distinguish quite clearly the differences, most people can't at first but can be trained to. I call this education.

In video this kind of DBT evidence does not seem to be required here in this forum, I can bet that most people would not pass a DBT between the best projectors if they are all adjusted for same brightness. Some people will and most likely would have been trained to see more than the obvious.. The saturation of the Red ands in Peter case the ever present green monster once it is a SIM Projector
...


Now I am by no means defending the High End Audio industry: Looking at it with an absence of passion it has become quite difficult for me to take several products seriously, amongst these cables , especially the high priced ones of which there exist more than an handful over $30,000 for an 8 foot pair.. When one take a pause it is not very difficult to find their price structure insane and ultimately detrimental. I continue to believe that we are nowhere near perfection on several fronts. I do think that we are very close to the asymptote in electronics but very far from it in speakers and media.


Last but not least if a thread is about subjective impression isn't it basic etiquette to leave it alone and not hijack it with request of DBT EVIDENCE? . If I ask anyone a recommendation on a good restaurant in San Francisco, it will be difficult to ask for evidence since the subjectivity of the reply was implied in the query. Same with several threads here.. What do you think of Mark Levinson amplifiers is not a requirement to produce DBT studies that has shown Mark Levinson to be the same as the 125 Akai receiver at Walmart.. I frankly doubt it.. One can always interject that in his/her opinion such amp is not worth the money, that's fine but an incessant call to DBT to "prove" is simply impoverishing this once vibrant forum... YMMV
 
#26 ·
...Did you see the part where I said that I compensated for that by increasing the volume higher for the lower performing source?...


Yes I did. That's not valid. You have to equalize A-B levels with instrumentation to conduct a valid comparison. BTW, I've never heard of the plus-minus 10 percent volume method.
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top