Originally Posted by Floyd Toole
Craig, Craig, have you not been paying attention? This topic has been covered more than once in these forums - and, need I say, in both of my books. Here is one directly relevant paper:
Olive, S.E. (2003). “Difference in Performance and Preference of Trained versus Untrained Listeners in Loudspeaker Tests: A Case Study”, J. Audio Eng. Soc., 51, pp. 806-825.
This is just one example of a large population sample (ending up with over 300 listeners) from many very different backgrounds showing very persuasive agreement in what they prefer. Olive and colleagues have since expanded these evaluations in the context of headphones using large populations of listeners from different backgrounds and different countries/cultures. Again, the result is monotonously the same. The preferred headphones had measured in-ear frequency responses that closely resemble the sounds in the ears from highly rated loudspeakers in typically reflective listening rooms. Surprise, surprise. Numerous AES papers exist on the topic. Here are a few:
Olive, S.E, and Welti, T. (2009). “The Relationship between Perception and Measurement of Headphone Sound Quality”, Audio Eng. Soc., 133rd Convention, preprint 8744.
Olive, S.E, Welti, T. and McMullin, E. (2013). “Listener Preference for In-Room Loudspeaker and Headphone Target Responses”, 135th Convention, Audio Eng. Soc., Paper 8994.
Olive, S.E., Welti, T. and McMullin, E. (2013b). “Listener Preference for Different Headphone Target Response Curves”, Audio Eng. Soc., 134th Convention, Preprint 8867.
Olive, S.E., Welti, T. and McMullin, E. (2014). “The Influence of Listeners’ Experience, Age and Culture on Headphone Sound Quality Preferences”, 137th Convention, Audio Eng. Soc., Paper 9177.
As has been pointed out, the listener training is to teach listeners how to identify resonances and to describe what they hear in terms that are useful to design engineers who will strive to attenuate any audible problems. Untrained listeners differ from trained listeners in three ways: (1) they make mistakes, leading to greater standard deviations in their judgments and (2) they are inclined to give higher scores, being less critical than experienced listeners, and However, the average sound quality ratings of all groups of listeners end up indicating a preference for the same loudspeakers - those with neutral timbre.
As has been stated numerous times, program material is not consistent and the circle of confusion exists in movies and music, necessitating occasional intervention with tone controls. However, starting with a reproducing system free from audible resonances - i.e. neutral -is a significant advantage.
In any sighted listening evaluation all bets are off.
Is this an accurate analysis?
"From what I recall neutral curve is preferred one however depending on listeners education and training there will be larger deviation from neutral. Amateur listeners will deviate way more then trained and experienced listeners."
Dr. Toole, you said:
"Untrained listeners differ from trained listeners in three ways: (1) they make mistakes, leading to greater standard deviations in their judgments..."
How are "mistakes" defined. What criteria is used to determine that responses that differ from those by trained listeners are "mistakes"?
"...(2) they are inclined to give higher scores, being less critical than experienced listeners,..."
Are higher scores deemed to be "mistakes"?
"...(3) lacking training they have problems describing what they are hearing."
If they were simply given the vocabulary to describe what they hear, would that make their judgements more reliable? Would it change their judgements?
Dr. Toole, you have a vested interest in interpreting the data to show that trained listeners are more reliable than untrained listeners. However, what if there is another interpretation of these data that says that the variability of the untrained listeners is more representative of the general public than the more tightly grouped trained listeners? Is that possible?
To others in this thread who think that I am trying to discredit Dr. Toole's and Olive's science, or that I am disrespecting it or them, that is not the case at all. I do indeed highly respect and value what they've done. I think spinoramas are an extremely valuable way to evaluate speakers and they offer a tremendous amount to the knowledge base. I also agree completely with the premise that neutral speakers are the best choice. I just view some aspects of the science with a more skeptical eye, and I am really trying to understand it better. I apologize if my questions and comments are not phrased more politely, but I don't know how else to question things without it coming across as "challenging the science."
PS. My wife and I are gonna go watch a movie now.