Originally Posted by LTD02
here you go:
Loudspeaker Measurements and Their Relationship to Listener Preferences:Part
Loudspeaker Measurements and Their Relationship to Listener Preferences:Part
both by floyd toole....................
To summarize, directly from both the Toole papers:
"Evaluating loudspeakers by means of measurements is rather like being a detective looking for clues to the existence and origins of misbehavior. The measurements that we use are not, in themselves, virtues or problems; they are merely indicators. Two ears and a brain do not process sounds as do microphones, measuring instruments, and the eyes. The auditory perceptions of musical sounds are not the same as visual analysis of data from the clinical sine waves, impulses, and pink noise. And yet, for now at least, there is no choice but to proceed with traditional methods bearing in mind always that evidence that offends the eye may or may not indicate the presence of a problem that is offensive to the ear."
"Using the highly reliable subjective ratings from an earlier study, loudspeaker measurements have been examined for systematic relationships to listener preferences. The result has been a logical and orderly organization of measurements that can be used to anticipate listener opinion. With the restriction to listeners with near-normal hearing and loudspeakers of the conventional forward-facing configuration, the data offer convincing proof that a reliable ranking of loudspeaker sound quality can be achieved with specific combinations of high-resolution free-field amplitude-response data."
"In all there were 42 listeners in these tests, but the data used here pertain to 28 who exhibited low judgment variability. These listeners all had hearing threshold levels within 10dB of the ISO audiometric zero at frequencies below 1kHz, and within 20dB, up to 6kHz. As noted in the previous paper, these were the listeners whose 'fidelity ratings' showed the greatest consistency within individuals and the closest agreement across the group of individuals."
"Not all listeners auditioned all loudspeakers and not all loudspeakers were included in each experiment. There were, in fact, six separate experiments, with certain 'anchor' products being common to several of them. All of the loudspeakers were conventionally enclosed two- or three- way systems with forward-facing drivers."
"In the course of each 30-min exposure to randomized presentations of a test group of four loudspeakers, listeners completed questionnaires, quantifying their perceptions of various audible attributes and arriving finally at an overall fidelity rating. This rating is on a scale of 0 to 10, where 10 identifies a reproduction that is perfectly faithful to the ideal, no improvement being possible. The number 0, in contrast, denotes a reproduction with no similarity to the ideal - a worse reproduction cannot be imagined."
"All the fidelity ratings used here came from monophonic listening tests. Over half of the speakers were evaluated in stereophonic comparisons as well, but these data are not included."
"The purpose of this investigation was to see if there were obviously recognizable characteristics in the objective data that could be construed as confirming the fidelity ratings resulting from subjective measurements using listeners with basically normal hearing. The conclusion is that there are."
"In fact there are strong suggestions that, with a few basic instructions, observers could rank-order the measured data in a manner that would closely parallel the subjective rankings of the products. Listeners, it seems, like the sound of loudspeakers with a flat, smooth wideband on-axis amplitude response that is maintained at substantial angles off axis. If this is achieved, the loudspeakers will exhibit smooth (but not flat) sound-power responses and directivity indeces. The phase responses will also tend to be smooth, but not of any overall shape."
"What we do not learn from these data are the priorities of these performance measures. Need they all be equally
good? Neither do we see the potential effects of the listening room, although the effects of one particular room are included in the subjective ratings of these products."
"At the outset of this protracted investigation there was some doubt that it would be possible, with any precision, to describe a good loudspeaker in purely technical terms. The problem was not a lack of competent measurements; it was a lack of agreement among listeners."
"After the system for precise subjective measurements was developed, reliable listener responses provided the means by which measurement data could be classified. With this insight, consistent patterns emerged, indicating the principal parameters, and some less consistent relationships were apparent, identifying parameters of lesser importance."
"In many ways the outcome is a relief in that it is kind of a reaffirmation of 'motherhood.' Given the proper circumstances, experienced listeners with normal hearing prefer loudspeakers with wide bandwidth, smooth amplitude response, and uniformly wide dispersion. To arrive at this apparently simple conclusion one must define precisely what is meant by 'amplitude response,' and it is not what is commonly and currently popular."
"There are two principal conclusions to this work. The first is that given adequate preparation and experimental controls, listening tests can yield reliable subjective data and that listeners with good hearing performance agree closely on the relative merits of loudspeakers. The second is the the loudspeakers preferred by these listeners are those exhibiting measured performances that are superior in certain well-defined respects."
"The original objective of the study appears to have been met in that it has been demonstrated that there are clearly parallel systems of subjective and objective measurements and it is possible to arrive at similar conclusions from measurements of either kind."
So, basically, what was done was to simply correlate certain measurable characteristics of the loudspeakers used in the study to the subjective perceptions (the speakers' "fidelity ratings") of the 28 listeners included in the study. Note that those listeners who were deemed to not "listen" similarly to a majority of the group (the outliers; 14 of the initial 42) were thrown out of the analysis.
I suppose an inference can be made that one should be able to look at the same objective measurements that were used in the study (and performed identically) for a loudspeaker that was not included in the study and predict what it's "fidelity score" might be with these same 28 listeners, in the same listening environment(s) and conditions. Whether that inference can be extended to include ALL possible listeners is arguable. And whether one could reliably use, or would even want to use, such objective information, if it was even available, to make a speaker buying decision is also arguable. I guess if you can get those exact measurements, performed absolutely identically, for a speaker you are considering buying, and you trust those 28 listeners to represent your listening preferences, then it might be a somewhat reasonable way to make your decision (or even narrow down your choices). Personally, I wouldn't be at all interested (or comfortable) in using that method to make a speaker buying decision.
There is also considerable discussion regarding how to properly perform the listening tests as well as how to properly perform the measurements, with the implication being that as long as both the subjective testing (the listening) and the objective testing (the measuring) are done "properly", a correlation between the subjective results and the objective results can be observed.
Not to diminish the efforts of the author but, frankly, I do not see what is so significant, startling, or otherwise remarkable about the results. Yes, I took the liberty to extract quotes of the papers' text that I considered most relevant. If there is something I should have included or if my summary of the work and assessment and/or interpretation of the results is incorrect or misguided, please feel free to correct me.
I have not yet read Sean's paper.