Originally Posted by motrek
I haven't seen the algorithm, but this is somewhat problematic. If you're creating an algorithm to see if two things are correlated then you can design the algorithm in such a way that you're basically manufacturing correlation. I'm not saying this would be done intentionally but there's a possibility that it happened unintentionally. This has been a problem for other branches of science in the past.
Related: There have been tremendous developments in machine learning just in the last few years. I wonder if the Harman algorithm couldn't be replaced with an artificial neural network that has even more predictive power re: loudspeaker preferences. Such a neural network could be made to analyze all the data collected in your anechoic chamber and not just the final spin-o-rama plots. Then again, the ideas of "flat" and "smooth" aren't that complicated, it might not be worth it to go to all that effort to end up at (probably) the same destination.
Indeed one can "fudge" algorithms. In our case it was an exercise prompted by Consumer Reports loudspeaker reviews which routinely trashed loudspeakers we knew sounded good (some of them Harman products) and praised products that routinely lost in our double-blind evaluations. CU did no listening tests. Their ratings and rankings were entirely based on a 1/3-octave measurement of sound power and calculations based on misinterpretations of the perceptual process. Some people in the industry knew the truth but could do nothing about it.
One day our CEO called me into his office and asked me to explain why our products did not always rank highly. Long story short, he and Sidney Harman authorized us to spend the time and money to force them to put up or shut up. Research was done. We let them preview the two definitive Olive papers before they were made public via the AES.
Olive, S.E. (2004a). “A multiple regression model for predicting loudspeaker preference using objective measurements: part 1 – listening test results”, 116th Convention, Audio Eng. Soc., Preprint 6113.
Olive, S.E. (2004b). “A multiple regression model for predicting loudspeaker preference using objective measurements: part 2 – development of the model”, 117th Convention, Audio Eng. Soc., Preprint 6190.
Consumer Reports stopped doing speaker reviews and started a process of improving their evaluations. It did not last long and the effort was abandoned. So, in a strange way, Harman did the entire loudspeaker industry a favor by removing a source of influential, but incorrect, consumer product evaluations.
We, of course, gained more confidence in the measurements we were doing. Could it have been taken farther? Probably. But, the present reality is that we can design loudspeakers with high confidence that they will rate highly in double-blind subjective evaluations - to the point where the highest ranked products end up in statistical ties. The weakest link is no longer the loudspeaker, it is the program - i.e. the Circle of Confusion. To this must be added the fact that everything we hear below about 400 Hz, which is determined by the room and arrangements within it, accounts for about 30% of the factor weighting in sound quality evaluations. Only in-situ measurements and solutions can work. This was the motivation for our work on multiple subs.
There seems to be little point in spending more time and money on predictive algorithms.