Originally Posted by arnyk
99.99% surely a false claim. You don't know, do you Amir? How would you know? Were you a little bird on their shoulders for their entire life?
No, it is pretty easy. All you have to do is read their paper. No mention of training or trained/expert listeners. Here it is in their own words:
"With the help of about 60 members of the Boston Audio
Society and many other interested parties, a series of
double-blind (A/B/X) listening tests were held over a period
of about a year. Many types of music and voice signals
were included in the sources, from classical (choral,
chamber, piano, orchestral) to jazz, pop, and rock music.
The subjects included men and women of widely varying
ages, acuities, and levels of musical and audio experience;
many were audio professionals or serious students of the art."
Nothing in there speaks to specific training such as what I have in finding artifacts that they were chasing. There is also no mention of training content for the listeners they did have.
And "many types of music" supposed to be all the qualifications needed for such a test? Not that any of them have actual high frequencies in them?
It's an ABX test. Q.E.D.
Really? You are saying just having an ABX box gives us positive and negative controls? How does an ABX box guard against me having the same input into the box instead of the two that were being compared? I would get a bunch of random guesses then, right? Is that why it is OK to not have controls?
A positive control where say, all the high frequencies above 10 Khz are taken out would have identified that problem. If the two cables are carrying the same signal, you would immediately know it from the first few testers not telling them apart. Or else, we would know they are unfit testers for high-res audio.
We use controls to catch mistakes. We use controls to weed out non-critical listeners. This is why they are used in medical field. We keep talking about that using that "science" in audio but in the next breath, assume there is no need because we have an ABX box?
You yourself in your key jingling test provided controls by going way down in sampling rate. Where was that in Meyer and Moran?
Amir's new rule: If a scientific paper doesn't explicitly say something happened, it didn't happen. There are a ton of scientific papers that don't say what the people writing the paper had for breakfast lunch, and dinner and so they were starving the whole time they did the experiment.
No paper is "scientific" if it leaves out key aspects of doing a proper listening test as JJ lists. By your rule, we could also assume a test is blind even if it doesn't say so? It seems we have no standard for what constitutes a "scientific paper" if it agrees with our point of view. We are corrupt to the core that way.
Here is how you document tests "scientifically:" The Placebo Method, A Comparison of In Situ Subjective Evaluation Methods for Vehicles, AES paper, Authors: House, Neal; Shively, Roger
Affiliation: Harman Motive Inc., Martinsville, USA
1.1 Listeners, Training and Scales
Eight trained listeners with known and acceptable hearing acuity and listening experience were
used for this study [5,7,9]. The listeners have had experience evaluating vehicle sound systems
using the blind and sighted methods mentioned above. The listeners have successfully completed
“ear training and preference testing ” exercises [5,6] and were above a 95% accuracy rating with
less than 0.5 variance for identifying the correct response.
The listeners were trained to rank and order their judgements using a “fidelity level scale” which
is an interval scale ranging from O-10. Generally, when ranking “high end” vehicle audio
systems with this method, there tends to be some scale compression as most of the rankings occur
in the 5.0-8.0 range. An example of the scale; with verbal attributes is shown in Table 1. This
scale is somewhat similar to the IEC 268-13 interval scale [lo].
All listeners were required to “recalibrate” themselves by completing the listening training and
fidelity scale testing prior to the vehicle evaluations. An anchor [control] system was available to the
listeners throughout the study. The listeners were given explicit instructions about the listening
and system calibration procedures.
This is how it is done by professionals in the industry. What we have in Meyer and Moran is what hobbyist do without any of those qualifications. And we have other hobbyists running around forums talking about "scientific method" while turning a blind eye on the most basics of performing such tests properly.
Don't need to. A lot of players have some kind of phrase repeat feature. My old CDP 101 surely did.
And have you tried to program it? I have. It is a pain in the neck to mark a specific segment that may be as short as half a second. It is nearly useless for this type of testing. This is beside the fact that nothing has been said about their player having such a feature and used in the testing.
If you don't like it Amir, then do better.
I did. I passed your test. I passed Scott/Mark's test. I passed Ethan's test. Get me any of the people in the Boston Audio Society and let's see how they do.
Honestly how can we with straight face act so biased Arny? You know the merits of what I am talking about. If the outcome was positive, you would have harped on them until end of universe. But since you like the outcome, all is well?
Please let me know if we could make our reputation any worse Arny. Heaven knows every time I think we have sunk as low as we can sink, we manage to take it down even further. "Scientific paper" doesn't have to say it used controls, trained listeners, etc.? Unbelievable.