I wouldn't mind banishing irrational people to the caves, but I guess nobody is perfect and its probably better if I have a LITTLE compassion.
"Science" is more of a general process based on logic and isn't some kind of formula for success in and of itself. It serves to answer questions with a greater degree of certainty than random guessing. It takes a logical mind to understand the difference between "X implies Y but Y doesn't imply X" or "If X is true, then Y is false, but if X is false then you can't draw any conclusion."
In the case of Machani's test, the way he constructed it, failing to distinguish the difference between players would be a BIG result, but the results as they stand do not mean much.
On the other hand, if he level matched and did a bunch of trials with unbiased observers and eliminated all the sources of error he could think of and failed to tell the difference, that would ALSO be an "unsurprising" result. What would be most surprising of all, and most enlightening, would be to conduct a scientifically sound test that leads to the conclusion that even in blind, controlled conditions, CD players are still distinguishable, THAT would be a big result. Bigger than failing to find any differences in a casual test.