Count 1 more for all receivers sound the same - Page 15

As far as I can tell, the news archive for avguide only goes back to about August of 2008, so no.

The paper appears to be exclusively available from the AES web site.

Upon reading farther, it was not a ABX test but it is similar in concept and nearly similar in execution. I had a hard time myself finding the article but I found some more information. Unlike some, i can admit where I am wrong BUT the test was setup and conducted just like a ABX test. How was the goal different? They wanted to see if the compressed codec could be distinguished from the uncompressed signal. I am sure ABX testers would have missed the distortion as well.

It STILL proves double blind tests are worthless for audio comparisons. It still proves that listening to the source is better than ridiculous test protocols.

Here are things wrong with ABX testing and this is the biggest problems with them and the people who "live" by them.

• Outcomes are just presumed, unless you choose it near 100% of the time, it is just called "guessing". True science has an outcome (yes or no) where as an ABX is "random guessing" and just plain inconclusive.
• ABX tests have been passed before but are usually 50/50.
• It is too difficult to setup properly and the results mean nothing if you go to buy a product anyway.
• Music is emotion, when put in a test environment maybe your emotion is blocked and that is why you cannot distinguish sounds? Without assuming there are many critics of DBT for a reason as far as audio.
• It still fails to show if product A is as good as product B. Even bypassing sound it does not show reliability of the products, feature differences, etc. In essense: utterly useless.
• You do not test equipment the way a ABX test does, so again, what good is it? How can it show me what is the correct product to purchase? It doe not.
• What if I am split between 5 different products? Would I have to sit there for 3 days testing all of them? And let's say the results are 50/50, what now? The ABX test failed to help me and actually compounded the problem.
• Audio is emotional, everyone has different tastes, look at Klipsch speakers, there are people who think they are the best thing ever and a lot of opponents. I like to use them for example because few speakers have such a "love/hate" like they do.
• More important than the amp or receiver, is the output device. ABX tests are useless for speakers and they dictate the sound more than an amp or receiver would.

What we are still missing are your scientific evidence, Arny. You say I am technically wrong but your argument is pointless when you cannot even remember what you argued about to begin with.
Edited by Ricsim78 - 2/12/13 at 6:09pm

They do sound the same if all you play is Grateful Dead. Its all one long song.

No argument there for me! 100% agree there.

Not just ABX tests, any kind of double blind test may work for other things but I fail to find the use in arguing about the "technicality" or use of them for audio. Ok cool, so I do a DBT and I find out a $190.00 second-hand Pioneer sounds "as good" as a $2000.00 Denon. So am I going to buy the Pioneer instead of the Denon, thanks to the Denon "failing" to better the Pioneer in a elaborate test? Heck no, I am going to buy the Denon still just because it will have the best room correction, features, amp quality, bells and whistles, etc.

So all that trouble, all that time to setup with all that elaborate equipment, carefully, carefully matching them and what now? I am still going to get the Denon. DBT are flawed for audio because:

• People have different ears and tastes Not to mention hearing ability.
• People are not as perceptive as we think, some more than others.
• Whether it was a ABX, ABC, XYZ, 123, whatever test, the DBT which uses a lot of the same parameters FAILED.
• No one has the time, resources, or exact components they are deciding among to bother doing a DBT.
• DBT are inconclusive, therefore the fact it is said they "help" or help you choose a product is a big fail. You are still biased and still going to buy the one you want (want most), especially if you take the amp quality out of the question. They supposedly take away product bias but as Arny pointed out, everyone is biased. This includes DBT participants.
• My test is not 100% accurate, no test in the world is because people can critique any test method. My test is more useful than a DBT test, that is for sure! My test is similar what magazine reviewers would do, I have yet to recall a review that includes results of DBT. Wonder why?
• Even Arny, the creator of ABX, would not use his own test to pick out a product. Why would you?

Edited by Ricsim78 - 2/12/13 at 6:12pm

So how do you choose a receiver? In your basement do you buy both that you want and spend hours with expensive equipment (level matching, calibrating, playing Liberache, etc.), then call over your neighbor's friends you do not know? That is a DBT definition in essence, you cannot know the person because supposedly that gives out "gimmes". Then you do the test (12 passes) and that person finds that they are hard to discern the difference. They guess 6 of 12 times. Cool, you wasted all that money, time, and effort for an inconclusive result! Not even mentioning that person will have a different result from you, and since you are the person choosing a receiver, how does their result help you? It does not. What did the DBT tell you? Nothing.

DBT are nothing more than interesting. They can have use (and may be best in some forms of testing like food products and medical products), but the point of the "Codec" argument is a listener detected a problem that all that time, money, and effort failed to find. You can argue Harley's take on things and I do not agree with him 100% but a lot of what he says is basically true (not his stabs, his reasoning why DBT are flawed.)

DBT do nothing to help you, they almost always come up inconclusive. So they may be industry standard or whatever, but what do they do for the average Joe like me and you? If you want to believe your $200 Yamaha is going to perform the same as a $1000 quality amp because a "test said so", so be it. The issue is there is a thing called specs. All amps have THD, weighting, actual output, distortion, and many things which make them different. A receiver is an amp and preamp together, but even taking the amp out of the equation there is still performance specs. Look at the specs of my Pioneer Elite SC-65 as opposed to my Denon 1612, now you tell me how even the amp section would sound the same! It has lower power output, higher distortion ratings, lower quality components, is a year older, has inferior just about everything. Now you tell me how it would possibly perform as well as my Pioneer or sound as good as it? In a DBT, they could come close but sitting in your living room there will be a night and day performance difference. Not to mention the Pioneer would likely outlast the Denon because it runs cooler compared to it.

Someone before made a comment I wasted money on my receiver, which I find funny because I could not have made a better purchase for me. It is the only receiver I have owned which makes me happy and does not give me buyer's remorse. I wish it did only cost $200 but I am willing to pay for quality. Lastly, our ears are the final judge, even if the differences are just "perceived and not there". If you think a $200 cable sounds better than a $30.00 one, than to you it does and that is the bottom line. If you can afford it and think it improves the sound, knock yourself out. The object of this is to buy the product you want that makes you happy.

No test is going to do that for you.
Edited by Ricsim78 - 2/12/13 at 6:15pm

You still have not told us why FR measurements are omitted in ABX testing.

While I do appreciate the testing you did, I think you are still missing the overall point of a DBT, or most any meaningful, scientific test.  In any test, for any variables you are not testing you need to hold them constant as well as not induce any external biases.  And yes this can be difficult to do.  I know you are saying you are trying to be open to both sides but the reality is you are not.  You don't seem to grasp the reasons why a double-blind ABX test is required to hear differences that could only be caused by the amps themselves and not any other external influences.  People pointed out the flaws in the testing you did.  They are not bashing you, the are just pointing out the reasons why your test is not valid to draw the conclusions you did because too many other variables are changing.  Maybe there are indeed differences in the receivers you tested, but you cannot draw that conclusion from your results because of the deficiencies of the test method.

 

I don't claim to be a know it all by any means but I do happen to have an EE degree and up until about 5 years ago was a Test Method Development Engineer (not in the audio industry) so I do know a few things on this topic and have learned a lot about various influences on test methods and I continue to find psychology and how the brain works/interprets things extremely interesting.  Very seldom do you find a perfect test, so being open to other views, opinions and criticism so you can learn and deal with the deficiencies of the test allow you to better understand the results and/or make the test better.

 

So again, I applaud you for you efforts and personally would love to see you take the feedback others have provided, improve your test method and come back with more results.  I know I'm too lazy to go to all that work and just want to enjoy my system.  :-)

What do you care? And why would FR measurements (and under what specific circumstances) make any difference? Welcome to this forum, you mean yourself? Hobby? What you want to believe? Come on this is a science based forum. You want religion, not a good place to find it LOL!

The reason I disagree with Harley's take as you quoted it is not only simple, he points it out himself. It is ridiculous to say "The test must be wrong because I find the result to be 'patently absurd.'" Isn't the point of the test to find out the answer to a question? If you know the answer, why perform the test? If the answer isn't what you thought it would be, shouldn't you begin by revising the assumption?

In that article, Robert Harley announced the result of any test he didn't like meant something was wrong with the test. Huh? You think his reasoning is "basically true." I don't even think it qualifies as "reasoning." I suppose it's a testament to his influence that anyone can read that and not scratch their head.

The problem (among others) with the "codec argument" is that it isn't the same test. The ONE person that allegedly identified the problem (and really, is it an audio problem if most people can't hear it?) was sighted. He didn't perform the same test. If you run 100M faster than I run 200M, does that prove anything at all?

To my eye, it looks as though you'e starting with the conclusion (the Pioneer sounds better) and then backfilling the reasons why it must be so (more power, lower distortion, higher-quality components, superior everything). This is what people mean by expectation bias. You know it's measurably better in all these areas, so you expect it to sound better. But this isn't proof. Proof would be if you could pick out the sound of the Pioneer without knowing which one it was and no other clues (such as level discrepancy). Then no one could accuse you of bias (unconscious or otherwise) and when we ask "why does the Pioneer sound better," you might point to all those things you mentioned as the possible reasons(s).

All that aside, one of the things I learned here is that just because the Pioneer does all those things better does not mean that they can be heard. If everything you say about those receivers is true (and I have no reason to doubt you), it still doesn't mean the difference is audible.

I'd never say anything like that. In fact, I think you bought it for the best possible reason. ("It makes me happy.") I also think that price, features, build quality, design, brand loyalty are good reasons. But none of that is the same thing as "It sounds better."

I bought my last receiver based 100% on features and price. I bought it locally, but I never even listened to it. It replaced an integrated amplifier. I can't tell one lick of difference in the sound, but the receiver is a hell of a lot more useful. And like you, it makes me happy. I couldn't be more pleased.

But this wasn't always the case. A long time before I came here, I bought one amplifier over another one after I was convinced that it sounded better in the store. (And I had my girlfriend there with me and she agreed. True story!) But once I got it home, I realized that it didn't sound any different than what it replaced. I also realized I bought the one I thought looked better. No doubt she was reacting to my enthusiasm for the one I bought.

A short time after that, I bought a CD player (oddly enough, a Pioneer Elite), which cost what was for me a ridiculous amount of money for a CD player. I later realized I chose it over another one (Cal Audio Labs, I think), because I liked the disc platter. But of course I convinced myself that it "sounded better," which in the end, it didn't. I will say it had terrific build quality and I still use it today. (Although very little since I've ripped all my CDs to a hard drive.) So, there you go. We're all subject to bias.

I've come to believe that I can justify just about anything I want to buy, but I've learned there are better and more honest reasons than sound quality.

I know the test I did was not 100% "accurate". It was more for fun and for my own than a "scientific test" which proved me right. I also was just curious and it is fun to do such things. I appreciate your comments and your advice and I do not mind getting advice or even being told I am doing something wrong. It is when people reply like ArnyK do that makes me lose faith in people. I will have to do a more controlled experiment but I know no matter what I do, unless it is a DBT or ABX test, it is worthless according to those that feel all receivers + amps sound the same. I appreciate posts like this, if only more people were like the quality of this thread would be way up. Thank you. Any advice on a better test would be appreciated from you.
LOL is right, are you okay? I was kidding with Mudcat. This is not science, this is good people mixed with egomaniacs. I was not even the one to mention FR, Mudcat did. I was just stating the facts of the attempts of belittlement here, it has happened it lot if you followed this thread from the beginning. Not sure where the religion comment came from, but yeah, whatever you say.

serious question for Ricsim78,

What exactly are you trying to accomplish in this thread? I have said my piece on this topic way back on page one and have since lurked on this thread, and as far as this lurker goes, you posts are neither proving anything valid nor are they coming across as anything more then attempts to keep a flame war going.

You cannot seriously be thinking that your opinion on any of this matters to anyone. If you seriously want to tackle the dbt guys, you need to utilize proven scientific methods to obtain facts. Otherwise everything you say is highly subjective and can only be considered anecdotal at best, which goes without saying that your conclusions and opinions are only relevant to you and no one else. So please enlighten this lurker as to what it is exactly that you are attempting to accomplish.

Honestly from my perspective your posts come across as very amateurish with a healthy dose of immaturity coupled with gross misunderstandings of what the scientific method is and how it pertains to testing and experimentation.

I can say that for myself, I like what I like, however I am interested in understanding "why" I like what I like. If one attempts to seek knowledge then a greater understanding on any subject or topic can be found. However if one continues to wear blinders and refuses to open ones self up to gaining knowledge then this will lead to a poor understanding of the world around us. Which to me seems like a very sad existence indeed.

Look at the bright side, if you have faith in this scientific testing, your AVR and my power amps sound as good as those $350,000 Wavac mono blocks!

Okay, to cut off the predictable repercussion from my previous post, I do not need a lecture on my methods for hearing differences. I understand test methods and reasons why you would carry out such things (of course you test products, if they did not there would be a lot more component failures. I understand what ABX tests are, the way they are conducted, and why they are conducted that way.

Are DBT tests useless? Heck no, but I do not think they are a good solution for audio (especially at the consumer level).
Are ABX tests useless? Heck no, they are interesting as are the results. But they do not tell the whole picture and that is why magazines and such do not use them for receiver reviews. Why? Because they tell you nothing about the equipment being used, rather they tell you if you level match them you likely cannot hear a difference. How does that remove bias or help your knowledge of if the product is good or not? Does it tell me if the firmware is solid, does it show everything functions as it should? Does it tell me that the receiver/amp has good sound quality? No. For me, that disqualifies the usefulness for MY needs. For example, I buy an Onkyo 818 because it went against a Marantz 7007 and sounded the same in a DBT. Does the DBT warn me that there is a 24 FPS bug (this is fixed, AFAIK, in a new firmware)? I would buy the cheaper Onkyo and have a problem that would make it hard to live with (unless they fix it). This is why i would rather go by professional and user reviews rather than DBT. (Of note I almost considered the Onkyo 818 until I learned of the bug, which at the time of purchase was not fixed. I then spent a little more and went with the Pioneer Elite SC-65 and I am glad I did). A DBT, ABX, or even science would not have helped me make this decision. Simple user reviews made me aware of the problem.

You can argue what I just said but they are facts (especially fact 2, which is why I have said they are useless.) Not useless in a whole but I do not see how they help a guy or gal who just wants a good receiver for their money. And that is what everyone I have asked has told me, in this very thread. Most want the features, as do I. I didn't buy my Pioneer to "beat" my Denon, I bought it because I wanted something new and I had a certain budget to work with and it had everything I wanted and then some in a receiver and man does it work/look nice! It also happens to outperform every other receiver I ever had. Not because I want it to, I would love to think you can buy a $200 receiver and have it sound as good as a even $500 unit. It simply does not, MultiEQ XT can make a huge difference over 2EQ, for example. Most of the low end models are missing or getting watered down features that would promote you to the next product tier.

Here is MY point everyone here is missing or if not, it is just being looked over. When you buy a receiver, you are not doing a DBT except for fun and even then you lack the equipment and control used in sophisticated tests. Instead, you open it up, set it up, run the room correction (usually), tweak it, use it, then tweak it more and leave it be until you upgrade or add something else. Once you use it as the manufacturer intended, which almost everyone does, you are going to hear differences from different/lower models. It could be the room correction used, it could be the amp, the room and speaker interaction, the amp power, the connection type. In essence, you will hear differences in the sound. If you do not, you should go to a hearing doctor or should buy yourself some better speakers. But even cheap speakers I have owned have revealed differences. This is my experience, and I know many like me. Even if the difference is "in my head" it is there and I am glad because if everything was the same, the world would be boring. That and I would have trouble justifying buying more audio equipment to myself and my girlfriend. Thankfully she hears the difference too!

Use DBT tests to choose equipment, buy what you want, forget sound quality when you buy an amp/receiver. I have said many times, you are free to do and believe what you want. My belief, which requires nothing more than common sense and experience, is enough for me. You can argue that all day long or post snazzy comebacks, quote journals, flash credentials. This is MY view. Take it or leave it, at least I am open minded enough to see that there may be another way.
Edited by Ricsim78 - 2/13/13 at 1:23am

They told me this is a science forum based on facts. What they have "proven" with these tests trumps all other opinions. They may recommend you to other forums were people believe in magic and fairy dust, or something like that.

You see, neither side of the argument is absolute. People have stated the flaws in my argument, I have done the same to theirs. I never said my tests are better or trump their tests. My test is more akin to what you would do at home, it is far from perfect. But a DBT for audio is far from perfect too, actually in most cases they are inconclusive. They can read that for themselves or likely already know it. So you go through all that testing, have others do the experiment, all that protocol and data, and the end result: Inconclusive and since you are not allowed to participate, how much wiser are you?. You are not farther along than you were before all that trouble. As I stated before, some ABX tests have been "positive" (vast majority chose correctly) but largely inconclusive. At least my AMATEURISH test came to a conclusion and also allowed me to see how good both worked through the same output and input.

If anyone tells me where to go, they all know where to go.
Edited by Ricsim78 - 2/13/13 at 1:17am

 

The point about this is that it shows you either a) did not read the paper that you quoted from or b) you did not understand it.

 

Some posts back you said "Scientists in the early ages likely knew more about the stars than most graduates, they simply did not know what they were and thought the stars were lanterns held by deities. Science is great and all but it changes when new thought patterns and findings occur."

 

Do you not see that you are just like those 'scientists'* you impugn? You refuse to change your beliefs even when presented with substantial evidence, just like many contemporaries of Copernicus did.

 

Decades ago, people used to sit down and use their unreliable ears/brain to try to find differences between amps. They reported that they could indeed hear differences - some amps were 'brighter', some 'warmer', some gave 'better soundstage and imagining', some 'brought out hidden details'. Indeed, in those days, the amps may well have sounded different to each other because reliable, high quality components were expensive, so some amp manufacturers skimped on them. Now fast forward 40 years. Components have long since ceased to matter very much because even affordable ones exceed all the required performance parameters and introduce no audible distortion. And in the meantime, a much more reliable way of evaluating differences between amps has been invented - the ABX test. It removes all listener bias, it is conducted every time using the identical control values laid down in scientific papers such as the one recently linked to here. It is used in all fields of science, including medicine, where non-biased reliable results are needed. 

 

Yet you cling to the 'old way' like those members of the Church who imprisoned people who could prove scientifically and unequivocally that the Earth was not the centre of the universe.  Why?

 

* Those you refer to were actually philosophers rather than scientists, just for the record.

It surely cannot be long before this thread is locked. It is a shame because the premise of it is a good one and worth debating.

Ignorance of the scientific method will be to blame.

posts deleted: if I missed one I am sorry... PM me

please stop responding to and quoting problematic posts ......just report to mods to handle

Thank you.

I had a few minutes free so I played around with the "Wayback Machine" from the Internet Archive. It contains snapshots of many stories from that site during that era, but not this one.

Thanks! The question was asked out of complete and utter ignorance. Until I joined this forum, I thought an amp and a receiver were the same thing...

Would the test be accurate if the listeners did not care at all about which wins? For example, if I am demoing two amps in my home and I really do not care which one is better than the other, would seeing them make a difference? I realize it is not a study test then, just curious about real world application. I include complete ignorance of brands and also include money not being an object - that way those two biases are missing.