Count 1 more for all receivers sound the same - Page 10 - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #271 of 540 Old 02-09-2013, 07:42 PM
 
cybrsage's Avatar
 
Join Date: May 2007
Location: Harrisburg, PA
Posts: 8,074
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 147
Quote:
Originally Posted by Kilian.ca View Post

Lots of small-scaled, amateurish and negative trials don't add up to prove anything conclusive, only suggest a trend. For professionally conducted studies that are negative or a mixture of positives and negatives or inconclusive there are methods (meta-analysis) to evaluate them as a larger group to look for hidden positives.

Sample size IS important in statistics. Small differences that are statistically non-significant might become significant in a larger sample size, because the sample size is factored in determining the p value. You only have to read some clinical trials to see what sort of numbers (hundreds and thousands) are involved and the differences are often very small, not 80% vs. 20%, this large magnitude of difference doesn't happen. Usually when the sample size is down to 10 or 20 very very few studies have statistically significant results. Also note that 20 people each doing 10 trials isn't the same as 200 doing 1 trial in statistics.

As I said before some of you just use small and amateurish trials as concrete proof and this is not how proper science works in real. I'm not against DBTs but just the way you over-interpret their true meaning and significance. If you think some of these audio trials are well run you might also want to read some of the large scale clinical trials or studies (some are freely available) to get some idea of what's it all about at that level.

I think the big key is that, if there were noticeable differences between amps the companies making those better amps would show the hard data proving it. They would eagerly go into double blind tests and come out on top, repeatedly. I know I would be throwing my amp around in double blind tests if I ran a company that created one which was noticeably better - making sure ALL the professional magazines knew all about it. The dearth of these tests speaks volumes.


EDIT: I realize that in science the absence of evidence is not evidence of absence - but in capitalism it is.
cybrsage is offline  
Sponsored Links
Advertisement
 
post #272 of 540 Old 02-10-2013, 02:24 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by runnin' View Post

Quote:
Originally Posted by kbarnes701 View Post


It's neither in my case. It's a genuine desire to help people via the medium of these forums, in the same way that I have been helped so much by them in the past. It's called "giving something back".  Of course, everyone is entitled to their views and beliefs and I have never said differently. The real issue is that you will find people on these threads who will pass of their opinions as facts. So they say things like "All amps sound different to each other and I know that for a fact because I can hear the differences with my own ears".

Now this is not only wrong, but it doesn’t even enter the arena of sensible discussion because someone's opinion is just that - an opinion. It's like me saying that "I know all French people smell of garlic because I have smelled it with my own nose". People who say these things will never offer any objective evidence to support their position (usually because there isn't any) and will just repeat, ad infinitum, that it must be so because their ears say so. Worse, when presented with objective evidence that they are wrong, they will refuse to accept it and cling to their original position regardless. Even to the extent of saying "You will never convince me otherwise". IOW, their mind is made up that they must be correct n matter what.

Why does this matter? Because for every person who contributes to a thread, there are many more who silently read it. And many of them are looking for information to help them with their purchasing decisions. And many of them are on tight budgets. And many of them have very little knowledge of audio or acoustic science. So when they read statements that tell them that one of the most important things they can do to improve their SQ is to buy a new amp, many of them will go out and do that. They will do this even though it is almost always a waste of their money and they would benefit far more from rearranging their rooms or adding some simple room treatments, at very low cost if they are made on a DIY basis (which is very easy). So I, personally, think it is important to provide the right information, backed by objective science, in order to provide a counter-perspective. Hopefully, some people will read this and think "hmm, maybe that $2,000 amp would be a waste of money after all" and they might investigate other ways which have been objectively proven to yield what they are after: better SQ.

So I, personally, do object to being told I am on a "crusade" or have "an endless desire to lecture others" because neither position is remotely true and you can verify my aims and intents on AVS by searching my posts, especially in the Audyssey thread where I usually hang out. To offer objective, evidence-backed views is not disregarding the "real world" in any way IMO. It is very much trying to help one of the most 'real world' issues there is: spending one's hard-earned cash wisely.

I wasn't responding or talking to you, so I'm puzzled as to why you think I was telling you you're on a crusade, etc. If I want to do so, I will use the quote feature, or address you by name. I have no interest regarding your intents here nor reseaching your posts. There are, however, those who take your position in the matter who are on a crusade, if you want to defend them perhaps you could get some of their quotes and defend them directly.

You yourself told me earlier this week that you get tired of repeating yourself, and yet here you are once again doing so. Using a strawman argument, like the suggestion that the most important things one can do to improve SQ is a new amp, only weakens your position, I've never said that nor seen it expressed here.

 

Your gracious reply speaks volumes and I thank you for it. 

kbarnes701 is offline  
post #273 of 540 Old 02-10-2013, 02:30 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by cybrsage View Post

Quote:
Originally Posted by kbarnes701 View Post

At the end of the day, it doesn't really matter much because I know that the real way to get BIG improvements in SQ is to get really, really good speakers (which I believe I have done) and really, really good subs (which I have definitely done) and concentrate on measuring and treating the room. Especially the latter as it is so cheap, comparatively, to do. That is where the real changes will be heard - not in modern electronics.

Treating the room is something almost no one does - heck, before I started building my dedicated theater room I never even knew rooms SHOULD be treated. It definitely does give you a great bang for the buck.

 

Absolutely. And if a room is not treated, then really, all arguments about whether this amp sounds better than that amp, or even this speaker sounds better than that speaker to some extent, become pointless. People often forget that they are listening to a system - and the room is the most influential part of that system.

kbarnes701 is offline  
post #274 of 540 Old 02-10-2013, 04:12 AM
Advanced Member
 
JD in NJ's Avatar
 
Join Date: Jan 2012
Location: Northern NJ
Posts: 645
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 35
Quote:
Originally Posted by cybrsage View Post

Religion is a good comparison. Many atheists feel the need to tell theists they must stop being happy while serving their god...that since the god does not exist their happiness is fake and must be thrown out. I say the best atheists are those who are happy for the theist - happy that the theist has found happiness in their belief.

Well that doesn't sound like a very good description of the arguments I've read and occasionally participated in, but if that's your perception I have one question: Are the best theists those who are happy for atheists who have found satisfaction in their lack of belief?

JD in NJ is offline  
post #275 of 540 Old 02-10-2013, 04:23 AM
AVS Special Member
 
Theresa's Avatar
 
Join Date: Nov 2010
Location: Minnesota
Posts: 2,430
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 50
Quote:
Originally Posted by JD in NJ View Post

Well that doesn't sound like a very good description of the arguments I've read and occasionally participated in, but if that's your perception I have one question: Are the best theists those who are happy for atheists who have found satisfaction in their lack of belief?

Isn't that the truth! Non-theists are expected to accept theists but the inverse is not true. To get back on topic, I have no faith in the belief that amps are unique in their sound. Amplifiers, at least non-tube ones and one's not designed as glorified tone controls, sound pretty much the same.
Receivers are different because of all the processing they are capable of.
Theresa is offline  
post #276 of 540 Old 02-10-2013, 05:22 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,349
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 725 Post(s)
Liked: 1155
Quote:
Originally Posted by tsaville View Post

Quote:
Originally Posted by kbarnes701 View Post

What would be your explanation for the findings published in the article Arny linked to, here:

http://webpages.charter.net/fryguy/Amp_Sound.pdf

I know this post is from several pages back, but I wanted to point out a few issues I have with their testing procedure:

1. The sample size is laughably small (25)

Compared to any number of equipment reviews that you probably agree with, the sample size is relatively huge. What's the sample size for a typical Stereophile or Absolute Sound review? One, two, maybe three?

As others have pointed out, the number of people who have participated in all listening tests of this kind with comparable results is up in the thousands or more.

It turns out that the first amplifer ABX test I did back in the 1970s was perfectly predictive of every comparable amplifier test that has been done since. One guy, one pair of amplifiers, 16 trials, done! While

I see a blind focus on numbers without any appreciation of what they cost or mean. What sample size do we need to use to determine what happens when a living human skull is hit by a .50 caliber bullet traveling at 2,000 FPS? Amplifier tests are equally certain - once the measured quality of the amplifier reaches a certain, now readily achievable level, every test comes out the same.

Large sample sizes only make sense when the results of any single test or small number of tests can be highly variable for uncontrollable reasons.

I repeatedly invite people to do just one good listening test that produces a different result. Its not that hard to do. Never happens.
Quote:
2. They intentionally brought in people with preconceived biases (never good for a so-called "scientific" test)

The above statement shows a lack of understanding of the simple concept of the bias-controlled test. The test design presumes that everybody has their own biases, and uses generally accepted means for controlling those biases. The presumption that there are people that lack preconceived biases is hopelessly naive.
Quote:
3. Their testing methods were inconsistent (ABX vs pulling wires to switch between amps)

I don't know what this means. For one thing it shows a complete lack of understanding because the ABX testing methodology works equally well comparisons using switchboxes and comparisons done by swapping wires.
Quote:
Although their results are interesting, I would hardly call this a rock solid scientific test. All that it tells me is that further testing is needed/warranted.

Given the incorrect and naive ideas proven to exist in the comments, one has to ask what are the qualifications of someone who would make those mistakes!
Quote:
The results may not be any different in a "real" scientific study, but the testing procedures followed in this particular article are pretty poor IMO.

Everybody has opinions, but many opinions can be disregarded based if they fail a simple consistency check, which the above posting fail, three out of three.
arnyk is offline  
post #277 of 540 Old 02-10-2013, 05:23 AM
Advanced Member
 
Mark Lem's Avatar
 
Join Date: Aug 2002
Location: Ohio
Posts: 582
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
So concert hall, stadium etc processing aside, is one receivers DTS HD Audio the same as any other?
Mark Lem is offline  
post #278 of 540 Old 02-10-2013, 05:23 AM
Advanced Member
 
Mark Lem's Avatar
 
Join Date: Aug 2002
Location: Ohio
Posts: 582
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
So concert hall, stadium etc processing aside, is one receivers DTS HD Audio the same as any other?
Mark Lem is offline  
post #279 of 540 Old 02-10-2013, 05:25 AM
AVS Special Member
 
Theresa's Avatar
 
Join Date: Nov 2010
Location: Minnesota
Posts: 2,430
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 50
Quote:
Originally Posted by Mark Lem View Post

So concert hall, stadium etc processing aside, is one receivers DTS HD Audio the same as any other?

Good question, I don't know.
As for sample size, its more than sufficient and the many repetitions show the same thing increasing certainty.
Theresa is offline  
post #280 of 540 Old 02-10-2013, 05:43 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by Mark Lem View Post

So concert hall, stadium etc processing aside, is one receivers DTS HD Audio the same as any other?

 

DTS HD MA and Dolby TrueHD are, AFAIK, 'standards', so they should be handled the same by every AVR they are found in. It is the DSPs that create the differences by adding additional processing - eg Dolby True HD + Dolby PLII Cinema or Dolby True HD + THX Cinema etc. Some DSPs upmix 5 channels into 7, for example, even when the original codec is 5.1 - eg Dolby True HD + DTS Neo:X, where the latter adds height and/or wide channels.

kbarnes701 is offline  
post #281 of 540 Old 02-10-2013, 05:48 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by Theresa View Post

Quote:
Originally Posted by Mark Lem View Post

So concert hall, stadium etc processing aside, is one receivers DTS HD Audio the same as any other?

Good question, I don't know.
As for sample size, its more than sufficient and the many repetitions show the same thing increasing certainty.

 

In my professional life (Advertising) I have often had to work with markt research organisations and one of the very common errors people make is to assume that a huge sample size is needed to give accurate predictions or indicators. You will hear people say of a particular piece of MR things like "How can they determine what the market for this product wants when they only surveyed 400 people - the market is millions?"  The answer of course is that so long as the sample is properly representative of the universe, then the conclusions will be accurate. Arny also makes an additional good point about where larger sample sizes may be needed. But this doesn’t apply to ABX testing, as he and you point out, because no matter how many times the tests are run, no matter how many testees there are, the result is always the same. (I like Arny's bullet in the brain analogy here!).

kbarnes701 is offline  
post #282 of 540 Old 02-10-2013, 05:59 AM
AVS Special Member
 
Theresa's Avatar
 
Join Date: Nov 2010
Location: Minnesota
Posts: 2,430
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 50
+1

What hasn't been mentioned is that for some tests "experienced" audiophiles were deliberately included in the sample and that they were no more likely to prefer high end models.
Theresa is offline  
post #283 of 540 Old 02-10-2013, 06:22 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by Theresa View Post

+1

What hasn't been mentioned is that for some tests "experienced" audiophiles were deliberately included in the sample and that they were no more likely to prefer high end models.

Indeed. In among the links to various AB tests that were in the post I made a while back, there is an amusing description of how the founder of one of the 'audiophile' amp manufacturers resorted to a form of cheating in order to be able to 'hear a difference'. I guess that you can have the most golden of golden ears in all the world and you still will not reliably be able to hear a difference where there is, ahem, no difference to be heard.

 

I never really understand what all the fuss is about. I am massively pleased that I have learned that amps make no difference to SQ (usual caveats) - it has the potential to save me thousands of wasted dollars. You'd think people would be pleased to learn this (unless they've just spent 10 grand on an amp of course). In the 'bad old days' of my devotion to this hobby, when I knew little about such things, and had not even heard of the concept of treating the listening room, I dread to think how much I spent/wasted in the fruitless search for audio nirvana, changing amps like I changed my socks, buying highly expensive interconnects, etc etc, all to no avail, for what are now obvious reasons.

 

I got so fed up with the constant chopping and changing and its concomitant waste of money, that I started to research the subject, bought and read a book on acoustics (F Alton Everest's) and then started using the Internet as a resource and gradually started to learn where the real differences could be made. For me, the great revolution was when I discovered the importance of the influence of the room and how addressing that could make such a huge difference to SQ - far more than any electronics or fancy wires.

 

I was also fortunate, back in the day, to be working in the arena of creating and producing cinema commercials and part of that involved spending time in professional sound editing suites, where discussing thing with the engineers over a beer after the day's work was done also yielded fascinating insights into what these guys believed was important. Of course, they had little truck with esoteric wires, 'audiophile' amplifiers and the like. The one thing that every editing suite I ever went in, in all my life, had in common: they were highly treated to counter modes, reflections etc. It was one of these engineers, who worked extensively for the BBC, who put me onto my M&K S150s when he first heard them years ago and was blown away by their sound. It took me some time to get around to buying these speakers but I have never regretted it and will probably never change them (unless I ever decide to go the active/powered route with their active MK counterparts). 

kbarnes701 is offline  
post #284 of 540 Old 02-10-2013, 06:30 AM
AVS Special Member
 
Theresa's Avatar
 
Join Date: Nov 2010
Location: Minnesota
Posts: 2,430
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 50
Quote:
Originally Posted by kbarnes701 View Post

It was one of these engineers, who worked extensively for the BBC, who put me onto my M&K S150s when he first heard them years ago and was blown away by their sound. It took me some time to get around to buying these speakers but I have never regretted it and will probably never change them (unless I ever decide to go the active/powered route with their active MK counterparts). 

I was surprised to learn from you how good the M&K speakers are. The dual tweeter configuration complicates things but M&K seems to have overcome any problems. You mentioned that you had thought of getting the active version, I would be interested in hearing what you think of them if you audition them.
Theresa is offline  
post #285 of 540 Old 02-10-2013, 06:33 AM
Member
 
ZOOM ZOOM's Avatar
 
Join Date: Nov 2011
Location: Iowa USA
Posts: 94
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 13
I may be a little late to the party, but this has been on my mind for the last week.
May I take us all back to thread #138 for a moment. Arnie had a link to an article which had John Atkinins, who edits Stereophile magazine telling a story about a double blind test. the test was between a high end tube amp, and a lesser expensive solid state amp.
Here is the direct quote from the article ("A man is in his 20's thinks he knows everything whereas a man in his 60's realizes he knows little. John, being in his 20's in 1978, agreed to join a Martin Colloms' ABX test of two different amplifiers. The test was between a tube and a solid-state amplifier. No one during this test heard any difference. Armed with this knowledge, John sold his then expensive amplifier and purchased a lesser expensive one at the time because he could not hear any difference in the ABX test. After six months John could no longer listen to music through the new, lesser expensive amplifier because it was not enjoyable as a music reproducer.").
What I got out of this article was a very good lesson in human emotion. Even though the two amps sounded the same, John was not pleased in having an amp that in his mind was inferior. (I personally would pick a tube amp over a solid state amp just for the ambiance.cool.gif) I will go out on a limb and state that most of the people entering these forums have a great joy for music and HT.wink.gif They also have preferences for speakers, amps, receivers, and yes even where to buy their cables. not based on science, but on personal preferences and emotion (My self as one example). On a human level this makes total sense. But from a science level? And we are in, AVS "Audio Video Science". biggrin.gif

http://www.enjoythemusic.com/hifi2005/atkinsonkrueger.html

ZOOM ZOOM is offline  
post #286 of 540 Old 02-10-2013, 07:00 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by Theresa View Post

Quote:
Originally Posted by kbarnes701 View Post

It was one of these engineers, who worked extensively for the BBC, who put me onto my M&K S150s when he first heard them years ago and was blown away by their sound. It took me some time to get around to buying these speakers but I have never regretted it and will probably never change them (unless I ever decide to go the active/powered route with their active MK counterparts). 

I was surprised to learn from you how good the M&K speakers are. The dual tweeter configuration complicates things but M&K seems to have overcome any problems. You mentioned that you had thought of getting the active version, I would be interested in hearing what you think of them if you audition them.

 

Triple tweeters in fact, Theresa!  Very hard to get right I would imagine, but that Ken Kreisel knows a thing or two... ;)

 

Here they are in all their pig-ugly glory:

 

 

 

The dealer where I bought the S150s has the active versions on demo. I have disciplined myself not to even think about an audition!  They cost about $4,500 *each* over here in the UK... if I ever move house and get a better room, with potential for a PJ and very big screen, I will push the boat out I think and buy three... 

kbarnes701 is offline  
post #287 of 540 Old 02-10-2013, 07:04 AM
AVS Special Member
 
Theresa's Avatar
 
Join Date: Nov 2010
Location: Minnesota
Posts: 2,430
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 50
Quote:
Originally Posted by kbarnes701 View Post

Triple tweeters in fact, Theresa!  Very hard to get right I would imagine, but that Ken Kreisel knows a thing or two... wink.gif

Here they are in all their pig-ugly glory:





The dealer where I bought the S150s has the active versions on demo. I have disciplined myself not to even think about an audition!  They cost about $4,500 *each* over here in the UK... if I ever move house and get a better room, with potential for a PJ and very big screen, I will push the boat out I think and buy three... 

Sorry, my memory of them is defective. Didn't realize the active ones are so expensive.
Theresa is offline  
post #288 of 540 Old 02-10-2013, 10:25 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,349
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 725 Post(s)
Liked: 1155
Quote:
Originally Posted by Theresa View Post

+1

What hasn't been mentioned is that for some tests "experienced" audiophiles were deliberately included in the sample and that they were no more likely to prefer high end models.

That was exactly the case with the http://webpages.charter.net/fryguy/Amp_Sound.pdf tests.

The listener group was almost entirely composed of experienced audiophiles and people with a professional involvement with audio.

I find it interesting that someone has already tried to turn this into a disadvantage of the test. Some serious thought seems to have been put into twisting it into something other than what it was.

The test results were looked at both individually and as group averages. We found I think it was 2 people who did really well, but this could be just luck. So they were tested again, individually. Both the second guess and the combination of their individual results from both tests turned out to show random guessing.

By the time we did these tests we had been doing audio DBTs for about 20 years. We knew from experience that no amount of polishing could change the results so we had no fear about biasing the tests towards positive outcomes every reasonable way that came to mind. We went out of our way to beat the bushes for experienced audiophiles who were positive that audible differences would exist, for example. These guys get to be a little hard to find in your peer group when you've been doing DBTs for 20 years! ;-) But we found them!
arnyk is offline  
post #289 of 540 Old 02-10-2013, 10:33 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,349
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 725 Post(s)
Liked: 1155
Quote:
Originally Posted by Mark Lem View Post

So concert hall, stadium etc processing aside, is one receivers DTS HD Audio the same as any other?

AFAIK the actual code that runs on the DSP to decode these sources is either provided by the licensing authority (DTS or Dolby) or is carefully checked out by them before they license the product to be sold. I will pose this question to the person who recently retired as the head scientist of DTS Inc. I think he's travelling the world right now so the answer might not come quickly.

Meanwhile you can see some of the mechanics of distributing this code here: http://www.analog.com/en/dsp-software/bf_dts_51_decoder/sw.html
arnyk is offline  
post #290 of 540 Old 02-10-2013, 10:51 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,349
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 725 Post(s)
Liked: 1155
Quote:
Originally Posted by runnin' View Post

Great post, in the real world people need to quit taking themselves so seriously and examine how their love for audio became some kind of crusade.

That is a question that applies equally to those who are on the pro and con side of the issue.

On many forums even the hint of DBTs exposes one mentioning them to a great deal of abuse. The abuse tends to be polarizing.
arnyk is offline  
post #291 of 540 Old 02-10-2013, 11:05 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,349
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 725 Post(s)
Liked: 1155
Quote:
Originally Posted by Kilian.ca View Post

Lots of small-scaled, amateurish and negative trials don't add up to prove anything conclusive, only suggest a trend.

Right on the face of it, the above post shows that its author is probably very strongly polarized against DBTs. Note that the post mentions only negative trials.

The phase "small-scaled, amateurish" applies best to the typical kind of audiophile test we seem posted all of the time. Usually there is only one listener, one or just a few trials, and no experimental controls at all. How can one call this sort of thing anything but amateurish?
Quote:
For professionally conducted studies that are negative or a mixture of positives and negatives or inconclusive there are methods (meta-analysis) to evaluate them as a larger group to look for hidden positives.

Yet another example of bias. The obvious false presumption is that the people who have done the DBTs in the past didn't use the best available means of analysis to perform their analysis. Talk about prejudging based on zero evidence!
Quote:
Sample size IS important in statistics. Small differences that are statistically non-significant might become significant in a larger sample size, because the sample size is factored in determining the p value.

If memory serves, the tests I linked involved about 400 trials. Statistically that is quite a bit of data. We have done hundreds of more tests and thus have real world experience relating to thousands of trials. In essence we re-confirmed the mean convergence theorem. The more trials we gathered, the more our results converged to random guessing.
Quote:
You only have to read some clinical trials to see what sort of numbers (hundreds and thousands) are involved and the differences are often very small, not 80% vs. 20%, this large magnitude of difference doesn't happen. Usually when the sample size is down to 10 or 20 very very few studies have statistically significant results. Also note that 20 people each doing 10 trials isn't the same as 200 doing 1 trial in statistics.

I would liken the above to the story about the Little Engine That Could. It seems clear that someone thinks that all we would have to do is enough trials, and eventually positive results would present themselves. Two comments: One is that we have quite a bit of evidence that more trials converges to random guessing, and the other is that even if we could obtain statistically signficiant results after doing a million trials, who with any common sense would spend a dime or even bother to cross the street to obtain a benefit that takes a million trials to prove?

Correct me if I'm wrong but isn't it true that the true believers in everything sounding different claim that the differences are obvious?

What is obvious - something that it takes a million trials to prove? I don't think so!

So if you want to talk about amateurish tests with a small number of trials, take those shoes to where they actually fit - the people who claim that everything sounds different.
arnyk is offline  
post #292 of 540 Old 02-10-2013, 03:04 PM
 
cybrsage's Avatar
 
Join Date: May 2007
Location: Harrisburg, PA
Posts: 8,074
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 147
Quote:
Originally Posted by JD in NJ View Post

Well that doesn't sound like a very good description of the arguments I've read and occasionally participated in, but if that's your perception I have one question: Are the best theists those who are happy for atheists who have found satisfaction in their lack of belief?

Depends on the belief system of the theist. There are some belief systems which state non-believers are in peril. Those who belong to those belief systems have a moral obligation to at least inform the non-believers of their peril - much in the same way you would inform someone about to cross the street that they are about to step into the path of a speeding car. In that instance, the best theists are those who inform the non-believer but stop there, for the choice is up to the non-believer and cannot be forced upon that person. Not believing brings harm to the non-believer (from the point of view of the theist), which will sadden the believer. In the case of an atheist, they believe there is nothing to believe in (when speaking of a soft atheist - a hard atheist believes there are no gods which is a belief system in and of itself). As such, there is no harm in believing if said belief brings happiness to the person.

To put it into silly mode:

If I believed that you must eat grapes or you will die, I should inform you of my belief (unless I am hateful and want you to die). I should not force you to eat grapes, and I will be saddened if you choose to not eat grapes and therefor die. If you do not believe eating grapes will do anything, you can tell me this, but if I choose to eat them and eating them makes me happy, then you should be happy that I found happiness by eating grapes (unless you are hateful and want to see me unhappy).
Quote:
Originally Posted by Theresa View Post

Isn't that the truth! Non-theists are expected to accept theists but the inverse is not true.

The atheist should be more inclusive since their view is that belief or non-belief makes no difference. Since it makes no difference, why bother to try to change anyone's mind? To the theist (in the religions where peril awaits the non-believer), non-belief is actually harmful to the non-believer. If the person truly believes their religion is true, they have a moral obligation to at least attempt to convince the non-believer they are making a horrible mistake. It is the right thing to do, if the believer honestly believes their religion is true.

It is like if someone asks if you want the chicken sandwich or the hamburger and you really do not care which you get - but you know the person offering you the choice likes hamburgers better. You should choose the chicken sandwich, since you do not care which you get, and let the other person have what makes them happier. The choice does not matter to you but actually does to the other person.
cybrsage is offline  
post #293 of 540 Old 02-10-2013, 04:39 PM
Advanced Member
 
PeterK's Avatar
 
Join Date: Jun 2002
Location: The very wet, humid, muggy South !!
Posts: 544
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 15 Post(s)
Liked: 50
I know I will never get the opportunity to participate in a true DBT of products I am considering, heck can't even find competitive products in stores any more. This thread has my head spinning. I tend to think less than 1% of the posters here ever get to participate in a well done test. Makes me think I should quit considering upgrading to the new Marantz 8801 and just get a Blowes tabletop unit!! eek.gif Kidding!

When I upgraded from an old JVC receiver with synthetic 4 channel surround, no sub, no center channel, no decoding and went to the opposite extreme of the top of the line Pioneer Elite $4200 receiver, I can say without any official tests that the sound was truly night and day better! Of course there were probably 1000s of improvements in that 15 year time span between products. When comparing current competitively priced products, any differences should be extremely small or imperceptible. Just wish I could witness the comparative testing myself.
PeterK is online now  
post #294 of 540 Old 02-11-2013, 04:23 AM
AVS Special Member
 
MUDCAT45's Avatar
 
Join Date: Mar 2010
Posts: 1,410
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 124 Post(s)
Liked: 167
Quote:
Originally Posted by arnyk View Post

Right on the face of it, the above post shows that its author is probably very strongly polarized against DBTs. Note that the post mentions only negative trials.

The phase "small-scaled, amateurish" applies best to the typical kind of audiophile test we seem posted all of the time. Usually there is only one listener, one or just a few trials, and no experimental controls at all. How can one call this sort of thing anything but amateurish?
Yet another example of bias. The obvious false presumption is that the people who have done the DBTs in the past didn't use the best available means of analysis to perform their analysis. Talk about prejudging based on zero evidence!
If memory serves, the tests I linked involved about 400 trials. Statistically that is quite a bit of data. We have done hundreds of more tests and thus have real world experience relating to thousands of trials. In essence we re-confirmed the mean convergence theorem. The more trials we gathered, the more our results converged to random guessing.

I would liken the above to the story about the Little Engine That Could. It seems clear that someone thinks that all we would have to do is enough trials, and eventually positive results would present themselves. Two comments: One is that we have quite a bit of evidence that more trials converges to random guessing, and the other is that even if we could obtain statistically signficiant results after doing a million trials, who with any common sense would spend a dime or even bother to cross the street to obtain a benefit that takes a million trials to prove?

Correct me if I'm wrong but isn't it true that the true believers in everything sounding different claim that the differences are obvious?

What is obvious - something that it takes a million trials to prove? I don't think so!

So if you want to talk about amateurish tests with a small number of trials, take those shoes to where they actually fit - the people who claim that everything sounds different.

Forgive me for not reading every detail. In the ABX set up what measurements are taken other than level matching?
MUDCAT45 is online now  
post #295 of 540 Old 02-11-2013, 04:44 AM
AVS Addicted Member
 
kbarnes701's Avatar
 
Join Date: Jan 2008
Location: Main Listening Positon
Posts: 17,448
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 1143 Post(s)
Liked: 1550
Quote:
Originally Posted by arnyk View Post

Correct me if I'm wrong but isn't it true that the true believers in everything sounding different claim that the differences are obvious?
 

 

"Night and day" differences - that's a phrase I have read 100s of times on AVS when a subjective test (sample size: 1) has been made. "Even my girlfriend/wife/kids could easily hear the difference" is another. Note "easily hear". On more than one occasion I have read "Even my wife/GF could hear the difference from another room". "You would have have tin ears/be deaf not to hear the difference..."  And so on and on...

 

So yes, the alleged differences are, so they say, easily heard. One would therefore expect them to be just as easily heard in a blind ABX test. But they never are. And then people pop up and suggest the ABX tests are 'insufficient' or 'amateurish' or 'inconclusive' or that we need to make many more tests and then, suddenly, the results will swing away from random 50-50 chance. Meanwhile, on their sample size of 1, with a trials number of 2 or 3, in random circumstances using no proper controls etc etc, they ask us to believe them when they say the differences are clear. Wow.

kbarnes701 is offline  
post #296 of 540 Old 02-11-2013, 04:52 AM
AVS Special Member
 
MUDCAT45's Avatar
 
Join Date: Mar 2010
Posts: 1,410
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 124 Post(s)
Liked: 167
Quote:
Originally Posted by kbarnes701 View Post

"Night and day" differences - that's a phrase I have read 100s of times on AVS when a subjective test (sample size: 1) has been made. "Even my girlfriend/wife/kids could easily hear the difference" is another. Note "easily hear". On more than one occasion I have read "Even my wife/GF could hear the difference from another room". "You would have have tin ears/be deaf not to hear the difference..."  And so on and on...

So yes, the alleged differences are, so they say, easily heard. One would therefore expect them to be just as easily heard in a blind ABX test. But they never are. And then people pop up and suggest the ABX tests are 'insufficient' or 'amateurish' or 'inconclusive' or that we need to make many more tests and then, suddenly, the results will swing away from random 50-50 chance. Meanwhile, on their sample size of 1, with a trials number of 2 or 3, in random circumstances using no proper controls etc etc, they ask us to believe them when they say the differences are clear. Wow.

Very possible. Doesn't the average Joe always crank a new amp to the max when newly installed. Now compare that to the old one played at normal volumes.smile.gif
MUDCAT45 is online now  
post #297 of 540 Old 02-11-2013, 05:21 AM
AVS Special Member
 
gurkey's Avatar
 
Join Date: Aug 2007
Posts: 2,751
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 22
Especially, when the same sample shows "night and day" differences until you tell them, that they listened to the same thing twice eek.gifbiggrin.gif
gurkey is offline  
post #298 of 540 Old 02-11-2013, 05:56 AM
AVS Addicted Member
 
arnyk's Avatar
 
Join Date: Oct 2002
Location: Grosse Pointe Woods, MI
Posts: 14,349
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 725 Post(s)
Liked: 1155
Quote:
Originally Posted by MUDCAT45 View Post

In the ABX set up what measurements are taken other than level matching?

Level matching at a reasonable range of frequencies is essential.

Often a SPL meter is used to document the actual loudness that the listeners experienced during the test.

It is often a good idea to run a few bench tests on the equipment being tested to make sure that it is meeting origional specs.

I have personally had SS equipment that underwent audible degradation in 2-3 years.

Some tubed power amps are notorious for falling out of spec within a year. One example that comes to mind is just about any of the Dyna power amps, if MacIntosh power amp clinics had any integrity at all.
arnyk is offline  
post #299 of 540 Old 02-11-2013, 06:21 AM
AVS Special Member
 
MUDCAT45's Avatar
 
Join Date: Mar 2010
Posts: 1,410
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 124 Post(s)
Liked: 167
Quote:
Originally Posted by arnyk View Post

Level matching at a reasonable range of frequencies is essential.

Is this done with EQ?

Often a SPL meter is used to document the actual loudness that the listeners experienced during the test.

It is often a good idea to run a few bench tests on the equipment being tested to make sure that it is meeting origional specs.

I have personally had SS equipment that underwent audible degradation in 2-3 years.

Some tubed power amps are notorious for falling out of spec within a year. One example that comes to mind is just about any of the Dyna power amps, if MacIntosh power amp clinics had any integrity at all.
MUDCAT45 is online now  
post #300 of 540 Old 02-11-2013, 11:28 AM
 
Ricsim78's Avatar
 
Join Date: Jan 2008
Posts: 382
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
I did a little test to see if they truly DO sound the same.

I do not care about test data, DBT, etc. What is the final decision on if something sounds good to you or not? Your hearing and your ears. Tests are fun and all and interesting, but to blindly follow the (in my opinion, lack of) results is folly. The thing that set me off about this thread is when you do not agree with people like Arny, automatically you know nothing about audio or whatever little stab they want to take.

Ok, I go and buy a receiver and what will I do?

A) Hook it up, set it up, run the calibration, tweak it, use it, tweak it more, then enjoy it.
B) Do DBT and automatically conclude it sounds the same as every every receiver I bought before it.

I will let you guess which one we would ALL be doing. I have done little experiments here for the sake of this thread, so over the course of the last few days I have setup I think everyone (well, everyone that has common sense) would agree is a very fair test. I have 4 Kef Q100 speakers, a Denon 1613, and my Pioneer Elite SC-65. I put the Denon and Pioneer right next to each other. I set 2 Kefs right next to each other, one hooked into the Pioneer, the other into the Denon on both sides. Same speakers. I setup my Sound card to output 4 channel sound, one outputting into the Denon, the other into the Pioneer. The receivers are sitting right next to each other. I set each speaker so it is at 75 DB. I did not use a subwoofer for this demo.

I can run both receivers at the same time, or mute 1 while the other is playing. The sound is set to a volume not loud, but matched just about perfectly. So let's go over the parameters of this test.
  • Same speakers both receivers set right next to each other (2 identical speakers setup side to side on the left and right side) The speaker output order (from left to right) Denon, Pioneer, Denon, Pioneer. Measured 6.5 feet separation
  • All speakers set to 75 db with a Radioshack meter
  • Same output device (my computer with Auzentech Prelude soundcard setup to output to both receivers the same signal (quality)
  • Volume matched (The Denon had to be cranked up more to get to the same volume)
  • Ability to quickly mute one receiver so audio memory is no issue.
  • Me, girlfriend, and my friend Jason participated in this (so no one can say it's JUST me)
  • The volume level was set at comfortable levels, no where near reference
  • Same speakers, same sound source, same speaker wire (14 ga. RCA), same RCA to miniplug adapter for the soundcard output. Obviously level matched as the sound just is mirrored in all 4 speakers.
  • No EQ, room correction, MP3 Restorer, or any audio enhancements engaged (we did turn on the MP3 enhancers later just to see which one did better)
  • I have no personal bias towards either brand, I love Denon/Marantz just as much as Pioneer and they remain my favorite companies for receivers. Girlfriend has no bias, my friend has owned a Pioneer Elite and a ton of vintage receivers and amps but he also likes Denon.

We played some Dream Theater, The Knife, Genesis, Metallica, Rush, Van Halen, and quite a few other songs. Quickly muting one and then doing the same on the other. We tried direct mode, stereo mode.

We all agreed they sounded NOTHING alike. I repeat, nothing

Denon 1613: Laid back, not as dynamic, not as good separation of instruments
Pioneer Elite SC-65: Crisper, more forward, more airy, more revealing. More unforgiving of bad quality MP3 samples.

We then played Lord of the Rings: The Return of the King (Blu-ray) through both. It was harder to tell the difference with a movie, but we all agreed on the following.

Denon 1613: More laid back, slightly muddier
Pioneer Elite: SC-65: More forward, more dynamic, more airy. Battle scenes seemed to be more immersive..

Now to be fair, they both sounded awesome but the Pioneer walks all over the Denon in overall sound quality. And there is a huge, immediate difference.

So "ABX test pushers" and naysayers, explain to me the results of this tests. Do not just say, "your testing method is flawed" because this is more useful than a double blind test. Muting one receiver to listen to the other immediately takes away "random guessing" and audio memory (or lack thereof).

This is a true, fair, real world test. Here are some other observations:
  • Due to the fact we used Analog output on the sound card, we bypassed the DAC of the receivers. Using the DAC of the receiver would have likely made them sound even more different.
  • The "MP3 Restorer" function of the Pioneer is way better than the Denon version.
  • If they truly sounded the same, we would have heard the same thing when we switched to the Denon as opposed to the Pioneer. The difference was huge.
  • The Pioneer does have D3 amps, but if they all sound the same then there should be no discernible difference. Considering people claim in ABX tests people had trouble telling a tube amp from a solid state amp, this should not be an issue.

As I said, receivers do not sound the same and this is proof enough for me. If you want to go by "audio guessing" (like ABX tests) that is on you but as I said, I live and hear in the real world. So does everyone else here. Industry standard tests have nothing to do with what you hear.

Edit: I plan to repeat this test with my friend's Denon 3313 so no one can say it's a low cost Denon vs expensive Pioneer. It was not just the overall sound quality, it was the sound signature which was totally different.
Ricsim78 is offline  
Reply Receivers, Amps, and Processors

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off