AVS/AIX High-Resolution Audio Test: The Results So Far - Page 2 - AVS Forum
Forum Jump: 
 147Likes
Reply
 
Thread Tools
post #31 of 457 Old 07-29-2014, 02:15 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,120
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 92 Post(s)
Liked: 303
I must respectfully disagree; I have already received a goodly number of responses with incorrect identifications. Of course, I can't know how many people got it wrong (or right, for that matter) if they don't report their findings, and of course, if someone doesn't even try, they're not going to submit their findings.

Which is why your data is meaningless. You haven't even been able to collect all of it. Without knowing how many people tried and failed, any conclusions drawn will be baseless.

I hope people had fun. There wasn't any other point.
tubetwister likes this.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
Sponsored Links
Advertisement
 
post #32 of 457 Old 07-29-2014, 02:18 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,120
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 92 Post(s)
Liked: 303
I want to know exactly what qualifies a system as "Hi-Res" capable?

And why. If we don't know what people are hearing (and we don't), how can we possibly determine that one system is reproducing what is necessary to determine a difference and another isn't? This is just an exercise in selection bias, on top of the original exercise in self-selection bias. You couldn't make this more bogus if you tried.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #33 of 457 Old 07-29-2014, 04:25 PM - Thread Starter
AVS Special Member
 
Scott Wilkinson's Avatar
 
Join Date: Sep 2001
Location: Burbank, CA
Posts: 1,228
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 249 Post(s)
Liked: 976
Quote:
Originally Posted by Cal68 View Post
Hi Scott

I just want to be sure that I understand your definition of an HRA system. If I download the music files to a thumb-drive and play them through the USB port in my Oppo 105D, will that constitute an HRA system? If the answer is yes, then I'm going to participate in this test and see if I can differentiate between HRA and non-HRA files! Thanks.
Quote:
Originally Posted by Scott Wilkinson View Post
The system you specify certainly qualifies as HRA, but only as a front end. You also need to consider the preamp, power amp, and speakers or headphones; they all need to be able to pass frequencies well above 20 kHz and a dynamic range beyond 93 dB. If any component in the system can't do that, it's a bottleneck that renders the system as a whole non-HRA.

For example, the Oppo's analog audio outputs are spec'd out to 96 kHz at -1.5 dB, while the headphone output is spec'd to 20 kHz, ±0.3 dB into 300 ohms. I suspect the headphone output might be able to go much higher, but I don't know, so plugging a pair of HRA-capable headphones into the Oppo's headphone output might or might not reproduce the ultrasonics in the native high-res files. I'll contact Oppo and see what they have to say about this.
I just heard from Jason Liao at Oppo, who tested the headphone output of the BDP-105. Here are his comments and results:

"This was using a BDP-105 running the latest firmware 75-0515 version. The test source signal was from an Audio Precision APx-BD1 Blu-ray, 6ch 192kHz PCM signal, 74-point 1/6 oct stepped frequency sweep at -20dB level. As you can see from the result the headphone output of the BDP-105 is able to reach 80kHz flat, and 90kHz with very less than 1 dB roll off probably due to approaching the Nyquist frequency. The test loads are 300-Ohm resistors."



So the BDP-105 headphone output definitely qualifies as HRA.

Scott Wilkinson
AVS Editor
Scott Wilkinson is offline  
post #34 of 457 Old 07-29-2014, 05:50 PM
Member
 
epepin's Avatar
 
Join Date: Jun 2014
Posts: 33
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 12 Post(s)
Liked: 11
HRA system examples?

Scott,
When looking at the specifications of a pre-amp/amp/avr for the appropriate frequency range, what numbers should we be looking at?
This Marantz (http://us.marantz.com/us/Products/Pa...oductId=PM8005) gives:
THD: 0.02% (20Hz – 20kHz, 2 channel driven, 8 ohms load)
and
Frequency response: 5Hz – 100kHz ±3dB (CD, 1W, 8 ohms load)
This Yamaha (http://usa.yamaha.com/products/audio..._u/?mode=model) gives
Minimum RMS Output Power: (8 ohms, 20 Hz-20 kHz, 0.02% THD) 90 W + 90 W / (4 ohms, 20 Hz-20 kHz, 0.02% THD) 150 W + 150 W
and
Frequency Response: CD, etc. to speaker out, Flat position 5 Hz-100 kHz +0 dB / -3 dB, CD, etc. to speaker out, Flat position 20 Hz-20 kHz +0 dB / -0.3 dB


Also, could you post some of the HRA systems provided in the responses so far?
epepin is offline  
post #35 of 457 Old 07-29-2014, 06:29 PM
AVS Special Member
 
[Irishman]'s Avatar
 
Join Date: Nov 2007
Posts: 1,416
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 26 Post(s)
Liked: 59
Quote:
Originally Posted by mcnarus View Post
I must respectfully disagree; I have already received a goodly number of responses with incorrect identifications. Of course, I can't know how many people got it wrong (or right, for that matter) if they don't report their findings, and of course, if someone doesn't even try, they're not going to submit their findings.

Which is why your data is meaningless. You haven't even been able to collect all of it. Without knowing how many people tried and failed, any conclusions drawn will be baseless.

I hope people had fun. There wasn't any other point.
So, your logic is that - because everyone on AVS didn't submit data (myself included) - the conclusions drawn from the data that was gathered are baseless?

I don't think you get it.
amirm likes this.

COMING SOONFinding the Ark of the Covenant by Brian Roberts, in the iBook Store on iTunes, a new investigation into the Hebrew’s Most Sacred Relic!
[Irishman] is offline  
post #36 of 457 Old 07-29-2014, 06:31 PM
AVS Special Member
 
JHAz's Avatar
 
Join Date: Mar 2009
Posts: 3,944
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 88 Post(s)
Liked: 154
Quote:
Originally Posted by mcnarus View Post
I want to know exactly what qualifies a system as "Hi-Res" capable?

And why. If we don't know what people are hearing (and we don't), how can we possibly determine that one system is reproducing what is necessary to determine a difference and another isn't? This is just an exercise in selection bias, on top of the original exercise in self-selection bias. You couldn't make this more bogus if you tried.
While it looks generically like the kind of thing you'd only try to publish if you were an "expert" in litigation, it wasn't proposed as a scientifically valid experiment. But idk how bias causes accurate assessment in what seems like a blind test. I don't have transducers that go supersonic but I will check out the files at some point for fun.

If there are folks who accurately identify the files, blind
the question is why. Is there something that makes the higher res files distinguishable? Is it the higher res, or something else? Fun at least to speculate
JustaSheep and isa like this.
JHAz is offline  
post #37 of 457 Old 07-29-2014, 06:43 PM - Thread Starter
AVS Special Member
 
Scott Wilkinson's Avatar
 
Join Date: Sep 2001
Location: Burbank, CA
Posts: 1,228
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 249 Post(s)
Liked: 976
Quote:
Originally Posted by JHAz View Post
While it looks generically like the kind of thing you'd only try to publish if you were an "expert" in litigation, it wasn't proposed as a scientifically valid experiment. But idk how bias causes accurate assessment in what seems like a blind test. I don't have transducers that go supersonic but I will check out the files at some point for fun.

If there are folks who accurately identify the files, blind
the question is why. Is there something that makes the higher res files distinguishable? Is it the higher res, or something else? Fun at least to speculate
That's exactly the goal and spirit of this test—to have some fun while investigating the notion of high-res audio and to stimulate some thought and discussion about it, nothing more.
isa likes this.

Scott Wilkinson
AVS Editor
Scott Wilkinson is offline  
post #38 of 457 Old 07-29-2014, 07:11 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,120
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 92 Post(s)
Liked: 303
So, your logic is that - because everyone on AVS didn't submit data (myself included) - the conclusions drawn from the data that was gathered are baseless?

No, my point is that because everyone who downloaded the files didn't submit data, we have a skewed data set. In particular, there could be a lot of people who listened to the files, decided that they couldn't tell the difference, and just didn't go further. Had they all reported their guesses, we might have a far higher percentage of wrong answers than we do. And since we don't know how much higher, we can't draw any conclusions.

A good test would require you to recruit your test subjects in advance and require all of them to complete the test. This was not a good test.
rogergraham likes this.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #39 of 457 Old 07-29-2014, 07:17 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,120
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 92 Post(s)
Liked: 303
While it looks generically like the kind of thing you'd only try to publish if you were an "expert" in litigation, it wasn't proposed as a scientifically valid experiment.

Then it should not be interpreted as a scientifically valid experiment. But it will be, because there are people here who will stretch any truth to sell product. I presume I don't need to mention any names.

If there are folks who accurately identify the files, blind the question is why.

Based on the data we have, we cannot say that they weren't just lucky guessers.

Is there something that makes the higher res files distinguishable? Is it the higher res, or something else? Fun at least to speculate

Have all the fun you want. But you could have done the speculating just as well before the test.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #40 of 457 Old 07-29-2014, 08:43 PM
Advanced Member
 
RLBURNSIDE's Avatar
 
Join Date: Jan 2007
Posts: 784
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 54 Post(s)
Liked: 133
Knowing that the results are not scientifically valid, why do it at all? Seems like a waste of time. Maybe as a prelude to get iron out a methodology prior to undertaking an actual double blind study, sure. I'm very interested in real results, not anecdotes from "golden ears" on the internet.

People lie about all kinds of things, especially how good their hearing is (on websites like these), that's beyond a doubt.

Last edited by RLBURNSIDE; 07-30-2014 at 09:18 AM.
RLBURNSIDE is offline  
post #41 of 457 Old 07-29-2014, 09:09 PM
AVS Special Member
 
NorthSky's Avatar
 
Join Date: Jan 2013
Location: Canada - West Island: Vancouver, South Direction: Go East
Posts: 2,488
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 1091 Post(s)
Liked: 398
Blues is on, the r.a.d.i.o.

Bests, ~ Robert § (Bob)

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
NorthSky is offline  
post #42 of 457 Old 07-30-2014, 03:39 AM
Advanced Member
 
Cal68's Avatar
 
Join Date: Jan 2008
Location: Maryland
Posts: 549
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 7 Post(s)
Liked: 49
Quote:
Originally Posted by Scott Wilkinson View Post
The system you specify certainly qualifies as HRA, but only as a front end. You also need to consider the preamp, power amp, and speakers or headphones; they all need to be able to pass frequencies well above 20 kHz and a dynamic range beyond 93 dB. If any component in the system can't do that, it's a bottleneck that renders the system as a whole non-HRA.

For example, the Oppo's analog audio outputs are spec'd out to 96 kHz at -1.5 dB, while the headphone output is spec'd to 20 kHz, ±0.3 dB into 300 ohms. I suspect the headphone output might be able to go much higher, but I don't know, so plugging a pair of HRA-capable headphones into the Oppo's headphone output might or might not reproduce the ultrasonics in the native high-res files. I'll contact Oppo and see what they have to say about this.
Hi Scott

Thanks for your response to my question. I do not have an Oppo headphone to lug into the Oppo 105D, but I do have the Oppo hooked up to an Integra 80.2 pre-pro, and from there through a McIntosh 303 amp to B&W 803D speakers. I don't know if this system qualifies as a HRA system.

My only reason for asking the question was to determine if I could participate in the test if I was an owner of an HRA system. I was not seeking to stir the pot in anyway!

Cal68
Cal68 is offline  
post #43 of 457 Old 07-30-2014, 10:27 AM
AVS Special Member
 
Speakerphile's Avatar
 
Join Date: Mar 2010
Posts: 1,952
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 14 Post(s)
Liked: 161
Quote:
Originally Posted by RLBURNSIDE View Post
Knowing that the results are not scientifically valid, why do it at all?

For the hell of it. Learn to have a little fun and don't take everything so seriously...

Though I am an employee of Magnolia Home Theater, the views and opinions I express on this forum are those of my own and not my employer.
Speakerphile is offline  
post #44 of 457 Old 07-30-2014, 11:45 AM
Senior Member
 
Fishtank's Avatar
 
Join Date: Aug 2006
Posts: 276
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 20
Quote:
Originally Posted by Speakerphile View Post
For the hell of it. Learn to have a little fun and don't take everything so seriously...
yeah what this guy said...thanks for the test and effort provided Scott. A lot of us enjoy this hobby at a lighter level and can appreciate opinion at some level for what it is.

I look forward to more anyhow. I definitely prefer open... optimistic approach to new technology. Where would humans be if we always discounted new idea's based on current knowledge or paradigms.

Fishtank is offline  
post #45 of 457 Old 07-30-2014, 12:14 PM
Senior Member
 
ien2's Avatar
 
Join Date: Mar 2013
Location: Oahu
Posts: 205
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 3 Post(s)
Liked: 46
@mcnarus and RLBurnside:

Science (up and down every single field) is completely littered with tests/experiments that are there simply to see if there is a trend worth pursuing in a more rigorous setting. This doesn't give meaningless results at all, the proposal was to see if people could tell the difference and given the results that he received, the answer is that there is enough evidence to warrant further investigation and mostly likely the answer is yes. It doesn't matter how many non-reporting people there is, this type of experiment the way it was setup only requires positive answers and doesn't even need to get out of the margin of error (yes it would be nice, but he's receiving explanations along with the answers which also count for something).

It'd be nice to able to get the breakdown of the general population on whether or not they can and at what the percentage is at various resolution levels but that wasn't the point of this.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
ien2 is offline  
post #46 of 457 Old 07-30-2014, 12:27 PM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,120
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 92 Post(s)
Liked: 303
Science (up and down every single field) is completely littered with tests/experiments that are there simply to see if there is a trend worth pursuing in a more rigorous setting.

True, but this particular "experiment" is insufficient to demonstrate any "trend" worth pursuing. At best, you've demonstrated that something in the way these files were coded produced a difference in the audible range that was sufficient to be audible. Big deal. And that's only if you misinterpret the results.

This doesn't give meaningless results at all, the proposal was to see if people could tell the difference and given the results that he received, the answer is that there is enough evidence to warrant further investigation and mostly likely the answer is yes.

No, there really isn't. You can't even say for sure that people weren't just lucky guessers. What scientist is going to stop for a second to think about this result? It is the product of wishful thinking, nothing more. A scientist needs something more than that to go on.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #47 of 457 Old 07-30-2014, 12:49 PM
AVS Special Member
 
eljr's Avatar
 
Join Date: Aug 2006
Location: Futuristic London
Posts: 4,143
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 181 Post(s)
Liked: 1039
Quote:
Originally Posted by mcnarus View Post
there are people here who will stretch any truth... I presume I don't need to mention any names.

.

if you are at a mirror, it should be obvious

What you got back home, little sister, to play your fuzzy warbles on? I bet you got little save pitiful, portable picnic players. Come with uncle and hear all proper! Hear angel trumpets and devil trombones. You are invited.
eljr is offline  
post #48 of 457 Old 07-30-2014, 01:56 PM
Senior Member
 
ien2's Avatar
 
Join Date: Mar 2013
Location: Oahu
Posts: 205
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 3 Post(s)
Liked: 46
Quote:
Originally Posted by mcnarus View Post
Science (up and down every single field) is completely littered with tests/experiments that are there simply to see if there is a trend worth pursuing in a more rigorous setting.

True, but this particular "experiment" is insufficient to demonstrate any "trend" worth pursuing. At best, you've demonstrated that something in the way these files were coded produced a difference in the audible range that was sufficient to be audible. Big deal. And that's only if you misinterpret the results.

This doesn't give meaningless results at all, the proposal was to see if people could tell the difference and given the results that he received, the answer is that there is enough evidence to warrant further investigation and mostly likely the answer is yes.

No, there really isn't. You can't even say for sure that people weren't just lucky guessers. What scientist is going to stop for a second to think about this result? It is the product of wishful thinking, nothing more. A scientist needs something more than that to go on.

Again, you missed the point.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
ien2 is offline  
post #49 of 457 Old 07-30-2014, 02:15 PM
AVS Special Member
 
Frank Derks's Avatar
 
Join Date: Jul 2002
Location: Region A,B,C
Posts: 1,891
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 98 Post(s)
Liked: 89
Outcome so far: 15 positives and 15 negatives.


50/50 so nothing conclusive.
tubetwister likes this.
Frank Derks is offline  
post #50 of 457 Old 07-30-2014, 02:24 PM
AVS Special Member
 
tubetwister's Avatar
 
Join Date: Feb 2013
Location: Sacramento delta N. Cal. US Don't trust any air I can't see ☺
Posts: 2,935
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 522 Post(s)
Liked: 362
Quote:
Originally Posted by Scott Wilkinson View Post
I just heard from Jason Liao at Oppo, who tested the headphone output of the BDP-105. Here are his comments and results:

"This was using a BDP-105 running the latest firmware 75-0515 version. The test source signal was from an Audio Precision APx-BD1 Blu-ray, 6ch 192kHz PCM signal, 74-point 1/6 oct stepped frequency sweep at -20dB level. As you can see from the result the headphone output of the BDP-105 is able to reach 80kHz flat, and 90kHz with very less than 1 dB roll off probably due to approaching the Nyquist frequency. The test loads are 300-Ohm resistors."



So the BDP-105 headphone output definitely qualifies as HRA.


As we know no dynamic headphone diaphragm/element ( mine go to 27K way more than I can hear ) or ordinary conventional speaker will resolve 80kHz flat .

I fail to see any practical or useful result of 80kHz resolution there other than maybe bragging rights (just because it can )or maybe introducing some nonlinear distortion down the line.

ofc I think SACD claimed at least 50kHz flat I couldn't hear also on my made for SACD Sony ES system or IMO any difference on equally well recorded and mastered CDR vs SACD .

TBH I expect much the same result from 192 or 96K downloads , so far on my cursory sighted evaluations of same that is still my opinion anyway others may perceive other results and are of course welcome to present their opinions also.

That being said I'm sure the Oppos are very fine player's in their own right (longstanding very good reputation ) and probably an excellent choice for several reasons including disc playback or whatever for many people but resolving 80kHz /90kHz flat or otherwise while being impressive into a resistive load (because it can ) IMO isn't one of them .

In any event the listening tests here taken at face value as intended ( meaning not seriously for scientific purposes ) might be fun for some including me !

Cheers and carry on men and women !

Hires Music formats ..............."Why does it sound like a CD ?" ............. can we make it louder "?
"The wireless music box has no commercial value. Who would pay for a message sent to nobody in particular?"
- David Sarnoff's associates at RCA the 1920's -

Last edited by tubetwister; 08-02-2014 at 06:34 PM.
tubetwister is offline  
post #51 of 457 Old 07-30-2014, 02:52 PM
AVS Special Member
 
tubetwister's Avatar
 
Join Date: Feb 2013
Location: Sacramento delta N. Cal. US Don't trust any air I can't see ☺
Posts: 2,935
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 522 Post(s)
Liked: 362
Quote:
Originally Posted by Frank Derks View Post
Outcome so far: 15 positives and 15 negatives.


50/50 so nothing conclusive.
FWIW Not an unexpected result from honest listeners or testing IMO anyway
RichB likes this.

Hires Music formats ..............."Why does it sound like a CD ?" ............. can we make it louder "?
"The wireless music box has no commercial value. Who would pay for a message sent to nobody in particular?"
- David Sarnoff's associates at RCA the 1920's -
tubetwister is offline  
post #52 of 457 Old 07-30-2014, 06:48 PM
Newbie
 
sinbon's Avatar
 
Join Date: Jul 2007
Posts: 9
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Looking forward to hearing how I did and relistening to the tracks! Not all of my system is HRA capable so should be interesting.
sinbon is offline  
post #53 of 457 Old 07-30-2014, 09:30 PM
AVS Special Member
 
JHAz's Avatar
 
Join Date: Mar 2009
Posts: 3,944
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 88 Post(s)
Liked: 154
Quote:
Originally Posted by tubetwister View Post

ofc that doesn't keep the industry in general , pundits (present company excepted of course) or re sellers from making spurious claims unless there is regulatory intervention that is perhaps long overdue much like that wich has occurred in many other industries to the benefit of consumers .
I wouldn't expect further government action in this field until a decade or so after the dietary supplement market is forced to meet fda type standards to prove efficacy. Iow, not in our lifetimes.

The value of this undertaking is probably much more personal than broadly meaningful. Fifteen reported perfectly right responses and fifteen perfectly wrong sounds like the tails of a normal distribution to me, potentially.

Statistically the question is whether either result occurs significantly more often than random chance would predict. Of course with self reported results we cannot know. Nobody should think the sky is falling if they flip a coin and get three or six or even ten straight heads. If twenty out of fifty flippers got ten straight heads, you would start to wonder if the coin was fixed. But 20 out of a thousand might be within a reasonable statistical range.

But this never was intended to provide viable results and its kinda fun. Maybe tomorrow I'll be able to try em my ownself.
JHAz is offline  
post #54 of 457 Old 07-31-2014, 04:23 AM
Advanced Member
 
mightyhuhn's Avatar
 
Join Date: Nov 2013
Posts: 611
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 253 Post(s)
Liked: 89
Quote:
Originally Posted by mcnarus View Post
So, your logic is that - because everyone on AVS didn't submit data (myself included) - the conclusions drawn from the data that was gathered are baseless?

No, my point is that because everyone who downloaded the files didn't submit data, we have a skewed data set. In particular, there could be a lot of people who listened to the files, decided that they couldn't tell the difference, and just didn't go further. Had they all reported their guesses, we might have a far higher percentage of wrong answers than we do. And since we don't know how much higher, we can't draw any conclusions.
i'm at that point right now too.
should i really pm a pure guess i mean i don't have any clue at all.
mightyhuhn is offline  
post #55 of 457 Old 07-31-2014, 10:01 AM
AVS Addicted Member
 
mark haflich's Avatar
 
Join Date: Dec 2000
Location: brookeville, maryland, usa
Posts: 19,859
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 406 Post(s)
Liked: 461
Quote:
Originally Posted by Scott Wilkinson View Post
I would LOVE to have many more members submit their determinations, but I can't force them to. I hope to get more responses over the next couple of weeks. I suspect that suitably high-res audio systems are relatively rare, but there has to be more than six AVS members who own such a system. Also, I think it's a great idea for those who don't to seek out those who do and see if they can try the test during a visit.

As far as I can see, setting up an consistent ABX test with different computers and software isn't as easy as you suggest. For example, foobar for Windows and ABXTester for the Mac work in different ways and generate somewhat different results, and I couldn't figure out how to combine them into a consistent framework. Also, identifying which one is "X" isn't the same thing as identifying which one is high-res, which is the point of this particular experiment.

Of course you can force them. Just ban them from the forum if they refuse. This isn't brain science. Geez.

Mark Haflich

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

call me at: 240 876 2536
mark haflich is online now  
post #56 of 457 Old 07-31-2014, 05:53 PM - Thread Starter
AVS Special Member
 
Scott Wilkinson's Avatar
 
Join Date: Sep 2001
Location: Burbank, CA
Posts: 1,228
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 249 Post(s)
Liked: 976
Quote:
Originally Posted by mightyhuhn View Post
i'm at that point right now too.
should i really pm a pure guess i mean i don't have any clue at all.
You can PM me with "I could hear no difference between the files at all." Several other members have done that, which is interesting by itself. But if you do, be sure to tell me about your system anyway so I know which database to add your (null) results to.

Scott Wilkinson
AVS Editor
Scott Wilkinson is offline  
post #57 of 457 Old 07-31-2014, 08:49 PM
Advanced Member
 
RLBURNSIDE's Avatar
 
Join Date: Jan 2007
Posts: 784
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 54 Post(s)
Liked: 133
Quote:
Originally Posted by Speakerphile View Post
For the hell of it. Learn to have a little fun and don't take everything so seriously...
This website is called AV Science forums, is it not? It's not "The View". Colour me silly for wanting to read scientifically valid results. (which are, in the end, the only independently verifiable ones. I don't care if people subjectively love the objectively inferior audio reproduction capabilities of vynil, because people often believe nonsense and that's why science and rationalism is superior to primitivism and emotion).

I don't take unscientifically surveyed audio test results seriously, but I do appreciate the amount of reasoning and effort that went into this to attempt to discern any difference between high res and "low" res digital audio. My only gripe is that it's going 90% of the way to a valid test, then skimping out on the actual double blind study with a sufficient sample size, that must follow such a grueling methodology in order to become statistically significant, is just them not achieving what they could given the limitations of the online methods. They are so close, so why not? A constructive critique of this would be : could we devise a rigorous way to prove that people aren't lying / cheating with their specs and gear and doing their tests blindly? I know there is a way to validate results over distances, it just needs to be implemented.

I already suspect the answer : most people cannot tell the difference reliably (this is from prior knowledge that similar studies, that actually are scientifically rigorous, have failed to show a large discernibility between 128 and 320kbps MP3s, let alone CDs and SACDs or DSD or 24/96 HRA vs CD). But I would love to find out that I'm wrong and all the investments I've made and intend to make aren't for naught.

Once you see the shadows on the wall after the left the cave, it's hard to go back. This test hints that we may be tilting at windmills with high res audio (and probably are), but I'd like to know for sure.

Wouldn't you? Science is the answer to the problem.

Last edited by RLBURNSIDE; 07-31-2014 at 09:01 PM.
RLBURNSIDE is offline  
post #58 of 457 Old 08-01-2014, 09:09 AM
Senior Member
 
granroth's Avatar
 
Join Date: Jan 2012
Location: Gilbert, AZ
Posts: 398
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 62 Post(s)
Liked: 103
Yeah, these results could not be less scientific, but my curiosity is still fully piqued. It's possible to interpret the results such that 100% of the people who evaluated the files with an HRA-capable system could detect the difference. There are only six people in the "sample", but all we need is ONE person who can reliably demonstrate the ability to detect ultra-sonic frequencies and some of my understanding of the science here is thrown out the window.

Put another way, THIS test is just a fun one and is AT MOST just suggestive of a possible result... but it's suggestive enough that I think it justifies an actual scientific test to determine if there is new knowledge to be had here or not. I believe that Dr Waldrep was considering doing such a thing and I hope so! This is all utterly fascinating.

(For the record, I didn't submit my results because I've proven time and again with ABXTester that I can't even tell the difference between an MP3 and a CD So the fact that I can't tell the difference between CD-quality and high-res is hardly a surprise and would just skew the results.)


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
granroth is offline  
post #59 of 457 Old 08-01-2014, 09:45 AM
AVS Special Member
 
mcnarus's Avatar
 
Join Date: Nov 2007
Posts: 6,120
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 92 Post(s)
Liked: 303
There are only six people in the "sample", but all we need is ONE person who can reliably demonstrate the ability to detect ultra-sonic frequencies and some of my understanding of the science here is thrown out the window.

First, there are not just 6 people in the sample. There are 6 people in Scott's cherrypicked subset of the sample. And cherrypicking is a big no-no in statistics.

Second, so far no one has demonstrated that they can reliably tell the difference between these two types of files, because 3 out of 3 isn't statistically significant.

Third, even if some people can hear a difference, it does not mean that they can detect ultrasonic frequencies. It means that, either in the way the files were made or the way they were played back, there are distortion artifacts in the audible range in one or both of the files.

(For the record, I didn't submit my results because I've proven time and again with ABXTester that I can't even tell the difference between an MP3 and a CD So the fact that I can't tell the difference between CD-quality and high-res is hardly a surprise and would just skew the results.)

No, it won't skew the results. In fact, we need to know how many people can't do this in order to know whether anybody can. That's why the whole exercise is bogus. If you take yourself out of the sample because you think you know what your result will be, then it is you who are skewing the results.
tubetwister likes this.

If you can't explain how it works, you can't say it doesn't.—The High-End Creed

mcnarus is offline  
post #60 of 457 Old 08-01-2014, 10:28 AM
AVS Addicted Member
 
amirm's Avatar
 
Join Date: Jan 2002
Location: Washington State
Posts: 18,085
Mentioned: 3 Post(s)
Tagged: 0 Thread(s)
Quoted: 755 Post(s)
Liked: 435
Quote:
Originally Posted by mcnarus View Post
First, there are not just 6 people in the sample. There are 6 people in Scott's cherrypicked subset of the sample. And cherrypicking is a big no-no in statistics.
I suggest studying Simpson's Paradox. Not only is that allowed but without doing so you can get the opposite results than the truth.

Quote:
Second, so far no one has demonstrated that they can reliably tell the difference between these two types of files, because 3 out of 3 isn't statistically significant.
But this certainly is:

Quote:
Originally Posted by amirm View Post
Thank you Scott! Much appreciated the effort you have put on this project Scott. For the first time I feel that the forum is moving forward toward better understanding of this topic.

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/10 18:50:44

File A: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\On_The_Street_Where_You_Live_B2.wav

18:50:44 : Test started.
18:51:25 : 00/01 100.0%
18:51:38 : 01/02 75.0%
18:51:47 : 02/03 50.0%
18:51:55 : 03/04 31.3%
18:52:05 : 04/05 18.8%
18:52:21 : 05/06 10.9%
18:52:32 : 06/07 6.3%
18:52:43 : 07/08 3.5%
18:52:59 : 08/09 2.0%
18:53:10 : 09/10 1.1%
18:53:19 : 10/11 0.6%
18:53:23 : Test finished.

----------
Total: 10/11 (0.6%)

Quote:
Originally Posted by amirm View Post
The third track was pretty easy. First segment picked was quite revealing:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/10 21:01:16

File A: C:\Users\Amir\Music\AIX AVS Test files\Just_My_Imagination_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\Just_My_Imagination_B2.wav

21:01:16 : Test started.
21:02:11 : 01/01 50.0%
21:02:20 : 02/02 25.0%
21:02:28 : 03/03 12.5%
21:02:38 : 04/04 6.3%
21:02:47 : 05/05 3.1%
21:02:56 : 06/06 1.6%
21:03:06 : 07/07 0.8%
21:03:16 : 08/08 0.4%
21:03:26 : 09/09 0.2%
21:03:45 : 10/10 0.1%
21:03:54 : 11/11 0.0%
21:04:11 : 12/12 0.0%
21:04:24 : Test finished.

----------
Total: 12/12 (0.0%)


Quote:
Originally Posted by amirm View Post
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/11 06:18:47

File A: C:\Users\Amir\Music\AIX AVS Test files\Mosaic_A2.wav
File B: C:\Users\Amir\Music\AIX AVS Test files\Mosaic_B2.wav

06:18:47 : Test started.
06:19:38 : 00/01 100.0%
06:20:15 : 00/02 100.0%
06:20:47 : 01/03 87.5%
06:21:01 : 01/04 93.8%
06:21:20 : 02/05 81.3%
06:21:32 : 03/06 65.6%
06:21:48 : 04/07 50.0%
06:22:01 : 04/08 63.7%
06:22:15 : 05/09 50.0%
06:22:24 : 05/10 62.3%
06:23:15 : 06/11 50.0% <---- difference found reliably. Note the 100% correct votes from here on.
06:23:27 : 07/12 38.7%
06:23:36 : 08/13 29.1%
06:23:49 : 09/14 21.2%
06:24:02 : 10/15 15.1%
06:24:10 : 11/16 10.5%
06:24:20 : 12/17 7.2%
06:24:27 : 13/18 4.8%
06:24:35 : 14/19 3.2%
06:24:40 : 15/20 2.1%
06:24:46 : 16/21 1.3%
06:24:56 : 17/22 0.8%
06:25:04 : 18/23 0.5%
06:25:13 : 19/24 0.3%
06:25:25 : 20/25 0.2%
06:25:32 : 21/26 0.1%
06:25:38 : 22/27 0.1%
06:25:45 : 23/28 0.0%
06:25:51 : 24/29 0.0%
06:25:58 : 25/30 0.0%

06:26:24 : Test finished.

----------
Total: 25/30 (0.0%)


So we now have 3 out of 3 positive detection of differences in Scott's clips.
Summarizing, I managed to consistently tell all three files apart from their downsampled 44.1 Khz/16 bit versions.

The probability of me guessing is 0%. I can run 1000 trials and still get the same outcome as you can see in the detailed reports.

Quote:
Third, even if some people can hear a difference, it does not mean that they can detect ultrasonic frequencies. It means that, either in the way the files were made or the way they were played back, there are distortion artifacts in the audible range in one or both of the files.
Or put much more simply, the process of converting the files to 16/44.1 was not transparent. Hence the reason it makes sense to get the hi-res files.

Quote:
No, it won't skew the results. In fact, we need to know how many people can't do this in order to know whether anybody can. That's why the whole exercise is bogus. If you take yourself out of the sample because you think you know what your result will be, then it is you who are skewing the results.
That is just wrong. The fact that I can hear the differences reliably cannot be undone with a million people who say they can't.

Now, if you are talking about what those million people can hear, then sure. But that is not the purpose of such evaluations. Those millions of people don't read this forum, are not audiophiles, and couldn't care less one way or the other. Including them in such testing will guarantee Simpson's Paradox coming true. And is a tactic used to generate mistruths in such tests.

But let's have your test results to see. Can you run the above double blind test and post your results?
nodixe and stereoeditor like this.

Amir
Founder,
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

"Insist on Quality Engineering"

amirm is offline  
Reply Audio theory, Setup and Chat

Tags
frontpage

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off