Digital Music Files: 16/44.1, 24/88, 24/96, 24/192 - Page 7

Yep, that's classic, all right.


Exactly as I was saying...



This statement brings up something that I've always found very interesting. The subjectivist is making a claim that the cables make a dramatic difference. Such a claim is not uncommon for cables. BUT, why the hell is it that the "dramatic" descriptor is abandoned the instant a DBT is suggested?. There is NEVER EVER a qualifier regarding the amount of time it took to switch cables--the difference is "dramatic", PERIOD. But such a qualifier magically materializes when the DBT is proposed. Sighted--switching time NO problem whatsoever. UNSighted--switching time is now a problem. Objectivists are always willing to give the subjectivists a break on this. Maybe we shouldn't, thus avoiding ANY possible objection to a switch box.

Another thing: Objectivists always give subjectivists a break on the percentage of correct identifiers; always getting it right is never asked for. But why is such a break needed for a "dramatic" difference? After all, no one would ask for such a break with other "dramatically" different things, such as chocolate vs vanilla ice cream. One of the excuses is that comparing is "hard work", ie fatiguing. But why is it "hard" to distinguish a "dramatic" difference? Would one have to "work hard" to tell if one is tasting butterscotch vs. strawberry (another "dramatic" difference), or hearing a clarinet vs. a flute (also "dramatic")?

I can imagine an answer might be, "Oh, I didn't mean 'dramatic' the way you think I meant". That would come across as an attempt to escape the consequences of one's claim, or an outright backtrack.

Just some thoughts.

Not necessarily as the answers may give some insight.

Plus it's fun to watch them squirm, almost as much fun as ecotards.

There's a rather heroic assumption built into this statement.

I meant into the poster, not anything technical.

"As I say, messing with people's heads can be a lot of fun. You should try it." O. W. Grant.

OK, but I'm not really dying to know more about posters like that.

Some interesting reading on this

http://www.math.niu.edu/~rusin/known-math/98/bees

Thanks for that.

Another Subjectivist Myth (that scientific theory ignores/conflicts with the real world) exploded.

Was exactly my point. I did not make myself clear.

If David is suggesting that O should not make up his mind either way until there is further evidence proving negative, then it is David James who is shifting burden of proof.

But if David James is suggesting O should not believe S until S provides evidence, then his point is no different than saying "I don't believe you", so he is not suggesting something new.

My intention was the option I put in bold and yes, it's the same as saying I don't believe you but not as abrupt and offers 'S' the chance to defend their assertion.

As for my 'O's comment "That's impossible", perhaps the example wasn't perfect. What I was trying to do was show how people who require evidence for claims may fall into the same trap.

As for shifting the burden of proof, technical it's not a shift, as the original burden still exists. What it does, however, is allow 'S' the opportunity to legitimately redirect the thread away from them to 'O's unsubstantiated claim. You see this played out all the time.

Can anyone provide a link to an ABX of "CD vs HI Rez" that doesn't include hi rez content that isn't really high rez or is the aforementioned Meyer/Morgan effort the best we can do as of May 2012?

James

No way is Meyer/Morgan the best we can or could do. Back in 2000-2002 I assembled a web site called www.pcabx.com. It's probably still on the wayback machine. http://archive.org/web/web.php


I custom recorded a number of hi-rez tracks that were specifically designed to make any possible differences as audible as is reasonably possible.

Today, anybody who wants to can download hi rez tracks from HDTracks.com, inspect them to ensure that they actually are hi rez. Then put them through a downsample/upsample process, and compare the original and 44/16 downsampled version using one of the many freeware software ABX comparators, such as the one that is an add-on to the Foobar music player.

Your only out-of-pocket costs will be only the tracks from HDTracks.com. All of the remaining software is freeware/open source/etc.

All you need is a PC with a real 24/96 or 24/192 audio interface and something to listen with, either headphones or high quality monitor speakers.

Cool, thanks. I was hoping for something more along the lines of another study, but I'm not finding much myself.

This stuff really interests me and the idea that an entire format (SACD) was r&d'd, created, marketed, and sold with no discernible sonic benefit (surround "perk" aside) is simultaneously unbelievable, humorous, and a bit disturbing to me.

And I suppose there's more of this kind of stuff in this hobby than you can shake a stick at, but an entire FORMAT is a real doozy.



James

Actually, two such formats were developed more or less concurrently: SACD and DVD-A

BTW I recently did a study of the 20 or so recordings that Meyer and Moran of JAES fame used in their subjective tests of high resolution recordings, and found that it is fairly certain that 7 of the 20 were not actually high resolution recordings, but were low resolution recordings that were upsampled prior to sale as high resolution recordings. Three or more others were IMO questionable. I believe that Meyer and Moran were mislead by the music industry who mislabelled these products.

It would appear that the music industry unintentionally booby-trapped the commercial development of high resolution recordings by believing their own hype. ;-)

Since I know of zero instances of high end audio reviewers actually detecting these recordings shortly after initial release, it appears that the release of these bogus recordings constituted another loosely-defined blind test. The breadth and signficance of this test is far more impressive to me! ;-)

Well in fairness I doubt anyone would necessarily "blindly" finger an otherwise well-produced/engineered recording as "substandard" just because it wasn't a 24/96 or 24/192 file...I'm sure they were excellent, actually. The question would be: is there would be anything to gain with an even higher "rez" sample? It's at the very least, an honest question.

The above is akin to having someone say a car is about to blaze by you at 220mph, you thinking "holy $hit is that fast" as it happens, and then finding out it was only traveling 210mph. Both speeds are decidedly fast, but one is still faster than another. Whether that additional 10 mphs matters is of course user-determinable...and even that value is very much philosophically debatable.

Not saying that's precisely the case with hi rez vs 44/16 files, just making the point that just because the/a difference doesn't make itself obvious doesn't mean one doesn't exist...even one that may/would be readily (not necessarily easily or universally) audible in an ABX.

This is devil's advocate stuff here, I'm not trying to get anyone ramped up, I understand there is much to the contrary...

thanks,

James

This article references the same flawed study that Arnik keeps promoting.

The previous poster was asking for alternative studies and this is not an alternative - it is exactly the same source when discussing ABX.

Do you really get the idea that I'm promoting Meyer and Moran when I point out that their study was obviously seriously flawed by the fact that the recordings that they used were almost half fatally flawed from the viewpoint of their thesis?

Here is what I just posted:


Since I'm obviously miscommunicating here, let me give you an executive summary:

(1) The Meyer and Moran study is fatally flawed from the viewpoint of supporting their thesis because about half of the recordings they used were not suitable for their study. They were not truly high resolution recordings.

(2) I don't fault Meyer and Moran themselves because they took claims by the music industry at face value.

(3) I do fault the music industry for misleading people into believing and buying recordings that weren't as advertised - they did not start out as high resolution recordings and therefore never could be high resolution recordings.

Again, I apologize for not being clear enough.

To use your metaphor, the SACD and DVD recordings I'm talking about weren't missing being better by 10 mph out of 220. They were inherently limited to being low resolution recordings, which in your metaphor would be like driving 55 mph or at best 70.

The recordings may have sounded better, but that was due to the fact that they were remastered. Remastering is more like redoing the interior of the car and making it look nicer or be more comfortable. The car is going the same speed (55 or 70) as ever, but it feels and looks better.

Presumably, M&M still have their raw data, and could run the numbers using only the 10-13 recordings that apparently were hi-rez. You might want to inform them of your findings and suggest this.


Depends on what their thesis was. If their thesis was that commercially available SACDs and DVD-As were indistinguishable from downsampled versions of same, then their study supports their thesis just fine. If their thesis was that downsampling true hi-res recordings has no audible effect, that is not so clear.* Calling this "fatally flawed" is a gross overstatement.

*It can be argued that, even given the problem you raise, their results provide some support for the stronger hypothesis. Fully half the recordings they used were indeed hi-res, and they had a sample size well up in the hundreds, as I recall. If the true hi-res recordings really were audible, you should have seen some evidence of that in the overall data, even given the "noise" of the low-res recordings. You didn't see that. I'd still want to run the numbers again or repeat the experiment using only true hi-res recordings, but I don't think one can dismiss the findings we do have as completely meaningless.

Unh, huh. I just did this study yesterday and it is posted in a place they may monitor.


This is the abstract of their paper, from which their thesis should be able to be discerned:


I would say that their goal was to test the following hypothesis:

"...audibly superior sound quality for two-channel audio encoded with longer word lengths and/or at higher sampling rates than the 16-bit/44.1-kHz CD standard.".

In order to test that hypothesis, it would seem that all recordings tested should have "longer word lengths and/or at higher sampling rates than the 16-bit/44.1-kHz CD standard."

We know and from knowing the men they know that it is not the word length and sample rate of the recording at hand that matters, but rather it is the shortest word length and lowest sample rate any place in the production of the recording that matters. Since some of the recordings were sourced from analog recordings, Shannon's law would apply. They should have sufficient dynamic range and bandwidth, at least vis-a-vis recording and production.



If you count different listeners and different recordings, which is appropriate, agreed.


Perhaps. There's really only one way to treat this situation correctly after the fact, and that is to exclude the ambiguous trials.



They didn't print the raw data, which is again appropriate. But it conceals a lot from us.


Agreed.


I am very comfortable dismissing the current findings because of the piss in the soup that was clearly unknown at the time of the original study.

Interestingly enough, the problem appears to have been knowable at that time. Their omniscience modules failed! ;-)

Depends. You need to do a statistical analysis and see what the sensitivity to measuring differences would be with the reduced sample size.

I'm not talking about the raw data. I'm talking about the published findings. If an audible difference were detectable in half the recordings they used, then their overall result should have been somewhat north of 50%, perhaps substantially north depending on how often the hi-res samples were used and how detectable they were.


Again, outright dismissal seems to me an overreaction. Many scientific papers are published with "limitations" noted. Those limitations tend to lead to the cliched conclusion, "more research is needed." But that doesn't mean there aren't valuable conclusions to be drawn from the research as presented. This seems to be the case here, at least to me. The use of some non-hi-res recordings is a limitation that should have been notedand presumably would have been noted, if M&M had been aware of the problembut its absence doesn't mean the whole thing can be dismissed.

It's been a while since I've read the paper. I want to go back and look at it again before speculating too much.

Well, certainly you aren't going to prove anything with the findings we've got. But results can be suggestive.

Take a hypothetical: Suppose, given our sample size in some experiment, we need 60% correct to get a positive result. Does it matter if the actual result is 50% or 59%? From the standpoint of statistical significance, no. But ask a different question: How likely is it that there's some identifiable subset of the sample for which the results ARE statistically significant? It's much more likely if that overall result is 59% than if it's 50%.

To put it another way, if you're a scientist who's got to decide whether it's worth doing more experimentation, the difference between 50% and 59% is something you'd take into account.

A final thought: What if M&M had gotten a positive resultif they'd done enough samples with hi-res disks, and those proved detectable enough, that they got over the significance threshold? Would anyone be arguing today that the result is fatally flawed because only half the disks they used were truly hi-res? Of course not. You'd still want to disaggregate the hi-res and low-res results (in part because if the low-res samples were detectable you'd have to suspect a methodological flaw), but no one would dismiss the overall finding.

From the original article (emphasis added):


IOW, they did not attain a positive result with any individual recording—including all of the recordings that actually were hi-res. This would constitute definite evidence that even true hi-res recordings are not audibly distinguishable from downsampled versions.

Now, evidence is not proof, but remember that negative evidence is never proof. That's an impossible and therefore unscientific standard. The burden of proof is definitely on the other foot. There is no justification for dismissing M&Ms findings out of hand on the basis of the recordings they used.

On a side note, it is interesting that younger test subjects, who could hear above 15 kHz, did worse, not better, than older subjects.

Your application has nothing to do with my analogy.

If perceptible differences/"improvements" exist between hi rez and redbook files they are all but certain to be slight...even fantastically so.

That being the case, the playback of a well produced/engineered 44khz/16b file onto an unsuspecting listener (in this case, the "reviewers" that assumed they had a "hi rez" SACD/DVD A) could very well- probably more accurately never- be "called out" as a "lesser" version, so to even faintly assuage that the occurrences even remotely resembled a "blind test" (even fantastically "loosely") of any scientific regard and they that should have ANY real significance is wholly inaccurate.

And I won't bother to get into the consequences (which, flatly, are much more far-reaching than I believe some here seem to comprehend) of inserting samples presumed to be something else into a scientific test/survey (in this case, 35%, lmao), but I would soberly advise some folks to take (or revisit) a college level stats course, or at least do some cursory reading on statistical theory/methodology/implication before forming and asserting an opinion that is all-but disconnected from what's considered good science by the community at large.

The fact that someone is insisting that one such as the example at hand here should be given any reasonable amount of weight within a forum that seems to enjoy its tie with science is dubious at best.

But heh, whatever helps you sleep at night, I guess.

I will now leave the thread because there's no way in he!! I'll be caught up in the kind of dross contained within the preceding pages. I will happily re-visit the subject when there's some valid data to ingest.

Thanks for playing and drive home safely.

James

I'm not arguing that audio needs to be held to the same standard, but in a scientifically rigorous environment the study would be rejected completely.

I once was involved in the design of a human study which required vetting the study though Phd level statistical scrutiny and national committee oversight.

The first step is stating the hypothesis, which in this case is that "hi res audio" can be identified over CD quality audio by a percentage of the population.
Further scrutiny would reveal that "hi-res" audio is not an isolated variable and the study should be broken up into studies that test an isolated variable.Thus, a rigorous study would test identically recorded and mixed examples distributed at different bit depths but the same sampling freq, or the same bit depth but different sampling freqs. Otherwise, the study would need to be designed to test co-variants which greatly expands the required sample size.

The second step would be estimating the size of the difference that one expected to find. That is, are we looking to identify 2%, 10%, or a 30% difference per individual, and also per population.

Then, and only then, can you calculate the number of samples that each participant would need to test, and the total number of participants that we need to sample.

My experience is that you need way more data than you think.

Granted, NSF was not going to fund this work. But, given what was generally known at the time, it was a credible piece of psychoacoustic research, and passed peer review for that reason. It isn't all that unusual for even the most careful study to be debunked later, as new knowledge and new experiments emerge.

But it's important to note that this study has not been debunked yet. People are claiming that a different choice of program material would have produced a different result. Such people have an obligation to prove it. As M&M put it themselves:

I´m still the same as the beggining of this thread, I will have to trust on my ears ar the end of the day.

Kind of like fish oil being good for heart. It is not proven, but there is research strongly suggesting it is in fact good for heart and can reduce cholestrol.