AVS Forum

AVS Forum (http://www.avsforum.com/forum/)
-   Audio Theory, Setup, and Chat (http://www.avsforum.com/forum/91-audio-theory-setup-chat/)
-   -   AVS/AIX High-Res Audio Test: Updated Results (http://www.avsforum.com/forum/91-audio-theory-setup-chat/1653170-avs-aix-high-res-audio-test-updated-results.html)

Scott Wilkinson 08-20-2014 08:25 PM

AVS/AIX High-Res Audio Test: Updated Results
 
http://www.avsforum.com/forum/imageh...5566367edf.jpg

With 50 participants, it's time to update the results of this informal but interesting—and, I hope, fun—experiment.

It's been over three weeks since I posted the first results from the ongoing high-resolution audio (HRA) experiment being conducted by AVS Forum and AIX Records. We now have a total of 50 participants—22 with HRA-capable audio systems and 28 with systems that cannot reproduce the extended information in bona fide high-res audio—so I thought it might be a good time to share the latest results.

Before I do, let me reiterate for the umpteenth time that this is not a scientific test; it's merely an interesting and fun exercise that allows AVS members to investigate the efficacy of high-res audio for themselves. I'm gathering data out of curiosity to see what we might learn and how the results might help inform more rigorous experiments in the future.

As you can see in the chart on the left above, the participants with HRA-capable systems were able to identify the native HRA tracks quite well. None were completely wrong, and none reported hearing no difference (ND) between the native HRA tracks and sample rate-converted tracks; the average correct score in this group is 78.9% so far. Participants without HRA-capable systems are more evenly distributed in a roughly bell-shaped curve; the average correct score in this group is 52.8% so far (calculated without counting those who reported hearing no difference between any pair of versions).

I'm also interested in the identification rate for each track. The chart on the right indicates that "Mosaic" has been the easiest to identify correctly for both groups, while "Just My Imagination" has been the most difficult for both groups. As expected, those with HRA-capable systems were able to identify the correct version of all three tracks much more often than those without such systems.

These results seem to suggest that a high-res audio system is required to derive the full benefit that HRA has to offer—and that HRA does, in fact, have something to offer beyond CD audio. As one respondent with an HRA system commented, "On 'Mosaic' and 'On The Street Where You Live,' I don't even need to listen to the full tracks; the giveaway is pretty much in the instruments that carry the high-frequency information. At the lower sampling rate, the sound is muted or compressed (especially in the decay) in the high frequencies, robbing the music of a sense of space and liveliness." BTW, he correctly identified all three HRA versions.

However, for many participants, the difference was subtle. As another respondent with an HRA system commented, "I have to say that the differences in these files were very, very small, and I would have no trouble listening to the 16/44 versions rather than the 24/96." He correctly identified two out of three HRA versions, and he is not alone in his assessment that the difference between the two versions of each track is small.

If you haven't yet listened to the files and sent me your determinations by PM, I encourage you to do so, whether or not you have an HRA-capable system. You can read about the experiment and download the files here.

Let the listening continue!

Like AVS Forum on Facebook
Follow AVS Forum on Twitter
+1 AVS Forum on Google+

fokakis1 08-20-2014 08:40 PM

It would seem that high end/performance audio gear would call for high resolution media. Likewise, these test results suggest that high resolution media may bring about a demand for purchasing higher quality components.

RayDunzl 08-20-2014 09:00 PM

Is there any surprise if you look at the type of system reported - headphone vs speaker?

dnoonie 08-20-2014 09:59 PM

Thanks for the update. I would love to attend a local HRA listening audio event. These samples and this requirements gathering test are great. I need to rewire my system to bypass all preamp stages, then I'll have a modest HRA setup.

Thanks for the reminder!

tnargs 08-20-2014 11:30 PM

Quote:

Originally Posted by Scott Wilkinson (Post 26749330)
These results seem to indicate that a high-res audio system is required to derive the full benefit that HRA has to offer—and that HRA does, in fact, have something to offer beyond CD audio. As one respondent with an HRA system commented, "On 'Mosaic' and 'On The Street Where You Live,' I don't even need to listen to the full tracks; the giveaway is pretty much in the instruments that carry the high-frequency information. At the lower sampling rate, the sound is muted or compressed (especially in the decay) in the high frequencies, robbing the music of a sense of space and liveliness."

Don't get too excited. I would be more likely to believe that the presence of ultrasonic signal, in the particular ultrasonic transducers in the systems tested, is causing unwanted excitation of said transducers down in the audible range. Disproving this mechanism would be, er, difficult, and until disproven the test is not suggestive of what you wrote above.

datranz 08-21-2014 12:54 AM

7 Attachment(s)
I think he meant the media and player is HR, the system is able to produce the extra detail because of the media and not the speakers. Most speakers, amps, and avr should be able produce ultrasonic up to 20k hz. easily.

pbarach 08-21-2014 04:46 AM

Quote:

Originally Posted by Scott Wilkinson (Post 26749330)
As you can see in the chart on the left above, the participants with HRA-capable systems were able to identify the native HRA tracks quite well.

I'm sure I missed this bit of information, but how are you defining an "HRA-capable system"?

HockeyoAJB 08-21-2014 06:05 AM

Quote:

Originally Posted by datranz (Post 26753218)
I think he meant the media and player is HR, the system is able to produce the extra detail because of the media and not the speakers. Most speakers, amps, and avr should be able produce ultrasonic up to 20k hz. easily.

Actually, he means the whole chain from source to speaker. If any link between the source and the speaker is incapable of passing on the higher frequencies and/or bit depth then the whole system fails the test. Also, frequencies below 20kHz are generally not referred to as "ultrasonic". Ultrasonic frequencies start above 20kHz. CD is perfectly capable of reproducing frequencies up to 22kHz (assuming the standard 44.1kHz sampling rate). To be hi-res, your system needs to be able to handle greater than 16-bit/44.1kHz audio and actually reproduce it without down-sampling or compressing the dynamic range.

FYI, most speakers are only rated up to 20kHz, so would not qualify. You need speakers that are capable of going higher (preferably at least into the 50kHz range).

Gaugster 08-21-2014 06:22 AM

Would anyone care to recommend some headphone brands that are suitable for hi rez?

Per Scott, I was able to identify all hi rez tracks using my HT speakers and audio chain. Would like to see how my laptop performs also with ABX methods. My current headphones only specs up to 15kHz.

JeffPerrin 08-21-2014 06:54 AM

Quote:

Originally Posted by HockeyoAJB (Post 26754953)
Actually, he means the whole chain from source to speaker. If any link between the source and the speaker is incapable of passing on the higher frequencies and/or bit depth then the whole system fails the test.

How about the source itself? To provide a little different perspective on the fidelity of the "chain," it may be of interest that the frequency response for much of the gear favored by recording studios is rarely capable of even capturing 20khz. Let's just look at the mics; Ribbon mics typically top out at 15khz (engineers love these mics for drum overheads). Most other mic's freq response drops like a rock well before 20khz. (e.g: the classic Shure 57 and the Neumann U87 - probably the two most popular mics used in recording history) The reason is lies in mic design. Higher freq response does equate to better sound. And today, many engineers - in a quest to capture that classic "warm" analog tone of days gone by - are increasingly employing tape-saturation plug-ins which emulate the frequency-limited bandwidth of analog tape.

So, in short, if these recordings do actually contain sonic information above 20khz, how do we know it's truly "musical" information and not simply noise or distortion introduced during a post-recording process or format conversion?

HockeyoAJB 08-21-2014 06:55 AM

Quote:

Originally Posted by Gaugster (Post 26755177)
Would anyone care to recommend some headphone brands that are suitable for hi rez?

Per Scott, I was able to identify all hi rez tracks using my HT speakers and audio chain. Would like to see how my laptop performs also with ABX methods. My current headphones only specs up to 15kHz.

Sony's MDR-1R and MDR-10R series headphones are capable of reproducing hi res audio. Bear in mind that the blue tooth versions are only hi res capable when plugged in. If you actually use the blue tooth feature then the frequency range is reduced to 20Hz-20kHz. As such, you will need a headphone DAC & amp (standalone or built into another piece of equipment) that can reproduce hi res frequencies.

HockeyoAJB 08-21-2014 07:20 AM

Quote:

Originally Posted by JeffPerrin (Post 26755705)
How about the source itself? To provide a little different perspective on the fidelity of the "chain," it may be of interest that the frequency response for much of the gear favored by recording studios is rarely capable of even capturing 20khz. Let's just look at the mics; Ribbon mics typically top out at 15khz (engineers love these mics for drum overheads). Most other mic's freq response drops like a rock well before 20khz. (e.g: the classic Shure 57 and the Neumann U87 - probably the two most popular mics used in recording history) The reason is lies in mic design. Higher freq response does equate to better sound. And today, many engineers - in a quest to capture that classic "warm" analog tone of days gone by - are increasingly employing tape-saturation plug-ins which emulate the frequency-limited bandwidth of analog tape.

So, in short, if these recordings do actually contain sonic information above 20khz, how do we know it's truly "musical" information and not simply noise or distortion introduced during a post-recording process or format conversion?

Very true. This is where provenance comes in, as coined by Mark Waldrep (creator of AIX records). I'm inclined to trust the equipment and methodology he uses (until proven otherwise). I can't say the same for any of the other record labels. Theoretically, you could filter out all of the frequencies below 20kHz and then play the song back, but since what's left would be beyond normal human hearing, it would be difficult to validate it as music as opposed to noise. I suppose you could manipulate it using software that modifies the remaining frequencies to bring them down into the audible frequency range (like recording your voice and using a synthesizer to raise or lower the pitch). You could then compare the timing of the modified audio to the original track to determine if the sounds were likely produced by an instrument or just added noise.

Unfortunately, this goes way beyond what a normal consumer is capable/willing to do themselves and would likely require you to purchase the music before you test it. What we need is are stricter certification requirements that force the recording studios to prove that their music is truly hi res before the SACD/DVD-A/Blu-ray associations will allow their logos to be displayed on the physical disc and/or before online services will agree to carry the music in "hi res" form.

JeffPerrin 08-21-2014 08:23 AM

Quote:

Originally Posted by HockeyoAJB (Post 26756281)
What we need is are stricter certification requirements that force the recording studios to prove that their music is truly hi res before the SACD/DVD-A/Blu-ray associations will allow their logos to be displayed on the physical disc and/or before online services will agree to carry the music in "hi res" form.

There's on rub. Since audio quality is a subjective matter, the very definition of hi-res will always be in contention.

But my point was that before the playback medium, even before the record label's involvement in the distribution method of such product, there is the artist and the recording engineer. If we're going to define "high resolution" simply as pertaining to (a required wide) frequency response, then much of the classic (and coveted) analog gear being used to make the original recordings themselves will negate 99% of commercial music from a "hi-res" category.

IMHO, "hi res" should simply be defined as a lossless conversion from the original master recordings.

HockeyoAJB 08-21-2014 08:44 AM

Quote:

Originally Posted by JeffPerrin (Post 26757497)
There's on rub. Since audio quality is a subjective matter, the very definition of hi-res will always be in contention.

But my point was that before the playback medium, even before the record label's involvement in the distribution method of such product, there is the artist and the recording engineer. If we're going to define "high resolution" audio simply by the wide frequency response afforded by high sample rates, then much of the classic (and coveted) analog gear being used to make the original recordings themselves will negate 99% of commercial music from a "hi-res" category.

IMHO, "hi res" should simply be defined as a lossless conversion from the original master recordings.

You could do that. However, if that is our definition then there is no way of knowing whether or not "hi res" equipment is actually required to play it back without loss of information or if it is actually any better than a previous CD release, which may or may not have all of the same info on it (depending on what was originally recorded). This calls the value of both the content and the equipment into question when you consider how much more expensive it is. If I'm going to spend 2 to 3 times as much on hi res media and the same on a playback system as I would on CD quality, I want to know that when I put that disc in or play that downloaded track, I am getting something more than what I could have gotten from a CD or $1.29 iTunes download. Otherwise, I'm just hoping that I might occasionally get my money's worth, if I'm lucky.

Dr. AIX 08-21-2014 10:10 AM

1 Attachment(s)
Quote:

Originally Posted by HockeyoAJB (Post 26758073)
You could do that. However, if that is our definition then there is no way of knowing whether or not "hi res" equipment is actually required to play it back without loss of information or if it is actually any better than a previous CD release, which may or may not have all of the same info on it (depending on what was originally recorded). This calls the value of both the content and the equipment into question when you consider how much more expensive it is. If I'm going to spend 2 to 3 times as much on hi res media and the same on a playback system as I would on CD quality, I want to know that when I put that disc in or play that downloaded track, I am getting something more than what I could have gotten from a CD or $1.29 iTunes download. Otherwise, I'm just hoping that I might occasionally get my money's worth, if I'm lucky.

Scott sent me the link to this latest set of results and I've scanned through the posts and comments. I write every day about these issues..16 vs. 24-bits, higher sampling rate and recording procedures. I've even coined and trademarked the term Ultra HD-Audio to refer to recording that were actually recorded with equipment running at 96 kHz/24-bit PCM and that maintains the natural sound of the tracks through post production.

I have a number of mics that can capture frequencies beyond 20 kHz (B&K, AKG for example) and I use them close to the instruments to get the most energy I can at all frequencies. Their is musical material up to 40 kHz and beyond...but remember it's only an additional octave.

There are three levels of audio resolution: Reduced Resolution, Standard Resolution, and High Resolution. The folks from the CEA, DEG, NARAS, and labels have hijacked the High-Resolution Audio term to include virtually anything ever recorded...any analog source, any PCM source, DSD and even CDs. It means nothing.

The provenance and production path are the things that make the most difference.

I've just completed a new Ultra HD-Audio Sampler with Sprint. It's only $11 and contains 18 HD-Audio tracks in a wide variety of genres. It's available on iTrax.com and definitely worth checking out. If you haven't heard real HD-Audio, you can't know.

amirm 08-21-2014 11:06 AM

Quote:

Originally Posted by HockeyoAJB (Post 26758073)
If I'm going to spend 2 to 3 times as much on hi res media and the same on a playback system as I would on CD quality, I want to know that when I put that disc in or play that downloaded track, I am getting something more than what I could have gotten from a CD or $1.29 iTunes download. Otherwise, I'm just hoping that I might occasionally get my money's worth, if I'm lucky.

As with anything else, you can ask on forums about the fidelity differences before buying. A lot of people do this on WBF Forum and compare notes in different formats. Not much of that discussion happens here but there is plenty of info elsewhere.

Let's remember that these tracks can be downloaded immediately with no waiting for CDs to arrive. And no ripping is necessary either.

Scott Wilkinson 08-21-2014 11:31 AM

Quote:

Originally Posted by JeffPerrin (Post 26755705)
How about the source itself? To provide a little different perspective on the fidelity of the "chain," it may be of interest that the frequency response for much of the gear favored by recording studios is rarely capable of even capturing 20khz. Let's just look at the mics; Ribbon mics typically top out at 15khz (engineers love these mics for drum overheads). Most other mic's freq response drops like a rock well before 20khz. (e.g: the classic Shure 57 and the Neumann U87 - probably the two most popular mics used in recording history) The reason is lies in mic design. Higher freq response does equate to better sound. And today, many engineers - in a quest to capture that classic "warm" analog tone of days gone by - are increasingly employing tape-saturation plug-ins which emulate the frequency-limited bandwidth of analog tape.

So, in short, if these recordings do actually contain sonic information above 20khz, how do we know it's truly "musical" information and not simply noise or distortion introduced during a post-recording process or format conversion?

Excellent question! This is why the files in this experiment come from AIX Records, where engineer Mark Waldrep is meticulous about using equipment capable of capturing ultrasonic frequencies and wide dynamic range. We are completely confident that the ultrasonic info in the native 24/96 files here is musical and not noise or distortion.

tuxedocivic 08-21-2014 02:38 PM

Please keep posting updates to the home page, thanks :)

mdrums 08-21-2014 06:11 PM

Cool stuff!

I wait for the day that my cell phone is also my Hi-Rez audio player too. I travel a lot and can't wait for the day that my cell phone and a pair of nice headphones is all I need to enjoy hi-res audio on a plane or my hotel room.

Scott Wilkinson 08-21-2014 07:23 PM

Quote:

Originally Posted by mdrums (Post 26774017)
Cool stuff!

I wait for the day that my cell phone is also my Hi-Rez audio player too. I travel a lot and can't wait for the day that my cell phone and a pair of nice headphones is all I need to enjoy hi-res audio on a plane or my hotel room.

Actually, the HTC M8 is a high-res-capable phone; in fact, Sprint is making a pretty big deal about this feature, and Mark Waldrep is a big fan.

Scott Wilkinson 08-21-2014 07:48 PM

Quote:

Originally Posted by RayDunzl (Post 26750090)
Is there any surprise if you look at the type of system reported - headphone vs speaker?

I haven't parsed the data that way, but it's an interesting idea. I'll check it out; thanks!

Orta 08-22-2014 02:26 PM

I personally couldn't tell a difference in any of these tracks using a Fiio Andes via USB on a PC set to 96/24 with Sennheiser HD650's or Momentum's. Same deal using a Nvidia GPU via HDMI on a PC set to 192/24 to a Pioneer 1019 and PSB Image T45's. Used WMP:HC and VLC to play the files. Not really sure if this qualifies as a high res chain.

amirm 08-22-2014 04:52 PM

Quote:

Originally Posted by Orta (Post 26795689)
I personally couldn't tell a difference in any of these tracks using a Fiio Andes via USB on a PC set to 96/24 with Sennheiser HD650's or Momentum's. Same deal using a Nvidia GPU via HDMI on a PC set to 192/24 to a Pioneer 1019 and PSB Image T45's. Used WMP:HC and VLC to play the files. Not really sure if this qualifies as a high res chain.

I would set the Windows Sound Advanced properties to 24/96 Khz. 192 Khz will force another resample in Windows.

mlknez 08-23-2014 08:17 AM

Quote:

Originally Posted by JeffPerrin (Post 26757497)
IMHO, "hi res" should simply be defined as a lossless conversion from the original master recordings.

Your opinion is the problem that I have with others definition of high-resolution. It should be a performance that is CAPTURED and DELIVERED using a standard above all possible human perception. Not just a performance that is as close to the original master recording as possible. 99.99% of music out there does NOT qualify as high-definition! This is true even on the high definition music download sites.


It the master recording does not completely capture the original performance, how can you possibly call it "high-resolution"


If I was bound by your opinion, I could record my band using a 1977 8-track cartridge tape recorder with a $4 Radio Shack microphone. That is now the "Original Master". I could use an analog to digital converter at 96Khz/24 bits and I could then sell that as "high-definition".

JeffPerrin 08-23-2014 09:12 AM

Quote:

Originally Posted by mlknez (Post 26811714)
Your opinion is the problem that I have with others definition of high-resolution. It should be a performance that is CAPTURED and DELIVERED using a standard above all possible human perception.

That's an audiophile's pipe dream. With such stringent stipulations, you're effectively telling the artist "which chisel" they should be using to sculpt their art. My suggestion is based on real-world expectation of what could be delivered today, and possibly even appreciated by the larger public. Telling the artists and producers which microphones or recording methods they need to use just isn't realistic.

"Straight from the source" or "master quality audio" will make a better selling point than "192 kHz". (The fact that the benefits of high sample rates have been hotly debated now for over a decade should speak volumes...)

JeffPerrin 08-23-2014 10:04 AM

Quote:

Originally Posted by mlknez (Post 26811714)
If I was bound by your opinion, I could record my band using a 1977 8-track cartridge tape recorder with a $4 Radio Shack microphone. That is now the "Original Master". I could use an analog to digital converter at 96Khz/24 bits and I could then sell that as "high-definition".

Yes! The 96/24 file in this case would be "hi-res" in that it best reproduces the sound of the music as the artist intended and how they heard and mixed the audio in the studio.

In your definition, the label executive would burst in the room, demanding a re-recording the performance on fresh strings, through a specific mic and pre-amp, etc... all to satisfy the "hi-res" sticker on the disc case. And in the end, the consumer does NOT get to hear the music as intended (hi-res or not), because the artistic process is derailed by industry specifications (which will no doubt change over time anyway).

Now if a boutique label DOES have an audience which demands such a meticulous and carefully followed recording process, that is one thing. I'm painting in broader strokes here and referring to the mainstream recording industry.

amirm 08-23-2014 10:18 AM

Quote:

Originally Posted by JeffPerrin (Post 26813250)
That's an audiophile's pipe dream. With such stringent stipulations, you're effectively telling the artist "which chisel" they should be using to sculpt their art. My suggestion is based on real-world expectation of what could be delivered today, and possibly even appreciated by the larger public. Telling the artists and producers which microphones or recording methods they need to use just isn't realistic.

"Straight from the source" or "master quality audio" will make a better selling point than "192 kHz". (The fact that the benefits of high sample rates have been hotly debated now for over a decade should speak volumes...)

Very well said.

amirm 08-23-2014 10:22 AM

Quote:

Originally Posted by JeffPerrin (Post 26757497)
IMHO, "hi res" should simply be defined as a lossless conversion from the original master recordings.

That is it. Our "demands" as consumers stops there. We can ask that before delivery someone doesn't truncate the music just because there is a format called the "CD" from 1980s. However the music is mastered, is what we should get.

And with that, we obviate the need for any of these listening tests. The tests are aimed at saying the truncated files are just as good. Well, if we get the original master, then we know it is as good as it gets :).

CRGINC 08-23-2014 10:26 AM

Please explain how one like myself who is over 60, can hear much beyond 10 KHz. I do enjoy multi-channel material sampled at 48 and 96 kHz with 48 being more common in my collection. I do notice increased dynamic range but to say I can hear 20 KHz is beyond my comprehension. I can understand hearing mixing products from sounds above my hearing range but I doubt that I can hear the actual audio much above my known hearing frequency range.

markrubin 08-23-2014 10:37 AM

Quote:

Originally Posted by CRGINC (Post 26815042)
Please explain how one like myself who is over 60, can hear much beyond 10 KHz. I do enjoy multi-channel material sampled at 48 and 96 kHz with 48 being more common in my collection. I do notice increased dynamic range but to say I can hear 20 KHz is beyond my comprehension. I can understand hearing mixing products from sounds above my hearing range but I doubt that I can hear the actual audio much above my known hearing frequency range.

agreed: but that does not mean you still cannot enjoy and notice the difference with HD audio

I took a hearing test as part of a promotion for Ultimate Ears custom fitted in ear headphones and was surprised how my hearing had dropped off over 12 khz: that was 5 years ago: I am now 67

yet I can still discern quality audio from HD Tracks and HD audio codecs


All times are GMT -7. The time now is 02:47 PM.

Powered by vBulletin® Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.

vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.