AVS Forum banner

1 - 20 of 222 Posts

·
Premium Member
Joined
·
3,258 Posts
Discussion Starter · #1 · (Edited)


AVS Forum and AIX Records invite you to participate in an experiment to see if bona fide high-res audio can be reliably distinguished from CD specifications.

The question of whether or not real high-resolution audio can be reliably distinguished from CD audio is a fascinating one—and not easily answered. To explore this question, I would like to enlist the help of AVS members who have audio systems capable of reproducing that difference.

But before I get to the experiment itself, I'd like to share some of the conclusions I've come to after posting several threads about it and reading the hundreds of comments they inspired. (If you haven't read them already, be sure to check out the threads on whether or not high-res audio is irrelevant, the high-res test initial and refined proposal, and amirm's high-res audio debate thread.)

1. The source material must be real high-res audio. The content must include frequencies well above 20 kHz and a dynamic range greater than 93 dB (CD's effective maximum after dithering to deal with quantization noise), and it must be recorded, edited, mastered, and delivered with at least 24-bit/96 kHz resolution. (I'm not going to consider DSD/SACD for this particular experiment.)

2. The playback system must be capable of reproducing the high-res material at full resolution, including the source device, DAC, amplifier, and speakers. The entire system must have a frequency response well above 20 kHz and a dynamic range greater than 93 dB.

3. Very few consumer systems are capable of reproducing frequencies above 20 kHz and a dynamic range greater than 93 dB, so trying to compare a real high-res recording with a CD-quality version of the same material on such a system is pointless. There will be no acoustic difference between the two files (other than perhaps some distortion if the level of the 24/96 content exceeds 93 dB), so there is no possibility of a perceptible difference between them.

4. It is unlikely that the human auditory system can directly perceive frequencies much above 20 kHz. However, there could be other mechanisms that allow humans to perceive ultrasonic frequencies. For example, those frequencies might interact and produce difference or interference tones in the audible range. Also, it could be that ultrasonic frequencies can be detected by other parts of the anatomy, such as the skin. So even if we can't "hear" ultrasonic frequencies directly, their presence might make a perceptible contribution to our sonic experience.

5. It is unknown if headphones can be used in lieu of speakers. Some headphones are specified to reproduce ultrasonic frequencies, but that is often in a free field, not on the ears, and I know of no independent measurements that verify ultrasonic performance in use. Also, if ultrasonic frequencies are detected by other parts of the anatomy, that mechanism won't work with headphones. (New anecdotal evidence suggests that some headphones might work, which I'll explain shortly.)

The Setup
Mark Waldrep (aka Dr. AIX), founder and chief engineer of AIX Records, has prepared three full-length tracks from his catalog that contain ultrasonic frequencies and wider-than-CD dynamic range. The original recordings were recorded, edited, and mastered at 24/96. The tracks have not been dynamically altered, and all metadata have been removed. Each clip has also been downsampled to 16 bits/44.1 kHz, the resolution of Redbook CD audio, using the "gentlest" triangular PDF algorithm in Sonic Studio's Sonic Process, a state-of-the-art sample-rate converter. Then, the downsampled version was upsampled back to 24/96 without adding audio data that wasn't in the lower-res file, making it easier to compare the two versions and more difficult to tell them apart by casual inspection of their file sizes. All files are in uncompressed WAV format.

Here are spectrograms from each track, showing both the 24/96 (red) and 16/44.1 (yellow) versions:


"Mosaic" from Guitar Noir by Laurence Juber


"Just My Imagination" from Stand by Wallace Roney


"On the Street Where You Live" from Tormé Sings Tormé by Steve March Tormé

We realize that putting one version through two sample-rate conversions—even state-of-the-art conversions—will raise some eyebrows, and we are concerned that this could somehow introduce artifacts that might make it easier to distinguish the sound of the converted file. The other alternative was to downconvert the 24/96 file to 16/44.1, then convert both files to analog and back to digital at 24/96 using state-of-the-art DACs (digital-to-analog converters) and ADCs (analog-to-digital converters), but this also would have raised eyebrows by introducing multiple digital/analog conversions. In the end, we decided to go with the double sample-rate conversion to keep everything in the digital domain.

The two versions of each clip are labeled A and B, and which one is 24/96 has been randomly selected for each track with a coin toss. The files are available for anyone to download at the end of this article.

System Requirements
Anyone can download the files and give a listen, but if you'd like to contribute data to the experiment, you need an audio system that is capable of reproducing the extra information in the 24/96 file. (UPDATE: This requirement has been eliminated; see the update note below for more.) What are the required specs? Let's start with system configuration. Basically, the fewer components in the signal chain, the better, so we think the best setup is a source computer connected via USB to a high-quality DAC. The DAC would be connected directly to a power amp, which would, of course, drive the speakers.


The best system configuration for the AVS/AIX high-res audio test is a computer sending 24/96 data to a high-performance DAC that is connected directly to a power amp feeding the speakers.

Of course, there must be an analog volume control after the digital-to-analog conversion to adjust the listening level; this can't be done in the digital domain because it would reduce the number of bits being used. Some DACs include such a control on their analog output, and some power amps provide level controls. If neither component in your system has this control, you must insert an analog preamp between the DAC and power amp.

Obviously, the computer must be able to output native 24/96 via USB, and the DAC must be able to convert that to analog without downsampling. Also, while virtually all DACs use very high internal rates, the output of the digital section must remain at the original 24/96. The analog output of the DAC must be able to support ultrasonic frequencies up to near 48 kHz and a dynamic range greater than 93 dB, and the preamp (if used) and power amp must have similar frequency-response and dynamic-range specs from input to output. Finally, the speakers must be able to reproduce those ultrasonic frequencies and wide dynamic range with minimal distortion.

As we were preparing for this experiment, Mark Waldrep tried playing the files using ABXTester on a Macintosh computer with a Benchmark DAC1 (which has an analog volume control) and Oppo PM-1 headphones, which are spec'd to reach 50 kHz in a free field. To his surprise, he was able to hear a clear difference between the A and B files, indicating that the experiment can be performed with these headphones and DAC—and, thus, probably others as well.

UPDATE: After much consideration, we've decided to eliminate the requirement that participants have a truly high resolution-capable audio system. Instead, anyone can submit their determinations to me via PM, but they must also include a list of equipment in their system (source, DAC, preamp, power amp, speakers or headphones). I am compiling separate results for true HRA systems and non-HRA systems to see what that might suggest.


The Experiment
To perform the experiment, all you need to do is play each file as many times as you wish and decide which one is high-res and which is limited to CD specs. Some have suggested that listeners use an ABX program such as foobar2000 for Windows or ABXTester for the Mac, both of which are free. However, each of these programs (and other similar software) operate somewhat differently from each other and return results that aren't exactly what we're looking for.

You can certainly use these programs to play the files if you like; in fact, the optional ABX Comparator module for foobar2000 makes it easy to switch between the two versions seamlessly during playback, which might help you make up your mind. (ABX Comparator also randomly assigns the files you identify as A and B to the labels X and Y as part of a double-blind test, but we're not using that functionality of the program in this case.)

After deciding whether the A or B version of each track is high res, you can send your determinations to me by PM with the subject line "AVS/AIX HRA Test." If you can prove that your system is capable of reproducing the extra information in the 24/96 versions—and some of you have already sent me your system specs—I'll include your determinations in the experimental data. Then, I will calculate the percentage of correct responses; if the versions are identified correctly much more than 50% of the time (the result expected by random chance), it would suggest that high-res audio can be discerned from CD audio, at least when played on an appropriate system.

Of course, the system requirements are pretty stringent, which means that for most people, high-res audio is irrelevant, at least in terms of perception. On the other hand, if you compare a commercial CD and a high-res file of the same music, the high-res version might well sound different—perhaps better—because it might have been mastered differently than the CD. But we are investigating whether or not the extra information in a well-recorded high-res audio file can be reliably perceived compared with a CD version of the exact same file.

Because there are so many uncontrolled variables—different systems, different rooms—this experiment cannot be considered a rigorous scientific test by any means. Even more important, we have no way of verifying that participants aren't cheating by examining the spectra of the files and/or misinforming us about the capabilities of their systems. All we can do is appeal to your sense of fairness and honesty in the spirit of scientific inquiry. We believe this is an interesting exercise, and we hope you can participate.

To download the files, simply click on the links below. All files have been zipped individually so that browsers initiate a download rather than try to play the files; in addition, all six files have been zipped into a single file so you can download them all at once with one click. (These are big files—the combined file is nearly 1 GB—so it could take some time to download them, depending on your bandwidth.)

Let the listening begin!

UPDATE: After some members identified a 0.2 dB difference in level between the native 24/96 and sample-rate converted versions of each track, we level-matched them and reposted all files as "Take 2." The updated files are available by clicking below.

All in one:

AVS/AIX HRA Test Files 2

Individual files:

Mosaic A2
Mosaic B2
Just My Imagination A2
Just My Imagination B2
On The Street Where You Live A2
On The Street Where You Live B2


Like AVS Forum on Facebook
Follow AVS Forum on Twitter
+1 AVS Forum on Google+
 

·
Registered
Joined
·
1,387 Posts
I would have loved to give it a try. My problem is: I am not that computer wise. I down loaded the zip files but I am not able to transfer them to a CD to place into my Oppo player.

But like I said: I am not wise on what to do. I am probably doing it wrong.
 

·
Registered
Joined
·
2,619 Posts
I would have loved to give it a try. My problem is: I am not that computer wise. I down loaded the zip files but I am not able to transfer them to a CD to place into my Oppo player.

But like I said: I am not wise on what to do. I am probably doing it wrong.

Use a USB memory stick.
 

·
Registered
Joined
·
36 Posts
ummm, Waldrep emphatically did NOT detect a difference between the two files. In fact, he couldn't have been more incorrect. His interpretation that he guessed incorrectly on every single sample meant that he actually got them all correct reveals he knows NOTHING about ABX. Absolutely nothing. More than one commenter in his blog even warned him about his egregious misinterpretation.

If you want to completely invalidate everything with this test, ignore this post.
 

·
Registered
Joined
·
4,889 Posts
The fact that the engineer who recorded, mixed and mastered the recordings can't tell the difference speaks volumes. I have the Juber recording at 24/96 and it does sound superb. I don't have the CD though.
 

·
Registered
Joined
·
1,387 Posts

·
Premium Member
Joined
·
3,258 Posts
Discussion Starter · #11 ·
i just started testing it and stopped at Just_My_Imagination_B.wav because Just_My_Imagination_A.wav has very loud background noise it's simply impossible not to hear it it's so loud and always the same noise. in a ABX i need only 2 clicks in 1 sec to make the right choice...

even if Just_My_Imagination_A.wav is the high res version i highly highly pref Just_My_Imagination_B.wav

so is this a mastering error of some kind or what is happening here?
Hmmm, I don't know; I will investigate...
 

·
Registered
Joined
·
337 Posts
i just started testing it and stopped at Just_My_Imagination_B.wav because Just_My_Imagination_A.wav has very loud background noise it's simply impossible not to hear it it's so loud and always the same noise. in a ABX i need only 2 clicks in 1 sec to make the right choice...

even if Just_My_Imagination_A.wav is the high res version i highly highly pref Just_My_Imagination_B.wav

so is this a mastering error of some kind or what is happening here?
I can't even get Just My Imagination A to extract. I think something is wrong with the file.
 

·
Registered
Joined
·
3,321 Posts
What about it? Where do you see it? I know of no such folder...
i found out what that is these are invisible folders on mac osx, on windows they are visible.

there are 6 300 byte+ files in as wave with the same name the test file have. so looks like they are simply cache data from mac or something like that.

i check this file with different playback system to make sure there is nothing wrong with my system.

I can't even get Just My Imagination A to extract. I think something is wrong with the file.
maybe a CRC should be added to all file or at least posted in the thread from the creator. but if the file is corrupted you normally got some else as added noise.

a CRC for the zip file should be enough if i think about it
 

·
Premium Member
Joined
·
3,258 Posts
Discussion Starter · #14 ·
i just started testing it and stopped at Just_My_Imagination_B.wav because Just_My_Imagination_A.wav has very loud background noise it's simply impossible not to hear it it's so loud and always the same noise. in a ABX i need only 2 clicks in 1 sec to make the right choice...

even if Just_My_Imagination_A.wav is the high res version i highly highly pref Just_My_Imagination_B.wav

so is this a mastering error of some kind or what is happening here?
I can't even get Just My Imagination A to extract. I think something is wrong with the file.
I just downloaded both versions of Just My Imagination and listened to both all the way through. They both extracted without problems, and I hear no background noise on either one. I'm on a Mac running OSX 10.6.8. I only listened through the computer's headphone output just to see if there was a problem, and I didn't hear one.
 

·
Premium Member
Joined
·
3,258 Posts
Discussion Starter · #15 · (Edited)
i found out what that is these are invisible folders on mac osx, on windows they are visible.

there are 6 300 byte+ files in as wave with the same name the test file have. so looks like they are simply cache data from mac or something like that.

i check this file with different playback system to make sure there is nothing wrong with my system.


maybe a CRC should be added to all file or at least posted in the thread from the creator. but if the file is corrupted you normally got some else as added noise.

a CRC for the zip file should be enough if i think about it
Hmm, well, the files were put together on a Mac and sent to AVS for posting from a Mac, so maybe that's where the invisible folder comes from. For now, just ignore any such folders and files.

I'll ask about adding a CRC to the files...stand by.
 

·
Registered
Joined
·
3,321 Posts
I just downloaded both versions of Just My Imagination and listened to both all the way through. They both extracted without problems, and I hear no background noise on either one. I'm on a Mac running OSX 10.6.8. I only listened through the computer's headphone output just to see if there was a problem, and I didn't hear one.
i did some checks.
this isn't happening with ASIO but with WASAPI (my default) and DS.
so it would be nice if someone could test this on his Windows PC.
WASAPI and ASIO should be Bit perfect.

i played the file back with my "TV SPEAKER" and i can't hear this problem.

i try a different playback program tomorrow.

Foobar2000 and MPC-BE WASAPI have this problem for me.
i can simply use ASIO so i don't have a real problem anymore but why is this happening.
 

·
Registered
Joined
·
1,372 Posts
What about it? Where do you see it? I know of no such folder...
It's ok, it's just a Windows thing. As that folder is invisible to us Mac users.

Fenton said:
Macs (which are now Unix) store various info, window options, etc. in little files which are normally invisible on Macs, but which show up on Windows. One thing you can do is use a utility to remove them before sending files to Windows clients. There are various free programs; the one I've been using is CleanArchiver, with the "zip" option. Leave it on your Dock, then just drop folders on it.
http://www.sopht.jp/cleanarchiver/
http://fmforums.com/forum/topic/47465-macosx-folder/
 

·
Registered
Joined
·
4,893 Posts
On the spectrogram for "Just My Imagination", what is that big peak at 44khz?
 

·
Registered
Joined
·
526 Posts
First, I LOVE AIX records. I have several of their recordings (Blu-Ray, DVD, and iTraxx downloads). I love them because they make quality multi-channel recordings. Furthermore, they provide them with different perspectives (audience and stage). My preference is always the stage perspective. I have 5 full range speakers and their multi-channel recordings sound tremendous.

I downloaded the test tracks on this thread and just finished listening to them. I could not tell the difference. Both sounded wonderful. I used Foobar with an Oppo 105 as a DAC. I don't think the rest of my system meets all the requirements (McIntosh MX-119 pre-amp, Spectron Class D amp, and Gallo Reference 3.5 speakers).

Edit: Please ignore this as I misread the original post: "If I understand correctly, the original 24/96 files were downsampled to 16/44 and then upsampled to 24/96 again? Isn't this an unfair test? You've limited what the 24/96 files can produce by downsampling it. And that information can't be gotten back by upsampling again. Am I missing something?"
 

·
Registered
Joined
·
4,893 Posts
If I understand correctly, the original 24/96 files were downsampled to 16/44 and then upsampled to 24/96 again? Isn't this an unfair test? You've limited what the 24/96 files can produce by downsampling it. And that information can't be gotten back by upsampling again. Am I missing something?
I thought that was the point: Comparing the original hi-res file to one that had gone through the Redbook CD audio 16/44 format to see if there is an audible difference.
 
1 - 20 of 222 Posts
Top