AVS Forum

AVS Forum (http://www.avsforum.com/forum/)
-   Audio theory, Setup and Chat (http://www.avsforum.com/forum/91-audio-theory-setup-chat/)
-   -   AVS/AIX High-Resolution Audio Test: Take 2 (http://www.avsforum.com/forum/91-audio-theory-setup-chat/1598417-avs-aix-high-resolution-audio-test-take-2-a.html)

Scott Wilkinson 07-10-2014 11:16 AM

AVS/AIX High-Resolution Audio Test: Take 2
 
http://www.avsforum.com/forum/imageh...1a40553627.jpg

After correcting for a 0.2 dB difference in level, the native 24/96 and SRC versions of the test files are again ready for downloading.

As many of you know, I'm conducting an informal experiment with the AVS community to investigate whether or not true high-resolution audio files—with information beyond what a Redbook CD can accommodate—can be reliably distinguished from the same files that have been downconverted to CD specs. The parameters of the experiment can be found here.

Shortly after I made the audio files available for download, several members reported that one of each pair of files was 0.2 dB lower in level than the other. This was determined to be the result of subjecting a copy of the native 24/96 file to two sample-rate conversions—first down to 16/44.1 and then back up to 24/96—in order to hide the fact that it has a lower resolution. It turns out that the sample-rate conversion software—Sonic Solutions Sonic Process in this case—lowers the level of the file being converted by 0.1 dB to provide some headroom so the calculations will not cause any clipping. (Most sample-rate converters do this, not just Sonic Process.) Because the "hidden" 16/44.1 version went through two conversions, it was 0.2 dB lower in level than the native 24/96 version, which is close enough to the just-noticeable difference (JND) that listeners might pick the native high-res version on that basis.

A number of solutions to this problem were proposed, including a couple of rather complicated, multistep conversion procedures. We also experimented with normalizing both files as well as simply increasing the level of the converted file by 0.2 dB and decreasing the level of the native file by 0.2 dB. In addition, we considered raising the level of the file that would be converted to CD specs by 0.2 dB before doing the conversion, but there wasn't enough headroom in two of the three files to do that without clipping.

In the end, neither Mark Waldrep (aka Dr. AIX), who provided the files, nor I believe that the procedure by which the levels are matched makes much difference for this experiment. Still, I wanted to process the files as little as possible, so I rejected the idea of normalizing all of them. Because I subscribe to the "cutting is better than boosting" school of thought, I decided that the best course of action was to decrease the level of the native 24/96 files by 0.2 dB rather than boost the level of the converted files. Yes, that decreases the dynamic range of the native file, but only by 0.2 dB, which still leaves it beyond what CD can handle in each case. I have checked all three pairs of files using foobar2000's ReplayGain Scan function, and each pair is now within 0.01 dB of each other, well below the JND of perceptibility.

Of course, some AVS members will object to this approach, but I seriously doubt there would be unanimous approval of any solution within the AVS community, so I made an executive decision to do it this way. Those who believe my solution is unreasonable are free to simply not participate and to argue their position in the comments here and in the other related threads; all I ask is that you keep it respectful and not personal. Just keep in mind that this is an informal—and, I hope, fun—exercise that invites AVS members to make their own determination about whether or not the extra information in a bona fide high-res audio file makes a perceptible difference over the same content that conforms to CD specs.

Some members have also suggested that the results of the experiment should not be limited to those with audio systems capable of reproducing the extra information in the native high-res files. It's fine with me if you want to submit your determination of which files are high-res regardless of your system's capabilities, but all submissions must include a complete list of all components used in the test, including makes and models, as well as how they are connected together. I will compile the results separately for those with truly high res-capable systems and those with systems that can reproduce no more than CDs can deliver.

The new audio files are now available by clicking on the links below. As before, we've zipped each file so all browsers will initiate a download rather than trying to play them, and we've combined all six files in a single zipped package. The "A" and "B" assignments are new, so they are labeled "A2" and "B2" to make sure there is no confusion with the previous files. To make the assignments, I used the random-number generator at random.org, which is based on atmospheric noise rather than a pseudo-random algorithm—hence, it is truly random. I set the range from 1 to 100 and let it generate a random number for each pair of files—if the number was odd, the native high-res version became A2; if it was even, the native high-res version became B2.

So let's try this again, shall we? If you have already submitted your determinations, please do so again with these new files, and be sure to include a complete description of your audio system. Remember, this is not an ABX test—simply listen to each pair of files as many times as you wish, at the same volume level for both files, and see if you can tell them apart. And please don't go poking around the files with audio software—just listen. If you can, decide which one is high-res, A2 or B2, and send me your determinations by PM with the subject "AVS AIX HRA Test 2."

Thanks to all who identified the level-difference problem and offered suggestions about how to correct it.

All in one:

AVS/AIX HRA Test Files 2

Individual files:

Mosaic A2
Mosaic B2
Just My Imagination A2
Just My Imagination B2
On The Street Where You Live A2
On The Street Where You Live B2

-----------

Like AVS Forum on Facebook
Follow AVS Forum on Twitter

+1 AVS Forum on Google+

Contuzzi 07-10-2014 04:43 PM

Are you going to let us know which were which on the last test?

CharlesJ 07-10-2014 05:08 PM

Great to hear this and the "executive decision" you provided. :D

eliwankenobi 07-10-2014 07:47 PM

downloading now, should be fun!

MLXXX 07-11-2014 01:12 AM

I recall that in testing my own hearing of music files in the past, using automated ABX software, I could demonstrably perceive 0.2dB as a difference, but not 0.1dB. [I recall that the subjective difference I perceived was not of loudness, simply an extremely small uncharacterised "difference".] So indeed it seems quite important to have corrected the files for comparison to bring them to equal amplitude within a very close tolerance.

As for sample rate conversions causing slight amplitude differences, I have never noticed that with the open source product Audacity [excluding any possible effects from introduced dither].

imagic 07-11-2014 08:30 AM

I could not tell any difference, but my system is not equipped for ultrasonic playback. I will say this, somewhere between 16,000 Hz and 17,000 Hz I lose the ability to hear a sine wave. I can't see how I'd be able to hear any recorded information at frequencies higher than that. just because it was recorded with music—not when an isolated tone is already completely inaudible to me at 20,000 Hz.

JimShaw 07-12-2014 10:18 AM

As for me through my system, I could not discern any difference.

It would have been interesting to have listened to the same files through a few of the extreme high-end audio systems at the T.H.E. SHOW at Newport Beach this year. Would I have noticed any difference?? Who knows but it would have been fun.



m

amirm 07-12-2014 10:32 AM

Quote:

Originally Posted by JimShaw (Post 25686473)
As for me through my system, I could not discern any difference.

It would have been interesting to have listened to the same files through a few of the extreme high-end audio systems at the T.H.E. SHOW at Newport Beach this year. Would I have noticed any difference?? Who know but it would have been fun.

m

I can reliably hear the difference on my laptop and headphone. No exotic system is needed. I hope everyone tries the files regardless of what system they have.

JimShaw 07-12-2014 11:36 AM

Quote:

Originally Posted by amirm (Post 25686769)
I can reliably hear the difference on my laptop and headphone. No exotic system is needed. I hope everyone tries the files regardless of what system they have.

After I read your post, I became intrigued. I asked my wife to sit and listen. She was able to hear a difference in all three through my audio system.

Maybe I should have my ears checked out.

CharlesJ 07-12-2014 01:47 PM

Quote:

Originally Posted by JimShaw (Post 25687865)
After I read your post, I became intrigued. I asked my wife to sit and listen. She was able to hear a difference in all three through my audio system.

Maybe I should have my ears checked out.

Yes, but nothing you can do about its condition. ;) :D

JimShaw 07-12-2014 01:53 PM

Quote:

Originally Posted by CharlesJ (Post 25690305)
Yes, but nothing you can do about its condition. ;) :D

You might be right but I will keep trying to convince my wife that this will do it for me...

http://www.mini-zenith.com/high-end-...ystem-2011.jpg

amirm 07-12-2014 02:12 PM

Quote:

Originally Posted by JimShaw (Post 25690425)
You might be right but I will keep trying to convince my wife that this will do it for me...

http://www.mini-zenith.com/high-end-...ystem-2011.jpg

Sounds good to me, pun intended! :D

CharlesJ 07-12-2014 04:59 PM

Quote:

Originally Posted by JimShaw (Post 25690425)
You might be right but I will keep trying to convince my wife that this will do it for me...

http://www.mini-zenith.com/high-end-...ystem-2011.jpg

Is that left side supporting that beam? ;)

p5browne 07-12-2014 05:39 PM

Age and Gender Selective Test!
 
For those under 30, are of the Female Gender under 60, but definitely not for those from the Construction Industry!

(May also exclude Younger Generation who like to listen with ear buds blasting out 110dB!)

HomerJau 07-15-2014 02:14 AM

Quote:

Originally Posted by JimShaw (Post 25690425)
You might be right but I will keep trying to convince my wife that this will do it for me...

http://www.mini-zenith.com/high-end-...ystem-2011.jpg

I couldn't sit in that room with such a low ceiling :)

dabotsonline 07-16-2014 07:23 AM

Apologies if this is the wrong thread, but is anyone using Genelec 8260A active monitors for this test? I know they're highly regarded, and the specs ( http://www.genelec.com/products/8260a/ ) state: 26 Hz - 40 kHz (-3 dB).

sjschaff 07-17-2014 06:42 AM

Interesting test
 
Heard a difference in the sound-stage presentation but that's about it. Need to try with IEMs and laptop.

Mpls-Sound 07-20-2014 10:36 AM

I could easily determine the high res version for the Mosaic and On the Street Where You Live files. Just My Imagination was a toss up though and I guessed wrong on that one.

My set up was a Mac Pro Quad-Core Tower, a MusicStreamer II DAC, a Yamaha HTR-5550 receiver and Pioneer SP-B22 speakers.

arnyk 07-20-2014 11:06 AM

Quote:

Originally Posted by Mpls-Sound (Post 25895938)
I could easily determine the high res version for the Mosaic and On the Street Where You Live files. Just My Imagination was a toss up though and I guessed wrong on that one.

My set up was a Mac Pro Quad-Core Tower, a MusicStreamer II DAC, a Yamaha HTR-5550 receiver and Pioneer SP-B22 speakers.

What were your ABX scores like?

Mpls-Sound 07-20-2014 11:27 AM

No sure what you meant by ABX scores but after I made my determination I opened the files and looked at the Spectrogram to see the difference.

That cut off at 22kHz isn't gradual either which makes me think what is noticeable is more a re-enforcment of the lower harmonics (those that are actually audible) and which is absent in the CD quality file. Like when we add a sub-woofer and experience a fuller overall sound.

High resolution files might actually justify those super tweeters that have been around for some time. I'm looking forward to the new Sony - Core Series bookshelf speakers, that have a frequency response of 53Hz - 50kHz.

LifeofDesperation 07-20-2014 02:24 PM

Quote:

Originally Posted by Scott Wilkinson (Post 25638361)
http://www.avsforum.com/forum/imageh...1a40553627.jpg

After correcting for a 0.2 dB difference in level, the native 24/96 and SRC versions of the test files are again ready for downloading.

As many of you know, I'm conducting an informal experiment with the AVS community to investigate whether or not true high-resolution audio files—with information beyond what a Redbook CD can accommodate—can be reliably distinguished from the same files that have been downconverted to CD specs. The parameters of the experiment can be found here.

Shortly after I made the audio files available for download, several members reported that one of each pair of files was 0.2 dB lower in level than the other. This was determined to be the result of subjecting a copy of the native 24/96 file to two sample-rate conversions—first down to 16/44.1 and then back up to 24/96—in order to hide the fact that it has a lower resolution. It turns out that the sample-rate conversion software—Sonic Solutions Sonic Process in this case—lowers the level of the file being converted by 0.1 dB to provide some headroom so the calculations will not cause any clipping. (Most sample-rate converters do this, not just Sonic Process.) Because the "hidden" 16/44.1 version went through two conversions, it was 0.2 dB lower in level than the native 24/96 version, which is close enough to the just-noticeable difference (JND) that listeners might pick the native high-res version on that basis.

A number of solutions to this problem were proposed, including a couple of rather complicated, multistep conversion procedures. We also experimented with normalizing both files as well as simply increasing the level of the converted file by 0.2 dB and decreasing the level of the native file by 0.2 dB. In addition, we considered raising the level of the file that would be converted to CD specs by 0.2 dB before doing the conversion, but there wasn't enough headroom in two of the three files to do that without clipping.

In the end, neither Mark Waldrep (aka Dr. AIX), who provided the files, nor I believe that the procedure by which the levels are matched makes much difference for this experiment. Still, I wanted to process the files as little as possible, so I rejected the idea of normalizing all of them. Because I subscribe to the "cutting is better than boosting" school of thought, I decided that the best course of action was to decrease the level of the native 24/96 files by 0.2 dB rather than boost the level of the converted files. Yes, that decreases the dynamic range of the native file, but only by 0.2 dB, which still leaves it beyond what CD can handle in each case. I have checked all three pairs of files using foobar2000's ReplayGain Scan function, and each pair is now within 0.01 dB of each other, well below the JND of perceptibility.

Of course, some AVS members will object to this approach, but I seriously doubt there would be unanimous approval of any solution within the AVS community, so I made an executive decision to do it this way. Those who believe my solution is unreasonable are free to simply not participate and to argue their position in the comments here and in the other related threads; all I ask is that you keep it respectful and not personal. Just keep in mind that this is an informal—and, I hope, fun—exercise that invites AVS members to make their own determination about whether or not the extra information in a bona fide high-res audio file makes a perceptible difference over the same content that conforms to CD specs.

Some members have also suggested that the results of the experiment should not be limited to those with audio systems capable of reproducing the extra information in the native high-res files. It's fine with me if you want to submit your determination of which files are high-res regardless of your system's capabilities, but all submissions must include a complete list of all components used in the test, including makes and models, as well as how they are connected together. I will compile the results separately for those with truly high res-capable systems and those with systems that can reproduce no more than CDs can deliver.

The new audio files are now available by clicking on the links below. As before, we've zipped each file so all browsers will initiate a download rather than trying to play them, and we've combined all six files in a single zipped package. The "A" and "B" assignments are new, so they are labeled "A2" and "B2" to make sure there is no confusion with the previous files. To make the assignments, I used the random-number generator at random.org, which is based on atmospheric noise rather than a pseudo-random algorithm—hence, it is truly random. I set the range from 1 to 100 and let it generate a random number for each pair of files—if the number was odd, the native high-res version became A2; if it was even, the native high-res version became B2.

So let's try this again, shall we? If you have already submitted your determinations, please do so again with these new files, and be sure to include a complete description of your audio system. Remember, this is not an ABX test—simply listen to each pair of files as many times as you wish, at the same volume level for both files, and see if you can tell them apart. And please don't go poking around the files with audio software—just listen. If you can, decide which one is high-res, A2 or B2, and send me your determinations by PM with the subject "AVS AIX HRA Test 2."

Thanks to all who identified the level-difference problem and offered suggestions about how to correct it.

All in one:

AVS/AIX HRA Test Files 2

Individual files:

Mosaic A2
Mosaic B2
Just My Imagination A2
Just My Imagination B2
On The Street Where You Live A2
On The Street Where You Live B2

-----------

Like AVS Forum on Facebook
Follow AVS Forum on Twitter

+1 AVS Forum on Google+

Where can I find the test results?

Myron Licht 07-21-2014 12:24 PM

computer speakers and pc(crap)

just my imagination A2
mosaic B2
onthe street where u live B2

Trent Hinkforth 07-23-2014 04:09 PM

On my low cost set up an old MacBook with Sony MDR-V150 headphones ($20) plugged in it sounded good as follows
Just My Imagination A2 Mosaic B2 On The Street Where You Live A2

On a higher end set up with a Presonus Inspire audio interface plugged in via Firewire 400 to the MacBook
with a pair of AKG K271 studio headphones ($289 list) it sounded as follows: the opposite
Just My Imagination B2 Mosaic A2 On The Street Where You Live B2



my guess is the higher resolution is
Just My Imagination B2 Mosaic A2 On The Street Where You Live B2


great music and a interesting test

Trent Hinkforth
or Mr Eq

Scott Wilkinson 07-24-2014 10:28 AM

Quote:

Originally Posted by dabotsonline (Post 25786185)
Apologies if this is the wrong thread, but is anyone using Genelec 8260A active monitors for this test? I know they're highly regarded, and the specs ( http://www.genelec.com/products/8260a/ ) state: 26 Hz - 40 kHz (-3 dB).

I think those would be great speakers to use for this test. I would make sure there's no bottleneck limited to 20 kHz anywhere else in the system. I encourage you to give a listen and PM me with your results, the make and model of each component in your system, and how they are connected.

dabotsonline 07-24-2014 10:33 AM

Quote:

Originally Posted by Scott Wilkinson (Post 26008418)
Quote:

Originally Posted by dabotsonline
Apologies if this is the wrong thread, but is anyone using Genelec 8260A active monitors for this test? I know they're highly regarded, and the specs ( http://www.genelec.com/products/8260a/ ) state: 26 Hz - 40 kHz (-3 dB).

I think those would be great speakers to use for this test. I would make sure there's no bottleneck limited to 20 kHz anywhere else in the system. I encourage you to give a listen and PM me with your results, the make and model of each component in your system, and how they are connected.

Sadly, I don't own them myself: I just mentioned them because I would imagine that many members on this forum - particularly if they're music producers - would own them, given that they're regarded as a fairly popular model amongst that demographic.

Scott Wilkinson 07-24-2014 10:39 AM

Quote:

Originally Posted by LifeofDesperation (Post 25900738)
Where can I find the test results?

I will be posting the results so far either today or tomorrow.

Scott Wilkinson 07-24-2014 10:42 AM

Quote:

Originally Posted by Mpls-Sound (Post 25895938)
I could easily determine the high res version for the Mosaic and On the Street Where You Live files. Just My Imagination was a toss up though and I guessed wrong on that one.

My set up was a Mac Pro Quad-Core Tower, a MusicStreamer II DAC, a Yamaha HTR-5550 receiver and Pioneer SP-B22 speakers.

I regret you didn't send me your results before looking at the files.

Mpls-Sound 07-24-2014 11:34 AM

Quote:

Originally Posted by Scott Wilkinson (Post 26008842)
I regret you didn't send me your results before looking at the files.

I don't see what difference it would make. I took the test honestly then looked at the files. I got 2 out of 3 correct.

Scott Wilkinson 07-24-2014 06:30 PM

Quote:

Originally Posted by Mpls-Sound (Post 26010234)
I don't see what difference it would make. I took the test honestly then looked at the files. I got 2 out of 3 correct.

I appreciate that. However, I'm compiling a database (spreadsheet, actually) of each participant's results and system in order to learn as much as possible. Would you be willing to PM me your results (the two you got right and the one you got wrong) and your system configuration, including make and model of each device and how they are connected? Thanks!

EDIT: I see your system in a previous comment in this thread, so never mind about that. But I would greatly appreciate you PMing me (not posting in any thread) which ones you got right and wrong.

mekduk 07-26-2014 07:55 AM

Am new here and immediately joined/followed up after watching Scott's conversation with Michael Fremer on the back episode of Home Theater Geeks vidcast. So the results are going to be sent to the individual concerned or will it be published on this thread?


All times are GMT -7. The time now is 01:00 PM.

Powered by vBulletin® Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.

vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging (Pro) - vBulletin Mods & Addons Copyright © 2014 DragonByte Technologies Ltd.