AVS Forum banner

1 - 20 of 620 Posts

·
Registered
Joined
·
15,391 Posts
Discussion Starter #1 (Edited)
Do modern DACs sound different from each other? [And what about analog RCA cables and USB cables?] Many audiophiles and professional magazine reviews claim they differ audibly but I have my doubts based on controlled experiments such as the one done at Tom's Hardware comparing a $2000 DAC vs. one that's about $2.

Obviously it would be impossible to test every single DAC against every other one, but I have another test. Let's test if a very common and inexpensive one, the $7.99 Apple iPhone DAC, A1749 (aka “Lightning to 3.5 mm Headphone Jack Adapter") is audibly perfect playing music, i.e."transparent", or if instead it in some way alters/distorts/obscures details in the music passing through it in some discernible way. I have set up just such a test to determine this, which anyone with web access is invited to participate in, and it is completely free. [Internal pics of the DAC and its long model number appear in "Step 9" here.]

This is based on a test called an SWBT (straight wire bypass test) or A/B Bypass Test. "Straight wire with gain test" is the similar amplifier version. Peter Walker of QUAD, Stewart Hegeman of H/K and Eico, and J. Gordon Holt, founder of Stereophile magazine, were all proponents of this test design. Here is how it works.

To test how accurately an audio device passes music from its input to output you conduct a blind listening test where you compare the device being inserted into, versus removed from, a stereo system's signal path while the device is set to “unity gain”, i.e., the volume level in is the same as the volume level out, measured precisely. Done.

These two signal paths, A and B, are compared under double blind conditions to be sure there is no cognitive bias influencing the listeners. The free download Foobar, with its ABX component installed, let's us compare two digital files prepared this way.

It is important to understand that a DAC and an ADC are the exact same thing but work in opposite directions. If you take the output of a DAC and send it to an ADC the signal goes: D to A to D. There should be no alteration if the DAC and ADC are both "perfect". That's what we do in this test for File B whereas File A is the original, untouched digital signal. File A is the "master" or "reference". So File A is Digital and File B is "Digital, to Analog, and back to Digital". Our test is: D versus "D to A to D". Do they sound the same or do they sound different?

Here are the two files' signal paths to be compared in Foobar ABX. Note that digital File A, the Reference or Master, is left in its original, pristine condition from start to finish and is never re-recorded nor adjusted. It is exactly as it appeared on the CD without alteration.

File A:

Reference digital song file (or a bit-accurate 30-sec. excerpt if posted online for legal reasons) --->

Played back on a High Quality DAC in the listener's preferred system.

File B:

Master digital song file (or a bit-accurate 30-sec. excerpt if posted online for legal protection)--->

Digitally transferred to my Apple iTunes folder-->
Digitally transferred to my iPhone X via Apple's free USB cord-->
Manually played on the DUT (device under test), an Apple A1749 DAC, $7.99-->
Monoprice 3.5mm stereo mini-plug to RCA jack adapter, $.67 -->
Monoprice 6ft. RCA cable 659, new, $1.05 (it was $.87 ea. when I bought ten of them on sale)-->
Hosa YPR-102 1/4" TRS to Dual RCAF Stereo Breakout Cables, $4.45 x2
Behringer UMC204HD 4-in-1 USB interface (Stereo XLR mic preamp, Headphone amp, DAC, ADC), $79.95 (when I bought it) being used with its best quality "insert" inputs to its ADC -->
Free, basic USB cable the above ADC combo box came with-->
Anker 4-port USB Hub-->
My Asus laptop's USB port-->
Audacity digital recording software, free -->
Audacity digital editing to adjust the recording's level and time sync to match the original file-->
The song is exported in 16-bit/44kHz PCM wav format to my laptop's hard drive as a NEW file-->
This recorded (and original Master) files are uploaded to Dropbox for ABX testers to access-->

Played back on a High Quality DAC in the listener's preferred system.

See the over dozen extra steps in File B, including passing through the inexpensive DAC? It is my contention all of those steps do technically degrade the sound ever so slightly, but by such a small amount they are audibly “perfect” or “transparent” to the ear. That is, everyone's ear, through any system.

UPDATE 05/06/20: File A is the CD digital sound and File B has additionally passed through this chain, including the $7.99 white DAC:



These extra steps, at least to my ears according to the ABX test, are indeed transparent. Give it a go yourself and post your complete score sheet, including its verification number along the bottom, to show how you did.

In a normal 16 trial Foobar ABX test, statistically speaking, a test score of more than 12 correct shows a strong likelihood the discernability was not merely due to chance. [It shows a p-value less than 0.05, which most agree is statistically significant.] Better than 12 correct out of 16 wins!

At the time I prepared the test the first person who had agreed to take the test got to pick the specific music of his choice, a well recorded Gershwin tune which I then bought as a CD. Another person expressed some interest in taking the test at the time too. The test taker made a great choice, in my opinion; the well recorded CD has nice dynamic contrast and a broad range of frequencies.

This test is just phase one. Assuming people find my recording chain is indeed “perfect” to their ears and indistinguishable from the master file, based on their posted, complete ABX score sheets-- a validation my overall test protocol works--then I plan to insert my mid-fi AVR into the recorded File B's overall signal path as well, to test its transparency. I may switch to an easier to use conventional DAC as well. Recording this little dongle one via a touchscreen device my klutzy fingers have trouble operating, manually, is an awkward process for me, but I'm glad I have now done it.

There is a video on all this in the works.

UPDATE 04/30/20: A video authenticating each file's creation and identity is now posted later in the thread:
The video shows each file's SHA-1 (digital fingerprint) right as I create each file so readers can verify the files I uploaded for testing are indeed the exact same files you see me create in the video. Their authenticity can be tested by anyone by checking that their SHA-1 "digital fingerprints" match what I state they are as I create them. Any alteration or substitution in a file would be revealed by a different SHA-1 number. This verification can be done with various programs or on-line calculators such as the free one you see me using during the file creation process at www.md5file.com

More info appears in the Dropbox download folder's READ ME file, linked to below. The two files, A and B, are explained and identified here.

Exact Audio Copy of the CD's track 04, cut down to 30sec. in Lossless edit mode:
FILE A: tr0430sec.wav
SHA-1: 2783dc8627da71672762b9dfb2ed971ac90f2370

Above "File A" after passing through an Apple $7.99 DAC, Monoprice ~$1 analog
RCA cable, and converted back to digital via an ADC connected by a free USB cable:
FILE B: tr0430sDACd.wav
SHA-1: b13ced990bde6228bab66aa745994f96a1d324e3


The question at hand is "Are the dirt cheap DAC, analog RCA, and USB cables audibly perfect, aka 'transparent'? Or do they in some way alter/degrade the sound?" The Foobar ABX test determines under double blind conditions if a listener can detect an audible difference between Files A and B, and if so with what level of statistical significance. The SHA-1 of the files are listed on the final score sheet to verify the correct, files were used. [I accidentally grabbed the wrong file at one point during the production myself, after all, but luckily I caught my mistake.]

 The two ABX files and a READ ME file are found in this free Dropbox download folder. At this link you may be invited to sign up for a Dropbox account, however there's no need to in order to download the files. Just ignore their ad and scroll to the bottom (or the side?) to find the direct download option:

https://www.dropbox.com/sh/sx7knsys07oa2cz/AACEGvJnZLZjdLtxkrWjbbxea?dl=0

I explain how to download, install, and use Foobar with its added ABX test component here:

UPDATE: My video explains from soup to nuts how to download and install Foobar and then its optional ABX Comparator component, how to take a test, and a brief discussion on how to read the final score sheet tally after finishing a test. Links discussed in it again provided here for easy access:

https://www.foobar2000.org/download

https://www.foobar2000.org/components/view/foo_abx

Once successfully downloaded and installed people can jump straight to the test taking instructions in the future, which start around 3m07s if they want. [That is, you can use it as a refresher course if you forget how to use ABX.]

There are several ways to verify the SHA-1 status of a file, if need be, but online I use the free calculator option at www.md5file.com. Again, they may invite you to sign up but there is no need to.

UPDATE: The running tally of all test results so far appears in post 570.
 

·
Registered
Joined
·
1,448 Posts
I appreciate all your work but is this because you cannot tell differences in dac's that others can?...if so and you have setup everything correctly you really should publish this to share with the world.

Sent from my MHA-L29 using Tapatalk
 
  • Like
Reactions: Goalline

·
Registered
Joined
·
20,837 Posts
I appreciate all your work but is this because you cannot tell differences in dac's that others can?...if so and you have setup everything correctly you really should publish this to share with the world.
You should take the test and do the same.
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #4 (Edited)
I appreciate all your work but is this because you cannot tell differences in dac's that others can?...if so and you have setup everything correctly you really should publish this to share with the world.
I take it you are one of these people who can tell that inexpensive DACs like this $7.99 one supposedly degrades the sound, so may we please see your completed score sheet?
 

·
Registered
Joined
·
8,499 Posts
thanks for doing this @m.zillch I will slowly get moving on taking your tests.
 
  • Like
Reactions: m. zillch

·
Registered
Joined
·
1,448 Posts
I take it you are one of these people who can tell that inexpensive DACs like this $7.99 one degrades the sound, so may we please see your completed score sheet?
Never said it degrades the sound..it's in the analog output stage with dac's that differences are heard.I can appreciate a fun little test like most but if your point of this is too PROVE there are not differences in dac's regardless of price then I'm of the mind you should publish it for all.Is that not a fair question?

Sent from my MHA-L29 using Tapatalk
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #7 (Edited)
thanks for doing this @m.zillch I will slowly get moving on taking your tests.

Torii, thank you. I really appreciate you agreeing to take the test (it means a lot to me) and I also think you selected excellent music.

I know I have said it before but it's worth repeating. There is no rush. I've worked professionally in high end audio for decades so this stuff is in my blood and I've been both taking and administering blind, precisely level matched audio tests for literally decades but there's a lot to take in for people new to Foobar ABX. Like riding a bicycle it seems easy once you know how, but for a first time user you need to learn first and that may take a few days, or more, before you can confidently say "Yes, I know how to ride a bike."

I have taken this particular $7.99 DAC transparency test several times and never have I had an "Ah-Ha" moment where I suddenly thought I was hearing a clear distinction. I took the test a few times and my score was always near 8/16, which means 8 right out of 16 total, i.e., no better than flipping a coin. Actually most tests, if there is indeed no audible difference, will be a little off from that, like 7/16, 9/16, etc.. In fact statistically it is more likely to get one of those two scores than a dead even 8/16 every single time.

A meaningful score which suggests the listener is probably indeed hearing a difference, i.e. statistically significant to the 95% confidence level, is a score over 12/16, meaning either 13/16, 14/16, 15/16, or 16/16.

The very best I ever got is as follows, and I personally think it was just a random, ever so slightly better than average score that time, means nothing, and it certainly is not statistically significant to the generally accepted 95% confidence level:

foo_abx 2.0.6d report
foobar2000 v1.5.3
2020-04-23 08:57:28

File A: tr0430sec.wav
SHA1: 2783dc8627da71672762b9dfb2ed971ac90f2370
File B: tr0430sDACd.wav
SHA1: b13ced990bde6228bab66aa745994f96a1d324e3

Output:
DS : Primary Sound Driver
Crossfading: NO

08:57:28 : Test started.
08:58:09 : 01/01
08:58:41 : 01/02
08:59:14 : 02/03
09:01:04 : 03/04
09:02:52 : 03/05
09:04:58 : 04/06
09:05:38 : 05/07
09:05:50 : 05/08
09:06:22 : 06/09
09:10:03 : 07/10
09:15:53 : 08/11
09:17:49 : 09/12
09:19:44 : 10/13
09:21:18 : 10/14
09:23:34 : 11/15
09:28:09 : 11/16
09:28:09 : Test finished.

----------
Total: 11/16
p-value: 0.1051 (10.51%)

-- signature --
b2f64b648fc3951c743fe2aa1689980369658038


When people post their score sheets it is important that it is complete, just like I have shown above, starting with "foo_abx 2.0.6d report" and ending with some long string of numbers showing the authentication signature. Please do this by cutting and pasting the text from your saved est score sheet file Foobar prompts you to save at the completion of each test.This way the text can be pasted into the included signature verification check tool appearing at the bottom of the score sheet on the Foobar ABX test window. You are welcome to do that with my test above, here:
https://www.foobar2000.org/abx/signaturecheck
 
  • Like
Reactions: Goalline

·
Registered
Joined
·
15,391 Posts
Discussion Starter #8 (Edited)
Never said it degrades the sound..it's in the analog output stage with dac's that differences are heard.
The Apple $7.99 DAC has a built in headphone amp so it indeed has an analog output stage included in this blind listening test as well, so if either the DAC chip or the headphone amp's analog output stage degrades the sound of music to the ear, then that also will be revealed in this test, with a score showing meaningful statistical significance, as I discussed in my last post.
 

·
Registered
Joined
·
11,079 Posts
Nice. I'm going to play around, with this challenge.
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #10 (Edited)
I can appreciate a fun little test like most but if your point of this is too PROVE there are not differences in dac's regardless of price
There are, I'm guessing, thousands of DACs out there (10's of thousands?). This is a test of just one which currently costs $7.99. If it and its analog output stage are indeed perfectly transparent in their reproduction of music, as I suspect, we can assume there are probably others as well. [At least I'll assume that; I can't speak for everyone.:) ]

It also would be trivially easy for a designer to intentionally (or accidentally) make their DAC sound distinctive in some way, say by slightly altering the frequency response or any number of other ways, were they of the mind they wanted to.
 

·
Registered
Joined
·
1,448 Posts
There are, I'm guessing, thousands of DACs out there (10's of thousands?). This is a test of just one which currently costs $7.99. If it and its analog output stage are indeed perfectly transparent in their reproduction of music, as I suspect, we can assume there are probably others as well. [At least I'll assume that; I can't speak for everyone.:) ]



It also would be trivially easy for a designer to intentionally (or accidentally) make their DAC sound distinctive in some way, say by slightly altering the frequency response or any number of other ways, were they of the mind they wanted to.
A test like this for differences a minimum of 2 separate dac's would prove beneficial wouldn't you think?...but you setup a test with ONE dac with audio files and your trying to see if others tell a difference right?...and if so I'm trying to get the point...help me out a little...again I appreciate your effort.

Sent from my MHA-L29 using Tapatalk
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #12 (Edited)
A test like this for differences a minimum of 2 separate dac's would prove beneficial wouldn't you think?...but you setup a test with ONE dac with audio files and your trying to see if others tell a difference right?...and if so I'm trying to get the point...help me out a little...again I appreciate your effort.
The test is to see if a very common and very inexpensive $7.99 DAC (actually free if you buy an iPhone) is audibly perfect to the ear in reproducing music, at least for the listener tested, at that time, with that music, with that specific gear they listen through. It does not prove what the other thousands of DACs out there will do, if they are also perfect, nor what would happen in other circumstances.
 

·
Registered
Joined
·
1,448 Posts
The test is to see if a very common and very inexpensive $7.99 DAC (actually free if you buy an iPhone) is audibly perfect to the ear in reproducing music, at least for the listener tested, at that time, with that music, with that specific gear they listen through. It does not prove what the other thousands of DACs out there will do, if they are also perfect, nor what would happen in other circumstances.
Ok and again what is the point?..and a serious question...what is perfect?...if you say had 2 dac's one 7.99 as you say and another iPhone dac...also 7.99 and you did this wouldn't you be able to pick out any differences (if there are any mind you)...since you would be listening on 2 separate dac's?...using just one dac of course it will sound the same...correct?

Sent from my MHA-L29 using Tapatalk
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #14 (Edited)
Ok and again what is the point?
To see if a very inexpensive $7.99 DAC, specifically this particular one, including it analog output circuitry, is audibly perfect to the ear when reproducing music by converting the digital signal to analog. Does it do it perfectly? Or does it have audibly discernible flaws, like poor sound stage, detail/imaging resolution, or added hum, hiss, buzz, noise, frequency response errors, etc.?
 

·
Registered
Joined
·
1,448 Posts
To see if a very inexpensive $7.99 DAC, specifically this particular one, including it analog output circuitry, is audibly perfect to the ear when reproducing music.
Ok I get that but isn't my perfect different than yours?...perfect is a point of view.

Sent from my MHA-L29 using Tapatalk
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #16 (Edited)
Ok and again what is the point?..and a serious question...what is perfect?
For people like me, seeking high fidelity (also called high accuracy or high faithfulness) that's an easy question to answer: complete transparency to the ear when reproducing music. I want to hear the music, no more no less. [See my signature for more.]

If there is any audibly detectable difference, whatsoever, be it added hum, hiss, distortion, coloration, sound stage alteration, lack of depth, frequency response error, poor channel separation, intermodulation distortion, phase shift, anything at all, that's bad in my book.
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #17 (Edited)
Ok I get that but isn't my perfect different than yours?...perfect is a point of view.
You can't argue a product aspiring to high fidelity is flawed, "not perfect", and "harms the sound" if it is completely undetectable, aka "transparent"; that is you can't tell if it is in, or out of, the signal path.
 
  • Like
Reactions: CharlesJ

·
Registered
Joined
·
20,837 Posts
Oh well... it would be better to take the test and discuss results than to bicker about the veracity of the test.
 

·
Registered
Joined
·
15,391 Posts
Discussion Starter #19 (Edited)
Ok I get that but isn't my perfect different than yours?...perfect is a point of view.
Here is the concept of the test expressed by the founder of Stereophile magazine, J. Gordon Holt, who as I mentioned was a proponent of the A/B Bypass Test:

". . . if a system is truly accurate it will sound musically natural to anyone—audiophile or musician. We like to believe that what we are all striving for is a system that will reproduce exactly what is on the recording, be that good, bad or so-so. Such a perfect component should perform its desired function—transduction, amplification or what have you—and nothing more. It should neither subtract from nor add to the input signal. It should, in other words, be the equivalent of a straight wire with gain, and when inserted into the signal path, it should produce exactly the same sound as when it is bypassed. "

- J. Gordon Holt
 

·
Registered
Joined
·
1,448 Posts
Here is the concept of the test expressed by the founder of Stereophile magazine, J. Gordon Holt, who as I mentioned was a proponent of the A/B Bypass Test:



". . . if a system is truly accurate it will sound musically natural to anyone—audiophile or musician. We like to believe that what we are all striving for is a system that will reproduce exactly what is on the recording, be that good, bad or so-so. Such a perfect component should perform its desired function—transduction, amplification or what have you—and nothing more. It should neither subtract from nor add to the input signal. It should, in other words, be the equivalent of a straight wire with gain, and when inserted into the signal path, it should produce exactly the same sound as when it is bypassed. Nothing could be more self-evident. "



- J. Gordon Holt
Right I get that and that's how it SHOULD be...and I'll tell you from experience that from using different dac's myself I've heard...I guess I'll call it a CLEANER sound..more detail comes through with certain dac's I've used than others...to me that's not distortion or coloration...but hearing all what's in a piece of music and or any audio being played.

Sent from my MHA-L29 using Tapatalk
 
1 - 20 of 620 Posts
Top