Audio DBT #1 Summary - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #1 of 254 Old 09-14-2007, 08:52 AM - Thread Starter
AVS Special Member
 
scaesare's Avatar
 
Join Date: Feb 2003
Location: Ashburn, VA
Posts: 4,637
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
PLEASE NOTE: DUE TO A WORKFLOW ERROR THE ORIGINAL RESULTS AND ANAYSIS ARE INCORRECT. CORRECTIONS BELOW WILL BE FORTHCOMING AS I HAVE OPPORTUNITY TO RE-ANALYZE THE RESULTS.

PLEASE SEE POST#3 FOR AN INITIAL EXPLANATION.


AUDIO DOUBLE BLIND TEST #1 RESULT SUMMARY

TEST PARTICIPATION

Firstly, thanks to all who participated. We got over 40 individual result sets from people with varying degrees of individual abilities, environments, and equipment setups. This includes two insiders who have significant experience with this technology.

There was also much good debate on the value of such a test, and some good suggestions for refining additional tests.

For the folks that opted to not participate, I find that unfortunate. Especially those that were invited directly, and didnt appear to have any technical obstacle that prevented their contributing. Dont blame us if your future assertions are greeted with at least a little skepticism.

Due to the disparity with the poll results and the comments, Ive opted to dismiss the poll results, as I noted in the thread. There were 68 voters, and about 40 comments. I hope all who may have voted in the poll who were serious were sure to post their results as a comment as well. Ill have to assume the other votes were ballot stuffers.


ANSWERS

Well, after the delay, lets get right to it.

He original file was a rip from CD to an uncompressed 16 bit .wav file. Here are how the files were encoded:


Track #1 192Kbps CBR .wma
Track #2 Original un-encoded .wav file
Track #3 320Kbps CBR .wma
Track #4 192Kbps CBR .wma after having a 1.5:1 compressor set at -24db threshold and overall output raised 3db


Track #1 192Kbps CBR .wma
Track #2 192Kbps CBR .wma
Track #3 320Kbps CBR .wma
Track #4 192Kbps CBR .wma after having a 1.5:1 compressor set at -24db threshold and overall output raised 3db

For anybody who downloaded the original encrypted dbtmap.txt file, and who wants to check that I didnt construct these results based on the test findings, the Blowfish encryption key is jabberw0cky. (decryption app available at http://www.gregorybraun.com/BlowFish.html).


TEST RESULTS

As the responses had many cases where people felt several tracks were indistinguishable, it wasnt possible to simply declare an overall rank.

Instead I tallied the responses by scoring them according to rank. If a track was declared The Original, or given a rank of 1, it received a score of one. If it was ranked second, it received a score of 2, etc..

If a track tied another track (thus being considered identical), the both received the score for the the position of the initial track. Tracks after that would receive their natural score.

For example: If some body posted a ranking such as:

Track 1 Original
Track 2 Compressed
Track 3 -- Same as #2
Track 4 Highly Compressed

Then it would have been scored as:

Track 1-- Score of 1
Track 2 Score of 2
Track 3 -- Score of 2
Track 4 -- Score of 4

Thus for the overall scoring, lower scores are better.

I also did a count of the total 1st-place scores that a track received. As there were a number of ties, the aggregate count exceeds the total number of responses.

Included in the spreadsheet are the respondent, the post # in the thread, the scoring, and any notes that seemed germane. Im attaching a copy of the spreadsheet for your usage as well.

Here are the results:


Please forgive the difficulty of representing tabular data in a post. Also comments are contained only in the attached spreadsheet:

Code:
AUDIO DBT RESULTS - 9/14/07\t\t\t\t\t
\t\t\t\tSCORING\t
\t\t       TRACK 1\tTRACK2\tTRACK3\tTRACK4
RESPONDANT\tPOST#\t192K\tOrig\t320k\t192cmp
MichaelHDDVD       27\t1\t1\t1\t4
RAFABAMAD\t29\t2\t2\t1\t4
tqlla\t             31\t1\t1\t1\t4
bkilian\t             33\t1\t1\t1\t1
Mulholland\t34\t2\t4\t3\t1
anttimonty\t35\t2\t2\t1\t4
Kram Sacul\t36\t1\t1\t3\t4
robena\t             37\t1\t1\t1\t1
SomethingMore\t44\t2\t3\t1\t4
online\t             46\t4\t4\t1\t
Rutgar\t             47\t1\t1\t1\t1
palofex\t             48\t1\t1\t3\t4
Facct\t             49\t3\t1\t1\t4
Customgamer1\t50\t3\t1\t1\t4
pepar\t             51\t3\t1\t1\t4
geocab\t             52\t1\t4\t4\t4
geocab's ex\t54\t4\t4\t1\t4
David Susilo\t61\t1\t1\t3\t4
Caurus\t             63\t1\t1\t1\t1
wakashizuma\t64\t2\t2\t1\t4
stahlej\t             67\t1\t1\t1\t1
Slaine\t             72\t1\t4\t1\t3
rto\t             73\t2\t3\t1\t4
MEC2\t            103\t1\t1\t1\t1
shamus\t            116\t3\t2\t1\t4
WayneL\t            133\t1\t1\t1\t1
2Channel      145\t1\t1\t1\t4
tteich\t            275\t2\t2\t1\t4
PlayDoh\t            286\t3\t2\t1\t4
primetimeguy       291\t1\t1\t1\t1
SimpleTheater\t295\t1\t4\t3\t1
wildfire99      300\t2\t1\t4\t3
tqlla\t            308\t1\t1\t1\t4
bobgpsr\t            320\t2\t2\t1\t4
awmurray\t325\t4\t3\t2\t1
raaj\t            326\t2\t3\t3\t1
user4avsforum\t329\t2\t2\t2\t1
tqlla          333\t2\t2\t2\t1
Borbus\t          340\t2\t2\t1\t4
amirm           360\t2\t2\t1\t4
FilmMixer     362\t3\t1\t2\t4
scaesare     362\t1\t1\t1\t4

SCORE
(Lower is better):      77\t79\t64\t120
\t\t\t\t\t
TOTAL 1st 
PLACE SCORES:\t          19\t21\t30\t14

Continued....

 

AudioDB11-Results.zip 8.962890625k . file

-Steve
scaesare is offline  
Sponsored Links
Advertisement
 
post #2 of 254 Old 09-14-2007, 08:52 AM - Thread Starter
AVS Special Member
 
scaesare's Avatar
 
Join Date: Feb 2003
Location: Ashburn, VA
Posts: 4,637
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
TEST ANALYSIS

Please keep in mind that this test was constructed to see if the results of lossy encoding introduced artifacts that were considered objectionable or detectable in a home viewing environment.

Firstly I'll address track #4(192k & compressed). I purposely cranked this one up a bit before encoding it @ 192Kbps. To keep it from clipping too severely I applied a moderately aggressive compressor to it first. The end result is what I consider the equivalent of the audio track mastered at a higher level, or the TV with the brightness cranked up on the showroom floor. It was an interesting salt in the mix. (Props to Caurus for his analysis of this track perhaps you can re-post your findings now that the test is over?)

Interestingly, a few folks found #4 to be superior, despite the distortion that the clipping incurred. Many folks identified this track as different. Not only due to volume level, but the different dynamic characteristics. It received an overall last place score if 120, and was only picked for first place 14 times.

I'm not going to include track #4 in the comments below.

Perhaps the most notable aspect of the results is that the original un-encoded file (Track #2) comes out THIRD in terms of overall score at 79. It was picked for first place 21 times. It also seems to have had the least comments on it alone, although one respondent called it clearly compressed and poor. And FilmMixer nailed it. (his 1 of 10!)

Track #1(192K) is interesting. Although it scored slightly better than the original at 77. It was picked 2 LESS times as being the original. Clearly, even at only 192Kbps, it's already a tight game.

Track #3(320K) received the best overall score at 64. And was picked as the uncompressed file more often than any other with 30 selections. Perhaps that the track is "different" even more so than it's lesser-bitrate cousin, track #1, despite having a "better" bitrate. Or it was the last "good" track heard before the distorted Track #4 was played. Or It's just the way the guesses came out... as the majority of additional comments seemed to indicate that many felt tracks 1-3 were a tie. This will be a good place to do some additional testing.

This really does cause some self examination though: If you actually like the results of moderate encoding better than the original(Although I know this is not what a purist may desire), are you being cheated if you don't get this track? Let the debate ensue...


So, with the inadvertent error I made, there was no original un-encoded file in the mix. Nonetheless, none of us were aware of this, and were still listening for an un-compressed track somewhere in the test.

As it turns out, both tracks 1 & 2 were 192Kbps encodes. And they scored very similarly, at 77 & 79 for overall scoring, and were picked as being at least as good as any other track (a 1st place score), almost evenly at 19 & 21, respectively.

This would seem to indicate that, compared to each other, we could appreciate no significant differences between them Which there weren't.
The 320Kbps encode was this actually the highest quality track in the test. And it had the best rank score at 64. It also was considered at least as good as any other track 30 times.

However, as was pointed out, although it was Ranked #1 thirty times in total, 17 of those times one or more other tracks was considered just as good. So out of all responses, 69% of the time a 320Kbps encode could NOT be picked over a 192Kbps encode.


And, interestingly, a 192K track (either 1 or 2) was picked as least as good as the other tracks 40 times, as opposed to the 320k track being picked as least as good as the others only 30 times. However, collectively they were ranked as better than the 320K track only 3 times, or only 7% of the time.

So, if we dismiss Track #4 (due to the deliberate modification of this track making it obviously different), it appears that just better than 2/3rds of people cant distinguish between 320Kbps and 192Kbps encodes. If the just less than 1/3rd that can, they almost always pick the higher-rate encode.

Again, with the dismissal of #4, the fact that fact that 1.3rd of folks picked 1 out of 3 tracks to be better than the other 2 is interesting.

The next test needs to obviously provide an uncompressed original to see if the story changes significantly with that comparison.



CONCLUSIONS

With 42 respondents we seem to have a reasonable (if small) sample size.

It appears that even at moderate bitrates there are no obviously objectionable artifacts for the most part. Not a single comment identified any of the common hallmarks of compression for the compressed tracks. Nor were there any more mentions of compressed dynamic range" for the lesser bitrate tracks than the others.

Given that the frame of reference for home viewing is often the theater presentation (often DD at 384ish Kbps @5.1), it would appear that we are already being given far superior audio tracks, even at standard DVD bitrates.

As way of comparison, the bitrates for these stereo encodes were 192 and 320Kbps. That equates to 96Kbps and 160Kbps per channel respectively. The tools available to me max out at 320Kbps for a stereo audio track.

To compare to 5.1 HD media encodes, a quick note: the .1 LFE channel takes only ~0.5Kbps uncompressed to encode. As such it's really a non-contributor to the overall bit budget. Thus I'll simply use 5 channels as the point of comparison.

For a 640K DD+ track, each channel receives 128Kbps of bandwidth. That's right in between test tracks 1 & 3.. both of which were considered as good or better than the original a majority of the time. That
is right in between the bitrates provided in this test. with 2/3rd of folks picking a 96Kbps-per-channel encode (the 192Kbps stereo equivalent), as just as good as a higher bitrate, it suggests that DD/DD+ at 640Kps is already very good.

For a 1.5Mbps DD+ track, each channel get a whopping 300Kbps. That's between TWO to THREE TIMES the bandwidth used for this test.

Additionally, you may factor in that a 5.1 encoder can allocate bits to channels as they are needed. As surround channels often have little info in them, and dialogue channels may often be operating almost solo, the effective bitrate for a channel may be far higher at any given point in time.

Now, there are undoubtedly some differences between WMA and DD+ or DTS. But they are all relatively modern codecs. I expect that efficiencies will at least be similar.

All in all, I personally would be hard pressed to feel like I am being short changed, even by a well-mastered 640Kbps track. And certainly not by a 1.5Mbps one. I think that mastering, room, etc.. likely play a much larger roll than the difference between lossless and say 1.5Mbps. And I don't think the fact I couldn't pick out the original tracks was a bad thing.. these codecs were modeled on our human perception, and it seems to be working quite well.

In conclusion, without the original, it's not really possible to determine how far "down the scale" we are comparing tracks. That having been said, it would seem that by at least by 192Kbps stereo (96k per channel) we are already at the point where throwing additional bits at the encode doesn't make much of a difference for better than 2/3rd of folks.

And finally, I'll set this last comment aside, as it wasn't part of the original test regimen: Because I DO have the uncompressed original file here, I went ahead and listened to it a couple of times in comparison to the encoded tracks after the test flaw came to light. I didn't do so blindly, and I still am hard pressed to say I hear much if anything different between any of the compressed encodes and the original. I really don't expect the gulf between these encodes provided for this test and the original to be very great. If anything, I expect that the margin between the 320K track and the original to perhaps be even less than between the 192 and 320 tracks.

Finally: if anybody wants, I can re-zip the file with a correct track #2 containing the original, and you can download to play with. I won't be tabulating results, as the cat is already out of the bag and it's nor longer a blind test... but it might be entertaining. Let me know.

I'm very interested in your opinions...

-Steve
scaesare is offline  
post #3 of 254 Old 09-14-2007, 08:52 AM - Thread Starter
AVS Special Member
 
scaesare's Avatar
 
Join Date: Feb 2003
Location: Ashburn, VA
Posts: 4,637
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
TEST WORKFLOW ERROR

In my original workflow, two of the last steps were to render all the files back out as 16bit uncompressed .wav files, and then rename them from their "working names (ie- "192k.wav") to the "generic names" (ie-"Track1.wav") in their own directory I could then zip up for all to download. It appears that I made a file copy or naming error here.

In this case, I renamed the rendered out files in-place, and zipped them up. IOW: I don't have the originally named files from the render. This I needed to determine if tracks 1&2 were both the original uncompressed file, or both the 192Kbps file.

Thus, I've had to re-render the files from the compressed versions back out to 16bit .wav. Doing that, it appears the Tracks 1 and 2 are BOTH the 192Kbps encoded files. Track 3 is correct in that it is the 320Kbps encode, and Track 4 is correctly labeled as the 192Kbps & compressed/boosted track.

Unfortunately this means:

There was no uncompressed original in the test.


The correct track definitions are:

Track#1 - 192Kbps
Track#2 - 192Kbps
Track#3 - 320Kbps
Track#4 - 192Kbps with 1.5:1@24dB compression and 3dB boost

I wholeheartedly apologize. As you can imagine, I feel terribly silly for making such a workflow error. Even more so, I am terribly disappointed that we didn't get the information I had hoped for after you all so graciously participated.

So. With my tail between my legs, I'm going to go and correct the original posts in this thread. I still think we get some value out of the results, so I'l try to re-assess what the results mean, and try to write that up.

After the several hours it took to perform/analyze this whole experiment, I am a little bummed out at the moment. But I hope after I can clean this up a bit that I can post some ideas for "Test 1a", and get every body's feedback. We also have some kind offers of assistance from folks here, that I'd like to seriously consider.

Specific thanks to Amir for speaking up to make sure accuracy prevailed, even though my bad results actually tended to make his company's codec look even better than it was.

Again with sincere apologies, thanks everybody for your efforts thus far. Sorry to have blown it after the anticipation build up.

-Steve
scaesare is offline  
post #4 of 254 Old 09-14-2007, 12:25 PM
Member
 
Mulholland's Avatar
 
Join Date: Dec 2006
Location: Sunny California
Posts: 64
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Must say that I am surprised by how far off so many of us were (including myself)!

I am not that surprised that I got #4 wrong once I realized that it had been adjusted in Db. However I must say that the rest of the results was really an eye opener for me.

Based on the fact that the original track ranked last (after elimination of track #4), it seems that lossy compression may be preferred by consumers in actual listening situations over lossless. Perhaps for future CD releases they should first compress the tracks before authoring just to get more desirable results for the consumers

Post-production house Insider, Representing my own point of view
Mulholland is offline  
post #5 of 254 Old 09-14-2007, 12:28 PM
Senior Member
 
PlayDoh's Avatar
 
Join Date: Apr 2003
Location: Texas
Posts: 275
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Thanks for your efforts, Steve.

While I'm also disappointed there were some who refused to participate, I can't say I'm that surprised.
PlayDoh is offline  
post #6 of 254 Old 09-14-2007, 12:35 PM
Senior Member
 
hellokeith's Avatar
 
Join Date: Aug 2006
Location: TX, USA
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:


Track #3(320K) received the best overall score at 64. And was picked as the uncompressed file more often than any other with 30 selections. Perhaps that the track is "different" even more so than it's lesser-bitrate cousin, track #1, despite having a "better" bitrate. Or it was the last "good" track heard before the distorted Track #4 was played. Or It's just the way the guesses came out... as the majority of additional comments seemed to indicate that many felt tracks 1-3 were a tie. This will be a good place to do some additional testing.

Any theory on why this got a better score than the original?


Quote:


FilmMixer 362 3 1 2 4

Was FilmMixer the only one who identified them all "correctly"?
hellokeith is offline  
post #7 of 254 Old 09-14-2007, 12:37 PM
AVS Special Member
 
raaj's Avatar
 
Join Date: Sep 2004
Location: Near the lovely City by the Bay
Posts: 2,794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 20 Post(s)
Liked: 51
Oh well, thank you Steve !!

Now you've gone and proved my hearing is shot ! Now I can stop feeling bad about my humble little sound system anymore.

Awww.. shucks !!


raaj is online now  
post #8 of 254 Old 09-14-2007, 12:39 PM
AVS Special Member
 
raaj's Avatar
 
Join Date: Sep 2004
Location: Near the lovely City by the Bay
Posts: 2,794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 20 Post(s)
Liked: 51
Could it be that the original track was mutilated and abused through digital processing, and that somehow re-encoding it in a lossy manner improved the sonics a bit?
raaj is online now  
post #9 of 254 Old 09-14-2007, 12:40 PM
Advanced Member
 
awmurray's Avatar
 
Join Date: May 2006
Posts: 786
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Track 4 really threw me off. Obviously (in hind sight) because it was louder... I did notice that #4 seemed to have less dynamic range than 3, but on the intro, it seemed noticeably "closer" so I picked it over 3. I didn't realize how much the volume difference could make.

Like I said earlier, I gravitated to 3/4 pretty quickly... maybe should have listened more to 1/2, huh?

Well, at least Amirm picked #3 as the original which I would have done had #4 not thrown me.
awmurray is offline  
post #10 of 254 Old 09-14-2007, 12:44 PM - Thread Starter
AVS Special Member
 
scaesare's Avatar
 
Join Date: Feb 2003
Location: Ashburn, VA
Posts: 4,637
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by hellokeith View Post

Any theory on why this got a better score than the original?

Well, it could be a "sweet spot" wherein it's exceeded a bitrate where even subtle objectionable compression artifacts are no longer present, yet is imparts a distinct "feel" to the music that is preferable.

Or, as I suggested above; it may be the last good thing people heard before #4.

I tend to suspect the latter.


Was FilmMixer the only one who identified them all "correctly"?


Yup.

-Steve
scaesare is offline  
post #11 of 254 Old 09-14-2007, 12:44 PM
Wireless member
 
pepar's Avatar
 
Join Date: Jul 2002
Location: Quintana Roo ... in my mind
Posts: 25,029
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 99 Post(s)
Liked: 155
I am pleased with my results. I got them all correct, except I thought the 320kbps track sounded as good as the original. If there is another test, perhaps one with a better source, I will set up in my home theater instead of my computer room.

Kudos to FilmMixer for nailing them all!

PS _ I "moved" this from the other thread as this one now seems like it's getting the most action.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
- it's never done!


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
pepar is offline  
post #12 of 254 Old 09-14-2007, 12:46 PM
Wireless member
 
pepar's Avatar
 
Join Date: Jul 2002
Location: Quintana Roo ... in my mind
Posts: 25,029
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 99 Post(s)
Liked: 155
3 - 1 - 2 - 4 was the correct order, which FilmMixer nailed. The "second best" order would be 3 - 1 - 1 - 4 placing the 320Kbps track as equal to the original with everything else correct.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
- it's never done!


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
pepar is offline  
post #13 of 254 Old 09-14-2007, 12:47 PM
Advanced Member
 
awmurray's Avatar
 
Join Date: May 2006
Posts: 786
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Conspicuously absent is thebland, who seems to have grade A+ equipment and is a huge lossless or die proponent. Hmmm....
awmurray is offline  
post #14 of 254 Old 09-14-2007, 12:49 PM
AVS Special Member
 
RobertR1's Avatar
 
Join Date: Nov 2005
Posts: 3,773
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 25
One thing to take from this thread. When FilMMilxer speaks about audio, we should listen
RobertR1 is offline  
post #15 of 254 Old 09-14-2007, 12:51 PM
Wireless member
 
pepar's Avatar
 
Join Date: Jul 2002
Location: Quintana Roo ... in my mind
Posts: 25,029
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 99 Post(s)
Liked: 155
Quote:
Originally Posted by awmurray View Post

Conspicuously absent is thebland, who seems to have grade A+ equipment and is a huge lossless or die proponent. Hmmm....

Not to defend him - or get flamed myself - but his position is that he WANTS lossless because he aspires to have the best.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
- it's never done!


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
pepar is offline  
post #16 of 254 Old 09-14-2007, 12:52 PM
Advanced Member
 
awmurray's Avatar
 
Join Date: May 2006
Posts: 786
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by pepar View Post

Not to defend him - or get flamed myself - but his position is that he WANTS lossless because he aspires to have the best.

Bring on the $1,000/foot pure silver interconnects, then!
awmurray is offline  
post #17 of 254 Old 09-14-2007, 12:52 PM - Thread Starter
AVS Special Member
 
scaesare's Avatar
 
Join Date: Feb 2003
Location: Ashburn, VA
Posts: 4,637
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by pepar View Post

3 - 1 - 2 - 4 was the correct order, which FilmMixer nailed. The "second best" order would be 3 - 1 - 1 - 4 placing the 320Kbps track as equal to the original with everything else correct.

No.

Correct order was 2-3-1-4.

Second best woulda' been 2-1-1-4. That would have been the golden ears that could pick out the lossless track, but felt both compressed tracks were the same.

-Steve
scaesare is offline  
post #18 of 254 Old 09-14-2007, 12:53 PM
AVS Special Member
 
bobgpsr's Avatar
 
Join Date: Jun 2004
Location: Cedar Rapids, Iowa
Posts: 2,733
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
My biggest fear in this music listening quality subject is that we have been getting used to lossy sound encoding to the point that we like what is familiar.

Perhaps if we listened to lossless (or very high bitrate lossy) more often using high quality speakers, etc we would be able to re-learn how to appreciate the real stuff more. I know I regret all the 128 kbps and lower mp3 rips that I made years ago on my PC. Been re-ripping to flac and wma lossless for the past two years (now that HDD storage is so cheap).

But picking out the diffs/problems with 256 kbps mp3 or 192 kbps wma is still very difficult for me. Whereas the difference with 5.1 channel 640 kbps DD+ versus TrueHD on POTO was for me detectable with better surround sound localization and "air". But still a very subtle difference.

And going to the symphony helps me accept what the Room Eq Wizard and YPAO are doing to get proper in-room freq response. No more West Coast sound for me (bright highs boosted low end). Live R&B music with "hit me in the chest mid-bass" is another story.
bobgpsr is offline  
post #19 of 254 Old 09-14-2007, 12:55 PM - Thread Starter
AVS Special Member
 
scaesare's Avatar
 
Join Date: Feb 2003
Location: Ashburn, VA
Posts: 4,637
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by pepar View Post

Not to defend him - or get flamed myself - but his position is that he WANTS lossless because he aspires to have the best.

I find it unfortunate that wasn't the REASON he gave for not partipating. He said it was that he didn't have HDMI in to his AVR yet. When I pointed out that these were stereo tracks (not 5.1), and thus he COULD particiapte with no loss of fidelity he chose to ignore that, despite continuing to debate in the thread.

-Steve
scaesare is offline  
post #20 of 254 Old 09-14-2007, 12:57 PM
AVS Special Member
 
shamus's Avatar
 
Join Date: Nov 2005
Posts: 6,254
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 13
Intersting results and a thumbs up to Filmmixer.
Im still not going to say lossy is "good enough" until we see some more tests!

A few suggestions for next test...
-Better material.
-Pm results.
-4 tracks consisting of 2 identical compressed and 2 identical uncompressed tracks.
-No more sneaky volume tricks!

Thanks again OP for all your hard work and looking forward to the next round!
shamus is offline  
post #21 of 254 Old 09-14-2007, 01:00 PM
AVS Special Member
 
shamus's Avatar
 
Join Date: Nov 2005
Posts: 6,254
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 13
Quote:
Originally Posted by scaesare View Post

No.

Correct order was 2-3-1-4.

Second best woulda' been 2-1-1-4. That would have been the golden ears that could pick out the lossless track, but felt both compressed tracks were the same.

????? are you sure?
shamus is offline  
post #22 of 254 Old 09-14-2007, 01:01 PM
Wireless member
 
pepar's Avatar
 
Join Date: Jul 2002
Location: Quintana Roo ... in my mind
Posts: 25,029
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 99 Post(s)
Liked: 155
Quote:
Originally Posted by scaesare View Post

No.

Correct order was 2-3-1-4.

Second best woulda' been 2-1-1-4. That would have been the golden ears that could pick out the lossless track, but felt both compressed tracks were the same.

FilmMixer's order was 3 -1 - 2 - 4.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
- it's never done!


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
pepar is offline  
post #23 of 254 Old 09-14-2007, 01:01 PM
Advanced Member
 
awmurray's Avatar
 
Join Date: May 2006
Posts: 786
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
scaesare,

This was a great idea and I look forward to doing it again...

IMO, those who refused to participate were afraid they couldn't even distinguish an original CD quality track (nevermind DD+ vs. LPCM) from compressed versions.
awmurray is offline  
post #24 of 254 Old 09-14-2007, 01:02 PM
AVS Special Member
 
SimpleTheater's Avatar
 
Join Date: Mar 2005
Location: Upstate NY
Posts: 2,698
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 12 Post(s)
Liked: 25
Quote:
Originally Posted by Mulholland View Post

Must say that I am surprised by how far off so many of us were (including myself)!

I am not that surprised that I got #4 wrong once I realized that it had been adjusted in Db. However I must say that the rest of the results was really an eye opener for me.

Based on the fact that the original track ranked last (after elimination of track #4), it seems that lossy compression may be preferred by consumers in actual listening situations over lossless. Perhaps for future CD releases they should first compress the tracks before authoring just to get more desirable results for the consumers

It could also be that compression masks other sounds and that not knowing what it SHOULD sound like makes it impossible to take this test. I just went back to track #2 and Sherryl's voice is noticeably quieter at the end of certain words - maybe compression is removing some sounds making the vocals clearer.

Considering I've never heard her music before, I had no clue what she was supposed to sound like. Maybe compression takes some of the edge off her voice making her sound better .

I will also work on getting better at A/B comparisons. It takes a lot of fast forwarding to hear the next track at 1:20.

Thanks for doing this test scaesare. I think following thebland's idea and giving us the master track first makes a lot of sense. Once we know what is should sound like it is easier to hear differences.

PS: Next time find a way to remove the checksum values (or at least mask them) on the files. After I posted my results I checked them and it was easy to determine what were compressed and what weren't. I'm glad I didn't cheat, but I admit I was tempted.

Why is there NO perfect equipment, only compromises?
SimpleTheater is offline  
post #25 of 254 Old 09-14-2007, 01:03 PM
AVS Addicted Member
 
thebland's Avatar
 
Join Date: Jan 2001
Location: Detroit, Michigan USA
Posts: 23,834
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 200 Post(s)
Liked: 133
I actually thought of partcipating but just did not get around to it..

Nice work though.

For the next round, lets all input on how to set up and design the experiment.

This was a first of a kind effort here and the poster deserves a lot of credit.

But, Round 2 could be better.

Same volume, positive controls, etc.... Lots to do... Great fun!


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


There are more than a handful of [op amps] that sound so good that most designers want to be using them as opposed to discreet transistors. Dave Reich, Theta 2009
thebland is online now  
post #26 of 254 Old 09-14-2007, 01:05 PM
AVS Special Member
 
SimpleTheater's Avatar
 
Join Date: Mar 2005
Location: Upstate NY
Posts: 2,698
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 12 Post(s)
Liked: 25
Quote:
Originally Posted by pepar View Post

I am pleased with my results.

I'm actually quite pleased with my results as well. Each and every time I listened ONLY track #2 stood out as significantly different than the others. I could pick out #2 any time - I just thought it sounded terrible.

Why is there NO perfect equipment, only compromises?
SimpleTheater is offline  
post #27 of 254 Old 09-14-2007, 01:05 PM
AVS Special Member
 
shamus's Avatar
 
Join Date: Nov 2005
Posts: 6,254
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 13
Quote:
Originally Posted by pepar View Post

FilmMixer's order was 3 -1 - 2 - 4.

Im confused....
shamus is offline  
post #28 of 254 Old 09-14-2007, 01:06 PM
Senior Member
 
hellokeith's Avatar
 
Join Date: Aug 2006
Location: TX, USA
Posts: 325
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by scaesare View Post

No.

Correct order was 2-3-1-4.

Second best woulda' been 2-1-1-4. That would have been the golden ears that could pick out the lossless track, but felt both compressed tracks were the same.

I think there is some confusion here because of the order you listed them in your table:

TRACK1 TRACK2 TRACK3 TRACK4
192K Orig 320k 192cmp
3 1 2 4

3 1 2 4 is what FilmMixer identified, that's what we're talking about, not the track names, which would be track2, track3, track1, track4. Either way, he got them perfect.

Ok FilmMixer, pony up your hardware specs now!
hellokeith is offline  
post #29 of 254 Old 09-14-2007, 01:06 PM
Advanced Member
 
awmurray's Avatar
 
Join Date: May 2006
Posts: 786
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by shamus View Post

A few suggestions for next test...
-Better material.

I thought the material was excellent. And I'll note that FilmMixer did too (he has lots of street cred. in this area )

Quote:
Originally Posted by shamus View Post

-Pm results.

Plus people shouldn't be reading the thread until after they've submitted their results anyway. The OP did say that in the original post, "I also suggest NOT looking at any results here prior to your own experiments."

Quote:
Originally Posted by shamus View Post

-No more sneaky volume tricks!

That would be helpful since it is difficult to adjust for when switching between tracks.
awmurray is offline  
post #30 of 254 Old 09-14-2007, 01:09 PM
Wireless member
 
pepar's Avatar
 
Join Date: Jul 2002
Location: Quintana Roo ... in my mind
Posts: 25,029
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 99 Post(s)
Liked: 155
Quote:
Originally Posted by hellokeith View Post

I think there is some confusion here because of the order you listed them in your table:

TRACK1 TRACK2 TRACK3 TRACK4
192K Orig 320k 192cmp
3 1 2 4

3 1 2 4 is what FilmMixer identified, that's what we're talking about, not the track names, which would be track2, track3, track1, track4. Either way, he got them perfect.

Right-o! and based on that way of expressing the order, 3 - 1 - 1 - 4, thinking track #2 and track #3 were the same, by my thinking would be 2nd place score. I'm making a big deal out of this because that's how I scored it. And I like lossless.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
- it's never done!


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
pepar is offline  
Reply HDTV Software Media Discussion

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off