I used all free-ware ABX double blind testing. I thought it worked great.
Funny that I found this thread and it is still active. I have been trying to figure out at what point I can't tell the difference in MP3s myself as I want to re-rip much of my library as I am actually buying some real speakers soon. (Best speakers I have right now are Kenwood 3 way on a 23 year old combo system I bought when I was young)
I have a LOT of 56 kbps MP3 files in my audio library - from 1995 or earlier when I was in high school and just found out you can edit sound on the computer. I know they sound bad, and was getting tired of them.
I just downloaded "foobar2000" and the "ABX Comparator" plugin/component for it.
I then used "Fairstars CD Ripper" software to rip it to _many_ MP3 formats, FLAC, and uncompressed WAV.
I did this on a Dell Inspiron laptop, stock sound card (Sigmatel)
I used Sennheiser HD202 headphones
I am planning to get floorstanding speakers like the Klipsch RF82-II, of course not necessarily those I have a lot of listening to do first, but that class of speaker - $700-$1200 per pair. And probably a $1000-$1500 receiver, Denon or Onkyo or Yamaha, something like that.
Anyway, I love - and hate
- this ABX comparator tool.
- I used Yello's "La Habanera".
- I used it where I always made "B" the WAV file, and changed the quality of "A". I also hid my results until I'd taken at least 10 tests.
- I started with the crappiest MP3 (56kbps) and the uncompressed WAV file. The difference was very much in my face. I didn't even bother testing myself. Same for my wife.
- Then went to VBR, max 224 kbps, quality 9. Ick. I could definitely tell, especially on the percussion. 10 out of 10 blind tests I guessed the WAV correctly.
- Then VBR, max 320 kbps, quality 9. Still ick. I couldn't tell between that and the max 224.
- Went to CBR 128kbps. Guys at work tell me 128kbps is crap. But I had a HARD time telling the difference. But I could. I took 14 blind tests, and missed 2 of them. It said 3.7% chance I was guessing. It was grueling, I spent 15 minutes. I was scrutinizing the cowbell, the cymbal crashes, the "sweeping sound" - whatever that is.
- I started testing myself with 160kbps MP3, and I just couldn't decide. I tried 10 tests, and had a 40% chance I was guessing. After 15 tests, the chances I was guessing went up to 43% or something like that. I'd go all over the file, trying all sorts of parts of the song. I gave up after about 20 minutes.
I'm not convinced I chose the best song. It may have been easier to tell differences with some live jazz or classical. But I didn't want to do it again. Too much work
My wife got exhausted trying to tell the difference with the song at 128kbps, though statistically the DBT proved she could, she said it was too "tiring". She's 6 months younger than me.
I'm going to assume with 160kbps I might be able to tell the difference with jazz or classical. And assume even maybe on a really good day I could tell the difference with 192kbps and really nice equipment.My decision was to re-rip my CD collection to VBR, min "auto", max 320kbps, quality 3, which seems to have a slightly smaller file size than 192 kbps CBR.
Or rip to 192 kbps CBR. Maybe 224 kbps CBR if I'm feeling paranoid. I'll probably re-rip anything classical/jazz that is under 160 kbps, and anything rock/alternative/etc. under 128 kbps.
So this was what I did to convince myself. Took me a few hours. My ears are 37 years old. I had a bad head cold on a plane once that caused unimaginable pain in my ears when they repressurized the cabin on landing. I had fluid in my ears for 3 weeks before they cleared up. I now have ringing in them all the time. Sometimes kind of loud, most of the time not too bad. But always there (first year of this was torture BTW). I wouldn't be surprised if many people can hear better than me at this point...
I'm a firm believer in the scientific principle of double blind tests. In this case if I am agonizing trying to tell the difference I figure I'm fine. Then upgraded one step beyond that.
Anyone see any problems in my approach? I figure that will work to set my own expectation - not anyone elses - just mine. I had my wife participate too since she'll also use the new equipment/mp3 library.
I think it's easy to take a double blind test. Do it with your favorite music on your best equipment or headphones. Decide for yourself what compression you are willing to accept.Edited by danielrg - 10/20/13 at 8:07pm