Then let me give you a simple one to get you started. I'll assume that, like most audiophiles, you believe you can here the difference between a lossless digital music file and one compressed to 256 kbps MP3. I know I can't hear the difference because I've done the test. Perhaps you can. The nice part of a test like this is that it doesn't require any level matching or comparator because you use the same playback equipment for the comparisons.
Rip a music track in some lossless format such as WAV to your hard drive. Then rip the same track to another file as a 256 mp3. If you have the equipment to play music files from your hard drive through your sound system then you're all set. If you don't, then you need to save the files out to CD's so you can play them with a CD player.
The bias controlled test is very simple. Have a friend or family member play the files in a random succession at whatever rate of speed you like without your knowing which is the wav and which is the mp3. You can have the computer make up a random schedule if you like. You need to identify each file as either the wav or the mp3. The friend scores your reponses as right or wrong. Do 20 or 30 iterations. The scores will tell you whether or not you hear a difference. 60% right or wrong or higher would be statistically significant. Less than 60% would verge on random and 50%, of course, would be perfectly random - i.e. no audible difference.
Then, if you prefer, take one of the files and listen to it every day for as long as you like until you feel you have gotten accustomed to it. Then do the bias controlled test again to see if the scores are similar to the original run. If they are similar, then the time spent getting accustomed to it was meaningless. If the scores are significantly different, then you're on to something.
This is an easy thing to set up. It costs only the price of a couple of CD-R's at most and you can do it at your preferred pace. Do it in minutes or do it months. It doesn't matter. All that matters is that it be bias controlled and random as described above. That should be easy and perhaps even entertaining and instructive.
Wow. You have a different definition of "simple" than I do.
I wonder how easy it is to listen long term (say, over two weeks) to something while ensuring you don't know what it is, and then repeating that aspect of the test many times. Does my wife have to change the CD in the player every time I want to play a CD other than the test CD during the two weeks? It seems like this is a test that could take a lot of effort, and perhaps a few months (or more) to complete if you need to do multiple trials of long term listening to compare to the multiple trials of short term listening. Not that I don't appreciate ideas as to how to do something like this, but it seems like a awful lot of effort, unless I'm missing something. At then at end, is there an argument anyway about what is statistically significant unless you do the trials many times (by which time I probably will be so old I won't be able to hear at all)?
I'm kinda interested in doing a test just to compare the compressed and uncompressed files, however. To my ears, lossless files of the same song on different MP3 players sound different, but then there is a different amp in the chain, so I'm kinda curious to compare the different compression rates on one player. Maybe I'll try that test when I have a moment.