The sad truth is that we really need two screen sizes---one large screen of 50" or more for "home theater" (Blu-ray movies) and a smaller one of 40" or less for normal programming such as news and information. (Who wants to watch giant talking heads?)
On larger sets, football and baseball look great when the camera is over the field---but my wife nearly got seasick watching football on a 50" set when the camera lingered on close-ups of the players walking back to the huddle, etc. Great big bobbing heads can be annoying after a while.
Steve S. makes a good point about sources. The smaller the screen, the more forgiving of the source. Regular DVDs (upscaled by a good Blu-ray player) start to get soft to a trained eye from 10ft. away on screens of 46" or more. If you watch older, classic movies not yet transferred to Blu-ray (and it'll be many years before a majority of good ones are), you'll want to stick to 47" or under. And these old movies look wonderfully sharp and clear on a 40" or 42" screen---perhaps better than in a real theater when they were first shown.
From a "perspective" point of view, if you want a genuine theatrical experience, 50" is minimal. There's just enough peripheral sightline on that size to be satisfying from 10ft. But you'd better be watching 1080p or you'll lose sharpness (and nothing is broadcast over cable in higher than 780p or 1080i). Of course it may depend on how experienced you are at evaluating this, or how much you care about sharpness vs. size. To me, sharpness and color accuracy---hence realism---are paramount.
Since I watch many different sources, I finally decided to compromise with a 42" set, which I am quite satisfied with (though I had to return three other brands, including Sony, Samsung, and LG, before I got a set free of picture defects).
As more and more good movies come out on Blu-ray (90% of the choices today seem like garbage) I may get a 55" set and create a home theater in a larger room. For now, the 42" is a great all-around size for all sources, and of course the Blu-rays are stunningly clear from somewhat closer than our ordinary 10ft. viewing distance.