Well, for me it would be too small. It may depend on what you did have currently/before.
Watching TV is one of my wife and I's main hobbies. We buy/rent pretty much everything and don't do theaters, so we are patient and wait for them to come out. We bought a 60" 4:3 rear projection CRT 10 years ago (first "married" purchase), so we kind of went big out of the gate. Unfortunately, that made everything else seem too small. Even the 55" seems a bit small unless it's 1080i/p and filling the entire screen (no black bars on top and bottom).
I just measured and our TV screen is 11 feet exactly from our face when sitting up on the couch. When reclined, a bit further. It's just about right. I have the face of the TV sitting about 5" closer than we had our 60" (it was 2.5 feet deep to the wall), but the image was 55" wide on the dot (60" corner to corner in 4:3). I have to say, after 10 years, it's weird to be able to see behind your TV... Didn't seen anything but a big black box when you walked by our old one.
I do notice that when we are in our seated positions, any lights to the right/left of the couch (on end tables), we can see each others light as glare on the screen. Our screen isn't a shiny one, more of a matte finish. My wife used to have a second light on behind the couch that she would use when doing her nails, but I had to ask her not to use it when I'm watching TV as the glare is really apparent with the bright light. I also notice the the black bars tend to be a bit gray on the furthest side from where a person is sitting (I sit on the right side, wife on the left). Isn't noticeable at all when an image is on that part of the screen. Only when the black bars are there.
Our setup:
From my dining room:

From the center of the couch (sorry about the flash):

From the center of the couch (really dark without the flash...):

I see people putting theirs up high and I don't get it. Having the center of the screen right as close to in line with where our eyes are seems to work out nice.
This is just from a guy that likes to watch TV. Dirty Jobs in HD is really dirty...
