I watch from between 8-9 1/2 feet away, and I decided to pass on a 55 inch and go 60 inch for that real WOW factor. Both would have been fine I think.
Do not forget too that you're not only watching full screen content! Once you throw on a letterbox movie, that viewing space goes way down. You do want it to be big enough so that when you do watch something on Blu Ray with large black bars, you're not thinking "Gosh that's suddenly really small".