Tough decision, Jtm79. Where you watch lots of SD, and DVD, it sounds like a good portion of your viewing is far-less-than-blu-ray quality. With that in mind, personally, I'd stay on the smaller end of things. I know many in here always say the larger the better, and I understand their point when blu-ray is concerned. But the thing you have to think about is - do you want a large portion of your viewing to look far worse than you may be used to, just for the benefit of your blu-rays looking their best?
For me, I totally love and appreciate blu-ray's quality potential (when done well from the studios), but I still watch lots of tv (mostly HD but still compressed) and I own a ton more DVD than blu-ray, and that will be the case for a long time to come. So for me I think about how I want all my favorite DVDs to look. With my 51" E8000, blu-rays look incredible, and DVD holds its own when upconverted. Certainly not blu-ray quality by any means, but a good anamorphic DVD upconverted looks darned nice on a 51" screen. At 60" or 64", it's gonna look rough no matter what. SD tv will look even worse.
So for you, is it worth it to have the 250 blu-rays (which is a nice collection) look their absolute best and really show off what blu-ray can do, at the expense of all your other viewing? For me, the answer is no. Of course it's up to you to decide how important your other viewing is. If you're a big tv fan and don't want to cry when you watch your DVDs, consider keeping it smaller. 55" is a bit of a jump from 51", not huge, but a bit noticeable, and your DVDs will look very nice. I don't know how DVD will look on a 60", but I can't imagine them looking good on a 65". That's just my thoughts.
Bottom line - blu-ray will still look awesome on a 55". SD/DVD will look terrible on a 65".