Speaker placement and room interaction can make a huge difference on sound. You aren't going to gain floorspace by getting bookshelves because you still need speaker stands, unless you plan to put them on a shelf. If that is the case, you should look for something front ported.
I don't think upgrading your speakers is going to make a large difference in sound quality, I would instead keep what you have and see if you can improve the placement. Can you relocate the TV or maybe wall mount it? Perhaps post a layout and we can see if you have any good alternatives. If you have limited placement options now, your new speakers will face the same issue...
Samsung PN60F5300 | Denon DBP-1611 | Roku 3
Denon AVR-1713 | EMP E55Ti | EMP E56Ci | EMP E5Bi | BIC F-12
Topping TP23 | NHT SuperZero 2.0 | Velodyne DEQ-8R