I have my HT setup to dual purpose for 2 channel as well, because frankly I prefer 2 channel music, and while I want reasonable HT performance it is not my priority. At the same time, I keep a dedicated 2 channel setup. Mainly because I can use it while other members of my family are watching something on the main setup. Over time my dedicated 2 channel setup is becoming my primary setup. It will eventually perform better than my dual purpose system.
The reasoning is more location than purpose though. I am certain I could get the same or better 2 channel performance out of my HT if I put the time and money into it. You can easily use a preamp or integrated with HT bypass so that the receiver is out of the chain all together for 2 channel listening. The biggest issue I have found, and the biggest advantage to separate systems is matching the sound stage. If I purchase speakers that will make me happy for 2 channel, I now "should" purchase a matching center to keep the sound stage for HT seamless. Ideally that center should also have matching amps. While you can sometimes go cheaper for rears, you should still have some synergy there. When I set up my HT with nautilus 804s for 2 channel, I ended up buying an HTM-2 for a more seamless front sound stage and I do not feel that I have gotten the same return on investment there. If I got the 2 channel speakers I really want, I would not want to stretch the budget for the center to match it.