I'm a firm believer in Bigger is Better having gone from a 45" to a 57" 2 years ago and the Wow is still there 2 yrs later. Even though mine predates 120Hz I'd personally never give up the size I have to jump to 120Hz that'll happen when I jump to the next gen 65".
In HT Size definitely matters and rewards one daily. Unless you have an extremely small room and lack HD Source I can almost guarantee that once you get the 46" panel in your home and you get used to the 46" within days or week you'll have buyers remorse in not going bigger.
Your Wallet so your decision but I'd go Big for that HT WoW!