So I tried to bring everything together into a single diagram, I think Im pretty close.

Red is the IR distribution. Blue is speaker, them running through the Nuvo keypads is understood, I just didnt want to get that detailed in a block diagram. Yellow is the fixed output from the Nuvo running to the AVRs. All the cabling to the video zones is going to be 6x CAT-6 and 2 RG6 with HDMI running through conduit where noted. The Study receiver is a bit of overkill, Im using it to push other sources to that room (plug and play) if the need arises until the matrix is in place. I put the LSI in the study because Im considering making that a 5:1 zone as well. Ill have my computer stuff in there, and there is the possibility that Ill game with that set up from time to time.
The LSI on the patio is because those speakers will not be ceiling mounted, they will likely be some kind of high mounted external speakers in the corners of the patio area projecting horizontally. I think that will work well for both TV and stereo music.
For usage, we are looking at 4 people living in the house; my wife and I, 10 year old son, and 18 year old daughter that is moving out in the next year or two. As stated before, my wife and I arent big TV people. We enjoy movies, and we watch the news, but we dont sit down and watch TV from 5-10 PM. We are serious sports fans though, watch a ton of soccer, college football, hockey, and the NFL. That is mainly a weekend thing though. The kids watch a fair amount of TV, but they are just as likely to play video games all day if they were allowed.
The family room will get the most usage obviously. We will probably do quite a bit of family room / patio stuff during sporting events. We grill a lot (outdoor kitchen) so the ability to keep track of the game going back and forth from the big kitchen to the outdoor one is important. That is why Im running the second monitor out from the family room AVR to the patio. Slaving to the famly room's channel shouldnt be a problem 90% of the time, and shouldnt be critical until we get the matix in down the road.
The whole family enjoys music, I have an Itunes server that will go along with the XM streaming to feed the NuVo stuff. I think that will be enough sources but we will adjust as we get a feel for how we will use it long term. Our TV habits are understood, how we will deal with having music at our fingertips all the time isnt.
Internet access will be plentiful, Im planning on putting in 3 WAPS in strategic locations (Study, Family Room, and Bedroom 2 closet) to ensure there is plenty of good wifi access in any point of the house or patio area. There will be plenty of CAT taps located to adjust that as we figure out usage as well.
Budget will dictate our priorities to some degree. Right now our plan is:
1) Wiring the living crap out of the house to include conduit on any external or soundproofed walls. Then adding some more.

2) Nuvo Essentia and Music Port. Still working out the priorities what 6 zones we consider a priority
3) Ceiling and in wall speakers (paint matching, no place holder plates)
4) Nuvo Extender for the additional 6 zones.
5) TV for the patio
6) CAT matrix and additional TVs.
As of now we think 1-5 will make the initial budget, but we are still early in the build plan. Worst case 1-3 will make the initial cut.
Please review and tell me where I may be going wrong.