I would recommend wiring everything to a single location and then deciding what you want to do at that point. If you have two separate areas then you can never share sources between them and can never really get to a properly integrated system. They will always be separate. Unless, of course, you have one rack directly above the other one and can just drop wires down or run them up.
Two 4x4s will always be less than one 8x8, but you may hit that 4 source limit very quickly.
My home (by example) - 2 cable boxes, PS3, 2 media players, 2 Blu-ray Disc players, AppleTV... That's 8 HDMI sources. Since I want to add HDMI aux. inputs in various locations, I will need something larger than 8x8 for my home setup.
You obviously can do it as you really want to, but pricing on HDMI matrices of really good quality is very high. Even a cheap 8x8 is still over $1,500 via eBay. The cheap 4x4 HDMI matrix switchers are pretty easily under $250 on eBay. Still, it may speak to the quality of what you are purchasing as it will impact how you enjoy the final system.