Boxes aren't usually used for LV; people usually use LV brackets (new construction) or LV rings (retrofit/old work, digging into drywall to use a buried cable or conduit). You want to avoid kinking category cables and HDMI cables.
Lighting designer was not used, or you told him/her that you didn't want lighting control.
For some people, aesthetics are more important than serviceability. A multigang bank of switches/dimmers can be reduced to a single gang keypad, when the loads are 'home run' to a central location. Centrally controlled systems are available from Lutron, Crestron, Vantage, LiteTouch, and Control4, so in reality, you're not really locked into a single vendor. These systems have been in use for decades, in high-end homes (and commercial buildings).
Installation of centralized lighting control, to remove banks of switches, generally requires new construction or significant renovation (down to studs). But, retrofitting centralized lighting control isn't impossible, especially for a room or 2. In fact, it's very easy, and not that difficult, if there is only drywall to contend with repairing, and the room is in proximity to an electrical subpanel. There is also an option for moving the gangs of switches to a nearby closet, and using a single keypad in the living space.
Lutron HomeWorks is the market leader, for lighting control in high end homes. The HomeWorks QS line offers wired and wireless options, for control. It can be retrofit, like z-wave, zigbee, and UPB, but you can't replace gangs of switches without rewiring.
I'm using Lutron Radio Ra2 in my home, the more affordable little brother of QS. It doesn't offer the option for hardwired lighting control, has comparatively limited functionality, and there are fewer keypad/dimmer choices (compared with RA2).
The decision to use centralized lighting control is personal. You've decided you want multiple gangs of switches, rather than keypads. Many people prefer that option, for the reason you cite.
As for retrofitting lighting control, z-wave isn't a bad choice. You might consider Lutron RA2 or HomeWorks QS. Find a lighting designer familiar with the various lighting control options, to help you decide. Or, become an expert, like you did with the fireplace. It's not rocket science, but there are many subtleties to choosing, designing, and implementing, and it's time consuming. Also, HomeWorks is dealer-only; not a DIY option.
This is a Lutron QS 'Architectural' keypad:
This is an example of a QS 'Signature' KP:
This is the only style of KP offered with RA2 (for Decora wall plate openings), paired with a Lutron thermostat:
Multiple button configurations (and colors) offered for both QS and RA2; that's just an example. Page 24 of this Lutron QS brochure
shows the keypad configurations, colors, and style options, for QS.