Thanks. Looks a lot like what I planned. I'm dealing with a 2x4 wall inside a 2x6 exterior wall so I have some more depth to work with. Anyway, mine would be what I'd call semi-permanent. I was going to do plugs but for all the trouble I've about decided it won't be worth it since I won't be removing them for any reason other than a catastrophic event (e.g. broken window).
The "easy button" is to simply add some furring strips to the window perimeter (jamb) of the wall-in-wall where I had the windows left open when I drywalled. Add 2 layers of drywall to plug the holes. Attach layer 1 with a few screws to hold in place. Then caulk layer 1 with acoustic caulk, and when it sets remove the screws. Add the second layer of drwywall and screw it in. New mud the crack but don't tape. The mud should "grow" a hairline crack where the joint isn't taped. I'd make an effort to not mud the screws. This way, I get a flat joint, same material as a wall, a hairline indicator of where the thing really is in case I (or a future owner) wants to remove it. All one would have to do is remove screws, maybe hit the perimeter with a MultiMaster vibrating cutter, pull the drywall out, voila, ready to case and trim.
Maybe skip the mud altogether and just cauld with acoustic sealant.
Just fretting about what to put behind the drywall. Thinking I'll have window, then a blind or something to make Mrs. happy, then insulation, then vapor barrier (or not), then drywall.