If it's just one hole, and it's slightly off, you could probably redrill it and fill the gap with extra epoxy when installing the hurricane nuts. Leave the bolt in the nut to keep out any extra ooze, or chase it with a tap later. As long as redrilling doesn't result in a hole too close to the edge.
When I built my THT's I did the following:
1. Place driver cone-side up, place cut out baffle over top, and ensure all around alignment.
2. Carefully clamp baffle to driver with small clamps so that you can flip the assembly upside down. You will need a jig that will keep the clamps elevated from your work bench. With small squeeze clamps, a pair of 2x4's should do it.
3. Re-check alignment and ensure it hasn't shifted. If it has, go back to (1).
4. Use largest bit that will fit mounting holes, but is still smaller than than hole required for the tee/hurricane nuts you are using. Drill 1 hole and run a temp bolt and nut. The mounting holes will act as a guide if the drill bit is large enough.
5. Re-check driver alignment, drill a hole, and run a second bolt at the other end. Tighten both bolts such that the clamps can be removed if you wish.
6. Drill the rest of the holes with the same bit used for the bolts.
7. Remove the driver and re-drill holes to size needed.
If you doubt the integrity of the baffle make another one. It's easy at this point. Almost impossible once it's glued into the cabinet,
