Frankly, if you are limited to one tool, you should buy a flat pack.
The most useful tool in my experience is a 1 1/2 HP router with 1/4" and 1/2" collets. A plunge base has its uses but is not necessary for most speaker building. If you just need to start a cut in the middle of a panel, you can ease the bit in at an angle. If you really have to do a straight plunge once in a while, you can just loosen the clamp on a fixed based, lower the bit, and retighen the clamp. A circle jig has its uses, but is also unnecessary. You can do the same thing with nothing but a piece of 1/4" plywood scrap with a pivot hole in the right place screwed to the based. Use a router with a straight edge and clamps, and you can true up an edge or even cut a sheet of plywood.
Of course you need router bits. The obvious one is a decent straight bit. I use 1/2" shank bits whenever I can. My all purpose bit is a 1/2" end cutting straight shear cutting bit. A 1/4" bit won't take the abuse a 1/2" bit will. I only use a 1/4" bit when I need a 1/4" slot. Spiral bits have their place, but are not necessary, and will break more easily than straight bits. Round over bits are necessary only if you are going to round over your edges. A flush trim bit slightly larger than the thickness of the panel to be trimmed may be useful. 2" is way too long for most purposes.
Save the money on the plunge base and circle jig and buy yourself a good drill and bits, too.