Thanks for the suggestions--I'm buying some of those stickers for other things, but the way these particular buttons are set up don't really lend themselves to stickers.
So I dug down to the circuit board and plucked off the LEDs. It was a little brutal, but the end result was exactly what I wanted. Here's a quick step-by-step in case anyone finds this thread in the future:
1) Unplug couch!
2) Pull off rubber pad at bottom of cup-holder. Should come up easily.
3) Unscrew the four screws at the bottom that hold cup-holder in place. They're firmly drilled right into the wood, so use your DeWalt.
4) Pull the cup-holder out of the couch and flip it over.
5) Using a tiny phillips screwdriver, remove the 7 screws (3 in the silver part, 4 in the black part) under the rim of the cup-holder.
6) Pull off the black plastic piece that covers the backside of the buttons.
7) Remove the circuit board.
8) Using small needle-nose pliers, carefully pluck the three offensive LEDs off the circuit board. (Alternately, use a soldering gun to remove the LEDs the "right" way. I was too lazy to do this.)
9) Put it all back together.
The buttons still work great, and the "LIGHT" button still turns on the big cup-holder light if you happen to want a light. Otherwise the whole thing is pitch black all the time.
I know nothing about circuit boards, so maybe there's some reason this method is a terrible idea. My results were great though. I've included a picture of the circuit board after I pulled off one of the three lights so you can see what the lights look like.