How to connect headphones to cable box?

Short answer is you can't. There's no amplifier in a cable box. All the cable box does is send audio and video signals to a source, such as TV or receiver.

Any kind of amplifier should work. Most amplifiers have a headphone out, but even if it doesn't, you could get an adapter that has, usually, a 3.5mm headphone female plug (that's the small one) on one end and two RCA male jacks on the other (right and left), which you could plug into the back of the amplifier if there are any "line out" or "audio out" jacks.

If you're JUST going to be using headphones, and have no intention of using speakers or surround sound setups, you could use a dedicated headphone amp, but those cost as much as a home stereo amplifier. If you don't have an amp and want something cheap, check out ecost or amazon for a recertified amp. You can usually pick up a unit for as low as $80-90.