You're also probably not even using much power. 1watt is actually a lot of power into a speaker, so you might only be using a small amount of power and due to speaker efficiency, get a lot of sound out if it. If your 50 watt amp was loud enough, changing it to a 100 watt amp won't make a difference. After all, if you were using it at 15 watts, you'd still be using it at 15 watts.
What a higher power amp gets you is more maximum volume. And headroom, if you are playing something with high dynamic range, it lets the loud parts get louder without the amp reaching it's limits.
Think of it this way. Those dinky $20 speakers are around half a watt of power, and can fill a room with sound. Not very good sound since many will top out and clip first. Invest in a better speaker able to do a couple of watts and it will sound much better as it can handle dynamics better (and be able to handle bass better, where power is usually needed). The base volume level doesn't require much power,but peaks in the sound do, as well as decent bass (which is why many subs contain their own amplifier - to offload the main amps from having to drive them).