Let's clarify a few things to keep from being confusing.
Right now Nintendo has 2 handhelds out on the market.
Handheld 1 is the Gameboy Advance(GBA), which comes in 2 forms.
The original Gameboy Advance looks like this:http://www.ebgames.com/ebx/product/230230.asp
It comes in a variety of colors and such, however, it has no backlight and it uses standard AA batteries.
Later on, Nintendo listened to people complain about the design of the GBA and they released a revision of the hardware. Internally, its made up of all the same stuff, but it offers a clamshell design(to help protect the screen), it offers a built in light, and it also has a built in rechargable battery. This new design is called the Gameboy Advance SP(GBA SP).
It looks like this:http://www.ebgames.com/ebx/product/232807.asp
It also comes in a variety of colors, becuase it has additional features that the original GBA didn't have(i.e. backlight, clamshell, etc.) its also more expensive, but well worth it in my opinion.
This past November Nintendo released another handheld onto the market. Its called the Nintendo DS. It looks like this:http://www.ebgames.com/ebx/product/241868.asp
This handheld is more powerful that the GBA. It offers things like 2 screens, touchscreen, wireless link, etc. It plays DS games and it also plays GBA games. It has a slot for each type of game. The DS is more expensive and bigger than the GBA SP. It is a great handheld and is something to consider as it offers more functionality and can play both DS and GBA games, but again, it costs more money. DS games typically cost between $30-$40.
Hopefully this helps clarify things for you. If you're looking for a cheap system and some cheap games, the GBA SP is a solid bet as its very well built and very tiny making it easy to carry in your pocket. The DS is an excellent system as well and it is more powerful and plays GBA games as well, but its more expensive and a bit bulkier. You may want to go to a game store or a Best Buy and check out the demo units they have on display so you can see them in person.