True, but without the movie fully cached, I was thinking that you might have to wait if you wanted to skip ahead several chapters (e.g., a movie you started watching a few days ago but never finished). But I just tested this out with a movie on my network, and here's what I'm seeing:
You start the movie and it starts at the beginning. Starts playing nearly immediately, with a buffer just slightly ahead at first. Assuming you have a decent network, the buffer should increase significantly after a short while. If you skip ahead to another chapter, it does that just as quickly/well as any other streamer I've used. The buffer then appears to start to increment from that point forward. My guess is that it's designed primarily to avoid pauses/stuttering. If I then try to rewind into an area that had not yet been buffered it, again, seems to do just as good a job as any other streamer I've used (and probably better than some). I should note that I'm testing this with a 802.11n connection, so it's quite fast. If you were trying to use this with 802.11g, you might have more trouble, but the on-board buffer could make even that workable for the average person under the more common situation of simply starting up a movie from the beginning.
That's all good news in terms of Apple prehaps relaxing the need to allocate enough of the buffer for an entire movie's storage, but here's the bad news: Their primary desire is for you to rent movies from them. And as you can read in this thread, there are a large percentage of people who do not have reliable 6Mbps or better download speeds. If you have a slow connection, the Apple TV will need to buffer at least half of the movie (and make you wait 30-60 minutes to start watching it) to ensure that you don't run up against the buffer 1/2 - 2/3 of the way through watching it. And to account for the possibility that you might fall asleep halfway through the movie and want to start it back up again the next day, I can see why they would want to allow the entire movie to fully load to the buffer, so that the next day you can start it up at whatever point you fell asleep.
That may be true about productivity apps on the iPhone, which are making use of the built-in UI controls (text boxes, etc.), but I suspect that graphically-rich apps/games consume significantly more than that, and those are the types of apps/games that might be more desirable for the Apple TV. For example, I just took a quick search in the iTunes store and found the graphically-rich FPS "Modern Combat 2", and that takes up 440MB.
OTOH, there could be a lot of popular apps which pull their graphically-rich content from the web (Facebook pulling photos, etc.) and those could simply make use of a smaller chunk of common cache for that. Loading another app could clear that cache out and then start to make use of it.
EDIT: I should also add that the Apple TV does have a USB port, so perhaps gamers will be allowed to connect external storage in the future. And there's nothing to stop Apple from coming out with a 64GB Apple TV in the future, aimed at that market. And now that I'm thinking this through, another obvious possibility: Under "Computers" you can store your movies and music as stored on a central computer. I see no reason (other than perhaps DRM/licensing concerns) why you couldn't store all of your Apple TV apps on that computer, and it will just copy one app at a time to the Apple TV's local buffer.
Having said all that, though, I still suspect that apps of this sort won't be coming to the Apple TV within the next 6 months, simply because Apple would have probably demoed or hinted at them if their release were that imminent.
Yeah, that's probably right, and I don't even know what, if anything, it really is caching. Unlike the original Apple TV, it's not syncing any of your music collection to the box, so it's always streaming that from an always-on iTunes computer. It's possible that it's only caching the metadata/coverart on-the-fly for whatever list of music/movies you're seeing on-screen at the time. Once you scroll down your list of movies further, it might let the previous screens' metadata get stomped on. But, as you said, that metadata probably doesn't need much space, so they can probably cache it all so as to keep the UI performing well.