The Observers seem to have used Peter's activation of the machine to modified things such that he never survived. Walter attempts to save him but Peter drowns in the attempt and therefore never grows to manhood. Walter's attempt does serve to cause the conflict between universes.
3. "Traces of him remain", as I remember an Observer saying, and Walter starts going crazy since he starts seeing Peter everywhere. Also Jones is kidnapping people, using shapeshifters and causing all sorts of trouble with his cool explosive rock he is mining.
Olivia is also having dreams about him which she shares with Walter.
The Observers task September (the Observer who started the problem) with the job of eliminating the traces of Peter. He creates a device and goes to the place where he would use it to presumably erase the "traces". Then chooses not to activate it.
4. I don't know what happened but Peter is back.
At the end of the episode 4, entitled "Subject 9", Peter materializes, naked, above Reiden Lake, the place where Walter crossed over to the other universe to save him (it was frozen at the time) and in which he drowns when the attempt fails. A father and son are fishing when Peter drops into the water and they pull him out. He ends up in the custody of Fringe Division, who slowly come to trust him as he helps them over the next several episodes.
5. Peter is understandably confused by his new timeline. Allegedly his influence magically makes Olivia forget herself and revert to Peter's old Olivia instead. This part makes even less sense to me than anything previously.
Olivia's regaining memories of the unedited timeline are presumably due to both Peter's presence and the evil Nina's dosing her with Cortexiphan for some scheme of Jones'.
What did I mess up? I think a key thing I missed is how Peter came back from the netherworld, although how did he even get to the netherworld if he died?
He didn't die and he apparently lived on due to the strong feelings of the people who cared about him, principally Walter and Olivia. Olivia is apparently losing all memory of the edited timeline and will forget her relationship with Nina (who raised her and her sister in this timeline but not the other).