You may not have the current, all-magnet panels. If your panels have 1/2" thick squishy-foam along their tops and bottoms, then they are the interference fit versions. It would be even more important for the two supplied Hangman brackets, spaced widely, to be used so that the fit within the frame is as designed. They should fit snugly and require a little compression to squish into the frame.
The current all-magnet versions are less sensitive to how the frame is hung.