Not sure where I read the MTU size should be 1464, but I logged into my router and made the MTU size 1464 on both WAN and LAN (carrier side and home network side) routed interfaces.
I had this problem since the PS3 days and it carried over to my PS4. Before making the change I was getting 6M download and 3 M upload from the PS3/PS4 test network connection. My service is actually 50 down / 10 up. After the change to my router config I got 37 down / 3 up, which is a huge improvement. These speed tests are not to be taken for scripture, but they should give you a ballpark idea of where you are at.
In any case, making that change on my home network equipment made a drastic improvement.
Also, I have a commercial grade router with complete command line access so I can assume that not all routers have the same flexibility and options. So this may not be applicable to all, but hopefully helps some.