I'll entitle this post "YAY" or "How I finally got glitch free playback with no frame drops using LAV/madVR/MPC-HC".
First, my system has an E4200 2.2Ghz CPU and an ATI 5570 w/ 1GB of memory. In other words nothing particularly modern or powerful.
While I never had any problem playing back 480p or 720p files, high bitrate BDs ripped to .mkv with HD audio would have ~300 frame drops in a typical movie. I also had some 59fps files that had crazy amounts of frame drops and were pretty much unwatchable.
First, a little research showed that using DXVA Copy-Back in LAV video pretty much was expected to choke at over 50fps so I went back to software decoding. While this did, of course, pump CPU usage up it was still around 80% so I figured I give it a try. This in an of itself didn't fix my problem with frame drops.
So, more research led me to believe that the problem lied in madVR's settings and in indeed it did.
Looking in the queues in MadVR (CTRL+J) showed my decoder queue was quite stable running at 11-12/12. So, my CPU didn't seem to be the bottleneck. The upload queue was also stable at 7-8/8 but the Render queue would drop to 0 and thus causing the Present queue to drop to 0 causing the frame drops.
In madVR I dropped the GPU queue setting slider (under Rendering/General Settings) down to 6. This really reduced the number of dropped frames so I also tried dropping it further to 4 or 5. That actually made it a bit worse so at least for my ATI card 6 seemed to be the sweet spot.
I also read that using Bilinear under all the Scaling Algorithims would help with an older system like mine so I changed my settings to those. I still had some frame drops though but it was getting pretty good.
Back under Rendering/General settings I ticked off Enable Automatic Fullscreen Exclusive Mode, Use Seperate Device for Presentation, and use D3D11 for Presentation. I can't say if any of this specifically helped but it seems to be recommended.
Even with all that I still had some frame drops.
What finally solved the problem was under the Rendering/Exclusive Mode Settings. I upped the How Many Frames Shall Be Presented in Advance to 6. Shockingly frame drops went down to 0! YAY! If someone could explain to me why this might help I'd be curious but I'm just so happy to have finally fixed my frame drop problem.
So, the moral of the story is that madVR settings really do need to be tweaked for specific hardware and shouldn't be glossed over.