I have a GTX 660 and it does the job just perfectly.
The 650 has significantly less shader cores, so it would be much slower. However, if its still enough for madVR i cannot tell you from lack of experience. Some of the advanced madVR algorithms are quite demanding, though.
If your budget allows, you could also go for a GTX 650 Ti to have some more performance headroom. If the 650 is your top limit, it will certainly work, but you might not be able to use all of the advanced features (especially future things being added, like debanding which will be in the next madVR release)