I don't have deep knowledge myself, for the most part I have been using VMR simply because it was the newer tech and I read it was better
So here it is, the DXVA end user's overview:
DXVA is a new set of interfaces between the operating system and the video card driver. If you want the card to help processing the video stream, you need a standarized way to pass it the unprocessed stream (deinterlacing, scaling, postprocessing) (more information is located at msdn.microsoft.com
The upside here is that newer formats like mpeg-2 in HD or MPEG4 are quickly overwhelming today's CPUs while most graphics cards are better suited for these tasks because they have more specialized raw computing power. This is going to be huge, cf. the comparison at xbit labs
where the GF 6800/6600 video processing unit reduced CPU load for HDTV content from 100% to 22%.
As HTPCs are moving into mainstream with all the digital lifestyle and convergence initiatives going on, I hope we're not only going to get lower processing requirements, but also higher quality scaling, deinterlacing and postprocessing in hardware.
Key to understanding VMR9 is to understand graphs - you can throw anything at DX and it will pick the filters and decoders automatically. If there are multiple renderers present for a specific format, one will picked according to their merit (priority). I am using Graphedit (Microsoft) to view graphs and Filtermanager (RadLight) to modify merit, if neccessary.
Generic media players like WMP10 or zoom player let the DX pick the filters, while players that come with codec packs like WinDVD are usually hard-coded to their own filters. Applications can also build their own graphs, the configurability of which is the key strength of zoomplayer.
ffdshow comes in here as a raw filter - it does not decode anything, it takes raw video and outputs raw video. So it gets to run after the decoding but before the hardware output. There are a number of things it can do to video like high quality resize, denoise and sharpen. (This reflects my opinion on how ffdshow should be used, and is not an accurate representation of its features)
As a raw filter you can manually add it via zoom player, and it will get added for anything that leaves building the graph to DX.
Going back to filters and graphs. The automatic picking of filters can be an unexpected and hard to troubleshoot source of problems. You decide to install a trial of Ulead VideoStudio and suddenly SageTV crashes - because it now uses a different set of filters. Sometimes you end up with a mpeg2-splitter, audio decoder and video decoder from different companies, and these often don't work well together, let alone in sync. I found myself editing or disabling filters more than once because something somewhere had disrupted video playback on my machine.
To make things worse, some codecs are using per-application settings. That means in order to get the much touted deinterlacing provided by WinDVD 6 in zoom player, you have to set some registry key, to get DXVA in Sage TV another registry key.
But enough whining, there's a learning curve, and the results can be rewarding, and as pleasing as a good hardware DVD player.