It looks to me like TMT3 installation is corrupted somehow.
Try uninstalling TMT3, then delete Arcsoft directories in
These are hidden directories, so you'll need to enable the option of "Show hidden files, folders or drives" in Folder options of Windows Explorer.
Install then TMT3 retail + .160 patch, and set it up. If it worked in Vista it should definitely work in Win7.
Of course it wouldn't hurt to uninstall/install Asus 17.75 drivers.
If I were you I would really try hard to make it work, as I personally don't think that a ATI 5xxx/TMT solution will be available - if it ever will - before 1Q next year, most surely with next TMT version (TMT4?). This is my personal speculation of course.
OTH 5xxx/PDVD could be a really bumpy road. We have yet to see 1st hand experience with PDVD patch for bitstreaming (not yet available AFAIK) by true first guinea-pig end users