Unless I'm misreading what you're saying, you're trying to plug the Line Out jack (which is analog stereo sound) into the SPDIF input of the MSI card. SPDIF is digital. I believe the only SPDIF jack on the DG965WH is optical. You could find an optical-to-coaxial converter and send that into the MSI card, or perhaps tap the motherboard's SPDIF signal internally (which is a total hack, but it's only a signal and ground-- should be doable) before it gets sent to the optoelectronics on the optical jack.
Are you on XP or Vista? There's an Intel Audio Studio application (XP only, I think) that should have come with the motherboard (or you can download it from intel.com) which I don't think allows you to retask the analog 8.1 channel outs on the back; perhaps it will let you retask one of them to SPDIF coaxial? I'm away from my machine, so I can't recall all the retask options available in the software, but at least that way you wouldn't have to hack.
I do not speak officially in any sense for
Intel Corp., Technology Manufacturing Group
but I do work there.