They will sound quite different I suspect. The RDC-7.1 uses Wolfson WM8716 (Virtually identical DACs are now used by Pioneer in their top end gear) whilst the DTC-9.8 uses Burrbrown (1796 I think).
With an HDMI source I suspect the DTC-9.8 would be plain better as the RDC-7.1 does not support high res audio over HDMI.
For DVDAudio and SACD, as long as you have an ILink player the RDC-7.1 might be better, ILink is lower jitter than HDMI and HDMI offer no benefits at all over ILink for these formats.
For digital sources I think you would have to ask someone who has tried both, the RDC-7.1 has a supposedly very accurate Apogee master clock and does all its DSP processing at 24/96 (Same as DTC-9.8).
For analogue I suspect the RDC-7.1 might be better, it has true analogue pass through and analogue volume control.
I know some TX-NR1000 owners tried the new 905 Onkyo and decided the NR1000 was better but of course there is the amp section differences too for that one. The RDC-7.1 also offers Nettune, more coax and optical inputs and both XM and Sirius radio which AFAIK the DTC-9.8 does not.
Overall I would only consider the RDC-7.1 if for some reason you wont be needing high res audio over HDMI.