Dialog level is actually varying the sound level coming out of the front center speaker. If the receiver is sending sound only to the front left and right speakers, there's nothing that can be varied, so the option is greyed out. If you enable Dolby ProLogic or DTS Neo surround sound processing for your Roku connection, then the Dialog level option should become available. The sound processing setting is remembered separately for each of the inputs on the receiver, so it probably is on for your PS3 when it sends stereo, but off for the Roku. When the receiver gets a multichannel audio signal, that includes a center channel, so the option becomes available.
Marantz SR7009 avr + MM9000 amp --> Atmos 7.1.4
Fronts=NHT 2.9+AC2, FH+TM=DefTech PM1000, LCR+TM amped