this depends on the design requirements for the total specular response you wish to achieve.
the problem with "too much diffusion" it is generally via the over-application of NON-broadband diffusion.
another issue is the construction of large arrays (to cover large sq area surfaces) via repeated applications of the same diffuser type/design (periodicity) - of which specular lobing is exhibited. this is why inverse panels (as you will note is available for calculation in QRDude) are utilized, and the array can be formed via a pseudo-random sequence such as the Barker Code.
QRDude technical guide snips much from Acoustic Absorbers and Diffusers (Cox/D'Antonio):
you could also apply Binary Amplitude Diffusers (BAD) panels around the room (which offer spatial but NO temporal dispersion, unlike the phase gratings). you could even curve the BAD surface to form a poly. plenty of options to assist in getting the energy as well mixed as possible.
Edited by localhost127 - 1/28/13 at 2:17pm