So your PS3 & TV has xvYCC, but is the movie you are watching using the full xvYCC color gamut ? If its a "feature film" did the cameraman use a camera which has xvYCC support ?
If its a cartoon/animation film, or video game then is the production company supporting xvYCC color gamut ?
I guess you need to answer those questions before you want to sensibly switch that control. I guess I would keep it on all the time...