Originally Posted by babgvant
If the video is encoded in 16-235, why is converting to 0-255 before output better?
Video is encoded in YCbCr. Converting YCbCr to RGB results in floating point numbers. Converting RGB floating point numbers to RGB cardinal numbers means that you have to round/dither. The more steps you have for that, the better. So using 0-255 gives you more steps to spread the floating point RGB data to than 16-235. The difference should be minor, though, I think. Besides, Y has the biggest effect on the resulting RGB pixels and Y maps directly to 16-235, too, which is an argument for using RGB 16-235. So I'm not really sure whether to recommend 16-235 or 0-255, based on the above alone.
If you output YCbCr as YCbCr, the situation is a bit different, of course, because your data stays cardinal. There keeping 16-235 is the best solution. But then, outputting YCbCr can be problematic on its own. E.g. if you upscale SD content and output it as YCbCr, the display will think it's native HD content and apply the wrong YCbCr -> RGB conversion matrix (BT.709 instead of BT.601), producing incorrect colors. Furthermore, even if you switch the GPU to YCbCr output mode, it's hard to say if the data will really stay YCbCr all the time. It's quite possible that the data is converted to RGB somewhere on the way and later back to YCbCr, which could result in banding artifacts being introduced once again.
There's one more key aspect to this: You know, computer desktop applications always use 0 as black and 255 as white. Also digital photos are usually stored with 0-255. So if you display a photo in a photo viewer application and the GPU is in 16-235 mode, the GPU will scale the 0-255 photo to 16-235 behind the photo viewer's back, probably without dithering, which will introduce banding artifacts. The same problem applies to games and to my video renderer madVR, too. So if you want to use the same PC for desktop applications/games *and* video playback, then using RGB 0-255 is the best solution because it will bring everything to the same definition of black and white without the GPU having to do violence to the data behind anyone's back.