[sspears]

I am starting a new thread to discuss pre-processing of video. In this context, I am talking about converting from a higher bitdepth down to a lower bitdepth. In the case of BD and HD DVD, it is often 10-bit 4:2:2 to 8-bit 4:2:0.


Below are two examples taken from the Elephants Dream project. You can download the publically available 8-bit uncompressed PNG files from their website. http://media.xiph.org/ED/1080-png/ Sorry, I don’t recall the exact frame number. I am not sure the process they used to create their 8-bit PNGs. Elephants Dream does have some pretty severe banding throughout.


I was able to obtain the original 16-bit OpenEXR files from them. In my two examples, the same path was used.

1. Open EXR file in After Effects CS3

2. Disable all processing

3. Export as 16-bit TIFF

4. Use Microsoft xScaler to convert from 16-bit TIFF (48-bit RGB) into 8-bit RGB (24-bit RGB)

5. The images were also converted from PC to Video levels.


In the left image I used one of the xScaler dither modes. In the other I used the xScaler rounding mode. There are a lot of different techniques to dither an image, this is simply our favorite. Don and I spent a lot of time working on this technology. It was originally written about five years ago.


The area to see the banding is on the light gradient to the right of the image.

It is simple to maintain this level of detail on a static image. The problem comes when there is motion. In this particular part of Elephants Dream, the image pans to the right. The dither does make compression more difficult in this example. I have other examples, like Coral Reef, where it actually becomes easier to compress.

 

[Kroenen]

That's very impressive Stacey.


You mentioned dithering can make compression more difficult when in motion. Paramount's HD DVD version of Beowulf, as you know has a lossy audio track while the UK versions have lossless.


Is it plausible that since the UK BD version most likely shares an encode with the UK HD version; that they decided to round down to 8 bit, instead of using xScaler, because the combination of the dithering and the allowance for the possible peaks of the lossless track, might have been beyond HD's peak bitrate limit.

 

[MovieSwede]

Kroenen I doubt the 1,5 mbs difference will have that much effect.

 

[sspears]

xScaler is SLOW. Its really only useful if you can run it in parallel. Microsoft never built an application that has incorporated it this way. One K and Deluxe have built their own parallel application that use it.


The example I provided is an extreme example. In a lot of cases, you can't see any banding from rounding.


You can increase the dither amplitude and reduce banding in the source, which is done by some.

 

[dr1394]

I've often thought that the best dithering would be some adaptive algorithm that only dithers on contours and leaves flat regions untouched.


Here's an image that's been dithered from 10-bits (by a very simple hardware algorithm) and also MPEG-2 encoded. The dithering helps the contours somewhat, but also adds noise to the flat regions.




Also, the resulting noise looks very much like DCT basic functions. I also wonder if there's a dither technique that takes into account how the dither will be compressed (by some DCT or DCT-like transform). Seems like a much more difficult issue though.


Ron

 

[Kroenen]

Ah gotcha. I was equating "compression being more difficult" with "requires a bitrate increase". I see what you're saying now.


Thanks Stacey. I appreciate the reply.


Thanks as well MovieSwede.


BTW Stacey do you have a blog?

 

[MovieSwede]

Kroenen


Thought I can add this.


While the difference between a lossy and lossless track is just about 1,5mbs in peak, there could be more stuff in the mix that effects peak and avarage bitrate.


But sspears can this stuff better then me.

 

[sspears]



Quote:
I've often thought that the best dithering would be some adaptive algorithm that only dithers on contours and leaves flat regions untouched.

Ron, if you look at the pixel values in the dark part of the ED dither image, you will notice they are RGB 16, not a 15/16 or 16/17 mix of noise.

In the latest version of PEP, we included a contour correction filter. Its not perfect, but it's cool. I will ask Joe Kane if I can use an image from his new disc. It looks like Deluxe re-did the encode to fix some source side issues, so the banding is present. I had fixed it in the original encode I did.

 

[sspears]

Here is another clip. In the ED example, I converted from 16-bit RGB to 8-bit RGB. In this next example, I will convert from 10-bit 4:2:2 down to 2-bit RGB. OK, perhaps 2.5 bit. There are 5 levels used in the 2-bit examples. You can open in your favorite app and look at the RGB histogram if you don't believe it.


First up is the 10-bit to 8-bit reference image.

 

[sspears]

This is the 10-bit to 2-bit using rounding. If you look real close, you might be able to see some banding in the image.

 

[sspears]

This is the 10-bit to 2-bit using random noise dither. Its using an R250 random number generator. Its a bit noisy, but the banding has been reduced. I had to use a full quantization of random noise to reduce the banding.

 

[sspears]

This is 10-bit to 2-bit using error diffusion with the random number generator. This is using half a quantization of random noise to remove banding. This is our favorite and is what was used in the Elephants Dream example above.

 

[TheLion]

Impressive thread! Thank you for sharing, Stacey.


Can you provide any information if the recent BD release of Ratatouille (Deluxe?) used the xScaler dithering routine? Given the presence of really critical scenes ([email protected] Saine...) I was pleasantly surprised by the complete (!?) lack of obvious banding artifacts.

 

[sspears]



Quote:
Can you provide any information if the recent BD release of Ratatouille used the xScaler dithering routine?

Yes it did.

 

[TheLion]



Quote:
Originally Posted by sspears /forum/post/13472275


Yes it did.

Thank you. This settles it then - xScaler is very impressive indeed

 

[sspears]

So we have looked at dither, now I will post some image reszing samples using the same tool.


One of the ideas that we have been pushing is to perform all processing in linear light. The video you watch today has a gamma curve. Linear light involves removing the gamma curve, perform processing and then re-applying the gamma curve. When working with linear light, you need extra bits for processing. An 8-bit gamma corrected image would need 12-14 bits if working in linear light. Personally, I would not do it in less that 16-bit floating point.


The first sample is a zone plate that cycles from 5 MHz to 20 MHz. This is the original image, which is 1080p.

 

[sspears]

This image is the 1080p zone plate resized to 720p using linear light. The filter kernel is a bicubic algorithm.

 

[sspears]

This image is the 1080p zone plate resized to 720p like everyone else does it, ie with gamma correction. The same bicubic algorithm was used.

There are a couple of things to point out. First, the edges are darker. This is the highest frequency portion of the zone plate. Second, the moire is more pronounced. If you look closely at the linear version, the moire is present, just more difficult to see because the high frequency detail is not pulled towards black.


The linear zone plate looks more like the orignal. Again, same tool was used to resize, I simply turned linear processing on and off.

 

[sspears]

This next sample shows what happens on think diagonal lines as well as on strong colors. Pay attention to the thickness of the text and diagonal lines. Also look at the edges of the text on the red and blue backgrounds in the resized samples.

 

[sspears]

The image on the left was resized in linear light. The image on the right was done with gamma correction. Note that depending on your browser, and resolution, it might be top and bottom image.

You will notice that the "Here is some white text" has become thin on the right image. You should also notice that the green text on the red background has dark spots around the edges. You can also look at the thickness of the diagonal lines.