AVS Contrast Project v2.0 (updated 1/26/07)

Background:

Contrast has long been considered a key display specification, but in todays environment with projectors having dynamic irises (DI), fixed irises, high-ANSI but average on/off CR and high on/off CR, but average ANSI contrast it's clear that a better description of contrast is needed. The traditional metrics used in the industry are sequential (e.g. Full White / Full Black) contrast and the ANSI checkerboard contrast test pattern. Sequential contrast is a useful metric for gauging the maximum contrast achievable in the darkest of scenes as luminance approaches 0 while the ANSI checkerboard contains 50% peak white luminance which is much brighter than a typical movie or video scene. One drawback to using only these two tools for gauging contrast is that both of these test patterns describe only the fairly extreme bright and dark ranges that would occur in a typical movie scene. Film based content is a dark media with ~20% average picture level (APL) so it's clear tht most scenes fall in between the traditional measures of sequential and ANSI CR. We will also show that trying to apply traditional sequential contrast is no longer an applicable figure of merit for gauging the static contrast for dynamic iris projectors at 0 luminance and we will provide an alternative definition that describes both non-DI and DI equipped projectors alike.

What this project attempts to do is gain a better understanding of projector performance in several interesting ways. One is through the definition of a new suite of contrast test patterns (The "Phelps/Petersen Contrast Test Patterns"), Another is through mathematical modeling methods implemented by Erik Garci and another is by relying on individual forum members to supply data on their own projectors which can be maintained and displayed in a database as an ongoing effort.

A word on Contrast
In the following discussion it's best to think of contrast as a function of luminance, the higher the luminance the lower the contrast. The measured (and in some cases modeled and predicted) contrast functions for several projectors can be found in the contrast chart and each point along the luminance curve gives a person a rough feel for the contrast with a specific amount of luminance. Rather than complicate this project by using pixels with grey luminance which in turn can deviate based on projector gamma, this project defines test patterns with pixels using only full white and full black. Varying degrees of luminance are achieved by using more white pixels organized into boxes of different sizes and numbers. Since contrast is always a relationship of white to black, it's useful to think of these test patterns as representing "static" or "simultaneous" contrast because like traditional ANSI test patterns, the contrast in these test patterns can be can be seen and measured at one time and even with a a dynamic iris projector the measured contrast is fixed (ie static)..

The Phelps/Petersen Contrast Test Patterns
These test patterns will soon be available for public download and they contain multiple test patterns that allow consistent measurements of contrast with varying degrees of full white luminance. These test patterns range in luminance from 0.1% to 100% (full white) and also include the equivalents of ANSI CR and Full On/Full Off patterns. These test patterns were designed for constant probe position so that all a person needs to do is set up instrumentation and then go through the test patterns one by one and record the white and black readings. Once the test patterns have been finalized a link or e-mail address ffor download will be provided here.

Mathematically modeled contrast
In addition to directly measuring contrast as a function of luminance, Erik Garci has created the contrast calculator which allows us to extrapolate contrast values from ANSI and On/OFF CR. In some of the curves below are several diagrams comparing theoretical performance of projector types as well as a comparison between actual and predicted values. Here is a link to the contrast calculator for those who would like to use it.
http://home1.gte.net/res18h39/contrast.htm

AVS Member Supported Contrast Dabase
In this link (AVS Member Supported Contrast Database) an Excel spreadhseet is available for download which contains the data for current measurements for several interesting projectors. As the project goes forward, we hope that others will use the test patterns and measure their projector performance and add their results to this database. White level, black level and derived contrast for luminance ranging from 0.1-50% can be found for various projectors in this file.

A word on Intra-Scene Contrast
Ultimately intra-scene contrast (the contrast between different areas in a specific scene) is what is most important to people. Unfortunately the projector video gamma, spatial relationships and pixel luminance histogram complicate an attempt at equating test pattern derived contrast with real world video content. The test pattern contrast data is still very useful in providing a consistent benchmark for gauging technology differences between projectors but we are also working on relating this data to a few specific scenes to help people gauge how scenes with varying degrees of luminance affect contrast. To this end, Erik Garci has provided an intra-scene contrast calculator (http://home1.gte.net/res18h39/intrascene.htm) and we hope to integrate these few select scenes into this contrast project at a later date.

Contrast at 0 luminance
As mentioned earlier, these test patterns with varying degrees of luminance (% APL) allow contrast to be measured directly in a similar fashion as the traditional ANSI benchmark and we call this static contrast because both white and black points can be measured at the same time and the values don't change over time (ignoring bulb drift, etc). Attempting to directly measure static contrast at 0 luminance is of course impossible because there is no white content to relate to black. In the past, sequential full on/full off contrast was a useful equivalent for contrast at 0 luminance, however this breaks down for dynamic iris projectors because the iris can never be open (full white) and closed (full black) at the same time.

Contrast at 0% APL - a new definition
Because the full on (iris open)/ full off (iris closed) contrast is never achievable at the same time with a DI projector, it's useful to extrapolate what the contrast would be at 0 luminance by creating a new definition that we're calling contrast at 0%APL. This is defined as the measured white level at the smallest available luminance / measured full black. As can be seen from the charts below, this definition accurately portrays the contrast as one approaches 0 luminance for both dynamic iris and non DI equipped projectors.


The chart below graphically shows the problem trying to plot both DI and non-DI data when attempting to extrapolate contrast at 0 luminance using traditional sequential contrast




The chart below graphically shows how using the new contrast at 0% APL definition leads to accurate values for contrast at 0 luminance for both DI and non-DI projectors alike.




Plotted Contrast vs luminance


The chart below shows a sample of contrast vs luminance (using 0% APL definition) for various projectors as well as a sample of DI settings.



Relative White level and Relative Black Level vs Luminance
Because contrast is a relationship between white and black, it's useful to examine white level vs luminance and black level vs luminance independently to see how simultaneous deviations in both can affect overall contrast. Unfortunately, due to setup and measurement differences, absolute comparisons of white and black levels between projectors cannot be made. These graphs do the next best thing which is to normalize the black and white levels so that the change in black level or the change in white levels from 50% APL (ANSI) can be compared. Please don't assume that projector A or B has better black levels or whiter whites from these charts as this is not what this data is displaying. Instead it shows how the deviation in white or black level from 50% APL as a function of luminance is changing. As an example a person can see from this chart that the black level at full off has decreased from 50% APL by roughly similar amounts between the HD1 and the VW50 Iris 1. This graphically shows the benefit of enhanced black levels with this iris mode. The Relative White Level chart also graphically shows the drawback that this mode creates with white levels. Interestingly enough, the overll contrast curve shows how these two relationships work together to boost overall contrast over the non-DI mode.


The chart below shows relative white level deviation vs luminance for a few selected projectors (DI and non-DI)



The chart below shows the relative black level deviation vs luminance for a few selected projectors (DI and non-DI)



As a side note, it's also possible to very roughly extrapolate absolute black levels between the HD1 and VW50 Iris 1 mode from this relative black level chart because both have similar 50% APL (ANSI) performance. In other words assuming both projectors had equal brightnesses at full field white (full on), the black level at 50% APL will be roughly similar because their 50% APL contrast performance are similar. So, if they both have similar black levels at 50% APL then this black level performance chart shows us that the deviation from 50% APL (where they are both equivalent) will be fairly close at 0% APL and nearly 4x better than the VW50 with Iris off. I should point out that this exercise is not the purpose of this project or benchmark, but it is an interesting exercise nonetheless.

Final thoughts
This project is unique in one respect in that it's a public work in progess and is therefore subject to change and revision at any time. For example, no attempt has been made to relate this contrast data with intra-scene contrast or to gauge other important aspects of projector contrast such as DI behavior (outside of full open and fully closed apertures). But this doesn't mean that work along these lines won't be done at a later date.

To see the full discussion on this project and to provide comments, criticism and to add data to this project, please use this thread:

http://www.avsforum.com/avs-vb/showthread.php?t=761806

Contrast measured at the Screen: (TBD)

Okay folks, here is some interesting data! Let's hear your take on what it all means!

A couple of points of clarification first:
1) The 0% APL description is defined as Reference white / Full Black. For a non-DI equipped projector it's close to Full White / Full Black (sequential contrast). But for a DI equipped projector Full White and Reference White aren't the same and we found that defining a new metric that we're calling "0% APL" is useful at describing what happens in dark scenes that approach full black.

On/Off Contrast is still a useful metric because it describes the full range of brightness to darkness, but for a DI eqiuipped projector the intra-scene contrast is better defined using the 0% APL metric.

2) 50% APL is actually not a test pattern but ANSI contrast. To be precise what's been used so far is Greg Rogers Modified ANSI CR method.

3) Preliminary Data for the VW50 at Iris Off is shown which doesn't include 15% and 20% APL test patterns. This will be corrected once Wm gets a chance to post more exact numbers.

4) The HD2K modeled values assume an ANSI CR of 100 which is probably fairly accurate, but Wm will hopefully be able to provide a final number after CES.

Hi lovingdvd, I'm still incorporating your numbers and will revise the graph shortly. The "Model" terminology means using Erik Garci's contrast calculator. So we've graphed actual HD1 data versus the "Modeled HD1" which by the way takes as inputs on/off and ANSI CR values which were measured.

For now it's nice to have the modeled and measured data displayed simultaneously because one of the goals of this project was to validate the model (which appears to be the case). Down the road as we start collecting lots of data we may have to split it or perhaps just display graphed data on the projectors that are most interesting to people. We can figure that out later.

Also the "20K" means the Sharp 20K. Since this is modeled results it's true for any projector that has those values of on/off and ANSI CR (6000:1 on/off and 800:1 ANSI).

So what are we learning from this data?

1) ANSI CR is an important measure of intra-scene contrast. Even a projector such as an RS1 with huge on/off CR has lower intra-scene contrast in the brighter scenes compared to a projector with very high ANSI CR.
2) On/Off CR is an important measure of black level but it dominates only the dark scenes.
3) Something very curious is happening between the VW50 Iris 1 and Iris Off modes. This is preliminary data, but it indicates that Iris 1 has better black level and better contrast in the dark scenes but the situation is reversed higher up the scale where Iris Off has significantly higher intra-scene contrast in the brighter scenes!

This is an important point. This data shows that projector contrast can be easily meaasured and it is predictable. Every scene from every movie and video will fall somewhere along the contrast curve. Where it gets dicey is determining where a specific scene with a specific APL falls along the curve. Movie APL doesn't seem to translate directly to the APL used in these test patterns so I'm anxious to see some examples and thankful for the work that Erik is going to put into that.

Also to answer Thunder's point. I wouldn't be surprised that once the dust settles with regard to movie APL vs test pattern APL that most dark scenes fall well below the 5% APL test pattern. So the RS1 vs high ANSI DLP contrast debate could be focused on the relative merits of contrast performance in these specific scenes.

There is also a third component to this discussion which lovingdvd has provided data for but I haven't posted yet, which is that measured screen contrast will also play a huge role. In a given room, the the washout effect will act to compress the contrast differences between projectors and this is likely more pronounced at higher APL. So for real world viewing high On/Off CR may yet still play a huge role.

EDIT: I can see Erik and I are already thinking along the same lines. It might be the case that we need to add a few more low APL test patterns to sample the 0%-5% region a little better.

I do have a version that shows two CRT graphs, one with a typical calibration of 10,000:1 on/off and 130 ANSI CR and another "Super CRT" with 100,000:1 on/off and 130 ANSI CR. If there is interest I can post those. Is 130 ANSI CR typical for a CRT?


I missed greg's posts and will go back and read them. As I posted earlier the contrast performance of a projector should be readily measurable and predictable. In this era of high ANSI (but average on/off) and high On/off (but average ANSI) contrast projectors and manual Irises, dynamic irises, etc we need to get a better feel for the relative projector performance between the two contrast extremes of on/off and ANSI. So this project is useful for that reason alone. What is of most interest to people is determining where on the contrast curve a specific scene falls and this is an area where we should get a better feel for soon.


All of the data coming so far is from Wm (HD2K, VW50 and HD1) and we have Sharp 10K data from Lovingdvd that I'm in the process of posting. So the data so far is using the same setup, equipment, etc.

When we start to add data from others our analysis will have to change accordingly. In other words we will need to look for trends, take into account equipment and setup, etc. If we are lucky enough to get say 5 different members to post results for projector X, we can look for deviations between members and apply statistical guides such as removing outlying data (or ask members to remeasure). I do agree with you though that this sort of data collection and interpretation is fundamentally different than data from a single consistent source. I do think that it's still meaningful and that it can be interpreted though, just that we have to be careful in doing so.