AVS APL Study - Adjunct to AVS Contrast Project

Thanks for starting this thread. I think there are two issues that need to be addressed. The first is determining the correct methodology for calculating APL and the second is how best to try and perform a rough comparison of an actual film image to a checkerboard type test pattern based on the amount of luminance alone.

To me this second issue is really the rub. I think at best only the roughest of ranges might apply because precision more than that requires taking into account the other factors of contrast such as the geometry of the images and histogram distribution. It might be better then to just try and break it down to three ranges: low, middle and high APL and try to find a rough determination on what range of checkerboard patterns might apply (based on percent area) to these three ranges.

The discussion of relating APL to film images has come up in several other threads. Including this one:
http://www.avsforum.com/avs-vb/showthread.php?t=765552

It was from this thread that Eric Garci introduced the intrascene calculator:
http://home1.gte.net/res18h39/intrascene.htm

As was mentioned in the earlier threads. Checkerboard patterns are unique in that the average luminance is the same even after gamma coding has been applied. For an ANSI pattern for example it's 50% before and after gamma coding. At 50% I also speculate that its much too bright of a pattern to try and compare with what is happening in intra-image contrast with real scene information.

To see how gamma can affect the average luminance calculation we can take an image of a specific scene such as the starfield image in Eric's calculator. The average luma before gamma coding is 4.29% but after gamma coding it's only about 0.12%. So let's see where that particular scene would fall with the measured projector data below:



From this data we can see that 0.1% places it at the extreme end of the chart at close to the on/off limit of the projector. This seems to fit well with the general perception that on/off plays a huge role in dark scene performance and it would seem to suggest a rough match.

In fact if we take any of the images in the intrascene calculator with the exception of the nurse scene they all tend to be very dark and tend to fall in the region of projector performance that suggests the importance of on/off contrast.

The nurse scene is an interesting example. Most people would characterize it as a bright scene. It contains an average luma of 50% similar to the ANSI pattern but after gamma coding it only has an average luma of 21%. If we compare it to the projector data above it suggests that this region would favor ANSI performance. What's interesting is that even though this is a bright scene it still contains only about 1/2 of the luminance of the ANSI pattern and at this place along the curve the benefits of high ANSI is reduced. In other words a projector that has say a 3x advantage in ANSI contrast might only have a 2x or less advantage in intra-image contrast at this point along the curve. This might partially explain why people seem to have a hard time seeing differences in images based on ANSI CR alone. It's possible that the ANSI CR metric may be implying larger intra-image contrast benefits that don't exist with typical scenes, even scenes that most people consider to be bright.

EDIT: I should also point out that the AVS Contrast thread that is referenced in this thread can be found here:
http://www.avsforum.com/avs-vb/showthread.php?t=852467

Would you guys please explain just what you mean by "gamma coding?" In video, gamma is the non-linear relationship between input signal going into the display and the output signal you actually see on the screen.

So, for example, a 2.2 gamma has this relationship of input to output:

Input - Output
10% - 0.63%
20% - 2.90%
30% - 7.07%
40% - 13.3%
50% - 21.8%
60% - 32.5%
70% - 45.6%
80% - 61.2%
90% - 79.3%

When it comes to film material, I am having a hard time understanding what is the input and what is the output. Clearly, what you see on the screen or on a monitor is the output, but wehat was the input. Not signal voltage.

Also what does gamma have to do with determining APL? Hadn't we already decided that the correct methodology was (average luma -16)/219? Where does gamma come into this?

Ok I'm going to show my ignorance here. But, one of the problems with the ANSI checkerboard is that it uses full white for the checker board. Now, as I understand it using less than full white would be better except the projectors gamma gets in the way. So, why not instead of using gray pixels for the white square make it a mixture of alternating single pixels of full white and black. Since the sensors available average groups of pixels you would in essence be measuring a gray field on average and the projectors gamma would not play a part.

It'll be really interesting to see these results.


It just means the output after gamma is applied to the input. Since it's non-linear the avg. calculation has to be done with every point in the image after gamma is applied rather than take the average input and then apply gamma.


I originally thought that APL was the average image luminance after gamma coding but there is a lot of literature to suggest that it's just average luma before the display gamma is applied. For the purpose of this discussion though I don't think the definition of APL is important so long as we consistently apply the gamma coded forumula in discussions about contrast as I think that's whats really relevant.

For what it's worth Joe Kane Productions defines APL as:
"APL (Average Picture Level)
The video signal level, during the active picture part of each horizontal line, is mathematically averaged over the period of a frame to come up with APL. A 100 IRE Window pattern has a lower APL than a 100 IRE Flat Field pattern. Television program material is said to have a 15% average picture level over a long period of time. The Window patterns used for setting gray scale in all three titles have a 16% APL so that they will closely represent normal program material in their current drain on the high voltage power supply. The width of the Window was enlarged the extra 1% to accommodate the shape of the Philips PM5539 Color Analyzer which was on the market at the time of A Video Standard. That wide aspect ratio shape was passed on to Video Essentials. The computer world uses a much higher number for APL over time, that of 35%. That is reflected in the ANSI Checkerboard Patterns, VE T17 C27. "

I also found some literature where others have done this sort of APL study before. Here is a reference from IEEEXplore:

"Distribution of average picture levels in television programmes
Quinn, S.F.; Newman, P.M.
Electronics Letters
Volume 1, Issue 9, November 1965 Page(s):261 -
Digital Object Identifier 10.1049/el:19650235
Summary:An analysis of measurements made over 14½ hours of television programmes is presented, to show the distribution of average picture levels as a percentage of total time. It has been found that for 90% of the time the average picture level is between 20 and 70% and that the mean value of the distribution is close to 40%."

Here is another definition:
"average picture level (APL): In video systems, the average level of the picture signal during active scanning time integrated over a frame period; defined as a percentage of the range between blanking and reference white level."

And another:
APL - Average Picture Level
By: Bruce Bahlmann


The average signal level with respect to blanking during the active picture time. APL is expressed as a percentage of the difference between the blanking and reference white levels. APL represents the video signal level, during the active picture part of each horizontal line, is mathematically averaged over the period of a frame to come up with APL. A 100 Institute of Radio Engineers (IRE) Window pattern has a lower APL than a 100 IRE Flat Field pattern. Television program material is said to have a 15% average picture level over a long period of time. "

OK, but if the average luminance values that we capture on the disc is the input, then gamma coded APL output will be smaller (50% input gets you 21.7% output @ 2.2 gamma).

Why then does this white paper show the relationship in the opposite direction? Look at pages 10-12. He shows the gamma coded output APL as LARGER than the linear average luminance input.

http://www.etconsult.com/papers/Blac...tt%20Cowan.pdf

Which is it?

Mark:

What Kane reports
"Television program material is said to have a 15% average picture level over a long period of time."

and what the Quinn and Newman paper reports
"It has been found that for 90% of the time the average picture level is between 20 and 70% and that the mean value of the distribution is close to 40%."

cannot both be true.

I don't think that we are working with a clearly defined set of terms.

The data I collected showed a fairly consistent average luma in the low-to-mid 20s. How does that relate to these quotes regarding APL? There can't be a gamma difference, because the numbers are too similar.

This is an interesting methodology and something that I've thought about before. There are a lot of issues that need to be resolved in order to gleen something useful out of this though. The MTF of the display device will play a big role in any alternating on/off pixel patterns. There is also the issue of trying to ramp luminance by using a pattern like that. Does one keep the same pattern and use larger/smaller checkerboards or alter the underlying on/off pattern for example.

Fundamentally though I don't think that there is a problem using full white and full black checkerboxes. The challenge is trying to understand how to interpet the information that they convey to images based on APL alone and I think you'll have the same challenge with other patterns like these alternating on/off ones.

That is true for checkerboard patterns where the brightest rectangles are 100 IRE. For a checkerboard that is 50/0 in %stim or IRE the ft-lamberts or luminance (post gamma) would be about 20% of that 100/0 checkerboard.

Going along with what Mark said about MTF, if you mean a single pixel checkerboard then I think this would be a problem with LCOS projectors. At least the latest JVC ones. I don't know the specifics, but they have some trouble with single pixel checkerboards. I think it is that the white pixels end up being smaller than they should be, with the average luminance in that section being dimmer than it should be. Using alternating white and black lines in either direction may or may not cause problems as far as trying to hit 50% ft-lamberts or luminance. Calculating based on the current ANSI checkerboard may actually have less problems with some technology.

--Darin

That paper has tripped me up before too. Someone suggested that the use of the term gamma coding means camera gamma rather than display gamma. It has the note (see on waveform monitor) which implies that this notion is correct. So I think they are comparing film luminance to post camera gamma coded APL. Since camera gamma has an inverse shape from display gamma it may explain why the relationship seems reversed in the white paper from what we would expect. What we really want to do is compare film luminance to projector luminance after the display gamma has been applied.

I think people use the term APL to mean different things depending on what part of the display pipeline they are talking about. To be clear maybe what we should do is coin our own term and call it display APL.

CORRECTION: The average luma before gamma coding is 4.29% but after gamma coding the average luminance is only about 0.12%.

CORRECTION: It contains an average luma of 50% similar to the ANSI pattern but after gamma coding it only has an average luminance of 21%.

Note that average luma is calculated before the display's gamma is applied, and average luminance is calculated after the display's gamma is applied.

Yup, 100%stim checkerboards like the ANSI pattern are the ones that I meant although I should have clarified what I meant.

Hi Erik, yes I should have said average luminance to be more clear.

I think the term "display APL" might be an oxymoron. I typically use the terms "average luma" and "average luminance." Since luma and luminance sound similar, the terms "average signal level" and "average display luminance" might be better. Maybe ASL and ADL.

"ADL" and "ASL" are perfect

This may be true.

What I am trying to figure out is what the data on nearly 20,000 frames from 3 films means?

The white paper reports this image



as having an average luminosity of 95.5 and an APL of 37%. This relationship corresponds almost exactly to the relationship between the average luma and the normalized values that I reported for the 3 films I sampled.

Yet, I gather that you believe that there is yet some further calculation required to account for gamma coding. What would that be? Applying a 2.5 gamma to .37 (if that is to be understood as the % stim) results in 8.3. I can't believe that you mean to say that we should interpret the display APL (or whatever you choose to call it) of this relatively bright image as 8.3 APL.

I really don't see where this is going.

Technically, for those patterns, the average luminance may be slightly greater than the average luma, depending on the display's on/off CR, but the difference is usually less than 1% so it can be ignored.

In typical usage, the term APL refers to the average level of the signal before the display's gamma is applied. If you use the term APL to refer to something else, then you run the risk of confusion.

I think it will probably be good enough for most images.

OK, that explains why the image below is reported as it is.

Thus, I assume that this means that the methodology of deriving APL from a normalized average luma would be correct absent any futher calculation would be the correct approach.

I thought it an interesting exercise to try and relate some of the checkerboard data from the AVS contrast project to the scenes and calculations from Erik's calculator.

The comparison uses the measured values below that correspond to the same data set that is shown in the graph at the beginning of this thread. These numbers were then plugged into Erik's calculator.

RS-1 on/off - 9587
ANSI - 397
gamma 2.2 (not measured directly - this is a "guestimate")
20K on/off 5000
ANSI 560
gamma 2.2 (also not measured)

I picked two images from Erik's calculator where the calculated ADL falls close to where we have measured data. The two patterns are the star field (~0.2%) and Saruman(~2.5%). With these two patterns I looked up the intra-image contrast data at the ADL location on the curve (or close to it) and if the image doesn't contain 100% stim pixels I extrapolated what the contrast would be using a 2.2 gamma. Here are the results:

Star Field:
RS1 (extrapolated data) - 4984 RS1 (Erik's calc) - 5402
20K (extrapolated data) - 2877 20K (Erik's calc) - 3073

Saruman:
RS1 (extrapolated data) - 2850 RS1 (Erik's calc) - 3482
20K (extrapolated data) - 2879 20K (Erik's calc) - 3334

Both of these match much closer than what I would have expected.

It would be interesting to collect more images where the ADL falls close to where we have measurement data and see if this rough relationship holds for other images. If the relationship holds this could go along way in giving us a rough idea how the strengths and weaknesses of different projection technologies play a role in contrast with real images. I'm guessing that this will also validate the importance of high on/off as the ADL in most of the images will tend to be very low.

The image is APL 37%, applying a 2.5 degamma would yield a luminance of 8.3% ( of the all white, 100% value). If an all white image was 10 ftL, that image would be 0.83 ftL. That is the nonlinearity, and why high on off is important. In order to get a 100:1 CR between a 10% object and black in said image would require a 30,000:1 pj

I think I am finally starting to see what you are doing. So is the calculated ADL just the APL with gamma applied? If so, then yes the ADL will certainly be very low, rarely above 15.

If so, then the only patterns from the contrast project that will be relevant will be the 15% of full field white and below, right? I say this because it looks like APL above 50 is pretty rare and 50 post-gamma is somewhere between 13-17, depending on the gamma used.

Yes, ADL is calculated from the luma after display gamma is applied. It will be very low and maybe this is the value people are quoting when they say APL is a dark medium and over time it averages 15 or less. I don't think the definition of APL that others use is all that important so long as we are precise in using ADL as that's what's really relevant to this discussion of display contrast.


Not necessarily. the nurse scene has an ADL of ~25%. I would have done the same calculations for the nurse scene as the star field and saruman but the closest checkerboard pattern we have is at 20%. If I use the 20% values we'll have higher numbers than the 25% predicted by Erik's calculator and in fact that's what we see.

I think that with enough images we'll see that many of the images will fall in a region that is heavily influenced by on/off while most of the brightest images will fall well below the ANSI pt. The Saruman is a good example of an image that falls close to the crossover point between the RS-1 and 20K where the ANSI CR advantage of the 20K is mitigated by the on/off advantage of the RS-1.

I think this also shows pretty clearly why current on/off and ANSI measurements are poorly suited for trying to gauge differences in intra-image contrast. For example, if we plug in on/off values into Erik's calculator for the VW50 and the Star Field image, the result will yield intra-image contrast values that are far above the native contrast limit of the VW50. On the other side of the spectrum, the ANSI cr measurement suggests intra-image contrast differences that are larger than they actually will be with the vast majority of images.

My point was that scenes as bright as the nurse scene seem fairly rare. 49 APL is outside the 95% range for all of the films I have sampled so far.

The calculator does not take into account the effect of a dynamic iris, so the result would be more accurate if you plug in the native on/off CR.

Tom, I think we're on the same page. If you were to pick 3 patterns based on ADL to represent dark, average and bright ranges would you pick?


Yup that's my point. It's not that the calculator is inaccurate it's that on/off as a predictor of intra-image contrast doesn't work with a DI.

Rather than using native on/off (iris open), it would be even more accurate if a person plugged in measured values with the smallest pattern where measurements could still be reliably done because this would include the iris benefits when the iris is in the smallest aperture position. In fact it's pretty easy to extrapolate the native on/off (iris closed) from that data by taking the measured white level with the smallest checkerboard (iris in smallest aperture) and then dividing by the full off black level. In the AVS contrast project I've termed this extrapolated value "maximum intra-image contrast". In some cases this may be the only way to measure static on/off if iris off measurements cannot be performed (or in the case of DLP where the iris alone can make a huge impact to contrast).

Just to dramatize how erroneous the numbers can be let's run the numbers for for the VW50 and the Star Field scene:

Assumptions: ANSI - 263 (measured). Gamma 2.2 (guestimate)

a) Iris 1 on/off - 12550 (measured)
b) Iris off on/off - 3089 (measured)
c) Iris 1 on/off - 4000 (note: extrapolated from measured max intra-image contrast metric)

Contrast numbers from the calculator:
a) contrast - 6747
b) contrast - 1863
c) contrast - 2379

Actual VW50 Iris 1 measured contrast at .2% = 2280 (white level corrected to reflect 80%stim at max luma).

So you can see how much more accurate we get using the value derived from the max intra-image contrast metric.

Interesting data. If I were to pick three checkerboard (full white) test patterns to try and use to gleen intra-image contrast comparisons based on this data I would probably pick 30% as the upper limit, 5% for an average and the smallest pattern that could be reliably measured (0.1% or 0.2% for the lower limit). I would use these as a replacement for the ANSI and on/off metrics. I would also supplement it with a test pattern to show the full benefit of a DI (probably a very small area pattern with a low %stim grey like 20%). What do you guys think?