Better to have 2.2 gamma or stable 2.3 gamma on a plasma? - Page 4

Here is a summary of some additional measurements I've made. An example of each pattern type (22 sets of 10%-100% stimulus sequences except full-fields[23]) are in the attached pngs. It took about an hour to go through all the patterns with my D3. The display and meter were allowed to warm-up for an hour before measurements and repeat measurements were made to assess probe uncertainty. Note that absolute accuracy is irrelevant for these measurements, we only care about stability and sensitivity from start to finish.

For comparison, all of the measurements are referenced to an average of patterns 3,7,12,17. Chromaticity shifts are calculated using the dE(u*v*) formula with L*=100. Gamma shifts are calculated and averaged over the range 20%-40% as that is where the largest shifts are observed and this range is also important for perceived contrast.

I've termed the reference xyY values the Operating Range average (ORavg). These are the standard 10% windows with black surround, 10% APL, 22% APL, and 35% APL. I chose these to define some sort of baseline against which all the patterns can be compared and outliers can be eliminated using a "consensus" approach. Hopefully the consensus will be consistent with previous assumptions about calibrating with patterns that simulate actual usage.

The first plot I made looks at the power load characteristics using three different measurements.



The blue curve is the %Luminance drop of the peak white pattern in sets 1-21. You can see that there is no indication of power loading in these patterns to within a couple of percent up to the maximum APL that was used (35%). The green curve is the power loading as measured using a variable area peak white pattern [set #22] and shows that this pattern starts loading down the display above 25% APL stimulus. The red curve was derived from full field measurements and referenced to the luminance of the ORavg. This curve shows us two things, first there is a luminance depression throughout the 10%-50% stimulus region. This is the effect we've seen plotted in other ways previously and leads to the gamma shifts. We also see that it recovers and does not show the typical ABL loading until average APL is greater than 50%. Why the difference between the red and the green curves? This is because in terms of power draw it's not the average stimulus that we need to look at but the average luminance.

Here is the same data plotted against average luminance.



This doesn't change any conclusions about the stability of the power circuit over the range these patterns were run (the blue curve is still well within the overall stable region) but it does point out that a constant APL stimulus does not imply a constant power draw on plasmas. If the area of the window used to construct the pattern is held constant, the average luminance over a 10 step pattern set will change even if the average stimulus is fixed.


So the first thing I looked at was the chromaticity shifts relative to the ORavg. The error bars in the figure were calculated as the +/- 1-sigma deviation of dE(u*v*) from an average of 6 repeated measurements of pattern set #1. The six measurements included one done at the beginning of the run and one done at the end.



The JND of this dE formula is 3 and you can see that there is only one pattern set that is at that level (barely) and also statistically significant. These were the full field patterns[#23]. This agrees with Tom's previous findings although the dE values on this display are somewhat smaller. This pattern set is the only one I can eliminate from consideration based on chromaticity shifts.

edit: Thanks to Tom I have fixed an error in my dE calculation which show more outliers in the data set. The peaks in this plot [1,10,15,19] are all 1% windowed patterns on fixed backgrounds and along with the full field patterns[23] show the largest shifts relative to ORavg. It's interesting to note that 1% patterns on multilevel backgrounds do not show this behavior.

The error bars for measuring gamma shifts were calculated from the same repeated measurements as above. In the plot below, positive gamma is low relative to the ORavg while negative is high[gamma shift=ORavg gamma-measured gamma]. The pattern numbers are generally arranged from lowest APL to highest except for the following:

1. Pattern #1 and #19 is my control pattern [1% area/25% APL]
2. Patterns #20,#21 are Chronoptimist's patterns which got added in late.

In this plot you see the separation as shown earlier in other plots between the low APL/low gamma response and higher APL/higher gamma response. The maximum shift is 0.17 between 1% on black background and full field patterns. You can also see higher variability among the black background [2-8] vs. fixed APL patterns. So given this plot I would choose the more stable patterns which are shifted negative relative to the ORavg in order to weight the calibration toward typical APL averages. Of those patterns the tightest grouping are the ones which use 10% area windows. I found that the 1% windows yielded lower gammas [see patterns #9,#15] indicating that regardless of the APL very small window sizes show a luminance boost. In addition I think you would find that 1% constant luminance areas are in the minority in real video content compared to larger areas. Anyone have isoluminance maps of average video levels?



Other considerations:

The data indicates no differences outside of the error bars between a random, fixed, or even the still image background I chose at random, to form the fixed APL level of the pattern for gamma calibration. Some dependence on multilevel vs. fixed backgrounds was seen in the chromaticity shifts of 1% windowed data. I would recommend a multilevel background in keeping with the "calibrate as you use" philosophy when testing displays with unknown linearity characteristics.

Given this data set, for plasma calibration I would opt for 10% by area windows on a multilevel fixed background designed to yield 22% constant APL.

I would also recommend this pattern set for any display technology in the sense that if you obtain different settings using this set compared to standard windows, these use a more theoretically defensible metrology approach. As everyone knows in most cases such differences will be very small. This data is good example of both arguments. The chromaticity shifts in the grey scale do not warrant switching from standard to APL on this display but the gamma shifts do.

patpg1.png 43k .png file
patpg2.png 93k .png file
patpg3.png 137k .png file
patpg4.png 20k .png file


Raw data used in calculations.
gammatest_raw.xlsx 16k .xlsx file
Edited by zoyd - 6/24/12 at 1:42pm

Zoyd:

I have got to say until I looked at the raw data file I didn't have a clue as to what you were doing in this latest post. You are comparing so many different types of patterns with so many independent variables, it is really hard to follow all of this. However, when I looked at your raw data it became is little clearer.

My only interest, is determining whether fixed APL normally-sized window patterns are a superior choice for plasma calibration compared to the standard windows with a fixed black background. The data I collected and now the data you collected seems to suggest so. I have some questions about what you write above. You write that:


I guess I don't see the value of collecting data that shows that a particular variable makes no difference, but then recommending one of those variables over another.

You also write that:


However, your data (and mine) just do not support this. From what I have seen LCDs and front projectors simply do not behave the same way as plasmas, and I see no reason to adopt any special test patterns for them. However, I'll keep an open mind on this and I am certainly eligible for persuasion.

Finally, you write that:


Since you looked at so many test pattern variations, I'll take your word for this, but if you look at the limited universe of:
1) Standard windows
2) 22% APL windows (I looked at 25%, but that difference is negligible)
3) Full field patterns

both the chromaticity values and the luminance values in the critical range of 20-40% of full fields is much closer to the 22% APL patterns that both you and I now seem to endorse, at least for plasmas, than it is to the standard window patterns, right?

Whether or not other display technologies behave like plasmas is not the point, my point is that if you see a measured perceivable difference between patterns that simulate actual content vs. those that don't I would choose the former. In other words always verify your assumptions about linearity when calibrating anything. I hope that multilevel backgrounds are not required but I can't assume that other than on the display brand I measured. Sorry for being a theoretical stickler but that's just my nature. So my recommendation is really to check that APL patterns don't give you a different answer, if they do then you have to revisit your assumptions.

Definitely yes for luminance I saw that. For chromaticity, the shifts I calculated were smaller than yours and I could not really differentiate between 1) and 2), my 35% APL pattern chromaticities were slightly closer to the full-field but not statistically significant. Are the values you posted above from my data set? Did you include L* in your dE calculations, that would drive those values higher and create more separation by APL. I used the following:

dE(u*v*) =sqrt( (u1*-u2*)^2+(v1*-v2*)^2)
u*=13L*(u1'-u2')
v*=13L*(v1'-v2')
with L*=100

I wanted to separate chromaticity shifts from luminance shifts to see if there were two independent effects at work but that doesn't appear to be the case.
Edited by zoyd - 6/24/12 at 1:46pm

Yes, I used your data.

Again, we are dealing with multiple independent variables here. If the goal is simply to simulate real-world viewing material, the the LAST thing you would want is a set of test patterns with a fixed APL. APL varies constantly during real-world program material, often dramatically. That's one issue. A completely separate issue concerns whether, if you have decided to use fixed APL patterns, should one use a fixed or random surround. Your own data shows that there is no meaningful difference between them. Finally, of course, one should always use the best methods that yields the best results, but I have seen zero evidence, either here or elsewhere, that this entire subject of customized window patterns is even relevant for anything other than plasmas. I never use luminance data when calculating the dE of white. I used a*b* instead of u*v*. However, with u*v* the dEs are even larger.



You can see this in the xy differences. A standard window deviates as much as 0.009 from a full field. That's a pretty big difference. The largest deviation between the full field and the APL pattern is 0.004.

I agree. If one were to take my point to an extreme you would calibrate with patterns embedded in some agreed upon standard test clip sequence as well as have the probe average the readings over some agreed upon number of frames. That's neither practical nor needed to obtain a good calibration but it is theoretically correct.

Regarding fixed vs. random, yes for this case it's fine but if something new comes along I would measure the difference again to verify that assumption is still applicable. I've rephrased my recommendations to make this clearer.

Are you saying that it's been verified elsewhere that APL patterns yield the same results as standard windows on all other display technologies? I would be surprised if there were a difference but I'd still test it if I had one.

I'm glad you did that calculation - I found an error in my spreadsheet and yes the dE separation between full field and APL patterns is smaller than the separation between full field and standard windows. There is an average offset in y of 0.005 between standard windows and all of the APL patterns while the full field is only 0.001 higher than that average (0.327). I've updated the previous post with the fixed chromaticity measurements.
Edited by zoyd - 6/24/12 at 10:05am

I wonder if we should do similar tests for color calibration as well. I suspect that at 75% or 100% luminance there will be small enough differences to be able to neglect pattern choice. And only two of the Munsell colors are below 40% stimulus.

Sort of, but it is more fundamental than this. For display types that do not have the current limitations that plasmas suffer from (LCDs and digital projection) there's no need to use windows at all. I use full fields. In fact, SMPTE specifically recommends using full fields for projector measurements.

Just to confirm, here are measurements from a Samsung LED using APL windows, standard windows, and full fields. It shows none of the chromaticity or gamma shifts we saw with plasmas.

If you get a chance, try measuring the Sharp Elite as you probably will find some issues with that display.

@D-Nice: What do you think causes the shifts on plasmas? I'm not convinced it's power circuit related, maybe something in the dithering algorithm does this? The chromaticity shifts appear mostly in the green direction.

I have found some pretty serious issues with this display already having to do with accurate color reproduction throughout the color space. However, at the time I wasn't looking at this issue. If I run across another one I can check it for this. You can check this as well. You probably see a wider diversity of displays than I do. Just looking at the difference, if any, between fields and standard windows would tell us a lot.

To make it completely obvious for everyone, I'll point out that typical window patterns involve two changing variables. When you switch between typical windows (constant pattern area) you are changing both the video level and the average picture level (APL) displayed. Various displays simply do not necessarily react similarly as APL changes. Some displays tend to vary light output almost entirely based on the video level displayed (ex. fixed backlight LCD), while other displays can vary light output based on both video level and APL (ex. plasma and CRT). The displays that vary light output due to changes in APL don't necessarily perform similarly in how they react to changes in APL, so CRT and plasma may not vary in exactly the same manner, and two plasma likewise can react a bit differently to the same change in APL.

Each video frame has a fixed APL, so the image you view on the screen basically has a fixed APL. As video plays the APL can change, but the on-screen image tends toward a fixed APL. The point of using a fixed APL pattern is to try to measure various video levels from the currently-displayed image. Instead of having both a changing APL and a changing video level, if you use a fixed APL pattern then only the video level changes during measurements. Sure it might make sense to take multiple measurement runs at different APLs to see how a display performs as APL changes to better simulate typical video material, but complexity sort of goes against what people are asking when they bring up questions like the ones that started this thread.
Edited by alluringreality - 6/28/12 at 7:24pm

I don't really understand what you are trying to say here. The fact that a single frame of video--30 (or 24) of which are shown every second during playback--has a fixed APL is certainly true. So too, a photographic image has a fixed APL. Unless we watch our displays with the pause button activated I don't see the relevance of this. Video is a dynamic medium. The image changes constantly in a variety of ways, APL included. I only mentioned this because of Zoyd's stated interest in creating test patterns that best mimic real-world video material. Fixed APL is not that.

However, on the other side of the coin, I now agree that plasma displays--at least during a narrow range of APL--seem to perform in a way that is not mimicked by traditional window patterns and that fixed APL patterns probably do a better job at this. Having said that, the data we have looked at strongly suggests that the APL patterns remain in the 20-25 APL range (14-27 level surrounds), which is both where the deviations in gamma and chromaticity appear and, coincidentally, where a large proportion of real-world video resides as well.

A constant APL measurement merely provides a measurement where only the video level changes. On the other hand a typical grayscale measurement run with windows consists of increasing video levels and increasing APL. When you are dealing with displays that can potentially adjust light output based only on APL, what are you really measuring when you measure a series of windows? My point is that with a fixed APL you are measuring something similar to a single frame of video, and if you wanted you could always measure more information (different individual frames) to get a more complete picture of how a particular display operates. During video such as a movie, on-screen video levels change and the overall average light output changes as the video plays, but does the image ever actually react similarly to window measurements? I would suggest that trying to compare video levels by using varying APL window patterns on displays that may change light output based on APL is a questionable practice, and at least a constant APL pattern makes it possible to directly compare a particular measurement across various displays, since the primary variable with a constant APL series is a changing video level.

Static calibration patterns are rather common. For example the dim DVE pluge or a typical color bar pattern is sort of like watching video on pause. If a grayscale bar pattern can be relevant, then I'm not sure why measurements would require windows with changing APL.
Edited by alluringreality - 6/26/12 at 4:21pm

So I threw these patterns together for you to test another variable that I don't think has been accounted for yet: http://www.filedropper.com/22aplsurround

They're all 10%stim patterns with a variable surround, at a constant 22% APL. (as that seems to be the number you decided was best)

In a perfect world, all these patterns should measure identically, but I'm curious to see how the brightness of the surround affects the readings, if at all. (going in 5% increments may be too much though, perhaps 10% would be a better choice) As I have said previously, I suspect that there might be some sort of contamination in low APL patterns when using brighter surrounds.

well at least we found something that is stable. Y readings for the 17 patterns were (last in list a repeat of the first):

These yielded an average gamma of 2.172 +/- 0.002

There also was no shift in chromaticity within the uncertainties of the probe for this stimulus value. x,y locations were within +/- 0.002 for all patterns.
Edited by zoyd - 6/27/12 at 4:25am

This is not just about window patterns. It is full fields as well, which constant APL patterns would rule out by definition.

The whole point of a grayscale run is to measure dynamic performance under a variety of conditions using a series of different test patterns. If this were not necessary, we'd just use a single gray test pattern and leave it at that. A Pluge pattern or a SMPTE color bar measures a static phenomenon (MLL and hue/chroma levels). The "questionable practice" you refer to has been in place since the advent of video itself and was developed by professionals working with international organizations devoted to laying out video standards for everyone. I wouldn't be so quick to dismiss it.

As I have said previously, I now think that there is value to these patterns for plasmas over a narrow range of operation, but let's not oversell them.

This is consistent with our findings that the problems seem to arise in the 20-40% stim range. 10% stim seems stable whether you use regular or fixed APL patterns.

I tried these constant average video level patterns on my SXRD with the auto iris turned on. Looking at the screen it's clear the iris is adjusting, and I get over 20% variation in Y between the lowest and highest Y measurements. I don't have a CRT to test, but based on this test I would guess CRT and other LCD-based displays with dynamic lighting may show brightness variation when using constant average video level patterns.

1.318171
1.345737
1.34998
1.315088
1.292322
1.402075
1.511101
1.538085
1.540841
1.549552
1.552612
1.557075
1.564798
1.569328
1.567161
1.56835
1.573973

Well if we're in agreement that the brightness of the surround isn't affecting low %stim readings, and that 22% APL patterns are best, I would like to propose using these patterns: http://www.filedropper.com/22apl
Size is kept constant from 10–30%stim (the only way to lower the surround brightness would be to make the 10% and 20% patterns smaller than 30%)

EDIT: Crap, looks like this was posted when I was typing that up. This is exactly why I personally am not a fan of keeping APL constant through the use of a bright surround.

With an SXRD display though, shouldn't you be using full field patterns? I don't believe they're current-limited at all, similar to LCDs and projectors.
It would be good to have a full field 10%stim measurement to see how the other patterns compare.
Edited by Chronoptimist - 6/27/12 at 4:06pm

The way in which gamma is typically discussed generally follows the lines of this quote from Charles Poynton (http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html), which relates video levels with luminance:

"If luminance is to be coded into a small number of steps, say 256, then in order for the most effective perceptual use to be made of the available codes, the codes must be assigned to luminance levels according to the properties of perception."

If you have a source that indicates the intent of people taking a typical grayscale run is to measure something else, such as both a change in video levels and a change in average video level, I would certainly love to read it. Personally I suspect that the initial intent of measuring windows was basically to compare how video levels relate to luminance.

The practice of measuring windows was also implemented at a time when CRT was basically the de-facto standard, but now there are various technology displays, often with numerous user adjustments. It seems clear enough to me that various displays can react differently to identical changes in average video level. So far I have not ran across a discussion of how to account for displays that vary differently to identical changes in average video level, so as far as I can tell the most clear-cut way of comparing various displays is to use constant APL pattern.
Edited by alluringreality - 6/27/12 at 6:02pm

I think generally when people refer to Average Picture Level they are typically talking about an average of the light at the display, or luminance in the terms Charles Poynton seems to use. At least that was my general intent, to have APL refer to an average of the light from the display. I only averaged two of your images, but it looked like your intent was to maintain a constant average for the video levels, which I think Poynton would refer to as luma. I'll assume you're aware, that due to the expected gamma adjustment between video levels and light output, an average of the light at the display is almost never identical to an averaging of digital video levels. The only reason I mention this is that when I was discussing this sort of topic with Zoyd we didn't seem to always be using identical terms, and personally I wouldn't say your images maintain a constant light output at the display. From the discussions with Zoyd I think I get your intent, but if it works out for plasma the same patterns may be useless for other display types.

Like I was saying in the last reply to Tom, what exactly people are trying to do with gamma measurements is not entirely clear to me. If you measure fields on my SXRD I believe it still varies light output as average video level changes. If that's the case, then measurements with the iris enabled wouldn't correlate with how the display measures with the iris fixed. Individual on-screen images are roughtly similar regardless if the display is set to the adjusting or fixed iris, but without checking I believe field measurements are rather different. My opinion is that measurements on my display with the adjusting iris generally do not seem to correlate with how other display types measure, so personally I thought it was rather silly when people were using video processors to make pretty "calibration" graphs from window measurements on their SXRDs with the auto iris on. I'm not sure how exactly fields measure with the adjusting iris on my SXRD, but I know using windows that what they were doing would have significantly altered the on-screen image.

EDIT:
I went back to check how the display measures with fields with the iris on. Some of my comments were incorrect. Using fields does generally reflect how the display measures with the fixed iris, but it doesn't reflect how the setting can change the on-screen image. Between using windows and fields with the auto iris the clear choice is to use fields, but using fields leaves out some on-screen changes that occur on my display that depends on screen brightness. Basically as the screen darkens the auto iris setting on my display tends to change on-screen gamma.
Edited by alluringreality - 6/28/12 at 7:24pm

I am not sure what you mean to say here. First, yes, of course, one of the purposes of taking a grayscale run is to measure gamma, which measures changes in output luminance as the input signal changes. This is just an example of the dynamic aspect of video performance that I referred to. Second, of course there is another reason that I hardly have to cite a source for, which to measure changes in chromatcity at different video levels. In fact, I suspect that is why we call this the GRAY scale rather than the video-level scale.
I have now written this several times, but I'll try again. Plasmas and CRTs are similar in that their current capabilities are limited. This means that you must use window test patterns of some sort because you simply cannot get adequate output in a full field above about 60% stim. For every other display technology that I am aware of there is no requirement to use window test patterns; a fortiori there is no reason to use APL window test patterns.

This entire discussion is only about plasmas (and CRTs). With displays--such as auto-iris SXRDs--that vary output based on the strength of the incoming signal, you almost have to calibrate with full field test patterns because otherwise the iris won't fully open. Also, there is no ONE way all auto-iris displays operate. They each have their own proprietary algorithms, some more sophisticated than others, for how the iris affects light output as the input signal changes. This is a feature, not a flaw.

Gamma calibration and white point calibration are the same process - to calibrate the white point you change the EOTF of each primary to obtain the desired chromiticity. Ideally the EOTF of each primary is a function only of the video level of that primary, Lr=f(vr) where vr=input code for red for example. In reality Lr, Lg, Lb depend on other variables, average video level, average luminance level, iris aperture and so on due to both engineering limitations (plasma ABL) and purposely implemented manipulation (dynamic iris). So think of the variation with these additional variables as a multi-dimensional operating space of the device. When calibrating a function that is supposed to depend on only one variable but doesn't, you have a choice of whether to calibrate for some sort of average performance, which is essentially integrating over the additional variables, or try and hold these other variables constant and calibrate at one point in this multidimensional space. The question becomes which method yields the lowest perceptual errors the most often over the widest range of real video content. We've answered that question for a narrow case of standard windows vs. fixed APL windows on plasmas, other cases would have to be similarly tested and evaluated. And I agree with Tom that this particular case is just outside of what would be considered ignorable for practical purposes, so while theoretically the more correct method from a metrology point of view, it will not dramatically change your calibration settings.

Yes, RGB gamma is the same thing as white balance, though typically we deal with gamma separately as a unified luminance response irrespective of the relative weight of the RGB components.

If you obtain the same gamma response for the RGB components, then white balance has been done. However, the reverse is not true. You could have a perfect RGB balance but a sub-optimum gamma response.

To Tom, zoyd, alluringreality, chronoptimist, D-Nice and the other specialists:
The posts on this thread stopped 3 weeks ago. I was following this thread with interest, but as a non-specialist I had sometimes difficulties understanding it and drawing conclusions.
Is it possible for one of you to describe in simple terms what the common conclusion is?

Your discussions also made me think about the following: if average APL of normal movie content is 20-25%, would it not be better to measure gamut at lower luminance level than 75% or 100%?

Gamut 'should' be the same for all luminance levels (ok, not for 'zero'), so yes, we check gamut at all luminance levels, using 21 steps from 0% to 100%
Any major gamut changes throughout the range shows a potential problem.


Steve
Light Illusion