AVS Forum banner

Time to define an alternative to ANSI contrast?

37K views 340 replies 32 participants last post by  Mark Petersen 
#1 ·
Greg Rogers excellent review on the Sharp 20k has made me think about the perceptual benefits of such an amazingly high ANSI contrast (> 800:1). In particular it has made me wonder about the benefits of high projector ANSI when a typical HT environment washes out measured screen ANSI and keeps it below 200-300:1 max. I have little doubt that there are benefits to such performance but it made me realize that perhaps the traditional ANSI approach of large checkerboards has become a poor metric for trying to gauge the perceptual benefits of not just high projector ANSI, but also the performance of DI equipped projectors. The biggest drawback to traditional ANSI is that using such large white checkerboards equates more to high APL scenes which means that there is no current metric for gauging low APL performance which for many has become a crucial determiner of projector performance.


Rather than using the ANSI large checkerboard approach (for both screen and projector measurements), what if there were a new standardized form of something similar to ANSI but for a mix of APL scenes? A good low APL test in particular could be a very useful and meaningful metric for testing todays high performance projectors and it also might allow us to better capture the elusive "image depth" and "pop" descriptions that are so hard to define particularly for these low APL scenes.


For a low APL test pattern for example, we could define a pattern that is 95-99% 0 IRE and with only a few (or one!) small white block(s). With a test pattern like that I would think that the limited amount of reflected white light would work to reduce room effects and it would therefore provide a more meaningful measurement at the screen than traditional ANSI. Similarly, a test pattern like that would better help to test the performance of a projector in low APL scenes and in particular DI performance and brightness compression impacts. Granted that DI algorithms are complex and would likely interact with such a test pattern, but at least it's better than having no metric at all.


Many manufacturers don't quote ANSI specs, so it seems as though this lackluster industry support for ANSI contrast could provide an opening for us hobbyists to define something more meaningful. If enough AVS members think there is merit in this idea we could collectively work to define these new test patterns and at least put them to work in forum reviews that hopefully people like Bob and Jason might utilize. In fact if there is enough merit in this idea it would be very intriguing if we could get Greg Rogers to help define and adopt this sort of a test pattern and methodology in his reviews.


Rather than waiting for SMPTE or other standards bodies to get around to defining such a test, why don't we put the Science into AVS Forum and work together to define something? How cool would it be to get an AVS inspired and defined methodology into widespread circulation? I'm very curious to hear everyones thoughts on this idea...
 
See less See more
#2 ·
The ineffectiveness of using an ANSI 8 white and 8 black checkerboard test pattern to describe real world APL scenes is also covered in this white paper:
http://www.etconsult.com/papers/Blac...tt%20Cowan.pdf


I like the 5% luminance (26% APL grey) test pattern shown in the paper, but even that might be too bright.


Also, my thanks to HoustonHoyaFan for digging up this gem of a white paper.


Edit: Thanks Darin for pointing out that 8x8 is not the same as 8 and 8



Also sorry HHF for not remembering that it was you who found this paper.
 
#3 ·

Quote:
Originally Posted by Mark Petersen /forum/post/0


The ineffectiveness of using an 8x8 ANSI test pattern ...

Just a technicality, but it is 4x4 (for 8 rectangles of each).
Quote:
Originally Posted by Mark Petersen /forum/post/0


I like the 5% luminance (26% APL grey) test pattern shown in the paper, but even that might be too bright.

If somebody does test with a pattern like this I hope that they will compare the result with what they would get by using the current ANSI CR and on/off CR and plugging them into the contrast calculator that attempts to estimate some of these other from those two tests that are out towards the extremes to measure the raised black floor effect and the washout effect. That calculator can be found here:

http://home1.gte.net/res18h39/contrast.htm


It looks like that 5% APL pattern you were referring to has 5% of the screen covered by white rectangles. If I were to estimate the value for that image in that calculator I would change the percentage of the screen covered by the checkerboard to 10% (twice the 5% because the checkerboard contains equal amounts of white and black). And I would change the room gain to 0 to just consider the projector. As an example, for something with 800:1 ANSI CR and 6000:1 on/off CR and those other choices, that calculator would estimate the CR in that image at about 3600:1.


BTW: I like that you didn't suggest having parts of the image use anything other than video black and video reference white, since gamma can have a big affect on how bright those intermediate values really are (and then how close things are to where on/off CR dominates compared to ANSI CR dominating).


--Darin
 
#4 ·

Quote:
BTW: I like that you didn't suggest having parts of the image use anything other than video black and video reference white, since gamma can have a big affect on how bright those intermediate values really are (and then how close things are to where on/off CR dominates compared to ANSI CR dominating).

Maybe including more than just black and white in the image would be a good thing. My first thought (without pondering the question for more than a few seconds
) would be to use 3 different patterns:


standard 0%/100% checkerboard

0%/10% checkerboard

90%/100% checkerboard


and then somehow weight the performance of the three into a single relationship. We would learn several things:


1. The extreme case that standard ANSI presents.

2. Black level detail.

3. White level detail.

4. An indication as to whether or not gamma has been applied correctly.


Or maybe 3 patterns something like these:


standard 0%/100% checkerboard (8 squares each)

0%/10%/20%/30% checkerboard (4 squares each - each row offset by one)

70%/80%/90%/100% checkerboard (4 squares each - each row offset by one)


I dunno...It's early and my brain isn't into gear yet...
 
#7 ·
Darin said:
Quote:
BTW: I like that you didn't suggest having parts of the image use anything other than video black and video reference white, since gamma can have a big affect on how bright those intermediate values really are (and then how close things are to where on/off CR dominates compared to ANSI CR dominating).

This is really important. I have some checkerboards at other APLs(i.e. the lit portions are something other than 100%/reference white), but I would never recommend that be implemented as some kind of standard measurement precisely because gamma is an additional variable to consider if you're not just using 0% and 100%, and leaves room for all kinds of manipulation.


I do like the patterns suggested in the linked PDF for that reason in that they vary the APL of the pattern by reducing the area of the white squares rather than changing their intensity to something less than 100%.


It would be interesting to see how this agrees with the CR calculator, because it may or may not make the need to measure these lower-APL patterns moot.


It would also be interesting to measure something like a single small square or circle of white, but measure spatially away from it in order to capture the nature of the spill. Some displays spill light in interesting ways nearer or farther away from the lit portion (for instance defined ring halos in AC CRTs) which aren't captured by an ANSI or modified-ANSI type measure.


As interesting as doing these things could be, it is paramount to reiterate the overwhelming importance of the two CR measures we have now: on/off and ANSI(or ANSI-related). I don't think anyone here needs to hear that, but a lot of people have misinformed opinions and think that one or the other of these CR measures is 'meaningless' or the like. I would hate to see some other kind of ancillary CR measure come to displace on/off or ANSI CR. On/off CR and ANSI are fundamental. Other measures like those proposed here are merely supportive. Very important to keep that crystal clear lest these ideas float away and get twisted into something we do not intend.
 
#8 ·

Quote:
Originally Posted by Bob Sorel /forum/post/0


4. An indication as to whether or not gamma has been applied correctly.

First you would have to get people to agree on what it means to apply the gamma correctly. Even the experts don't seem to agree on that one. Some differences of opinion are fairly small (like where the gamma for CRTs falls between about 2.2 and 2.5) and then there are differences of opinion about applying a linear tail at the bottom or not (I don't believe it is supposed to be applied on the playback side).


--Darin
 
#9 ·
this is a good idea.


Before any attempt at encompassing various measurement factors lets start small. Start with an agreeable general consensus as to a test pattern that is currently not being well adressed. This test should also be scalable to add-on additional test standards as it becomes more generally accepted.


This way this idea can start before it gets bogged down with unresolved issues.
 
#10 ·

Quote:
Originally Posted by darinp2 /forum/post/0


If somebody does test with a pattern like this I hope that they will compare the result with what they would get by using the current ANSI CR and on/off CR and plugging them into the contrast calculator that attempts to estimate some of these other from those two tests that are out towards the extremes to measure the raised black floor effect and the washout effect. That calculator can be found here:

http://home1.gte.net/res18h39/contrast.htm

Darin, thanks for reminding me about the contrast calculator. I think both you and Chris are right in that we should see how well it correlates with actual measurements, although I have doubts that a mathematical model can accurately take into account all of the variables. As I recall there was a long thread that discussed this calculator can you find a link to it?

Quote:
Originally Posted by ChrisWiggles /forum/post/0


As interesting as doing these things could be, it is paramount to reiterate the overwhelming importance of the two CR measures we have now: on/off and ANSI(or ANSI-related). I don't think anyone here needs to hear that, but a lot of people have misinformed opinions and think that one or the other of these CR measures is 'meaningless' or the like. I would hate to see some other kind of ancillary CR measure come to displace on/off or ANSI CR. On/off CR and ANSI are fundamental. Other measures like those proposed here are merely supportive. Very important to keep that crystal clear lest these ideas float away and get twisted into something we do not intend.

I agree that on/off CR and projector ANSI are valuable figures of merit. It's really screen ANSI that I have the problem with. That and there also seems to be a good low-APL figure of merit that is completely missing.

Quote:
Originally Posted by Darinp2 /forum/post/0


First you would have to get people to agree on what it means to apply the gamma correctly. Even the experts don't seem to agree on that one. Some differences of opinion are fairly small (like where the gamma for CRTs falls between about 2.2 and 2.5) and then there are differences of opinion about applying a linear tail at the bottom or not (I don't believe it is supposed to be applied on the playback side).

I think if you ask experienced calibrators what is the best gamma they will say, "the one that looks the best" and it will likely differ depending on display technology. I think that using 0 IRE and 100 IRE is the best way to go because it takes gamma out of the equation although I could see the usefulness of a fine black detail test pattern (all 0 IRE with a few small low IRE boxes) and a fine white detail test pattern (all 100 IRE but with a few small high but not 100 IRE boxes). Rather than to define a multitude of test patterns though, I think we should pick something that is the most needed and then to work up from there.

Quote:
Originally Posted by Head Shot /forum/post/0


Before any attempt at encompassing various measurement factors lets start small. Start with an agreeable general consensus as to a test pattern that is currently not being well adressed. This test should also be scalable to add-on additional test standards as it becomes more generally accepted.

Yup I totally agree. I like Bob's idea of a weighted average of several test patterns and perhaps thats the way to go eventually but for now how about if we work out a really good low APL test pattern which when combined with projector ANSI and on/off CR can provide some insight into how a projector performs in the crucial low APL environment.
 
#11 ·
Mark - as a side note keep in mind that a higher ANSI CR pj will have higher ANSI CR at the screen even with room reflections factored in. For instance, I measured just 80:1 with my Ruby but 130:1 with my Sharp 10K - same room, same treatments. Later I found that I could raise my ANSI CR with the Sharp to 180:1 but that is another story. Bottom line is that to get the most ANSI CR one should take steps to control light reflections and use a higher ANSI CR pj if that is a top priority.
 
#12 ·

Quote:
Originally Posted by lovingdvd /forum/post/0


Mark - as a side note keep in mind that a higher ANSI CR pj will have higher ANSI CR at the screen even with room reflections factored in. For instance, I measured just 80:1 with my Ruby but 130:1 with my Sharp 10K - same room, same treatments. Later I found that I could raise my ANSI CR with the Sharp to 180:1 but that is another story. Bottom line is that to get the most ANSI CR one should take steps to control light reflections and use a higher ANSI CR pj if that is a top priority.

lovingdvd, what would be really interesting would be to plot screen ANSI vs projector ANSI and see if there is a rolloff in screen ANSI in projectors with higher ANSI values.
 
#13 ·

Quote:
Originally Posted by Mark Petersen /forum/post/0


lovingdvd, what would be really interesting would be to plot screen ANSI vs projector ANSI and see if there is a rolloff in screen ANSI in projectors with higher ANSI values.

While you guys are throwing around new contrast measurement ideas, how about some form of standardized screen AND room contrast measurement? I can see screen measurements being a real can of worms; large block ANSI, small block or modified ANSI and inter pixel contrast measurements. I would expect glass bead, translucent or pearlescent screen materials to have a significant impact on adjacent single pixel areas.
 
#14 ·

Quote:
Originally Posted by lovingdvd /forum/post/0


For instance, I measured just 80:1 with my Ruby but 130:1 with my Sharp 10K - same room, same treatments.

Are you sure those numbers are accurate?


According to one review , the Ruby's ANSI CR is 251:1 in a non-reflective room. To bring that down to 80:1, the room would need to have a reflectivity of 0.01718, theoretically, according to the contrast calculator . However, no projector can exceed 118:1 ANSI CR in such a room, not even a hypothetical projector that has infinite ANSI CR in a non-reflective room.
 
#15 ·

Quote:
Originally Posted by Erik Garci /forum/post/0


Are you sure those numbers are accurate?


According to one review , the Ruby's ANSI CR is 251:1 in a non-reflective room. To bring that down to 80:1, the room would need to have a reflectivity of 0.01718, theoretically, according to the contrast calculator . However, no projector can exceed 118:1 ANSI CR in such a room, not even a hypothetical projector that has infinite ANSI CR in a non-reflective room.

Yes I am quite sure about the numbers. However note that, for whatever reason, I only measured about 110:1 directly out of the pj lens. This may have to do with the fact that I was using service menu Iris tweaks to achieve 20,200:1 on/off, or perhaps the ANSI CR drops as the bulb ages (my bulb had 525 hours at the time of measurement).


Thanks for the link - very interesting calculator.
 
#16 ·

Quote:
Originally Posted by lovingdvd /forum/post/0


Yes I am quite sure about the numbers. However note that, for whatever reason, I only measured about 110:1 directly out of the pj lens. This may have to do with the fact that I was using service menu Iris tweaks to achieve 20,200:1 on/off, or perhaps the ANSI CR drops as the bulb ages (my bulb had 525 hours at the time of measurement).

Let's recalculate, assuming 110:1 for your tweaked Ruby in a non-reflective room. To bring that down to 80:1, the room would need to have a reflectivity of 0.00684. In order for the Sharp to have 130:1 in that room, it would need to have 234:1 in a non-reflective room. Isn't the Sharp's ANSI CR supposed to be higher than that?
 
#17 ·

Quote:
Originally Posted by Erik Garci /forum/post/0


Let's recalculate, assuming 110:1 for your tweaked Ruby in a non-reflective room. To bring that down to 80:1, the room would need to have a reflectivity of 0.00684. In order for the Sharp to have 130:1 in that room, it would need to have 234:1 in a non-reflective room. Isn't the Sharp's ANSI CR supposed to be higher than that?

I measured 279:1 out of the Sharp. Please note that as mentioned this is a Sharp 10K which is over 3 years old. I don't know what the Sharp 10K ANSI CR was measured at but it is very far behind what modern DLP pjs are delivering. Does this make more sense now?
 
#18 ·

Quote:
Originally Posted by lovingdvd /forum/post/0


I measured 279:1 out of the Sharp. Please note that as mentioned this is a Sharp 10K which is over 3 years old. I don't know what the Sharp 10K ANSI CR was measured at but it is very far behind what modern DLP pjs are delivering. Does this make more sense now?

It makes a lot more sense.


Given 80:1 for the Ruby, I calculate 143:1 for the Sharp. Or, given 130:1 for the Sharp, I calculate 76:1 for the Ruby. They are very close to your measurements.


Thanks.
 
#19 ·

Quote:
Originally Posted by Erik Garci /forum/post/0


It makes a lot more sense.


Given 80:1 for the Ruby, I calculate 143:1 for the Sharp. Or, given 130:1 for the Sharp, I calculate 76:1 for the Ruby. They are very close to your measurements.


Thanks.

How are you using the calculator to do this? For example, now that we have that baseline - can the calculator be used to estimate what my ANSI would be at the screen using the RS1 if it puts out 300:1 ANSI CR?


Also recently I made some changes to my room to reduce reflections. I don't have the Ruby anymore to remeasure that. But I know my Sharp 10K went from 130:1 to 180:1 after I made these improvements. Based on these improved room conditions what would you then anticipate my ANSI CR to be at the screen with the RS1 yielding 300:1? Thanks!
 
#20 ·

Quote:
Originally Posted by lovingdvd /forum/post/0


How are you using the calculator to do this?

Just enter 110:1 ANSI CR for proj. A (Ruby), and 279:1 ANSI CR for proj. B (Sharp). In this case, it doesn't matter what you enter for on/off CR and gamma, so you can just leave those at their default values. Then enter 0.00684 or 0.00825 for reflectivity. Click the "Calculate" button, and look at the calculated contrasts for the 100% Checkerboard (on the bottom line of the table).
Quote:
Originally Posted by lovingdvd /forum/post/0


For example, now that we have that baseline - can the calculator be used to estimate what my ANSI would be at the screen using the RS1 if it puts out 300:1 ANSI CR?

Sure. Just enter 300:1 ANSI CR for one of the projectors, and click the button.
Quote:
Originally Posted by lovingdvd /forum/post/0


Also recently I made some changes to my room to reduce reflections. I don't have the Ruby anymore to remeasure that. But I know my Sharp 10K went from 130:1 to 180:1 after I made these improvements. Based on these improved room conditions what would you then anticipate my ANSI CR to be at the screen with the RS1 yielding 300:1? Thanks!

According to the calculator, your reflectivity went from 0.00825 down to 0.00395, so a projector that has 300:1 ANSI CR at the lens would have 188:1 at your screen.
 
#21 ·

Quote:
Originally Posted by darinp2 /forum/post/0


It looks like that 5% APL pattern you were referring to has 5% of the screen covered by white rectangles. If I were to estimate the value for that image in that calculator I would change the percentage of the screen covered by the checkerboard to 10% (twice the 5% because the checkerboard contains equal amounts of white and black).

Hi Darin (or Eric), are you sure about this? The notes in the calculator says, "The Checkerboard percentage is the stimulus (or signal level) of the white rectangles." Which implies that only the percentage of the white checkerboard is taken into account.
 
#22 ·
A projector that can do an 800:1 checkerboard versus a projector that can do a 100:1 checkerboard will show visual improvements in other images; but the checkerboard itself will look about the same.


So the 8X8 checkerboard test is a useful measure even if you can't see the difference yourself in the actual checkerboard shown. The problem with Checkerboard testing is that it requires a near perfect room or for the screen to be covered with black velvet (the later is sometimes called the Putman method)


To actually see the difference that an 800:1 ANSI projector can achieve versus a 100:1, you need to use an image that doesn't throw so much light on the screen. About two years ago, I created a very simple image to show the difference. It is a 128x72 white pixel square in a 1280X720 black image (it is thus 1% white). I called it the "AVS Contrast Test"




compared to a completely black image: http://mrwigggles.250free.com/1280_black.jpg , you will see a big difference in how much light spill that little square makes. And with these images you can visually see the difference between an ANSI 800:1 projector and a 100:1 projector which might have been invisible in the actual ANSI test.


This simple image was mainly meant for visually spoting differences but you could measure with it as well. Take a reading in the center of the top left white square and then in bottom right corner and you will get an "AVS Contrast Ratio"
for your set-up . (I say "set-up" and not "projector" because some of your result will be based on your room.) A good "set-up" should do 800:1 or greater on this test. If you get less than 400:1, you might want to consider changing projectors or painting your walls to a darker color (or if you forgot to turn the room lights completely off, that might help.)


There might be better tests out there, but it is important to always use black and white in any hypothetical test image. "Dark Gray" squares or dots might seem like a better choice but you don't know how the projector's gamma curve is going to handle it. Also this image is a torture test for auto-iris projectors, the projector can't close the iris to improve the black without white square going down as well and thus reducing the ratio.


It's a stupid simple image test, but it works well.


-Mr. Wigggles
 
#23 ·
MrWigggles

Why didn´t you put the white rectangle in the centre of the image and measure a ratio of the center brightness to the average of the corner brightness values?
 
#24 ·

Quote:
Originally Posted by MrWigggles /forum/post/0


A projector that can do an 800:1 checkerboard versus a projector that can do a 100:1 checkerboard will show visual improvements in other images; but the checkerboard itself will look about the same.


So the 8X8 checkerboard test is a useful measure even if you can't see the difference yourself in the actual checkerboard shown. The problem with Checkerboard testing is that it requires a near perfect room or for the screen to be covered with black velvet (the later is sometimes called the Putman method)

Yes exactly! The way that I see it, the problem with ANSI (both projector and screen measured) is that 1) The checkerboard itself is way to bright to represent even a bright APL movie scene and 2) the screen measurement will always be way off because of so much light spill. Bottom line: ANSI CR has value in and of itself, but trying to relate the benefits of ANSI in typical movie scenes and viewing environments is next to impossible. A low APL CR test on the otherhand can give someone a clear indication of what the contrast of a projector will look like in specific scenes.

Quote:
To actually see the difference that an 800:1 ANSI projector can achieve versus a 100:1, you need to use an image that doesn't throw so much light on the screen. About two years ago, I created a very simple image to show the difference. It is a 128x72 white pixel square in a 1280X720 black image (it is thus 1% white). I called it the "AVS Contrast Test"

Too funny, this is exactly the same idea that I'm having now... only 2 years later
I like the idea of defining it as a low APL test which implies that we could also define mid and bright APL patterns down the road too and perhaps do a weighted average between the three (similar to Bob's suggestion).

Quote:
you will see a big difference in how much light spill that little square makes. And with these images you can visually see the difference between an ANSI 800:1 projector and a 100:1 projector which might have been invisible in the actual ANSI test.

Bingo! Being able to see the difference between an ANSI 800:1 and a 100:1 projector is one reason a test pattern like this would be so valuable.

Quote:
It's a stupid simple image test, but it works well.

It may be a simple test, but if we can get it applied to projector reviews a pattern like this will provide some really useful information across projectors and technologies that is currently missing. As I said earlier in this thread it may help us to get a better handle on quantifying subjective comments like, "image snap", "depth" and "pop" especially with various DI schemes. If a person knows that projector X is capable of only 200:1 in a low APL contrast test vs 800:1 they can mentally get a feel for the contrast depth difference in these sorts of scenes. More importantly, it may also point out low APL contrast problems in projectors that may have decent ANSI and on/off CR.
 
#25 ·
Say, I think a minor but important thing to quantify is what the average average picture level is.


I.e. what *is* the average APL of most films over time? ANSI is 50% APL, but that's still way higher than I think most images. I don't think I've ever seen a good number or range for the average APL of most movie content. That's important to have an idea of so that you can whip up a test pattern at that APL or a few in the general range of average APL.
 
#26 ·

Quote:
Originally Posted by ChrisWiggles /forum/post/0


Say, I think a minor but important thing to quantify is what the average average picture level is.


I.e. what *is* the average APL of most films over time? ANSI is 50% APL, but that's still way higher than I think most images. I don't think I've ever seen a good number or range for the average APL of most movie content. That's important to have an idea of so that you can whip up a test pattern at that APL or a few in the general range of average APL.

Check out the white paper in the second post that HHF dug up. It has a couple of good examples. One thing I'm not clear about from the white paper is their specific definition of luminance and APL. From the table on page 10 it shows a luminance of .5 which I assume to be the 4x4 ANSI checkerboard which has an APL of .73 but I don't know how they have derived this relationship.


To proceed from here seems easy enough:

Step 1) Define the representative value for low APL that we want to target(e.g. 5%, 1%, 10%, etc.).

Step 2) Define how we want the test pattern to accomplish this value of APL. Examples would be one white checkerbox, 4 smaller white checkerboxes, etc.

Step 3) Photoshop a 720p and 1080p version and post it so people can start using it.

Step 4) Get forum members to use the test pattern and collate the values that people measure.
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top