Reading and interpreting calibration charts and data for dummies - Page 2  

Bob

I think your color accuracy of the Eye One is going out the window before your luminance accuracy as it gets darker. Red is a darker color and Blue is even darker - and this is generally where I see inaccuracy with Eye One measurements.

So yes a brighter image which ever way you can manage helps. Also with the firehawk watch your measurement angle -if you are right at the screen you are avoiding the sensor shadow you are probably at a severe angle.

I like to use a 2' wide image size - as it makes the lumens math easy (2.25sqft) my Spyder2 can take the 300-400ftL this gives me. ColorFacts has a training mode - the idea is you read a white screen with the EyeOne to find the color error of the screen - then read from the projector with a correction factor for the screen.

As far as I can remember, all DLPs have shown green being towards yellow to one degree or another in the CIE charts I've seen reproduced, so I was assuming this was a technology issue rather than a preference from the manufacturer. I've also seen the measured triangle to show red slightly more saturated as well as the green towards yellow so thought that this was down to the displays colour replication being skewed in that direction, and having moveable primaries would fix it perhaps. Or am I missing something?

If the CIE 'shark fin' is the colour gamut visible to the human eye, will we benefit from being able to reproduce the colours that exceed the triangles of PAL, NTSC, HDTV? Some say the colours (such as red) are over saturated if they are measured outside the system gamut, yet it's still someway form the maximum range of the visible gamut (or is it just exceeding the syatems gamut range and crushing maybe?). Why don't we realise the currently constrained gamuts aren't the full gamut or is the eye/brain easily fooled and readily acceptable to this 'trickery' so believes the colour range to be 'true color'?

Sorry for all the questions but these are a few that I've been meaning to ask for a while now - I want to learn this stuff and even when I think I'm getting to grips with just a small part of it the experts quickly show me there's so much more to know. Keep it up guys!!

Gary.

Excellent link - thank you very much.

Gary

Wide gamut HDTV is actually discussed in Poynton's HDTV book that I have - but I would imagine until we get laser displays that no manufacturer wants to implement. What we have is a manufacturing compromise - the original NTSC green was a much deeper forest green and was replaced ages ago.

It is not so much a technical issue that DLP Green is often Yellow - the JKP Samsung is correct and some others have a deeper Forest Green (Optoma H30). The problem is lumens - luminance is primarily composed of Green - so the brighter green you have the brighter your display. Since nobody wants to market a display less than 1000 lumens (even though calibrated they are always less except Infocus ScreenPlay) - that really is not possible to do with displays that have Forest Green. So does the market want an accurate display - or do they want a 10' widescreen that works in the daylite?

I thought the HT mag review of the JKP Samsung was rather tepid solely because the brightness and contrast was nothing special - even though the colorimetry, greyscale, and gamma, and video decoding were perfect.

So until the market gets focused away from the marketing numbers and into the real engineering numbers - I don't think things will change. Once they do get it - maybe then wide gamut HDTV will become a reality because the market will clamour for it. But that will require an all new source cheap.

Why do dogs eat dogfood? Because it is what we give them!

Green is also much lighter than red, so I'm not sure it wouldn't overemphasize green errors compared to red errors. But I'll try some plots to see how it maps to my subjective impressions on the next few projectors I review. I'm not sure how intuitive deltaC*, deltaH* plots would be for magazine reviews, but that's a different issue. Didn't you email me an example of those plots several months ago? I seem to remember that, but I can't find them now.

Greg Rogers
AccuPel
Widescreen Review

Kevin - I definitely like the revised MATLAB charts. They are really looking good now! (I'm not sure how I missed this thread...).

Bob - Definitely check your measurement technique. However, the rest of it is looking pretty good if you can validate the rest of the measurements. As a note, when I was putting my H77 through its paces using both the EyeOne and Spyder2, I definitely noticed a drop off at lower light levels with the EyeOne (Jeff recommended >5 Lux or so if I remember correctly!). Check out Kevin's technique to make sure that you are getting enough light into the meter.

One quick question, though: how important is focus here? I would imagine that some projectors may be dim enough and have a long enough throw that focusing at 2' - 3' may be more than the optics can handle.

Later,
Bill

gregr

Yeah - go ahead and use the one I linked to as it is the latest version. I just finished grad school course work and looking at these again - thought it time to put them out for public comment. Anyone is free to use - even ghbliss/umr/Ursa - though Excel is not very chart friendly! I fully intend to make sure ColorFacts gets it into a version update ASAP since it is what I use. No benefit to keeping it to myself if nobody else uses it and I have to explain it every time.

I have the specific equation references from the texts if anyone needs them - all I did was invent the visualization to focus on what I think is objectively/perceptually important. This is something I have lots of practice from using Matlab in my Acoustic Engineering assessments- my last assessment was 10Mb of .pdf charts/text and several 80Mb movies!

On delC*, delH* - while Green is much lighter than Red it is balanced out by the Green area being much smaller than Red on the u'v' chart. On the SP7210 it makes it clear that the Red (which is oversaturated) is worse error than the Green (which is desaturated towards yellow) - but it is hard to see either of those things looking at the CIE charts. On CIE31 Red looks perfect while Green looks bad, and CIE76 they look to be about equal error. But watching the SP7210 projector - the Reds are the first thing you notice - and you are hard pressed to see the Greens compared to the SP4805 (which is better color all around - just slightly desaturated)

The u*, v* should be obvious - it is not much different from an existing ColorFacts x,y chart - still need to double check my math on the angles for the different colors. Note I use u*, v* and not u', v' - this is intentional since black has no color by definition it should not show any error even though your sensor always shows more color error - and color at 10% is very hard to see so it's error range is limited - and so on out to white. The key was adding the radial circles at consistent target points - I hate how ColorFacts does an arbitrary zoom. You can easily see how white peaking and yellow/white segments or auto-iris causes color shifts using this chart.

Notice I do not do RGB bars and 6500K lines - as they are meaningless! Customers only expect them because we keep telling them to! I hate the RGB bars almost as much as the useless 6500K chart - as we have already pointed out in this thread - was it calculated with the primaries measured or not - and is it relative to an arbitrary gamut? How is the customer supposed to know that +Blue+Green means it is pushing Cyan?

delC* and delH* I think came from the Color Science using MatLab book - not the simple h angle that poynton uses - but a normalized version so that you can use it in delE units (the radius is delE). I have thought of surrounding color bubbles to help visualize this chart better - as it is the one I have to most explain. It assumes the L* between the color reference and target is the same - Colorfacts does not report this - and that would require a 3D target chart. I just took AV/Pro training from SethS - and the Runco VX2 has the ability to do not just blue only like the SP7210 - but RGBCMY only modes so that you can measure the video color decoder levels! I need to experiment with lens and filters to see if there is some way to more easily measure this on other projectors.

I toyed with gamma math for a while - I finally get the same numbers as ColorFacts. I did the average of each "IRE" step for a while like WSR/gregr and umr- but I think these is more meaningful using lightness variation - turns out we cannot see these slight slope differences. I want charts that represent what we see - not what we can't see. This is why the gamma chart is flat - if gamma is flat it will look tonally smooth. Infocus does exceptionally well on gamma - I have reviewed others that really kink this chart. I decided not to grade the gamma value itself - as that is purely subjective to your environment.

I may still assign letter or percentile grades just for those people that have to have a number and cannot figure this out from the target lines/circles. And I need to add brightness/contrast comparisons - I am toying with some novel ideas of presentation there as well - like validating marketing claims. The idea is to present these charts for the best calibrated preset, best contrast, brightest, and ISF calibrated so that people can really see what the marketed numbers are costing them. I think that is what is lacking in reviews currently (I know - more magazine space is required for ads not charts!)


Ursa

I have no problems focusing such a small image - but I don't think it matters when you are measuring color and brightness. I don't quite have room in the office setup to measure long/short throw differences with the lens - but something I want to look into.

Kevin - That's about what I thought. As for this:

If they do RGB% the way I do it, then it is merely the average color balance throughout the grayscale (10% stimulus and up for me). Others I have seen normalize the primary components, but that leads you to having no deviation at 100% Stim, which is not correct. I'll let Jeff comment on how he does it, but I do not know how ColorFacts does it.

Later,
Bill

I use the reference primaries for the RGB deviation although this is user configurable if someone wants to modify the spreadsheet to use display primaries instead. It seems to me to be a toss-up as to which you choose to use.

Using reference primaries:

Pro - no pre sampling required, gives a consistent reference for color error between displays
Con - less accurate for guiding a user on setting gray scale

I also did some testing on one of my displays that allows manual selection of color space for each input type to test the color space hypothesis for Bob's results. I tested a 480p and 720p component input from my AccuPel generator using 75% color windows with SD and HD color space processing on each. I could not see any shift in the primaries at all while the secondaries moved along the line between the primaries. With my display and signals I could not verify that Bob's results are the result of improper color decoding.

Not only is accuracy decreasing at lower light levels, but the influence of light that is contaminating the measurement also increases. Low light levels require dramatic efforts to exclude extraneous light.

Sorry, but I can't find the link you are talking about. I would really like to look at your delC*, delH* charts, but I can't find them.

If you look at the MacAdam Ellipses on a u'v' chart they are similar in size near green and red. Of course the same is true for a u*v* chart for a constant L*. So the big difference between the luminance (Y) of green and red still concerns me (L* being related to the cube root of Y). The problem with using delC*, delH* alone to express the color difference is that doesn't fully incorporate delL*, i.e. the total color difference is delE* = SQRT [ (delL*)^2 + delC*^2 + delH*^2]. This is more easily expressed as the L*u*v* color-difference equation, delE* = SQRT [ (delL*)^2 + delu*^2 + delv*^2], which I use to express grayscale differences in my reviews (given that the measured grayscale luminance is equal to the target luminance, delL* = 0). But delL*=0 doesn't hold when comparing the relative perception of color differences around two colors that have widely different luminance values.

I'm sure this is getting too technical for this thread so perhaps we should take the color science discussion off line.

I plot (or sometimes just describe in words) the "equivalent gamma" at 10 IRE steps over the grayscale, not an average of anything. It makes a huge visual difference if the gamma at 10 IRE changes to say 2.2 from 2.4 between 20 to 90 IRE. But many projectors have a much less constant gamma than that, and the differences are very visible.

It's really not ad space that limits the number of charts in my projector reviews, which are usually 7-8 pages of pretty small print (that's probably 3x the length - in words - of the reviews in most magazines). Complex charts, that many readers won't understand, reduces readership and magazines must have readers (to get ads) to survive. Widescreen Review is certainly the most technical home theater magazine available in the US. WSR could increase its circulation by becoming less technical, but fortunately the publisher prefers to maintain the technical level of the magazine. I think my reviews include much more technical information than any other magazines, and as much as I might like to just write an engineering lab report for every review, we have to serve a slightly wider customer base. That said, I'm always open to finding a better way to show information, if I can make it understandable to our readers. Which takes me back to the beginning of this posting. Where are your charts posted? :)

Greg Rogers
AccuPel
Widescreen Review

Greg,

Off topic post...

I really appreciate your reviews they are always very well done. I would be very careful about changing how you present the information. Deviating greatly from other sources both current and historical would make it more difficult to compare displays on an equal basis.

Stupid AVSes new bulletin board does not underline links anymore.

It was in this post

http://www.avsforum.com/avs-vb/showt...&&#post6143279

click on the 'Infocus by krasmuzik' words

I was thinking of HomeTheaterMag when I said 2" charts - I think you get 3" charts and more of them! Anyways I reupped WSR because of your reviews - sorry about being flippant on that! In fact I was just at WSR headquarters for SethS AV/Pro training - amazing theater!

My goal here is indeed to reinvent the way that calibrations are charted. My experience with customers is they are unable to translate the existing charts no matter how scientifically simple I get on them into what they see - and they are not useful enough to me for doing detailed comparisons of displays.


My comment about gamma was the chart itself - we don't see gamma "transitions" - we see lightness variations from an ideal. I am striving to use perceptual measures in all my charts now in a way that are meaningful to both the calibrator and customer without having to teach color science to my customer.


delE* = SQRT [ (delL*)^2 + delu*^2 + delv*^2],

Rather than report delE directly in a confusing 3D chart (delE is simply the distance from origin (D65) to the cartesian L*u*v* points) -I would rather report the delL* from an ideal gamma curve (my last chart), and the delu*, delv* from D65 (my first chart) in the colorspace. The reason is the major problem in greyscale are those two things - gamma tracking and color shifts. delE is a single number that says it is off - but does not say why it is off - which is what I end up having to explain. Using these charts I can point to them and say something like "your bright whites shifted Cyan, and your dark blacks are a bit Red, and your middle greys are too light" - rather than - "You are within 10 dE".


delE* = SQRT [ (delL*)^2 + delC*^2 + delH*^2].

This I do use for the color target - ColorFacts does not dump out delL (I have to write an automation script then I can get it). I suppose if the color decoder is off (red push) you could have some delL* not showing up on the delC* delH* chart - but I would again prefer to avoid a confusing 3D chart and think of a good way of showing that in 2-2D charts.

On all my charts I use the L* of the ideal target color or greyscale point rather than 100%. Color L* I get from the HD/SD standard (convert Y), and greyscale L% I get from the ideal gamma fit.


umr/Ursa/ghbliss/Milori etc. are all welcome to participate if they want - I don't think discussing known color science should be tool/review proprietary - but maybe they think otherwise. Others have said they glean something amongst the scientific ramblings - so I help someone learns something! My goal is objective perceptual measures that can be used to compare pre/post calibration or A vs. B displays that Joe Blow can understand - so if they chime in and say hey I am confused - then I know there is more chart work to do.

And if my posts seem to be not very lucid - I have a good excuse besides fumble fingers - I had dental surgery last week and still on dental narcotics!

umr

I very much disagree with that- already you cannot compare CIE charts across magazines and even across reviewers. The may use a different mode than the other guy, a different screen, a different measurement setup. I searched the archives of HTMag and WSR - and found only one review that even correlated on projector mode setup (Sony HS51) - and the results did not even agree!

So I don't think that arguing that this is the way it always has been so don't change or I cannot compare is valid. It is still going to be true that you cannot compare even if you change charts for the same reasons.

The reality is most people reading these reviews do not even understand the colorscience behind the charts they are looking at - and have no idea how to compare a CIE chart or 6500K chart. I know because I have hung out in the <$3500 forum which most calibrators don't do - explaining the color science is way to difficult. That is why I do want to change the way I chart to objective/perceptual measures - because it helps my customers understand better what I did for them. I am not at all concerned that they cannot compare my work to a magazine review they read! What I am comparing against are the standards - and these charts do present that.

"Mark says the "constrain color gamut" option should be used to optimize grayscale calibration. I would trust that he knows how his product works."

Go it, good timing also as I'm doing 50 grayscale tunings in the next five weeks. :)
Got colorfacts! Don't use contrain to gamut for CIE readings just grayscale tunings.

Thx Greg.

I tend to lean toward simplicity. I don't see how an angular coordinate as a very useful tool for most customers. I also do not see much added value to me over what I am currently using given the limitations found with most displays.

I prefer delta E CMC as a color tolerancing function. It tends to be more difficult to get lower values with this metric while it does not generate extremely large values when the colors are way off. I null out the luminance part of this calculation to reduce this to a color only error. This system also had great success in the textile industry. delta E 1976 has not been the best system for most people who are critically dependant on color accuracy. This is not to say WSR should switch to delta E CMC.

For luminance errors I prefer to look at a slope error that is calculated in my software, but not shown with the data that Bob posted. This error is calculated between each 10 IRE segment. I prefer this over the effective gamma between each value because it will work for the linear segment assuming you want to consider it. I also like to look at a gamma plot and the actual relative luminance values for each measured point versus the target level. Using a luminance tolerancing metric would also be valid, but I see little value in adding that metric to my arsenal.

Customers seem to be more comfortable with CIE 1931 charts, color temperature, RGB deviations and gamma plots if they are to some degree technical. The final result is probably all that matters to most customers in the end. The charts and tables just tend to confirm how much of a change was made.

guitarman

so the mail-in thing really works - I was afraid they would all be buying the SpyderTV instead! Guess I better get on the ball before you service the entire AVS market! :D

I would still not use the option - after discussion it sounds like Mark was really saying it makes the RGB % be relative to the standard you are tuning to - yet your displays primaries are likely not that standard. So you should first measure your RGB primaries - then your RGB% should be relative to those and be better for greyscale tuning.

krasmuzik,

I did not say the comparisons were perfect, but switching to a completely different coordinate system makes them even more difficult.

I am not arguing. I was stating my opinion.

I see no reason to contribute to your posts since you seem to have all the answers.

The large number amount is linked up to the upgrade firmware deal. Now you're saying leave it unchecked all the time? Did we get that wrong or misunderstood with Mark saying check it for grayscale tuning?

Actually 1976 has both the L*u*v* and L*a*b* standards - they are more suited to different applications. L*a*b* was because textile/paint etc are more suited to surface colors under various illumuniations. Lu'v' was more suited to self-luminous sources. Poynton does not discuss this - but it is covered in detail in the MatLab color science reference I have. Many of the more advanced error measures from the last decade are based on the CIELAB for the surface color industry.

And of course if I am talking to customers I don't talk in coordinate systems polar, cartesian, CIE or whatever.

I show them the greyscale target and say - see the colorwheel overlaying the target - that is the color that your greyscale was pushing.

I show them the color target and say - see the colored dots - this represents the colorfulness (B&W to laser intensity) and hue (rainbow) errors w.r.t. to the HD (or SD) standard.

Then I show them the gamma plot - and see how the lightness does not track?

This explanation of the color charts has no color science in it whatsover - you really do not have to explain the math behind the charts. It is a target - Joe Blow can understand a target. Most importantly being perceptual math - it is a target that represents what he sees.

Unlike if you show a 6500K chart - you have to explain the concepts of black body curve, and CCT and get into the science simply to explain that if that graph is on target - it may still not be calibrated as it might be Magenta or green. Then you show them the RGB histogram and they go - so where is the Magenta you talked about? OK Red+Blue = Magenta - then you argue about Magenta is not Purple as that is surface color mixing not light mixing......



This is much better than showing the RGB histogram (and I know guitarman is fond of that one) - unless you know your color science that Green+Blue is Cyan you cannot interpret it - and if the are various method for computing RGB% it really does not say anything.

Yeah, I agree that RGB% measurements don't have much useful meaning unless they are made relative to the display's actual primary colors, as I said way back earlier in this thread.

Anyway, I would hope that the constrain option doesn't really affect the grayscale measurements at all, just the RGB% numbers. If I had time (which I don't) I would dig out ColorFacts and see if it affects the actual x,y,Y grayscale measurements.

Greg Rogers
AccuPel
Widescreen Review

umr

I am not trying to be argumentative here about your software! I am going off my experience getting customers to understand decade old conventions of the ISF way of presenting things - which currently does not even recognize or teach that the perceptual measures exist - even though Video Experts like Poynton do discuss them. I don't have all the answers which is why I hope to have a discussion about better ways of doing things - not reject them out of hand because they are "different".

It used to be 6500K charts were what all the magazines presented - before someone had the nerve to drop it and present the CIE chart with RGBW plots - and Greg has expanded that out alternating between the different CIE versions to highlite RGBCMY differences (which the other magazine does not even measure - but happily say it was only 500K off - and as a calibrator that tells me nothing!). The other magazine has no concept of gamma or even dE - and gregr discusses it - should he not because it is a new chart?

If we sticked to the old way of doing things we would show up with our composite LD and our blue filters - and look at a few testpatterns and say yep you are good - that will be $450 please.

Greg,

I think you would find the differences between using the displays primaries and the reference primaries on the RGB calculations are small enough to not render the values useless. These errors will diminish to zero at D65 with the method I used to formulate the solution.

Here is an example:

RGB percentages using reference primaries.

78% 88% 124%
81% 99% 111%
84% 105% 104%
86% 105% 103%
86% 102% 105%
88% 102% 105%
90% 101% 105%
90% 99% 107%
91% 98% 107%
92% 97% 108%

RGB percentages with display primaries.

73% 89% 118%
78% 99% 109%
84% 104% 103%
86% 104% 102%
85% 102% 104%
87% 101% 104%
89% 101% 104%
89% 99% 105%
90% 98% 105%
90% 97% 106%

Display primaries used in example...

Red Green Blue
x 0.7000 0.29 0.18
y 0.3392 0.7 0.05

Kevin,

I am not saying a new metric may not be warrented at times, but it needs to be tempered with what is lost. Obviously adding something is no problem at all unless it confuses people.

You realize that quote was from Mark was years ago in the archives? What numbers and firmware are you talking about here?

umr

Yes the RGB% accuracy makes me wonder what the whole point of the CF gamut option was to begin with - all RGB% is for is to say "you are off so fix it"!. Where is Mark anyways - he surely can ask Heath what the history on this was. I cannot say I noticed any greyscale changes when I switched it off - but it sure opened my eyes about oversaturated primaries (as guitarman would concur - I think we both were saying Optoma/Infocus had perfect reds). But it would be nice to know for sure that leaving it off is no harm.


I at least want to do away with the 6500K chart - it is misleading to the point of detrimental. I cringe everytime a calibrator review presents the 6500K chart as if it were the summary of the performance.

If you think of the color/greyscales targets as zooms onto the CIE chart RGBCMY points - it really is not different - just a closer view. The CIE shows you the gamut triangle - this gets you zoomed into to see what the color errors really are (I have even tried overlaying the little magazine charts against a light source to try to compare and it is a useless effort....). Now if they publish the x,y values - I enter it into the graphs and compare.

I do wish everyone would publish the xy values they measure.

I would not say that RGB chart is useless. I use it all the time and get great results with it.

The color temperature chart is next to useless, but many people want to see it.

This whole metric discussion is a major side issue. Without good measurement technique and quality instrumentation all the metrics are useless. Metrics can bring value once you have good measurements, but until you do they are worthless.

Thanks, I got it. You've done a lot of good work and thinking here, but I probably come to different conclusions on the grayscale/gamma graphs.

That's a reasonable point, but plotting the L* error from a constant gamma (your gamma graphs) could really confuse the reader. In most situations it will be more important to have the right gamma with slightly bigger L* deviations, than the wrong gamma with slightly smaller L* deviations. In other words, a perfectly constant gamma of 1.8 is not going to be better than a gamma of 2.4 that varies slightly above and below 2.4. So the chart with the "best shape" doesn't necessarily convey the best performance, and the reader must somehow decide how to trade off different L* errors between charts with different average gammas. There is no way they are going to be able to do that. There are other practical issues such as a projector that has 15 different gamma curves, as some do, you would need 15 different charts to be readable with decent resolution.

With respect to grayscale, which is where I use delE, when the calibrated grayscale is within a delE of about 2-3, it is so close to correct it doesn't really matter much whether it is off because it is slightly red or slightly blue, i.e. most users can't see a difference. But when it off by more than about delE=3, then the user can see the color shift and so that is something that is useful to describe. (I generally do this with words in the review to describe what happens at the dark end of the grayscale for instance.) I have thought about adding the h (hue) angle to the delE table so the hue color is known to anyone that wants the information, and then plotting a polar graph of delE and hue angle, which is essentially the same thing as your u*v* plot. The problem with the plot is that lots of points typically overlap each other and it is difficult to impossible to tell what point corresponds to what IRE value. Therefore, it is usually easier and clearer to simply say something like the delE value was 6 (significant) and the grayscale become blue at 10 IRE.

BTW, I should just note that the nearly worthless color temperature graphs that you see in WSR were there before I arrived (I added the delE measurements), and they have remained there because other magazines publish them. If we took them out, then less knowledeable readers would assume that we don't provide all of the information they can get from other magazines. So in almost every review I have to say that the delE values are what is most important for grayscale accuracy.

Anyway, I will play around with your delH*,delC* graphs for the color primaries and secondaries and see if they are an improvement (IMO) for describing the perceptual and subjective dislike of color errors.

Greg Rogers
AccuPel
Widescreen Review