[Dominic Chan]

Here's a concrete example based on your settings, assuming Rec 709, 2.4 gamma, and a measured White of 100 nits. Your 60% gray, RGB triplet 153,153,153, within the 0-255 range, will have a target Y of 29.357 nits. Under the covers, the graphics card is rescaling that pattern to RGB triplet 147,147,147 (most likely) within the 16-235 range, so that's what's actually displayed and measured. That pattern would actually have a target Y of 29.133 nits, if you were to configure your calibration software for range 16-235 (and were using a pattern generator path that passed it unaltered to the display). So with 0-255 there's a mismatch between what the calibration software is calculating for targets, and the patterns that are actually being displayed and measured.

With 8 bit precision there will be inevitably some small rounding errors.

As mentioned previously, (Measuring Greyscale and Primary/Secondary Colours Using...) you will likely get more accurate results by having the software perform the scaling, but that involves overriding the EDID and is something I do not believe a “beginner” (the intended reader if the guide) should get into.

Setting the software to 16-235 can potentially lead to range mismatch errors which are far more serious than rounding errors. That is the default of HCFR and many new users have run into that error.

For the ultimate accuracy you will need to use a disc player with bit-perfect patterns (Ted’s Calibration Disc), or a pattern generator that’s verified to be “bit perfect”:

The cheap Test Pattern Generators accuracy thread

Finally I could satisfy my (and I hope your) curiosity to know which test pattern generator (or TPG) in the low price range is really accurate. To do this I bought a medium-high end TPG: DVDO AvLab TPG, which has inside a signal analyzer whose accuracy has been verified and confirmed with an...

www.avsforum.com

 

[thoth]

With 8 bit resolution there will be inevitably some small rounding errors. Please justify that assertion, as I can think of no reason to believe it's true. As mentioned previously, (Measuring Greyscale and Primary/Secondary Colours Using...) you will likely get more accurate results by having the software perform the scaling, but that involves overriding the EDID and is something I do not believe a “beginner” (the intended reader if the guide) should get into. For many years, Samsung TVs have had a setting (HDMI Black Level) to explicitly control which incoming RGB levels to expect. My old LG monitor has a similar setting. I've read LG TVs have a similar setting. I suspect most modern displays have one. The Nvidia control panel for a long time has had the ability to explicitly configure the RGB levels to output. I assume AMD has something equivalent. I've never encountered a need to hack EDIDs. ​

 

[Dominic Chan]

​

Please justify that assertion, as I can think of no reason to believe it's true. For many years, Samsung TVs have had a setting (HDMI Black Level) to explicitly control which incoming RGB levels to expect. My old LG monitor has a similar setting. I've read LG TVs have a similar setting. I suspect most modern displays have one. The Nvidia control panel for a long time has had the ability to explicitly configure the RGB levels to output. I assume AMD has something equivalent. I've never encountered a need to hack EDIDs.​

You and I have both expressed our respective opinions and I do not see this thread as the right place to get into a prolonged discussion on the range setting. The readers can make their own decisions.

 

[coolrda]

Urgh. I really have to get a windows laptop now that my VM software is again out of date.

Something like this would be as cheap as buying a new VM app and Windows license. Would it work fine?

I use Bootcamp. Far superior to any VM. I simple reboot and in a few seconds run HCFR, JVC Autocal, Omnimic, REW, MiniDSP and all the other Win10 programs natively. Works flawlessly and none of the VM issues.

 

[Lttlwing16]

Here's a concrete example based on your settings, assuming Rec 709, 2.4 gamma, and a measured White of 100 nits. Your 60% gray, RGB triplet 153,153,153, within the 0-255 range, will have a target Y of 29.357 nits. Under the covers, the graphics card is rescaling that pattern to RGB triplet 147,147,147 (most likely) within the 16-235 range, so that's what's actually displayed and measured. That pattern would actually have a target Y of 29.133 nits, if you were to configure your calibration software for range 16-235 (and were using a pattern generator path that passed it unaltered to the display). So with 0-255 there's a mismatch between what the calibration software is calculating for targets, and the patterns that are actually being displayed and measured.

So just to confirm I've got my settings correct.

Pgenerator : RGB Full
Sony 950H : Video Signal sRGB Limited (for Pgen input)
ZRO - 16-235

 

[thoth]

So just to confirm I've got my settings correct.

Pgenerator : RGB Full
Sony 950H : Video Signal sRGB Limited (for Pgen input)
ZRO - 16-235

I would have guessed that's correct, and seems to be what's recommended over in the PGenerator thread, but I suggest you read through a recent thread in the Light Illusion forums relating to this topic. Based on that, it sounds like there's no way to accurately calibrate 16-235 using ZRO.

 

[Lttlwing16]

I would have guessed that's correct, and seems to be what's recommended over in the PGenerator thread, but I suggest you read through a recent thread in the Light Illusion forums relating to this topic. Based on that, it sounds like there's no way to accurately calibrate 16-235 using ZRO.

Perhaps you mean this post from Lightillusion forum :

"
Thanks for the response.

First, I have to correct my previous post, the target Y's for Patch Scale 16-255 are correct for the fractional conversions for Super White; for that case I should have been looking at sliders 234,234,234 as mapping to 100%, not 235,235,235. Your statement about a stretched EOTF doesn't make sense to me; it isn't stretched, it's extended. My TV, when configured for RGB Limited/Video range, is perfectly capable of displaying whiter-than-white values, and although I calibrate for 0-100% accuracy, I want to measure 0-109% to see what happens up there. Which raises another point. As 16-255 is an extension of 16-235, one might think that a patch set developed for 16-235 could be reused directly for 16-255, but that isn't the case, which is at least confusing.

Second, your statement that 0-255 is nearly always the correct setting is odd to me, if what one wants to do is calibrate for a consumer video chain, rather than PC use. Let's look at three scenarios. The one I'm most interested in is using PGenerator, which has been validated as bit accurate for RGB 0-255. The recommendations I've seen, which make perfect sense to me, is to use PGenerator to tunnel 16-235 (or 16-255) patterns through it, with the display configured to receive RGB Limited/Video content. But for that to work with ColourSpace, you need to configure Patch Scale to 16-235 or 16-255. Alternatively, if using the ColourSpace patch window, then configure the PC to output RGB Full (which seems to me most likely to be accurate), configure the Patch Scale to 16-235 or 16-255, and again configure the display to receive RGB Limited/Video. But neither of those setups are accurate. The setup you suggest, where the graphics card rescales from 0-255 to 16-235, is also not accurate. For example, with Rec709, Max Luma Target 100 nits, Patch Scale 0-255, putting the sliders at 128,128,128 yields a target Y of 19.1253, but the graphics card will (or should) rescale that pattern to 126,126,126, for which the correct target Y is 19.1548. Indeed, with that setup, all grayscale target Y's are incorrect except for sliders 0,0,0 and 255,255,255.

If my analysis is correct, then it appears ColourSpace Manual Measure is simply unsuitable for proper consumer video calibration."

 

[Dominic Chan]

Perhaps you mean this post from Lightillusion forum :

"
Thanks for the response.

First, I have to correct my previous post, the target Y's for Patch Scale 16-255 are correct for the fractional conversions for Super White; for that case I should have been looking at sliders 234,234,234 as mapping to 100%, not 235,235,235. Your statement about a stretched EOTF doesn't make sense to me; it isn't stretched, it's extended. My TV, when configured for RGB Limited/Video range, is perfectly capable of displaying whiter-than-white values, and although I calibrate for 0-100% accuracy, I want to measure 0-109% to see what happens up there. Which raises another point. As 16-255 is an extension of 16-235, one might think that a patch set developed for 16-235 could be reused directly for 16-255, but that isn't the case, which is at least confusing.

Second, your statement that 0-255 is nearly always the correct setting is odd to me, if what one wants to do is calibrate for a consumer video chain, rather than PC use. Let's look at three scenarios. The one I'm most interested in is using PGenerator, which has been validated as bit accurate for RGB 0-255. The recommendations I've seen, which make perfect sense to me, is to use PGenerator to tunnel 16-235 (or 16-255) patterns through it, with the display configured to receive RGB Limited/Video content. But for that to work with ColourSpace, you need to configure Patch Scale to 16-235 or 16-255. Alternatively, if using the ColourSpace patch window, then configure the PC to output RGB Full (which seems to me most likely to be accurate), configure the Patch Scale to 16-235 or 16-255, and again configure the display to receive RGB Limited/Video. But neither of those setups are accurate. The setup you suggest, where the graphics card rescales from 0-255 to 16-235, is also not accurate. For example, with Rec709, Max Luma Target 100 nits, Patch Scale 0-255, putting the sliders at 128,128,128 yields a target Y of 19.1253, but the graphics card will (or should) rescale that pattern to 126,126,126, for which the correct target Y is 19.1548. Indeed, with that setup, all grayscale target Y's are incorrect except for sliders 0,0,0 and 255,255,255.

If my analysis is correct, then it appears ColourSpace Manual Measure is simply unsuitable for proper consumer video calibration."

I only had a quick glance at that, but the issue seems to only arise when you use the sliders and let ZRO calculate the patch values. If you use your own patch set defined by the CSV file (as described in my guide), ZRO should use the user-defined RGB triplets.

 

[thoth]

I only had a quick glance at that, but the issue seems to only arise when you use the sliders and let ZRO calculate the patch values. If you use your own patch set defined by the CSV file (as described in my guide), ZRO should use the user-defined RGB triplets.

I think you are incorrect, and the patch set contains slider values, not RGB triplets. Try it yourself, use your patch set and try switching between 0-255 and 16-235 and see if the RGB triplet in the pattern window remains the same.

 

[Dominic Chan]

If the Patch Scale is set for 0-255, the RGB triplet in the pattern window is exactly what is defined in the CSV file, unmodified. If the Patch Scale is set for 16-235, then ZRO will scale the values to fit the 16-235 sub-range.

I don't know why anyone would expect the patch values to stay the same when the patch scale is changed.

If you don't want ZRO to do the scaling, then set the Patch Scale to 0-255, and use the set of 16 - 235 values in the CSV file.

EDIT: Note, however, ColourSpace is not designed for 16-235 patches and always requires black to be 0 and white to be 255.

 

[thoth]

If the Patch Scale is set for 0-255, the RGB triplet in the pattern window is exactly what is defined in the CSV file, unmodified. If the Patch Scale is set for 16-235, then ZRO will scale the values to fit the 16-235 sub-range.
I don't know why anyone would expect the patch values to stay the same when the patch scale is changed.

So then you are confirming that the patch set values are equivalent to slider values, they are not RGB triplets to be displayed unaltered. In which case, using a patch set does not solve the inaccuracy, it's exactly the same as using the sliders.

 

[Dominic Chan]

So then you are confirming that the patch set values are equivalent to slider values, they are not RGB triplets to be displayed unaltered. In which case, using a patch set does not solve the inaccuracy, it's exactly the same as using the sliders.

This is the last time I will respond to you, as you seem to be picking a fight (which I have already declined) rather than seeking an answer.
As mentioned in my previous reply, just set the Patch Scale to 0-255 and ZRO will use the user-defined patch values without any modifications. Then you can set the patches in the CSV to 16-235 (Limited), 0-255 (Full), 16-255 (SuperWhite), whatever.

 

[thoth]

This is the last time I will respond to you, as you seem to be picking a fight (which I have already declined) rather than seeking an answer.
As mentioned in my previous reply, just set the Patch Scale to 0-255 and ZRO will use the user-defined patch values without any modifications. Then you can set the patches in the CSV to 16-235 (Limited), 0-255 (Full), 16-255 (SuperWhite), whatever.

I'm not trying to pick a fight, just provide information. I guess you either still don't understand the problem, or you don't care about the resulting inaccuracy, or both. OK, so be it.

 

[alfred67]

Then you can set the patches in the CSV to 16-235 (Limited)

So the CSV file should look like this for 16-235 (limited)
0,16,16,16,0% grey
1,24,24,24,10% grey
2,47,47,47,20% grey
3,71,71,71,30% grey
4,94,94,94,40% grey
5,118,118,118,50% grey
6,141,141,141,60% grey
7,165,165,165,70% grey
8,188,188,188,80% grey
9,212,212,212,90% grey
10,235,235,235,100% grey

 

[Dominic Chan]

So the CSV file should look like this for 16-235 (limited)
0,16,16,16,0% grey
1,24,24,24,10% grey
2,47,47,47,20% grey
3,71,71,71,30% grey
4,94,94,94,40% grey
5,118,118,118,50% grey
6,141,141,141,60% grey
7,165,165,165,70% grey
8,188,188,188,80% grey
9,212,212,212,90% grey
10,235,235,235,100% grey

I didn’t check all the numbers, but yes, it should look something like that. This assumes, of course, that the graphics card or pattern generator are set to 0-255.

EDIT: The values are not correct. See my subsequent post with the correct values.

In any case, questions like this are probably more appropriately discussed in the main ZRO thread.

 

[Light Illusion]

If I may - as I replied within the Light Illusion forum, all measurements within ColourSpace need to reference back to a 0-1 range. This is associated with LUT generation.

This does mean all 'source' patch data also need to use a 0-1 range.
For manual calibration this means the defined patches need to be 0-255 range, with any rescaling performed via the graphics card under EIDID control, or using the Patch Scale option, etc.

So any .csv sequence needs to also be 0-1 range.
This because the Target values will always assume black is an internal patch of 0, not 16.
And white is 255, not 235.
(This is all because the main ColourSpace code enables any 'profile' to be used to generate a LUT, and ColourSpace ZRO is a sup-set of the main code. We are NOT going to modify ZRO so it deviates from the main code base.)

Yes there will be some 'rounding' of the 8 bit values.
That is expected.
Is it an issue?
Mathematically, yes.
In reality?
Nope.
No display/probe combination is ever more stable than the rounding error.

You can prove the rounding issues by taking single step measurements, as you will see on occasions same 16-235 output value for 2 adjacent 0-255 input values.
But, you never do that in manual calibration.
You use a few 'grey-scale' points, likely 10 or 20(ish) throughout the 016-235 range.
So in reality it is not an actual issue.
(You could make a .csv that is a fill set of 255 steps, but the display/probe instability would cause far more measurement errors than the 12-235 rounding!)

But, we will we adding 10, 12, 14, 26 bit 'bit depth' options in the future.
(Presently being worked on...)
That will greatly alter the rounding issues.

Additionally, it is likely we will add some form of 'Sub-Range' capability.
But, that introduces issues with LUT generation, as outlined above... so is not likely to happen quickly, as we will need to make the code base compatible across the main ColourSpace program, as well as ZRO.

I tried to explain all this (in a simpler way) on our forum, but I guess I failed.

But as I keep saying, it is really better to ask on our forums, as we (specifically, me) do not frequent other forums, such as this, that often.

Hope that helps.
And I hope that ends the argument.

Steve

 

[Dominic Chan]

So the CSV file should look like this for 16-235 (limited)
0,16,16,16,0% grey
1,24,24,24,10% grey
2,47,47,47,20% grey
3,71,71,71,30% grey
4,94,94,94,40% grey
5,118,118,118,50% grey
6,141,141,141,60% grey
7,165,165,165,70% grey
8,188,188,188,80% grey
9,212,212,212,90% grey
10,235,235,235,100% grey

I checked the values and found they are not correct (except the end values). These are the values I calculated:
0,16,16,16,0% grey
1,38,38,38,10% grey
2,60,60,60,20% grey
3,82,82,82,30% grey
4,104,104,104,40% grey
5,126,126,126,50% grey
6,147,147,147,60% grey
7,159,159,159,70% grey
8,191,191,191,80% grey
9,213,213,213,90% grey
10,235,235,235,100% grey

Each step covers 10% of the range (16-235), rounded to the nearest integer.

EDIT: Note, however, ColourSpace (including ZRO) is not designed to use 16-235 patches; i.e., for Limited Range the patches should still be 0-255, with the scaling being done in the graphics card, or in the ColourSpace Patch Scaling.

 

[Dominic Chan]

If I may - as I replied within the Light Illusion forum, all measurements within ColourSpace need to reference back to a 0-1 range. This is associated with LUT generation.

This does mean all 'source' patch data also need to use a 0-1 range.
For manual calibration this means the defined patches need to be 0-255 range, with any rescaling performed via the graphics card under EIDID control, or using the Patch Scale option, etc.

There are people who prefer not scale the range, for a number of reasons that I'm not getting into here.
I did run a test with 16-235 patches, and compared the results with the 0-255 patches. The display in the respective range, of course, and there is no scaling done by ZRO or the graphics card. Here's the comparison:

Full (0-255); Limited (16-235)
0, 0.01; 16, 0.01
26, 0.3; 38, 0.3
51, 1.4; 60, 1.4
77, 3.4n 82, 3.4
102, 6.4; 104, 6.5
128, 10.7; 126, 10.8
153, 15.9; 147, 16.0
179, 22.9; 159, 19.6
204, 30.9; 191, 31.1
230, 41.8; 213, 41.8
255, 51.4; 235, 51.8

So any .csv sequence needs to also be 0-1 range.
This because the Target values will always assume black is an internal patch of 0, not 16.
And white is 255, not 235.
(This is all because the main ColourSpace code enables any 'profile' to be used to generate a LUT, and ColourSpace ZRO is a sup-set of the main code. We are NOT going to modify ZRO so it deviates from the main code base.)

As can be in the measurements above, when the display is set for Limited Range, 235 is white, just like for Full Range 255 is white. In the above example, the measurements are actually correct, even though ZRO interprets the 16-235 patches "incorrectly" (for the reasons you provided).

And I hope that ends the argument.

Well, not quite However, this is off topic for this thread, which is intended to describe the measurement process based on 0-255 patches in ZRO, as described in the guide. I have also added a note to Post 37 to that effect.

 

[nathan_h]

I use Bootcamp. Far superior to any VM. I simple reboot and in a few seconds run HCFR, JVC Autocal, Omnimic, REW, MiniDSP and all the other Win10 programs natively. Works flawlessly and none of the VM issues.

yeah I should see how I can turn one of my vmware win7 vms into a boot camp partition.

edit: after some research, that looks like a pain, so i'll do a fresh install of win10 using bootcamp. thanks for the idea.

 

[Light Illusion]

There are people who prefer not scale the range, for a number of reasons that I'm not getting into here.
I did run a test with 16-235 patches, and compared the results with the 0-255 patches. The display in the respective range, of course, and there is no scaling done by ZRO or the graphics card. Here's the comparison:

The measured value will be basically identical, as you found (with some variation due primarily to display/probe instability), but the Target Values shown within ColourSpace ZRO will be incorrect, as internally ZRO will still be operating in the 0-1 range.

The reasons for this have been explained.
But, as stated, we will be making changes when we add the ability to use different bit depths.
For now, it is what it is, as that is how ColourSpace has been designed to work at the present time.

Steve