Hi, I'd like to know what are the RGB values associated with the grayscale % test patterns form DVE (or AVS HD 709 calibration disc).

For example, I know that the video black test pattern is RGB 16-16-16 and that the 100% gray test pattern is RBG 235-235-235.

I've come up with a conversion equation and I'd like to know if it is OK :

video black (0% gray) = RGB 16-16-16
10% gray = RGB 38-38-38
20% gray = RGB 59-59-59
30% gray = RGB 81-81-81
40% gray = RGB 103-103-103
50% gray = RGB 125-125-125
60% gray = RGB 147-147-147
70% gray = RGB 168-168-168
80% gray = RGB 191-191-191
90% gray = RGB 212-212-212
100% gray = RGB 235-235-235

I'd also like to know the RGB values associated with the primary and the secondary colors % saturation test patterns of AVS HD 709. For example, what is the RGB equivalent of the 100% saturation red test pattern. Is it RGB 235-16-16 or RGB 255-0-0?

I'm asking this because I want to make my own test patterns to calibrate my TV in RGB 0-255 range using the PS3 XMB and games in RGB full range mode (my TV supports RGB 0-255 with an HDMI connection).

I think this can't be done with calibration discs because their native output is Y/Cb/Cr and choosing to output them in RGB with the PS3 imply that it will remap Y/Cb/Cr 16-235 to RGB 0-255, messing up the test patterns RGB values in regard of what the HCFR calibration software is expecting.


In this regard, I have another question. If I make myself test patterns to calibrate my TV in RGB 0-255 range (PS3 XMB and games), will it work with the HCFR software? I will try to be as clear as possible.

For example, for the 0% gray, be it video black or computer black, HCFR is expecting a 0% gamma response output (you adjust TV brightness to acheive black at video black or computer black). But when calibrating using the DVD pattern mode of the software (16-235 RGB range), I think things are not the same for % gray above 0%.

For example, for a 10% gray pattern, HCFR is expecting a 10% gamma response output in respect to the 16-235 RGB range. If I'm correct, I can't just make myself a 10% gray test pattern of a RGB value of 25-25-25 (255*10%) because HCFR will tell me that my gamma is not correct. I think it is expecting a gray test pattern of a RGB value of 38-38-38 (10% of 235 corrected to reflect the fact that 0% is at 16). How can I calibrate?

Can someone shed a light on what I said? Is what I said make any sense?


Thanks in advance

If you are creating patterns to be used on the XMB, they need to be for the 0–255 range, no matter what your output is.

If the PS3 is outputting ‘Full RGB’ then it will use them as is, but if you are outputting ‘Limited RGB’ then it will scale those values.

So when outputting Full RGB, 90%stim is 229,229,299 and when outputting Limited RGB it will be scaled to 213,213,213. Gamma shouldn't change.


If you're wanting patterns at video levels for whatever other reason, the easiest way to do it is to create the pattern for the 0–255 range, bring up Photoshop's levels tool (ctrl/cmd + L) and set the output levels to 16/235 and it will adjust things accordingly.


What you can also do is create a test pattern for video levels with the above method, and then if you want BTB/WTW information, add it to the pattern and save it. Then set the PS3 to output 0–255 and the display's input to 16–235. That way you get video level patterns without clipping BTB & WTW. (assuming the display doesn't clip it)

Andrew's already covered much of the practical answer to your question. At the theorectical level, yes, the grayscale patterns are called 10%, 20%, ... stimulus for a reason. These are the pre-gamma values. Your measurement software, e.g., HCFR, compares the measured result to the stimulus percentage after gamma adjustment.

The saturation patterns are trickier and are discussed in this thread (http://www.avsforum.com/avs-vb/showthread.php?p=15619419#post15619419), where Tom Huffman gave the values used to create the saturation patterns on the AVSHD disk. Essentially the inverse of a 2.2 gamma is applied to create the patterns, such that the points will measure with equal Y on a display with a gamma of 2.2, at the 25%, 50%, 75% and 100% distances from white to the primary. The RGB values are described there in the range [0..1], since that is what is used to calculate the YCbCr values actually recorded on the disk.

for a 10% gray pattern, HCFR is expecting a 10% gamma response output in respect to the 16-235 RGB range. If I'm correct, I can't just make myself a 10% gray test pattern of a RGB value of 25-25-25 (255*10%) because HCFR will tell me that my gamma is not correct. I think it is expecting a gray test pattern of a RGB value of 38-38-38 (10% of 235 corrected to reflect the fact that 0% is at 16).

In this area thinking in terms of RGB doesn't necessarily help you to understand things. When you're using an external source like a calibration disk, all that HCFR really does is to take some measurements, make some calculations, and then give some reports on what the meter measures. HCFR doesn't necessarily care what type of input (video or computer) you feed your display. All that really matters is if you measure the expected pattern.

Sure with video levels 16 is expected black and with computer levels 0 is black, but for HCFR that really doesn't matter because all it wants to do is to measure whatever the display produces for an input designated as black. Again for 10% stimulus HCFR really doesn't care what RGB level you're sending to the display, it merely expects to measure 10% of the encoded range from black to white. If you wanted you could even calculate point gammas yourself using http://www.avsforum.com/avs-vb/showthread.php?p=10469695#post10469695 and again RGB values wouldn't matter, all that would matter is the measured light for a given stimulus.

If you are creating patterns to be used on the XMB, they need to be for the 0–255 range, no matter what your output is.

If the PS3 is outputting ‘Full RGB’ then it will use them as is, but if you are outputting ‘Limited RGB’ then it will scale those values.

So when outputting Full RGB, 90%stim is 229,229,299 and when outputting Limited RGB it will be scaled to 213,213,213. Gamma shouldn't change.


If you're wanting patterns at video levels for whatever other reason, the easiest way to do it is to create the pattern for the 0–255 range, bring up Photoshop's levels tool (ctrl/cmd + L) and set the output levels to 16/235 and it will adjust things accordingly.


What you can also do is create a test pattern for video levels with the above method, and then if you want BTB/WTW information, add it to the pattern and save it. Then set the PS3 to output 0–255 and the display input to 16–235. That way you get video level patterns without clipping BTB & WTW. (assuming the display doesn't clip it)


Thanks for the answer, but I think I don't quite understand correctly. I will go point by point and give examples.

1- When I calibrate my TV for movie viewing (Blu-ray/DVD), I do that:

a-I set my PS3 video output to Y/Cb/Cr and SuperWhite to On so that BTB and WTW signals are not clipped at the source level.

b-I set my TV HDMI black level to extended (RGB 0-255 range) so that BTB and WTW signals are not clipped (or the grayscale remapped) and so that it is at the same black level as the source.

c-I use AVS HD 709 and HCFR software to calibrate (amongst other things, I will calibrate my black level at video black, even if BTB and WTW are not clipped, but not visible - so I set my brightness so that the luminance at 10% gray is 0,603% of peak luminance (100% gray - RGB 235-235-235 - so that my gamma is 2.22 at 10% gray)

d-These settings will be good when PS3 RGB range is set to limited(?)


2- I want to have an other calibration setting for PS3 games and XMB so that I get the full RGB range of 0-255 (for the games that may support it) - I want my TV black level at computer black. So if I understand, I will do that :

a-I make myself gray patterns like this (using Paint, because I don't have Photo Shop :D):

0% gray = RGB 0-0-0
10% gray = RGB 25-25-25
20% gray = RGB 51-51-51
30% gray = RGB 76-76-76
40% gray = RGB 102-102-102
50% gray = RGB 127-127-127
60% gray = RGB 153-153-153
70% gray = RGB 178-178-178
80% gray = RGB 204-204-204
90% gray = RGB 229-229-229
100% gray = RGB 255-255-255

b-I set my PS3 RGB range to full (if I understand, there is no BTB and WTW information because black is at 0 and white at 255).

c-I set my TV HDMI black level to extended (0-15 and 236-255 are not clipped or the grayscale remapped).

d-I use AVS HD 709 and HCFR software to calibrate.

Here is my question : HCFR does not have a 0-255 RGB range mode when using external images, so that the readings will be messed up. At 100%, HCFR will be expecting RGB 235-235-235 and the gray pattern I will use will be RGB 255-255-255. So even if out-of-the-box my TV has the correct gamma for, say, 90% gray, HCFR will report that it is lower than 2.22.

So how can I calibrate with HCFR to have my black level at computer black?


3- Other questions

If the PS3 is outputting ‘Full RGB’ then it will use them as is, but if you are outputting ‘Limited RGB’ then it will scale those values.

So when outputting Full RGB, 90%stim is 229,229,299 and when outputting Limited RGB it will be scaled to 213,213,213. Gamma shouldn't change.

When doing so, is there a compression process happening, in the sense that the grayscale is remapped and some information is lost. For example, if I have a 0 to 255 gradient, can remapping 0-255 to 16-235 introduces banding?

Is it the same thing when my TV HDMI black level is set to 16-235? I found out that when using AVS HD 709 and setting my TV HDMI black level to 16-235, BTB is clipped, but not WTW, and that also RGB 16-16-16 is remapped to RGB 0-0-0 and RGB 235-235-235 is remapped to RGB 255-255-255.

Why does the TV just can't clipped BTB and WTW without remapping the grayscale - and me setting the approriate black level with brightness?

Same thing with the PS3...

What is the difference between clipping and remapping?


Thank you VERY much for you assistance :)

Andrew's already covered much of the practical answer to your question. At the theorectical level, yes, the grayscale patterns are called 10%, 20%, ... stimulus for a reason. These are the pre-gamma values. Your measurement software, e.g., HCFR, compares the measured result to the stimulus percentage after gamma adjustment.

The saturation patterns are trickier and are discussed in this thread (http://www.avsforum.com/avs-vb/showthread.php?p=15619419#post15619419), where Tom Huffman gave the values used to create the saturation patterns on the AVSHD disk. Essentially the inverse of a 2.2 gamma is applied to create the patterns, such that the points will measure with equal Y on a display with a gamma of 2.2, at the 25%, 50%, 75% and 100% distances from white to the primary. The RGB values are described there in the range [0..1], since that is what is used to calculate the YCbCr values actually recorded on the disk.

Thanks for the link, I will go see it.

In this area thinking in terms of RGB doesn't necessarily help you to understand things. When you're using an external source like a calibration disk, all that HCFR really does is to take some measurements, make some calculations, and then give some reports on what the meter measures. HCFR doesn't necessarily care what type of input (video or computer) you feed your display. All that really matters is if you measure the expected pattern.

Sure with video levels 16 is expected black and with computer levels 0 is black, but for HCFR that really doesn't matter because all it wants to do is to measure whatever the display produces for an input designated as black. Again for 10% stimulus HCFR really doesn't care what RGB level you're sending to the display, it merely expects to measure 10% of the encoded range from black to white. If you wanted you could even calculate point gammas yourself using http://www.avsforum.com/avs-vb/showthread.php?p=10469695#post10469695 and again RGB values wouldn't matter, all that would matter is the measured light for a given stimulus.


I'm not sure about it because I did the test. I calibrated my TV with AVS HD 709 (black level at video black without clipping BTB or WTW and everything else calbirated).

I then did myself gray patterns images in the 0-255 RGB range (10% increments) for calibrating the XMB (PS3 in full RGB range and my TV in full RGB range) and the HCFR readings were messed up, especially the gamma curve.

If it did not matter, my TV should still be calibrated when using the PS3 image viewer and my full range RGB test patterns(?)

Thanks for the answer, but I think I don't quite understand correctly. I will go point by point and give examples.

1- When I calibrate my TV for movie viewing (Blu-ray/DVD), I do that:

a-I set my PS3 video output to Y/Cb/Cr and SuperWhite to On so that BTB and WTW signals are not clipped at the source level.

b-I set my TV HDMI black level to extended (RGB 0-255 range) so that BTB and WTW signals are not clipped (or the grayscale remapped) and so that it is at the same black level as the source.

c-I use AVS HD 709 and HCFR software to calibrate (amongst other things, I will calibrate my black level at video black, even if BTB and WTW are not clipped, but not visible - so I set my brightness so that the luminance at 10% gray is 0,603% of peak luminance (100% gray - RGB 235-235-235 - so that my gamma is 2.22 at 10% gray)

d-These settings will be good when PS3 RGB range is set to limited(?)
This is not correct. When you are outputting video levels, you should have the display set to accept video levels. Ideally the display would not clip BTB/WTW, but many do.

You want to set brightness on the display at the point where BTB information is just clipped. It is preferable to have WTW information visible, but not essential—there shouldn't really be any information above 235.

If the display is well designed, these settings should also be correct for RGB, but may not be.

2- I want to have an other calibration setting for PS3 games and XMB so that I get the full RGB range of 0-255 (for the games that may support it) - I want my TV black level at computer black. So if I understand, I will do that :

a-I make myself gray patterns like this (using Paint, because I don't have Photo Shop :D):

0% gray = RGB 0-0-0
10% gray = RGB 25-25-25
20% gray = RGB 51-51-51
30% gray = RGB 76-76-76
40% gray = RGB 102-102-102
50% gray = RGB 127-127-127
60% gray = RGB 153-153-153
70% gray = RGB 178-178-178
80% gray = RGB 204-204-204
90% gray = RGB 229-229-229
100% gray = RGB 255-255-255

b-I set my PS3 RGB range to full (if I understand, there is no BTB and WTW information because black is at 0 and white at 255).

c-I set my TV HDMI black level to extended (0-15 and 236-255 are not clipped or the grayscale remapped).

d-I use AVS HD 709 and HCFR software to calibrate.
This should work just fine.

Here is my question : HCFR does not have a 0-255 RGB range mode when using external images, so that the readings will be messed up. At 100%, HCFR will be expecting RGB 235-235-235 and the gray pattern I will use will be RGB 255-255-255. So even if out-of-the-box my TV has the correct gamma for, say, 90% gray, HCFR will report that it is lower than 2.22.

So how can I calibrate with HCFR to have my black level at computer black?
It doesn't matter what the RGB values are. HCFR's range only affects internally generated test patterns if you were to use it as a signal generator.

If the display/source are set to "limited" then 235,235,235 should look identical to 255,255,255 when set to "full" as both are 100% white.

3- Other questions

When doing so, is there a compression process happening, in the sense that the grayscale is remapped and some information is lost. For example, if I have a 0 to 255 gradient, can remapping 0-255 to 16-235 introduces banding?

Is it the same thing when my TV HDMI black level is set to 16-235? I found out that when using AVS HD 709 and setting my TV HDMI black level to 16-235, BTB is clipped, but not WTW, and that also RGB 16-16-16 is remapped to RGB 0-0-0 and RGB 235-235-235 is remapped to RGB 255-255-255.

Why does the TV just can't clipped BTB and WTW without remapping the grayscale - and me setting the approriate black level with brightness?

Same thing with the PS3...

What is the difference between clipping and remapping?

Thank you VERY much for you assistance :)There is "compression" going on when you have PC level (0–255) data being displayed at Video levels (16–235) on the PS3, as there are less steps of gradation available. (255 vs 219) Potentially this could introduce posterisation (banding) if it's not handled well.

The levels are remapped, as it is a much better alternative to clipping. If you were to simply clip the data, you would lose all information below 16 and above 235, meaning that a significant portion of shadow/highlight detail becomes areas of solid black/white.

Heres a gradient with 0–255:
http://i42.tinypic.com/35n4za1.png

Remapped to 16–235:
http://i42.tinypic.com/28kh5cl.png

Clipped when converted to 16–235:
http://i43.tinypic.com/6ga9oo.jpg

I'm sure you'll agree that remapping is much more preferable to clipping. (and when the display uses 16 for black and 235 for white, the remapped image should look the same as the original 0–255 one, whereas the clipped one won't)

Thanks for the reply.

This is not correct. When you are outputting video levels, you should have the display set to accept video levels. Ideally the display would not clip BTB/WTW, but many do.

You want to set brightness on the display at the point where BTB information is just clipped. It is preferable to have WTW information visible, but not essential—there shouldn't really be any information above 235.

If the display is well designed, these settings should also be correct for RGB, but may not be.

I don't understand. You say that my TV HDMI black level should be set to Low (16-235) when calibrating for movie viewing (video black)? When I do that, the TV will clip BTB and WTW signals and the image gets darker, that's why I set it to High (0-255) so I can see the BTB and WTW information on the pluge pattern and then set brightness so I can't see the BTB bars.

Or maybe my TV low black setting is in fact 0-255 range and the high black setting is in fact 16-235 range (I have a 42LG50).

In general, should you see BTB info when the display is set to 16-235 or 0-255?

I want to add this to the pot.

When under XMB or anything besides Blu-Ray, DVD, or any video disc based:
RGB Limited (PS3) RGB Limited (TV): Levels are 0-255 remapped from 16-235
RGB Full (PS3) RGB Limited (TV): Levels are 16-235 from a 0-255 remapped from a 16-235
RGB Full (PS3) RGB Full (TV) Levels are 0-255 untouched
When using Blu-Ray, DVD, or and video disc based:
YCbCr Super-White Off: 16-235
YCbCr Super-White On: 16-255

^^ Don't have enough information about YCbCr but through experiments and grayscale patterns
that's what I've come up with.

I don't understand. You say that my TV HDMI black level should be set to Low (16-235) when calibrating for movie viewing (video black)? When I do that, the TV will clip BTB and WTW signals and the image gets darker, that's why I set it to High (0-255) so I can see the BTB and WTW information on the pluge pattern and then set brightness so I can't see the BTB bars.

Or maybe my TV low black setting is in fact 0-255 range and the high black setting is in fact 16-235 range (I have a 42LG50).

In general, should you see BTB info when the display is set to 16-235 or 0-255?
If you have video levels (16–235) going in, the display should be set to use them.

BTB information ideally would not be clipped, which means you can make BTB information visible by raising the brightness control, but for normal viewing it should be set to the point where it is just clipped.

If you have Video levels in and then use PC levels at the display and the brightness control to clip black, there's a good chance you're having a negative effect on the display's gamma curve.

I want to add this to the pot.

When under XMB or anything besides Blu-Ray, DVD, or any video disc based:
RGB Limited (PS3) RGB Limited (TV): Levels are 0-255
RGB Full (PS3) RGB Limited (TV): Levels are 16-235
RGB Full (PS3) RGB Full (TV) Levels are 0-255.
When using Blu-Ray, DVD, or and video disc based:
YCbCr Super-White Off: 16-235
YCbCr Super-White On: 16-255

...I thought thar I understood grayscale levels, but now I'm not sure :eek:

How can you test whether the TV or the source (PS3 in my case) is in 16-235 or 0-255 range or accept 0-15 and 236-255 in regard to the medium range?

I don't want any remapping, I want to adjust the gamma myself. So if a blu-ray/dvd has a 16-235 range, I don't want my system to remap it to 0-255 and if a PS3 game has a 0-255 range, I don't want my system to remap it to 16-235. I want to adjust things accordingly (but is it the right thing to do?).

For blu-ray and DVD, I thought that when BTB and WTW information was NOT clipped (calibration disc), my source and my TV were in the 0-255 range, that is, were accepting 0-15 and 236-255 information, which is not the same as remapping 16-235 to 0-255.

Is there ressources (books, articles, etc.) that I could read to understand the concept, because I'm not a pro like you guys?

Thank you very much for your input.

If you have video levels (16–235) going in, the display should be set to use them.

BTB information ideally would not be clipped, which means you can make BTB information visible by raising the brightness control, but for normal viewing it should be set to the point where it is just clipped.

If you have Video levels in and then use PC levels at the display and the brightness control to clip black, there's a good chance you're having a negative effect on the display's gamma curve.

Thanks for the reply.

When you say that the display should use video levels 16-235, do you mean that it must not remap them? If I set my PS3 and my TV so that BTB and WTW information on a blu-ray test pattern is NOT clipped (I can see them with brightness cranked up, but I will calibrate so that 0-15 is NOT visible), would you say that I have the right HDMI settings?

In that case, is my TV and PS3 in the 0-255 range?

For PC levels, what I understand is that I have to remap them so that the brightness I set for video levels is right (no black crush) or that I have to have another calibration setting that reajust the brightness so I can see the 0-15.

When you say that the display should use video levels 16-235, do you mean that it must not remap them? If I set my PS3 and my TV so that BTB and WTW information on a blu-ray test pattern is NOT clipped (I can see them with brightness cranked up, but I will calibrate so that 0-15 is NOT visible), would you say that I have the right HDMI settings?

That is correct.

n that case, is my TV and PS3 in the 0-255 range?

No, you're calibrated to the proper video range 16-235, but also maintaining the entire range 1-254 without clipping. However, values below 16 (black) are not visible when properly calibrated.

...I thought thar I understood grayscale levels, but now I'm not sure :eek:

How can you test whether the TV or the source (PS3 in my case) is in 16-25 or 0-255 range or accept 0-15 and 236-255 in regard to the medium range?

If your TV has a feature called RGB Range, HDMI Black, etc. then those are the one controlling the grayscale output. If you don't then probably have a TV with a level of 16-235. Then if you don't have a TV that supports RGB Full, the PS3 will remap that 0-255 to 16-235. Even if you have a test pattern that shows 0-255 (when the PS3 is set to RGB Limited and the TV is limited to 16-235) on the XMB the 0 value is really 16 and the 255 is really 235 because it was remapped.

If you want a test pattern to put on your XMB you can have it here:
http://www.pbase.com/jackcnd/image/75285378

I don't want any remapping, I want to adjust the gamma myself. So if a blu-ray/dvd has a 16-235 range, I don't want my system to remap it to 0-255 and if a PS3 game has a 0-255 range, I don't want my system to remap it to 16-235. I want to adjust things accordingly (but is it the right thing to do?).
PS3 will not remap a Blu-Ray/DVD to 0-255 unless you set it to RGB
(PS3 will remap it to 0-255 if set to RGB Full on a TV with 0-255)
PS3 always remap game from 0-255 to 16-235 level on a 16-235 TV.
PS3 will not remap game if set to RGB Full and TV has 0-255.

It's confusing as hell when you talk about it but it's pretty simple.

For blu-ray and DVD, I thought that when BTB and WTW information was NOT clipped (calibration disc), my source and my TV were in the 0-255 range, that is, were accepting 0-15 and 236-255 information, which is not the same as remapping 16-235 to 0-255.
Refer above.

Is there ressources (books, articles, etc.) that I could read to understand the concept, because I'm not a pro like you guys?
This is through trial and knowing what outputs what. PS3 outputs 0-255 on RGB Full. PS3 outputs 16-235 on RGB Limited. XMB, Games are 0-255 levels, DVD, Blu-Ray are 16-235 levels. Now when something is combined with something opposite, think of it getting remapped.

Thank you very much for your input.
You're welcome :)

P.S. I clarified my last post a bit.

http://www.avsforum.com/avs-vb/showthread.php?p=15619419#post15619419

This link might help. These are some locations for the saturation windows.

Dan