question about 709 colour space calibrating material

I've also been trying to decide what to do in this area. I would like to just calibrate my HD players with HD material, but none seem very suitable. I had been using my computer to provide patterns, but I found that doesn't calibrate the same as my HD players. When the ISF tech was here he said both my HD players calibrated the same, so if I can get a good disk for HD DVD I'll be pleased.

I have HD DVE and half the time it locks up my XA2. It also only offers 20% grayscale window jumps. If you want to do color measurements you basically have to move the meter around. On my TV that's a pain due to how much color variation my SXRD has. Compared to GetGray, the disk is a total joke for grayscale and color calibration. Of course the only similar patterns I've seen for Blu-ray are the Sony color bars, so overall it's not that bad, but for HCFR use it's rather lacking.

w6rz on a custom HD DVD seems the best option right now. The major item is that there's no gray pattern to match the solid color patterns in order to set color saturation. With a digital TV I could probably measure gray off the color bar and use full-field side measurements, but I don't know if that's an option with a projector. Really the only other thing I miss from my computer is above-white to tell when red runs out. After a lot of searching I've found what seem to be correct RGB to Jpeg Ycbcr converstion factors and Ycbcr to 709, but I haven't ran another MovieFactory disk yet to see if it comes out like I might expect and matches HD DVE.

EDIT: There are 5% gray patterns in 720p DVE HD as the following reply mentioned

I believe the HD DVE has 5% grayscale window jumps in the 720p section. These can be used in 1080p with n problems.

If your HD dvd player does what it's supposed to do it will twist SD 601 to HD 709 when it upconverts so you can use the getgray disk for both grayscale and color.

If it follows industry standards it doesn't matter, then again it might not.

I thought all the 1080p patterns were the same as 720p, so I'll have to check out the 5% HD DVE grayscale.



My understanding is that for color saturation you need to use the same percentage gray as the colors. All the w6rz colors are 75%, so to set color saturation by TomHuffman's guide I think you need a 75% gray.



I don't remember the exact tests I did, but on an early look I seemed to be getting somewhat different readings between getgray and DVE HD on my XA2. I would have to check again to be sure, but I think xyY luminance was one measurement that might have varied by over 10% between the disks. Because light output is one thing I had been intending to set, and because I rarely watch SD DVDs, that seemed reason enough to use an HD disk. I might get around to doing the grayscale on my XA2 today, so maybe I can post some HCFR files of what different disks show on my HD DVD player.

Because you're dealing with an HTPC, you might have it easy and not have to deal with what I'm seeing on my XA2. As far as I can tell on my computer, everything seems to get converted to RGB. As long as your playback program converts correctly and doesn't mess up HD conversion (by default Zoom player beta and The KMplayer seem to mess colors up and Vista WMplayer seems to get things right), then you can probably just use getgray and get almost the exact same results as an HD pattern such as w6rz. There might be very slight RGB differences due to how video is encoded, but with the better programs they seem to convert HD and SD to almost the exact same RGB values.

If you want to be completely sure, then you might want to check out the color pulling technique I learned from Arfster in the HTPC thread about the ATI 2XXX video cards and their driver expansion issue. The basic idea is that if you can capture an image of the screen, you can post it into a paint type program and pull RGB colors. I'm not sure if it's possible to screen capture with PowerDVD though, so you might have to do it with another program and then compare by eye to check PowerDVD.

Usually what happens with HD conversion problems I've seen on the computer is that in one way or another HD will have higher RGB values for similar 75% or 100% HD patterns than SD. Generally 75% will have RGB values around 180 unless you're expanding to computer levels, and if something gets messed up they often get shifted higher (around 192 or so for unexpanded levels). That sort of change is enough that that I can comparatively pick it out by eye if I have both colors on screen at the same time.

If you can't pull colors with PowerDVD but you know another program works correctly, for example Windows Media Player in Vista seems right, then open the same pattern in both programs and check by eye that colors match. If you're doing this then you'll probably want to set PowerDVD to match the system and avoid otherwise adjusting colors in PowerDVD. Due to the huge following the program seems to have, I would expect that it probably gets things right. If you want to check though, computers are nice because you can tell what RGB values you're looking at and it's easy to create your own patterns if needed.

There's really not enough information there. If you're using w6rz patterns, do not use the 601 patterns. For SD you can use GetGray. For HD you can use the w6rz 709 patterns or HD DVE.

The impression I get is that on a computer everything gets converted to RGB. Both the playback software and the video card drivers can alter RGB values. I don't know about your video card, but I know a 2XXX ATI card would cause different 601 and 709 results without modified drivers.

If your meter is still showing different values for SD (not 1080p 601) and HD, then I would check RGB values. Playback a HD pattern with WMP, press print screen, then paste into a paint.net type program. If you use the dropper tool and then look at color you can see your RGB values. Unexpanded 100% gray should be aroung 235, and unexpanded 75% gray 180. Black would be 16. Colors will be a mix of those values. If you're expanding the signal then 235 goes to 254 and 16 to 1.

PCs are both easy and hard to deal with. Calibrating from the DVI port to the projector can be done using bitmaps from the basic file viewer included with Win2K and later. You then need to figure out what your playback app is doing, which can be much harder. I use a "color sniffer" (these are usually for web design) to diagnose these issues, specifically ColorCop from Datastic. It's free. It works -- but only using VMRx, not overlay. However, figuring out where in the software the problem lies can be a real PITA (is it the playback software? is it the driver?).

We are going to release a free pattern generator app for HTPCs that will be a bit more comprehensive than what is currently available once we get the last few bugs ironed out. It won't solve the software issues I mention above, but it will ease the calibration from the TMDS chips to the display.

Bill

In RGB terms, they should. If your desktop is at something greater than VGA, then the playback app should assume that the signal will be interpreted as HD, and it should "twist" the Rec601 to hit the appropriate coordinates inside the Rec709 gamut.

Bill

Just calibrate using HD patterns with 709 color. If that means getting HD DVE, then buy it. All the necessary patterns for grayscale and color calibration with HCFR are there in 720p.


Additional information:
If you already own GetGgray, you can run this test to see if it measures the same as HD material.
1 - Measure the grayscale and measure colors with GetGray. Normally you would use 100% colors for HCFR primary and secondary measurements, but for this test instead you need to use 75% colors so you can compare with w6rz.
2 - Save your file.
3 - Now measure the grayscale and colors with w6rz. Use 75% 709 color patterns.
4 - Save as a new file.
5 - Now if the two measurements for GetGray and w6rz are similar, then you can just use the GetGray disk for everything. If they're clearly different, then buy HD DVE.

Pleas note that if they measure the same, then this means something is wrong inside your playback software. The Rec 601 (i.e., GetGray) material should be measurably inside the triangle produced using Rec709 material if things are working correctly.

Bill

Bill,
Can you explain how an upconverting player "twists" the space? I was under the impression that Rec 709 simply moves the primaries in relation to Rec 601.

Lets say the upconverting DVD player encounters a frame with pure red Rec 601 primary. Shouldn't it "remap" that color to the red primary as defined in Rec 709?

If that is the case, then wouldn't alluringreality's propsed check work correctly (assuming we are using an upconverting player to calibrate for HD material)?

Please correct me if possible (I may be too far off for correction as a result of me being an idiot )

Honestly I've only done the GetGray vs HD test on my standalone player. It doesn't match there. GetGray has lower Y values on colors. That test was just how I would go about the same on the computer. I expected they would match based on what I've seen from my computer, but maybe that's incorrect. My assumption was that everything on the computer gets put into RGB and I've seen similar RGB values for SD and HD so I'm not quite sure why they wouldn't match but I haven't actually checked. All I know for certain is that Zoom Player's behavior doesn't match WMP's behavior on my computer.

It ought to be a relatively straight-forward matrix multiplication (3x3 matrix). If they do anything differently, then they either know way more, or way less, than I do. Somewhere in Charles Poynton's website, he shows what that matrix is (I could calculate it quickly, but formatting the table is more effort than it is worth).


That is essentially what happens if it doesn't do the "twist". What you want it to do is to desaturate the "pure" (HD) red so that the color red that is produced is the correct shade of red (SD) that was intended in the encode.

In other words, the Rec709 red primary is located at xy( 0.64, 0.33). For SMPTE-C (effectively Rec 601), red is located at xy( 0.63, 0.34). This means that the HD red is more saturated than the SD red in both x and y. With SD content, though, you want that "pure" SD red, and all shades of red in-between, to be based upon the SD red primary, not the HD primary. If you don't, then ALL reds would be somewhat incorrect not just a pure red.


Not an idiot. This stuff is just a little complex and almost completely undocumented in the enthusiast space. The interesting thing is that once enough users have calibration equipment and decent meters, we ought to be able to tell as a community who is doing it right and who is not. Then manufacturers will have to pay more attention so that they don't miss a fairly trivial step in the process.

Hopefully this little bit helps.

Bill

Thanks a lot Bill, very informative. Let me make sure I have it straight.

We have Rec 601 material encoded on the DVD, but the TV is expecting Rec 709 because we are feeding it a HD signal (from the upconversion). Because of this the DVD player has to shift the digital "red" as it is encoded on the disc to agree with the Rec 709 colorspace. The resulting output should conform to the Rec 601 standard on the TV screen, namely xy( 0.63, 0.34), which is inside the gamut defined by Rec 709.

If the DVD player does not convert correctly, then the TV sees digital "red" and interprets that to mean Rec 709 "red" at xy( 0.64, 0.33). This results in the final picture being slightly oversaturated and off hue.

So now that I understand (or think I understand) what is happening, what are we actually doing when we use Avia or other SD discs to calibrate with an upconverting player? Are we calibrating the Rec 601 primaries/secondaries correctly within the Rec 709 space? In which case the Rec 709 primaries/secondaries would perhaps fall into place as well provided the TV has an accurate color decoder and gamut?

I hope the way I asked those questions is making sense

First thing, his answer above is slightly confusing. There are two different things going on. One is the primary chromaticity difference that is called for in the standard, which has to do with the location of the primaries, which are slightly different between SMPTE C and 709.

The second problem, the one you are dealing with, is the matrix coefficients between 601 and 709 that move between YCbCr <-> RGB. Whether or not the matrix flow is correct, you also need a display with the proper primaries, and these are distinct questions.

As for your question with regards to upconverting dvd players, it depends on the dvd player. Hopefully it twists to 709 if it is outputting component. Most displays will just apply 709 to decode component if it is an HD resolution. Some displays allow you to choose the matrix manually, and if you have an upconverting player that is not twisting to 709, this can be advantageous since you can then force a manual decode in 601 for the HD input and get the right colors. Remember, if you output RGB, say from DVI, or from HDMI set to RGB out, then the decoding is going on in the player and the display is not involved since it is seeing already decoded RGB. This is also discussed briefly in my guide, linked in my sig.

But again, the issue of primary chromaticity and color decoding are two different questions, and its easy to confuse the two together, but they are distinct.

>Let me make sure I have it straight.

What you described and what Bear5k said generally appears to match what I've read. Although that doesn't seem to match with my XA2 or computer measurements off my SXRD. My SD and HD measurements of red, green, and blue from the standalone player shows no xy differences. The individual xy positions always matched in measuring primaries.

My HD DVD player output appears to hold xy points between SD and HD. What varied with my HD DVD/upconverting player is Y. I'm not too concerned with SD so I didn't closely look at the relative percentages, but my impression was that if HD was set to match gray then SD colors would have been undersaturated in relation to gray. Basically the normalized Y for the HD primaries was close to 1, and it seemed red, green, and blue Y for SD would clearly add to less than 1.

> what are we actually doing when we use Avia or other SD discs to calibrate with an upconverting player?

What matters is that you're calibrating for the source. So if you're using an SD disk, then you're calibrating for upconverted SD. I care mostly about calibrating for HD, so that's why I'm going to use an HD disk with my HD DVD player.

I have not tried changing my DVD player controls, but in using TV controls on my SXRD I get no impression that my XA2 is doing any xy alterations for primaries between SD and HD. I have seen my computer introduce other colors into pure red, green, and blue color bars, so I suppose that could cause a shift in xy. Otherwise I'm not sure I've seen much of anything that might indicate changing xy measurements. Although I'm becoming familiar with matrix calculations for changing colorspace types so that I can more easily use HCFR information in a spreadsheet, so far I haven't seen much that I've been able to clearly explain the behavior I'm seeing from my standalone or computer with such things as changing portions of red, green, and blue along with different xy positions for primaries.

Correct. That is an attribute that is physical to the display. Unless the display has an ability to move its primaries around in a CMS system or the like, they will not change.


Again, I think you may be misunderstanding. The output has no relevance to the display's chromaticity. Only the color decoding. The color decoding does not affect the location of the display's primaries at all.


Again, the matrix decode doesn't have anything to do with the xy location of the display's primaries. That's a distinct and separate question, and is unrelated to whether the color decoding is correct or not. The issue of accurate color decoding is more significant than whether a display's primaries are 709 or SMPTE C, which is of minimal significance.

No, you have a mistaken understanding. What the player outputs has no bearing on what the color of the primary is. All it does is output a signal that indicates say, pure 100% red, which in video RGB would be say (235,0,0). The display will then turn on the red 100%, and everything else 0, and whatever the color of that display's red is, is what color that will be.

Getting that RGB triplet from YCbCr accurately is just the function of the color decoding. It can't do anything for what the display's primaries are. Two different questions.

Thanks a lot Chris. I'm new to the calibration scene, so I'm trying to learn (I didn't mean to hijack this thread a bit, I apologize). That clears things up a lot.

What you are saying is that the chromacity of the primaries for a given TV are either set to the HD (709) or SD (601) standards and can't be changed unless you manually change them using a CMS. A set without a true CMS has little or no capability to change the chromacity of the primaries, you are stuck with them the way they came.

What we can change is the luminance of the primaries, which is what is indicated by the RGB triplet and is controlled by the color decoder. That leads me to another question.

Since the chromacity of the primaries are set for a given TV, when we talk about all this colorspace twisting we are really talking about how the DVD player or TV decodes the RGB luminance values from the component signal, right? And the only reason the decoding is different from SD to HD is the fact that the encoding was also different, such that a certain value for a component signal corresponds to a different RGB triplet depending on which set of equations were used, correct?

Do you have the equations? I did a quick Google search but didn't find them.

Thanks a lot for your help (and Bear5k), it has really improved my understanding.

I'm just trying to figure out what I don't get. It's quite possible there are some big lapses in my understanding considering how long it took me to get XYZ to RGB conversion in excel to match HCFR last night. What got me thinking was the comment about the computer test "if they measure the same, then this means something is wrong inside your playback software."

My impression has been that correct RGB decoding on a computer will have similar SD and HD RGB values. Is that correct or incorrect? By pulling colors some software does that and some software adjusts RGB values with HD material. HCFR seems to show that regardless of SD 601 or HD 709 colorspace, colors should have the same RGB values on a computer. I can't come up with a reason why the computer would be using the same RGB values and producing substantially different measured results between SD and HD material. If they shouldn't be similar RGB values for 75% SD color bars and 75% HD color bars, then can someone clue me into what sort of RGB values I should be seeing for test patterns on my computer between SD and HD material? Can anyone explain why HCFR appears to use the same RGB colors if that's incorrect? If colors with the same RGB numbers somehow measure differently, why is that?

Are any content producers even using the wider gamut for color correction or is it all based off of SMPTE C primaries anyway?

http://www.poynton.com/PDFs/coloureq.pdf and http://brucelindbloom.com/
seem the most thorough information I've seen about color conversion online.