xvYCC calibration

I see a lot of confusion about xvYCC around the place. But kit is now coming on the market that supports it (or claims to), so could we dispel some of the confusion by producing proper tests for it?

I've not come across any tests, but I believe we should have the ability to exercise it in AVS HD 709. Certainly my Panasonic DMP-BD60 and TX-P42V10 claim to support it, so let's put it to the test.

The important points:

• HDMI 1.3 supports xvYCC
• AVCHD does support xvYCC
• Blu-ray and DVD don't support xvYCC
• Due to the way xvYCC is implemented, I think it likely that some DVD/Blu-ray kit will pass xvYCC data unintentionally through HDMI YCbCr (in some modes) if encoded onto the disc.

We should be able to produce fully-legal xvYCC test patterns for the AVCHD version of the AVS HD 709 disc, and dodgy ones for the Blu-ray version. I'd be very interested to see the results, and test it on a colorimeter.

Is anyone interested in working with me on this? I can help with the colour theory, getting hold of the IEC 61966-2-4 standard, and the maths to produce the underlying YCbCr data for encode. But I'd like work with someone from the AVS HD 709 team to do the final encode and authoring. In particular I want someone who knows about AVCHD.

And a final note - xvYCC isn't just marketing snake-oil. There is a sound technical reason for it, and it's a good idea. But we need tests to confirm that it's being done properly, and we're waiting for Blu-ray to figure out how to add it in a backwards-compatible way. Other formats like AVCHD that support it are in use now, so get them tested, and distinguish marketing spiel from good kit.

Two handy links:
http://en.wikipedia.org/wiki/XvYCC
"New" Extended-gamut Color Space for Video Applications; xvYCC (IEC61966-2-4)

That's certainly a start. That's testing higher intensities of the standard primaries, yes? To some extent, that's not really exercising the new facilities in xvYCC, it's been kind-of-unofficially been something that people have quietly done in standard YCbCr - RGB values outside the 16-235 range convert to those values.

But as xvYCC formally endorses that practice, lets do it.

I don't think there's any reason to limit yellow for a test pattern - Y' is allowed to go beyond 235. And likewise, I'd be inclined to make the top white bar be maximum too (254=109%).

Some displays may conceivably collapse that overbright yellow to white, but that would be part of the test.

I would be inclined to have one more step, and smaller increments - say 95%, 100%, 105%, 109/110%.



The next step is to test increased saturation - actually producing new colours outside the standard Rec.709 gamut.

Not sure of the best way to approach this - just doing 95%, 100%, 105%, 110% saturation of the standard primaries isn't going to be terribly effective, as that'll just hit the limits of the CIE colour space pretty fast.

I suggest maybe selecting a new set of monochromatic primaries - maybe 630nm, 540nm and 450nm, which I've seen suggested for laser display systems, and producing a set of saturation test bars that gradually approach those colours - starting with the best that standard Rec.709 can do, and working outwards:

Attachment 150071

(Diagram borrowed from Wikipedia, and scribbled on). The diamonds are the new primaries and secondaries, and we produce test bars with different saturations of those - a few steps along the radial lines.

The catch is that although xvYCC can reach basically any colour in the CIE diagram, its luminance is more limited the further out it gets. My calculations say that we'd need to use 70% colour bars for those primaries.

Another test would just be an encode of that CIE diagram - alternating between normal and extended-gamut renditions.

But one issue is that HDMI xvYCC requires that the gamut be signalled by the source. That implies that there must be some way of encoding the gamut in AVCHD so it can be passed out through HDMI, and we have to do it. I'd guess it's embedded in the H.264 video stream, but we need to find out how that's done and whether the encoding tools available can do that.

Hi Doug!

The point won't be so much to calibrate the xvYCC space, as to confirm that it's working, and to see how wide a gamut your display has. Display gamut could be probed with a colorimeter, and that information could be reported by reviewers. Calibration will be best performed with normal 709 data.

I'd also like to see some A/B comparison animations - image with xvYCC, image without.

So I guess the title of the thread's wrong. It should be "xvYCC testing".

A display saying that it "supports" xvYCC doesn't tell you much. That just implies that it can show some colours outside 709, but it doesn't tell you how many.

It also doesn't tell you how well the display handles xvYCC colours outside its gamut - for a non-xvYCC display there's a clearly constrained set of colours coming in, and it's no big deal handling all of them - they'll all be within its gamut. But soon as a display tells a source "I handle xvYCC", the source could throw it almost any colour outside the display's gamut, and the display will have to have gamut mapping to cope. Maybe a good quality test for that could be devised.

And the xvYCC data can be lost in various places:

1) A repeater (ie A/V receiver) could fail to pass xvYCC metadata packets upstream or downstream, either disabling xvYCC (because an HDMI source isn't supposed to send it without permission), not telling the display that it is xvYCC, or losing the gamut mapping. Particularly likely if any sort of scaling is enabled.
2) A repeater could simply clip "out of range" YCbCr values, or lose them due to some intermediate processing that was performed in RGB space.
3) RGB data could be being sent instead of YCbCr, either due to user selection or a negotiation glitch.
4) The source or display may not support xvYCC properly or at all, even if it claims to. A metadata error may stop it being sent.

Some simple tests would immediately show any of those problems.

As for "special players", well xvYCC is not officially supported by Blu-ray, DVD-Video or HDMI 1.2 and earlier. In practice, I suspect these will often dumbly pass the out-of-band values in the video stream, but the display may not handle it well, as it won't have any of the metadata required by HDMI 1.3. The display may not recognise it as valid xvYCC, and may clip the data itself, out of caution.

The two main ways of getting xvYCC data in at the minute are AVCHD from certain camcorders, and EXIF JPEGs from certain cameras. The JPEGs should form part of a separate test - many new displays can display JPEGs from SD Cards, but do they show the extra colour information in sYCC photos, or just clip to sRGB?

I agree this is a test that's a little ahead of the time in that there's no commercial sources yet, but there are some sources, and there's kit on the market with facilities that can't be tested.

I think it likely that many xvYCC displays or A/V receivers bought now will still be in use by the time a future Blu-ray extension supports xvYCC and xvYCC discs start appearing.

I for one would like to know whether a receiver I was considering could pass xvYCC properly.

Where are you getting that from? I've got IEC 61966-2-4, and that explicitly endorses Y'>235 for "specular" elements. See Annex A - it says the value range for Y' is limited to -0.068 to +1.09, ie 1 to 254.

It says that "surface colours" should be in the range 0 to 1, but this is a test signal, not a photographed surface.

Besides, if we're not allowed <16 or >235, you'll have to revoke most greyscale tests for being illegal. Plus an awful lot of commercial discs - >235 is indeed commonplace for specular highlights, apparently, according to what I've read on this forum.

Yes, I've seen that. That does say that you're allowed Y > 235 (so agreeing with me as far as that goes), but it is also an assertion that Y >= 235 means "white", with an implication that Cr and Cb must = 128 if Y >= 235. I don't believe anything in the formal xvYCC standards actually says that. Nothing prohibits you from asking for a tinted white - having Y = 235 and a bit of chroma added. That allows you better encode something with a different white point. And that would extend to slightly coloured specular highlights with Y > 235.


To get outside the 2D Rec.709 xy gamut, yes. You can also broach 3D gamut brightness limits with over-high RGB, as your test pattern does.


No, that's not it. The "negative" RGB is really just a mathematical trick. It's not like the "sub-blacks" you can see by boosting the brightness. It's not done with a level difference.

If we wanted to specify an arbitrary colour, we'd do so in XYZ. (Digital cinema actually does this - they compress XYZ data instead of YCC). The XYZ->RGB transformation is well-defined, as is the RGB->YCC transformation. So when we want a colour, we can go XYZ->RGB->YCC. But on the way through, we've always limited colours to what can be expressed in RGB, using Rec.709 primary colours.

If, in that intermediate step, we just ignore the RGB limits, we can encode an XYZ value in YCC that is beyond Rec.709 RGB. A display can perform the reverse YCC->RGB->XYZ transformation to figure out what colour we wanted.

An RGB display with Rec.709 phosphors could not display those colours. To get a super-green it would have to mix "green minus red minus blue", which makes no physical sense. Light doesn't work like that.

But the point is that that modern RGB phosphors (or transmissions through LCD) are often more red, green and blue than Rec.709 primaries. When displaying Rec.709 data, they have to mix them a little to get the colours right. So when asked for Rec.709 green, they add a little blue and red to their green to reduce the saturation back to Rec.709 standard.

These displays could display a more pure green than Rec.709, if only there was a standard way of asking for it. xvYCC is a way of encoding such colours. A modern wide-gamut display could handle this "green minus red minus blue" by not mixing in the red and blue it normally would for Rec.709 green (and adding back some green to keep the level right).

The main problem is that by not limiting ourselves to colours based on Rec.709 RGB, suddenly there's no limit on what we could ask the display to do. We could ask it for a colour well beyond what it could support. We could ask it to "subtract" more than its phosphors allow. This imposes the need for the display to have an algorithm for mapping daft requests to something it can handle.


Yes, the Panasonic TX-P42V10B. It has various non-standard ways of showing extra-gamut colours. Some modes are more saturated than standard, and there's a "Digital Cinema Colour" option, which I believe just unilaterally decides to use DCI P3 primaries instead of Rec.709. Both of those are of no interest to anyone who cares about proper calibration.

It also claims to support xv.Colour, which should allow show extra-gamut in a well-calibrated fashion. But I've no current way of testing.

My Panasonic DMP-BD60 player which came free with it supports xv.Colour and AVCHD.

That's a so-so analogy, and the technical description of xvYCC is wrong. You can't actually pass xvYCC data to a non-xvYCC device and get sensible results. To do that there would have to be a core Rec.709 image, and some separate "super-colour" extension that would slip by unaware devices without them noticing. That's not how xvYCC works in current implementations - there's only one image, and it will contain both in-gamut pixels, which will be encoded the same way as Rec.709, and out-of-gamut pixels, which will use previously out-of-range values.

If you send xvYCC data to something that's not expecting it, weird stuff will happen. When they get the out-of-range pixels, they could do anything. Well, they wouldn't explode, but the result may well not be visually pleasing.

That's why they can't just put xvYCC data onto a Blu-ray disc. They could have done, if Blu-ray players had been required to be xvYCC-aware from the start. Then they could have relied on all Blu-ray players to sensibly deal with non-Rec.709 colours on non-xvYCC outputs. I believe AVCHD did have it in from the start, so they can use xvYCC on it.

Once something is xvYCC-aware, then there's no essential difference for it between handling xvYCC and Rec.709. The backwards compatibility is that any colour that is in Rec.709 is expressed the same way in xvYCC709, so any Rec.709 data is valid xvYCC709 data. But not the other way round.

Here's a specific example: lets say we had a cyan saturation ramp, going from 0% saturated cyan, up to 100% saturated Rec.709 cyan, then a bit beyond (all equal brightness). You'd see these colours in the image:

Colour	x	y	X	Y	Z	R'	G'	B'	Y'	Cb	Cr
0% cyan	0.313	0.329	0.421	0.443	0.483	161	161	161	161	128	128
25% cyan	0.291	0.329	0.392	0.443	0.512	141	166	166	171	131	115
50% cyan	0.269	0.329	0.362	0.443	0.542	116	171	171	159	134	100
75% cyan	0.247	0.329	0.332	0.443	0.572	84	176	176	156	139	81
100% cyan	0.225	0.329	0.303	0.443	0.602	16	180	180	145	147	44
110% cyan	0.216	0.329	0.291	0.443	0.614	-22	182	182	139	151	24
120% cyan	0.207	0.329	0.279	0.443	0.626	-43	184	184	135	155	12

When a non-xvYCC device saw that YCC=135,155,12 value, and translated it into RGB=-43,184,184, it could do anything. If we were lucky, it might treat it as 0,184,184, which wouldn't be too bad. But it could just as easily overflow a calculation and treat it as 213,182,182, and we'd get a lovely shade of pink.

A reasonable thing to do might be to map it back down to 100% cyan, RGB=16,180,180, but that relies on having a device that knows how to intelligently map extra-gamut colours.

Now, if they do eventually add xvYCC to Blu-ray, they will have to come up with the sort of "core+extension" scheme you suggest for xvYCC on the disc, to keep compatibility with existing players. It could certainly be done fairly easily. Non xvYCC players would ignore the extra data, and xvYCC players could add it in to generate xvYCC if the display supported it. But I don't think such a scheme exists yet.

(And on the DTS-HD Master Audio/TrueHD analogy - yes in DTS-HD Master Audio there is a DTS core which the player can simply extract, but there isn't a Dolby Digital core in TrueHD. The Dolby Digital output is a real-time encoding of the TrueHD data by the player. Dolby TrueHD isn't based on Dolby Digital).

Well, like I said, there is a lot of confusion out there. I wouldn't believe everything you read (especially in marketing). I'm going from the actual standards documents: ISO/IEC 61966-2-4, CEA-861D, HDMI 1.3.


Yes, it will be discarded by the camcorder, or whatever plays the AVCHD output, because they know about xvYCC. The HDMI specification says that the source can't feed xvYCC to the TV, unless the TV declares that it knows about xvYCC.


Different sense of "compatible". Yes, any conventional RGB content is encoded the same way in xvYCC. And that makes it easy to reduce xvYCC to 601/709. But that's not saying anything about how non-xvYCC-aware devices will respond when fed extra-gamut colours.


That statement's a bit too strong, if by "upwardly compatible" they're trying to suggest that they can fling xvYCC data to non-xvYCC devices. But then, it is a marketing-type document. Even if you could argue that devices wouldn't respond too badly (and that is probably the case for most), and maybe you might do that with xvYCC in some other applications, the HDMI 1.3 spec has explicitly ruled out that behaviour. No passing of xvYCC data to non-xvYCC displays.


This isn't an issue, because an HDMI source is not permitted to send xvYCC data to a non-xvYCC display. The source is required to reduce it to standard 601/709.

That's because there's no way for a display to not "use or process" the negative values. They would receive pixels with those values, they would process them and as there's been no prior agreement on what they mean, you don't know how they would process them. A display can't just "skip" pixels with negative RGB values - they'll do something.


Well, that problem does exist. If xvYCC was as compatible as you thought, then:

1. you wouldn't have needed support for it in HDMI 1.3 - they'd have just flung it across, without any changes to the spec. But no, they changed the spec, required xvYCC to be properly flagged, and prevented sources from sending it without prior agreement;
2. we'd see Blu-rays with it already - there'd be no reason not to just put xvYCC data on the disc. But they're not. If for no other reason than doing so would cause existing players to violate the HDMI spec (see point 1).

Now, certainly we can violate the specifications, produce Blu-rays and DVDs with unflagged xvYCC data, and fire them at non-xvYCC displays using non-xvYCC players to see what they do. I was intending to do that, as the results would be interesting. I suspect most will cope reasonably well.

But what I really want to do is produce some properly-flagged data for xvYCC-aware devices.

So, back to the original point of the post - I wasn't looking for people who were confused about xvYCC, I was looking for people with some actual concrete technical information - mainly info on AVCHD, and how to properly flag xvYCC content for it so we get valid HDMI output.

Anyone?