Setting contrast with Spears and Munsil disc - Page 5

And Final Cut Pro manual.

In fact, 8-bit Y′CBCR video does not use all 256 codes to represent picture information. Black is stored as code 16 and white is code 235. Codes 1-15 and 236-254 are retained for footroom and headroom, respectively. These areas allow for quick spikes in the signal caused by filtering in analog-to-digital conversions and, in the case of white levels, can prevent clipping for highlights that may exceed 235 (white). Levels above 235 are sometimes referred to as super-white levels.

http://www.mastersofcinema.org/reviews/onibaba.html

Here's an interesting comparison of two masterings of "Onibaba" with pictorial examples. The comments on "superwhite" are interesting. Their assertion that the master that doesn't range out of 235 is doing a good thing is slightly misguided in my view even though they actually view it as a positive attribute of the master that does range above 235.

In trawling about I find lots of blunt assertion from pro/semi pro practitioners that video is 16-235 and everything outside that is "illegal". However if you read the standards for rec.709 and bt.601 what they continually state is that 236-254 is defined as headroom/superwhite/excursion.

(rather than just latching on to the obvious numbers 16-235 which is where I think a lot of this confusion stems from...people like absolute numbers)

Even if you ignore the fact that YCrCb maps higher into RGB and look just at the numbers in the spec sheet the chroma planes range from 16-240.

So that leaves us with 3 possible conclusions: ( this is referring to where the values end up on your display visually)

16 is black 235 is peak white and anything else is a bad thing.

16 is black and 254 is peak white and you should show the lot.

16 is black and peak white is 240 because the chroma planes range above 235 ( ignoring the mechanics of how YCrCb maps into RGB..someone else can do the maths I tend to fall asleep...I can't help it).

Last but not least what does this mean in the real world?

Watch some material with your display setup for peak white at 235 and then watch it at 254 and see what you think and post your observations.

I'm sure we can manage to do this without having a massive punchup if we remain a little open minded.

Things we think we know about video:

black ...16

ideal display gamma in a dim environment 2.35
(assuming a range between 2.2-2.5 depending on viewing situation and display performance)

Grayscale ...D.65
(B/W notionally justifiable as D.54 from a preferential/historical perspective)

Gamut... (according to standard and display capability and metering/adjustment capability)

white... see above (personally I'm sticking with 254 until I'm persuaded differently)

I agree with you. I like to use 16-235 is possible but I still want to see upwards of 250ish on the contrast scale. I have a denon 4310 that if set to 0 contrast it will clip down to about 237-238. I have to set the contrast at -2 for every input to see 250-254.

Yes the white bars at the bottom of the image shown in http://www.spearsandmunsil.com/artic...stcontrol.html

As I quoted earlier CEA is pretty clear on this and things that clip at 235 are almost certainly wrong but I believe it's not a completely unreasonable reading of 861-D. By that standard data above reference white is not bad but may be optional.

I thought that when S&M looked at discs they found lots of pixels both below 16 and above 235. But I also think the percentage of such pixels is low (the max. intensity histograms in the MoC article don't help here). It may also be worth noting that "bad" pixels are found in hard matted borders which certainly points to defective transfers.

I don't know if anyone has done a similar analysis of BD. I'd find that much more interesting than the DVD numbers.

The question there is whether the percentage is low because its an error or whether its down to the fact that not every shot ( not even the majority) will have picture content that has areas that warrant by their nature excursion above 235.

This use of the word "occasional" in lots of the standards descriptions seems to infer unimportant which I don't necessarily agree with nor even suspect that was the original writers intent: ie that because its occasional the inference is you shouldn't worry about hanging on to it.

Its "occasional" because its not there in every single shot which is perfectly normal given the image content usually depicted.

Certainly the S&M examples seem to show the majority of the 235 "excursion" in areas of the image that you would reasonably expect to find it rather than lurking around as individual pixels.

What might be more useful to know is how high it tends to go. Given that if you don't want to calibrate for 235 but you don't want to hit 254 either what is a reasonable point to hit to maximise display contrast.

Here's a method I've used over the years:

1) Capture a frame direct from framebuffer from mainstream, telecined, consumer material at StudioRGB levels. Ideally full-frame and lossless. Use this as example A.
2) Using image A, create image B by applying a clamp at BTB/WTW.
3) Contrast and compare the A/B images in a A/B manner. Observations?




I've done this and posted such A/B images here dozens of times over the years...
Dave

Here's some stuff I've grabbed that was lying about. Its supposedly off D1 originally but I can't vet the transfer so don't take it as gospel.

native:




235 clamp



difference

My point regarding percentage was to speak to error introduced by clipping. If the percentage is sufficiently small then visual error (some sort of perceptually adjusted RMS difference between clipped and unclipped) may also be small. Or not. I'm just speculating.


I don't think this is it at all. The standard clearly establishes the expected range and but observes that values above that range may occasionally (but legitimately) be present. From an engineering perspective this means don't be surprised if you find these values in the data stream. And particularly don't do something ridiculous (e.g. the DCD decoder in my AVR will crash given certain values -- epic fail).

I'm not sure this is totally fair. I noted using mainstream, consumer material. D1 isn't mainstream consumer. You have a somewhat unique position if you have access to D1 RGB while the average Joe only gets the 8-bit YCbCr that it the final product. Most don't have access to this pre post, intermediate material.
What title/timestamp is this from?
Also, jpg is lossy. It will change the levels. I'm sure I would noted such dramatic differences as I would expect the PS3 swhite ON/OFF twiddlers would.

Dave

Normally I ignore posted images but the delta frame is just the ticket. In this particular case the sun clearly looks different, the clouds and the flare not so much (on my laptop).

However that said I'm not sure either version will change my emotional response to the larger film. Much like knowing the Hoth exteriors were shot on a Norwegian glacier adds no emotional impact.

I think it's fair to say that on a lot of content the impact may be rather minor, or not the end of the world. But it certainly is not ideal, and in quite a few cases can definitely be visible.

Also, as Mr D. already pointed out, paying attention to what % of the pixels are a certain value can be very misleading. If you look at what % of pixels are 235, you'll likely find that to be small in many instances, but that doesn't mean it isn't important. It becomes a kind of slippery slope argument, however, but what percent of the pixels should we be okay with clipping off? The entire question is, in my mind totally pointless, because it's far simpler to say: preserve everything unclipped. It isn't hard to do, and there is no reason for really getting into a discussion of how much you want to clip off, or what % of pixels you are okay with destroying. Nothing good comes with that, so why not just say: preserve the original image the same as it was in the studio?

There are many things that damage the video, and some of them may be slightly obscure and not the most obvious to an average viewer, but that seems like a lowest-common-denominator excuse to me.

By the same token we could say that the chroma upscaling error (the chroma bug) is also a rather insignificant problem. And honestly, most people would never know, just as most people would never know if peak whites are being clipped off. But it still does visual damage to the image, and for people who do care, that makes a difference. And at the end of the day, it should be handled properly, and a properly designed device will handle it properly because there is no advantage at all for clipping them off.

I have to agree with this; perhaps if someone could point to a specific scene in a specific movie (DVD or BD) that shows such a difference I would be more inclined to believe that there is a difference in the real world, not just on test patterns designed to show up to 254 (and other video formats I don't use).

Calm yourself I never said it was. Let's just se what we can figure out.

No kidding, that's why I was constantly fiddling with super-white on this movie!

For what it's worth, it was kind of fun using the frame reverse/advance controls on the PS3 to capture the muzzle flashes of gunshots. The flash of on some of these was only a single frame long and the size and brightness of the flash was quite impressive on freeze frame. Something you'd never notice without isolating a single frame.

This is perfectly reasonable.

Let's see what we can figure out guys without it dissolving into an argument.

Mr. D-

I measure peak whites in your original D1 image @254,254,252. This is pretty hot for material that has a reference target of 235. Certainly MUCH higher than anything I've measured with consumer material in the past. My waveform monitor would have flagged this as a limit violation bigtime. Might just as well be sRGB

Can you tell us what this is from as in studio/title/timestamp? If this is from mainstream material, I'd be interested in checking it out further and making measurements to see what levels got encoded and released into the wild for those of us in mainstream consumer land.

Dave

Its stock footage , generic vdeo footage. I cannot tell you anything more about its provedance other than it originated off D1 . I did not put it on the D1 I did not take it off the D1. I am offering it up for discussion not to prove a point.

Instead of pulling this particular image apart ( and its fairly indicative of what I regard as the whole 235 vs 254 debate) keep looking and we'll try and come to some sort of conclusion.

OK. While it's a data point, it doesn't seem to be a data point that is very germane to me. I can't attempt to duplicate, or do further analysis on special-case material. Most of us don't work in a studio or have access to D1 material. I have wondered before if your work environment gives you perspectives that isn't exactly 'mainstream' or 'real-world' like the rest of us. It seems in-studio D1 sample material would exemplify that.

Posted example images are few and far between. It's not indicative of my experience and measurements with mainstream, telecined, consumer material at all. I've never seen anywhere close to 254,254,252 in mainstream material before. That's why I was interested to duplicate and verify for myself.

Without some specific titles/timestamp examples from mainstream, telecined, consumer material , I've pretty much given up on looking. I've looked at dozens and dozens of titles over the years and posted dozens of A/B caps. I did mention the waveform monitor and histogramer that can easily flag out of limit or out of gamut conditions. I can't prove a negative.

Dave

I only looked through about 10 to 15 commerical DVDs, but I agree that the brightest levels in the posted image are not in line with the sort of decoded RGB values I ran across.

Dave: that's not the point.

If you feel things should be clipped at 235, then the onus is on you to explain what advantages come from imposing a clip at 235. You advocate deviating from video standards, yet you have been asked by many people over the course of several years of these repetitive discussions to forward some reason why anyone would want to deviate from video standards.

You can't come up with anything except for: "well, gee I can't really see the difference."

Some years ago you posted a blinded A/B set of images, I can't even recall what it was it's been so long ago, and even on my computer monitor (not my reference system), I could immediately point out the difference on the image that was clipped at 235. Your claim at the time was that nobody would be able to see the difference, that there wasn't anything meaningful beyond 235. Well there clearly was and I could clearly see the difference.

And one can clearly see the difference in the image Mr D. just posted. And you can see the differences and evidence from consumer material in images Stacey Spears posted way back in 2004! All of this you completely ignore on some asinine and bizarre crusade against video standards. And it's amusing to me that this all occurred because at the time you didn't even know that these video standards existed.

And here is a question Don Munsil posed to you in 2004 which is among the endless questions which you have never ever answered: "I've never seen a title that didn't have below-16 and above-235 values, I will turn it around and ask you to find one that stays in that range for the entire running time. I doubt there exists such a beast. I have 100% confidence that I could dump any frame of any normally-lit scene in any top Hollywood DVD and find below-16 and above-235 values."

One clearly can see the clipping on some content, and a lot of content doesn't stop dead at 235, nor should it, and when that happens, which is pretty frequently, you can often see some impact from that.

But you can't ever come up with ANY good reason why we would actually WANT to do that, and throw away perfectly legitimate video information that the standards explicitly provide for and allocate to video data.

So as was asked of you probably years ago by now: put up or shut up.

Stop arguing guys.

What we need to do is actually look at the stuff and see what conclusions we can draw.

Dave I appreciate you have done some tests but lets just see what the conclusions are of a bunch of people rather than one individual.

We are all interested in maximising the contrast range on our displays ( or we should be). There is no point hanging on to a bunch of level that doesn't do anything but compromise contrast . Likewise there is no point clipping off range that produces a more pleasing image especially if your display behaves like a torch compared with a BVM.

Can we drop the "I'm right and you are wrong" attitudes otherwise we will never reach any meaningful conclusions about this topic and it will continue to crop up again and again.

Alright guys I've been looking at some stuff. None of which I can post on here however I'll keep looking for something I can verify the path of and actually post.

I'm seeing level on some masters that seems to roll off at 240 in areas I expect to top out at 254.

I'm seeing other masters with level through to 254.

I haven't come across anything that tops out at 235.

Also these are big hollywood blockbusters and whilst not necesserily the masters that the consumer versions are generated from ( I cannot guarantee this) they are directly from the DI companies for the films and as such should represent the video masters generated there.

Like I said I have handled video material direct from DI suites at numerous companies and I've never seen it roll off at 235 its always had level right to 254. not on every single shot I'll grant you but that almost goes without saying.

Coming down on the side of level all the way to 254 at the moment.

I've not found anything that tops out at 235.

Lowest I've got is 240 and that's quite an old transfer ( Superman Returns actually).

Later stuff all has level up to 254: Bond films, Potter , Golden compass , King Kong... I cannot however 100% confirm that all of this material is direct from the DI ( some of it most certainly is) but I am certainly not seeing a 235 limit on any footage I've checked.

I'll try and get some braodcast material. (I'm film so I don't actually have much "virgin" video footage around, tonnes of 10bit log and most of the video footage I have is usually generated direct from that as a client temp so absolute levels are usually secondary to the image itself being pleasing or illustrative to the client .

Will look at grabbing some BD over the weekend. I did it before and verified I was capturing full range with test patterns (within a code value).

Found one thing that appears to be clamped at 16-235. TV advert ~ (actually a filmfour sting we did years ago). Raw footage as delivered is definitely clamped to 16-235...can't say its what I'd describe as "nice" looking though.

I wonder if it would worthwhile to start a new thread. Possibly not in the calibration area. I think a few issues get conflated when this discussion starts up and it might be worthwhile separating them.

1) What do the standards say and what do they mean.
2) What are the motivations for limiting above reference excursion.
3) What are the visual implications of limiting above reference excursions.

(It would be nice if there was dE for clipping but I suspect that may be an area of limited research.)

In any case most of agita seems to be related around item 1 so as I've previously suggested perhaps someone could chase down authoritative discussion about the specific wording of the standards rather than assuming specific intent. I would assume there are accessible texts somewhere although Poynton notes that there are circumstances where texts are not error-free.

My own (selfish) interests remain the visual experience. Since I can't change what's on the disc or what some component does I'm very interested in the visual implications of various "defects" of which clipping is only one.

A perfectly reasonable thing to want.

I'm not sure how readily we should move to another thread , as long as this one doesn't dissolve into an argument.

I'll be doing some investigating over the weekend with BD material. I've reached the conclusion that video fresh from a telecine or DI has level above 235 to at least 240. I'm also 99% certain it can extend to 254 without being in error from what I've seen on the majority of footage I've looked at and according to my interpretation of the various mentions of headroom and excursion in the specs.

However I'm also interested to see how much exists above 235 on consumer video. (there shouldn't be any differentation between pro and consumer video in terms of level...video is video).

I have run with white at 235 in the past and frankly hated it but lets just see empirically how much material actually goes over 235 and if it is over where is a reasonable point to clip if you don't extend all the way to 254.

I'm sure we can come to some reasonable conclusions without having a punchup guys.

Chris, I'm not advocating this (in fact I have my RS20 set, in Enhanced HDMI mode, with Br = -5 and Cst = 5, getting all the white bars up to 251 or more), but IF there were nothing above 235 then the advantage of clipping everything above that is obvious: one would have a higher Cst setting and thus a brighter pj.

Mr. D, though, is presenting ample evidence that there IS material well above 235, so that I certainly agree with what you advocate.

I agree this is the main reason to look at what levels actually appear in common video material. If you calibrate grayscale all the way out to 254 you're giving up say 20% of available brightness compared with letting above reference white color shift.


Personally I mainly watch movies sold in the US, and I do not agree that he addressed common levels in that material. Based on what I've looked at, a shot of the sun similar to the example would likely go to 235 rather than 254. I was mainly looking at near grays when I went through DVDs, so I was not focusing on how far out colors might commonly go, but at this point I cannot generally recommend calibrating a display for the image that he posted.

Thanks alluringreality, very good points.

I for one have really enjoyed and benefitted from all the discussion in this thread, the pros and cons. And even though there are differences of opinion, etc., I think it has more or less kept well within the bounds of civility (and certainly so compared to what I've seen on other occasions in the Forum!).