Help, no "blacker than black" bar, PLUGE  

What is your display?

Your HTPC doesn't display it because it is blacker than black. It will only appear if you rise the brightness of your software DVD Player, but since the brightness is no more at default, you can't calibrate your display anymore.
You can't make it appear without messing up all the calibration.

if I'm not wrong, a HTPC cannot display blacker than black on Video Essential test pattern, correct me if I'm wrong, anyone? This is because of the complex path taken by motherboard and video card leading to some inherent losses in dynamic range therefore an inevitable clipping of the blacker than black bars. What u can do is calibrate the projector using the next available black bar beside the blacker than black. It is usually around 2-3 ire. Adjust until you can barely see it and you'll be fine.
Can someone please comment on this blacker than black bar and whether you can display it on ur pc. If so, what's ur video card, mobo and configuration, thanks.

You can, in fact, get below-black to display on an HTPC. You need to use drivers that keep 16 as the studio black level, as used in all digital video interfaces. Sadly, lots of apps and drivers were written with the mistaken assumption that RGB should always use 0 as reference black and 255 as reference white, thus cutting off the below-black (and above reference white) data and screwing up the image.

The biggest problem with calibrating your monitor to 16 is that everything else you display using the same PC will look too dark. If all you use the PC for is video, it's all good. But if you want to play quake or something, you have to recalibrate. The desktop, surprisingly enough, generally looks fine with the video calibrated to 16 black, unless your background image is quite dark, and even then you can easily adjust it with GIMP or Photoshop or any other image editor so the blackest details are at 16 instead of 0.

To get the standard video levels, you need to use VMR7 or VMR9 and not use the hardware overlay. There are various DShow filters out there that misguidedly "fix" the video levels when converting from YCbCr to RGB, so try to test with a clean, simple path. The decoder should be using DXVA direct to the video card.

Windows Media Center by default will use video levels, though even then it's possible to mess it up. But out of the box, everything including the UI is calibrated to 16=black.

Don

Some decoders and color-spaces will pass the pluge and others won't (if the black level is sufficiently raised) I forget what camp the TT Sonic decoders fall into but I believe the WinDVD 5 filters will pass the pluge when in YV12 color space. There was once a thread that documented many filters and their ability to pass pluge along with the debate about whether DVDs actually had blacker than black information encoded on them. The blacker than black pluge is more trouble then it’s worth IMO.

Dave

As it should for RGB or sRGB color-space. Y16=black and Y235=white only in component color-space not RGB. The are many, many color space definitions. PCs can and do color-space conversions at many levels.

respectfully disagree... if you're talkin RGB or sRGB...

Dave

Studio Video R'G'B' does in fact have reference black at 16 and reference white at 235 and this is correct when playing back video content.

Poynton covers this in his book and for those that don't have his book, you can read more about it here:

http://msdn.microsoft.com/library/en...aceconversions

See "Color Space and Chroma Sampling Rate Conversions"

In BT.601 8-bit Luma is defined from digital level 1 to digital level 254. If you only have values from 16 to 235, then you are not getting all 8-bits of video. You have something like 7.78-bits.

When a PC converts to RGB using the 0-255 scale, they clip at 16 and 235. You have just lost .2-bits of real data. The 16 is stretched down to 0 and 235 up to 255 and now you have introduced non-linearity into the image. This is visible in the form of contouring. This could be smoothed out with noise shaping, but it is not being done by graphics cards.

When a display is properly calibrated, you won't see the blacker than black, but it will in fact be used. Any math operations on the image will use those values. Be it resizing to match the panel size or dither in the case of a DLP.

That is incorrect. The valid range in RGB colorspace is 0-255. In digital video (ITU-R BT.601), the valid range is 16-235.

The computer outputs RGB, so the decoder should expand the range from 16-235 to 0-255 for display. There's no cutting off. Black in 601 is 16. Black in RGB is zero. Besides, there's no point in displaying blacker-than-black. It doesn't exist in digital.

Values below 7.5IRE (blacker-than-black) are illegal in analog NTSC. You *can* go to 0IRE, but it's wrong and not suitable for broadcast. If I'm shooting video somewhere, I will simply look at my scope and open/close the iris on the camera so that I keep my black level above 7.5IRE. When that shot gets digitized (DV, DVD -- which may be immediately or later), it will be done so that anything below 7.5IRE maps to 16. There's no crushing or clipping since that's what I intended (for 7.5 to be black) when I did the shoot.

But what about Overlay/VMR? Well, internally your video card/drivers may operate in YUY2, YV12, etc., but the output from that HD15 connector to your ECP is still RGB.

See the discussion at the bottom of this page with regards to setup in the digital domain, since I don't think I explained it too well...
http://www.adamwilt.com/DV-FAQ-tech.html

Let's back up a bit here. Stacey and I work in the digital media group at Microsoft. We have all the relevant specs at our disposal. I can tell you with 100% confidence that there exists a standard called Studio RGB which keeps black at 16 and reference white at 235. (Peak white is 254, so there really doesn't exist an "above white," but that's a whole other can of worms.)

Not only does this standard exist, it is the appropriate standard to use when outputting video, and we have spent a tremendous amount of time and energy getting computer engineers to understand this - something that video engineers have understood for a long time. Windows Media Center uses this spec. NVidia will in the next version or so of their drivers be using this spec for all YCbCr conversion, as will ATI. Get ready, because this is the future of video output on computers. Read the references that Stacey provided. Poynton is the expert on digital video, and just quoting randomly from his "Digital Video and HDTV", page 309:

"In studio equipment, 8-bit R'G'B' components usually have the same 219 excursion as the luma component of Y'CbCr..."

Yes, it is true that in computing we have generally defined 0 as reference black and 255 as reference white, but when outputting video it makes much more sense to use the studio RGB standard of 16 and 235.

As for FSOne's comment that below 7.5 IRE is "illegal" in analog NTSC, that's just completely wrong. First off, 7.5 IRE setup is not used in all flavors of NTSC. In Japan and a few other countries, black is set at 0 IRE. But even in this country, below-black is perfectly legal - it's fundamental to all video interfaces. TV stations broadcast color bars with PLUGE patterns that go below black, and VCRs record below black. DVD players that are properly made produce below black just fine.

Every DVD I have ever examined has had significant below-black and above-reference-white information, often in every frame. On a real CRT, the above-reference-white information is absolutely visible, and the below-black is often visible because of the variations in black level retention.

Believe me, I'm not just smoking crack here, and neither is Stacey. I realize that for many people familiar with standard computer practice it's jarring to hear that RGB is not always 0-255 nominal range, but it's true.

Don

"Does anyone know how to get that to appear using my HTPC?"
Have you tried both SW mode and DXVA HW acceleration mode?
I seem to remember some time ago trying this with a player (can't remember which one) and one mode worked while the other didn't.

OK. I'll buy that. It's unlikely that any of us will ever have access to the non-linear, non-compressed, pre-encoded R’G’B’ studio feed on our PC’s however…

My point is that other than special test patterns, where is the blacker than black information? What images will have those levels encoded? It's not real-world. (for most of us, anyway)

Dave

Wow! can you give some examples? titles/time stamps? It's Deja vu all over again :D

Dave

I went and dug up some old threads and I am going to repost some information that I hope you find interesting.

First is from Alan SH.

I am glad you asked.

Here is the same photo from above (Thanks Dr1394 for capturing), but all values below 16 and above 235 have been marked as white. This is from the DVD release of Talented Mr. Ripley.

"Windows Media Center uses this spec. NVidia will in the next version or so of their drivers be using this spec for all YCbCr conversion, as will ATI."

Don,

Will this be done for the overlay or just VMR?

This was also provided by Alan SH.

It is up to the driver manufacturer. Both nVidia and ATI started doing this with VMR when MCE 2004 launched. They are using the BT.601 matrix to go from Y'CbCr to Studio Video R'G'B'.

Me thinks "The Talented Mr. Ripley" may be improperly encoded. But EVERY DVD? I'm shocked! If I re-calibrated to correctly accomondate "The Talented Mr. Ripley" I think I'd be disapointed...

Dave

Dave,

It is, in fact, real-world and you can absolutely see the below-black detail, if you're using a CRT (or a fixed-pixel device that properly emulates a CRT).

First off, the below-black samples affect the waveform output from the D/A converter. In the same way that hard clipping of audio samples produces unfortunate frequency-domain artifacts that sound bad, clipping the video waveforms produces visible artifacts.

Secondly, no CRT has perfect black-level retention, and when there is a bright image on the screen, the black level goes up, revealing some of the information that is ostensibly "below black." The CRTs that telecine operators use have exactly this characteristic as well, so when the operator adjusts the shadow detail down so it's just barely visible it inevitably shifts data into the below-black range. Think of the 16 level not as a hard barrier but as a target level. The real black level floats around quite a bit.

This is easily verified (and I have done so). I have clips right here on my desk that have lots and lots of below-16 and above-235 values in them. These are not weird clips or unusual clips. I didn't seek them out as test material - they're just clips I happen to have around. I've actually never seen a video clip that stayed entirely within the 16-235 range, with the exception of test patterns. In my experience, real video ALWAYS goes outside the "nominal" range.

Similarly, I can and do demonstrate that clipping at 16 and 235 is visible. It's not just a little bit visible, it's awful. Your white detail goes all to hell. Clouds look like white plastic cutouts in the sky. Snow scenes lose all their detail. People wearing dark clothing look like silhouettes.

The real lesson here is that computer graphics should have been allowing for headroom and toeroom all along. Video engineers know much more about how to display images on a CRT than computer engineers, but they were not consulted when the computer RGB standards were created. The result is this flawed 0-255 standard, which was not really designed around showing high-end video and images in the first place.

Going forward, it's clear that the video concepts of headroom and toeroom will seep into all computer graphics interfaces, and eventually all graphics will use "video" levels or at least a format that allows for head and toeroom. Until then, we have these two standards to deal with. I wish we didn't, but there you are.

Don

I have access to a lot of HD D5s and they all have values below 64 and above 940 (10-bit video, so 16 and 235 in 8-bit)

The recent IMAX WM9 title, Coral Reef has a lot of information below black and above white. Both Don and I were suprised at the amount of information.

Over xmas I actually went to visit a couple of post production houses, including Point360. They do a lot of film transfers and they don't like seeing head and toe room crushed.

MPEG-2, H.264 and VC-9 all support encoding the entire video range. from 1-254.

if you calibrate your display using DVE or AVIA Pro, then TMR will look fine. Be sure to use the patterns on DVE/AVIA Pro that will let you know when you clip above 235. (so you don't.)

To be clear, we are not saying that you should see the BTB after setting brightness. :) Set brightness using PLUGE and all will be well.

On that note, we don't recommend you use THX Optimode patterns. They are usually only correct for a given title.

Since I've never seen a title that didn't have below-16 and above-235 values, I will turn it around and ask you to find one that stays in that range for the entire running time. I doubt there exists such a beast. I have 100% confidence that I could dump any frame of any normally-lit scene in any top Hollywood DVD and find below-16 and above-235 values. Some very dark scenes might not have above-235 values, but I doubt it.

Every video clip I've looked at has had below-16 and above-235 values. I just dumped a random frame from Coral Reef. It's a perfectly normal-looking HD frame (1280x720). It had over 3000 above-235 pixels and over 500 below-16 pixels. Obviously they together represent only about .4% or so of the pixels in the image. It's actually a pretty dark and murky-looking underwater shot. These are the values that came directly off the master D5 tape. No level adjustments were done at all, and it's the same master used to create the DVD and WMV-HD.

If you clip at 16 and 235, is it immediately obvious? No, if you don't know what to look for, and most people don't know what to look for. Once you see the difference between properly presented video and clipped video, it's pretty hard to not see it. It doesn't jump out in every scene, but as soon as you have significant white highlights, it's all over.

Don

The last time this issue was debated, there was a guy that had access to a TEK/GVP MPEG stream analyser. He scoured several DVD's and couldn't fine any non-test pattern DVD's that had any data encoded below Y16 or above Y235.

I don't think I'd trust my eyes to concluslivey pick this out. Especially on a CRT with dynamic gamma...

I hope that dude is still around. I'm sure not gonna raise my black levels or lower my contrast for what I'd still maintain are improperly mastered or encoded disks. Maybe ignorance IS bliss.

Dave

Even if the Y levels stay within the nominal range, adding in Cb and Cr often pushes the decoded RGB levels outside nominal.

I've seen a few films where very few pixels had Y values below 16 or above 235, but because of adding in the Cb and Cr values, when they were converted to RGB, there were a lot of values outside the nominal range that would have been clipped if they were converted to PC (0 - 255) levels. On your TV, they're not clipped.

Dave, why the heck do you think the digital interfaces were designed with such large headroom and toeroom? Do you think the SMPTE engineers were on crack? They were well aware that they had limited dynamic range to work with and yet they chose to devote a full 34 codes out of 253 to "out of range" values. Don't you think they had a reason?

Don

I can't. I don't have the necessary MPEG stream analyzer. I do have a o-scope however and in all my testing and calibration I've not seen measurable data outside those ranges EXCEPT VE Pluge...


If ITU-R BT.601 defines black as Y=16 and white as Y=235, then wouldn't a disk that had data outside that range be improperly encoded by definition?


Lucky guy. HD D5's. Not exactly my world however...


Nor do I. Those purists don't much like digital either. That's why they love film. (and vinyl...)

Yea, it was called the sync pulse...

An O-scope is not going to show you nearly enough detail to see the tiny excursions you typically see in most content. A PLUGE pattern has a nice big block of below-black, something you don't see in real content. Real content does have a pixel here, a pixel there, sometimes a few pixels in a row. You wouldn't think clipping them would have much effect (and let's face it, it doesn't make the picture unrecognizable), but if you know what to look for, it can be seen.

No. On the contrary, it's made abundantly clear that all devices MUST pass the values outside those ranges. That's why the video range exists.

235 is "reference white", which just means it's the white level you calibrate to. "Peak white" is 254. All values up to and including 254 are supposed to be visible. On the black end the lowest value that is visible varies with the picture content, but typically varies from about 10 to 16. Below 10, you're basically keeping the values around to avoid hard clips that change the adjacent waveforms, and to allow for wiggle room in professional environments. In a pinch, we could get by with a final consumer standard with a lot less toeroom - we keep the studio standard because it's convenient.

No, they love digital as well. They just hate computer engineers that keep clipping their video levels.

Don

The sync pulse isn't encoded in digital video. It's synthesized by the video encoder.

Feel free to not believe any of this. It doesn't really matter - the levels output from computers when doing video playback are moving to 16-235 nominal (1-254 peak to peak). That's the reality. Strange that so many companies and engineers are making such a huge change when according to you it's unnecessary.

Don

Back to the real-world and cutting to the chase...

are you guys suggesting that PC decoders and color-space conversion is busted or flawed or that our display and video cards are mis-calibrated? or are we gonna debate wheater a "a pixel here, a pixel there" or .3% of the pixels is "significant" as you say.

I don't think I'd be happy limiting my brightness or raising my black level to accomondate .3%, but maybe I already am. My o-scope ain't .3% and my eyes sure aren't.

Dave