Gamma encoding

For mathematical pixel filtering operations you should ideally linearize using the exact transfer curve of the original video standard preferably entirely in 32bit floating point so that when you come to reconvert back from linear to video you can exactly undo the conversion. ( you'd be amazed at the people who rely on just eyeballing it back to roughly where it was before...mainly inferno and flame artists).

However in practise even if you just do a simple degamma type operation you'll get significant visual improvement in the filtering interpolation and a simple gamma operation is by definition easy to reverse exactly.

I generally stick with 2.2 for the pixel maths although generally I prefer a display with a gamma somewhere between 2.2 and 2.5. The fact its not perfectly linear when you perform the filtering operation won't make a huge difference in reality.

( you know the display is the 2.2 bit? The material itself is the inverse of this hence 1/2.2 which gets you 0.444...something or other so to linearise video material you apply a 2.2 gamma correction to it and then to reconvert you apply 1/2.2 gamma correction....do it exactly if you can , I punch in 1/2.2 in pseudo code rather than round it off)

PAL is never 2.8 in the real world. just doesn't happen.

Not sure if this'll help in your particular situation, but there's some pretty good info on Gamma here:

http://www.libpng.org/pub/png/spec/1...aAppendix.html

Near the bottom of the page it says this...
I believe you are correct that in the latest SMPTE standards for video, the in-camera transfer function is designed to slightly undercorrect for a dim surround. According to the link above that transfer function should closely approximate a power-law curve of about .52. On a 2.2 gamma display that would produce an end-to-end exponent of about 1.14.

However, the older NTSC transfer function was 1/2.2 (or .45).

Telecine operators that I've spoken to confirm the above numbers. The ones I've spoken to say they currently use about .50 correction when transfering film to video. But in the past they used corrections of about .40 or .45. This is one reason why alot of older films and NTSC video material look brighter. Video engineers and colorists will frequently tinker with the gamma after the transfer though, so the above numbers are by no means engraved in stone on most content.

If the PAL content you're dealing with was really recorded with a transfer function/correction of 1/2.8 (.36), then it would appear quite bright compared to most other video content on your display... unless of course the camera operator or video colorist made some adjustments to compensate for that.

Another error that you sometimes see in PAL to NTSC transfers is over-compression of the video levels. This is likely the result of multiple applications of setup and/or compression to video levels, and can easily be seen in histograms as abnormally elevated black levels/reduced white levels.

Also, I would not get too hung up on the display gamma in making your calculations. Some people say 2.2, some say 2.5, others say 2.35. Fwiw, the studies I've looked at put average CRT monitor gamma pretty close to 2.2. Most older CRT TVs were probably also in that neighborhood. (For the record my 2002 Sony XBR800 tube appears to have a native gamma of about 2.25, however the analog and digital processing in the display can alter that in many different ways). Current TVs will probably vary from about 1.8 to 2.6. So the mean value is probably still pretty close to 2.2.

However some folks here prefer (and often advocate) something closer to 2.5, because it can give the picture a greater sense of depth and dimension in a darker room on some displays. There are a variety of factors that can come into play though in determining the best gamma for a display, including the background lighting, and also the black and white level performance of the display. On many displays, increasing the gamma will simply result in crushed shadow detail.

Wikipedia also has a fairly good article on the subject.

The players should be transparent, so encode the source on to the disc as you want it sent to the monitors.

I know rec.709 is for HDTV, but the camera gamma equation for encoding video is published and well know. It has a linear toe slope and an offset, but then encodes the majority of the video with with a power function of 0.45 (wich gives an end to end gamma of about 1.12 when displayed with gamma of 2.5).

This PDF is the resource I used to get this information
http://www.poynton.com/PDFs/GammaFAQ.pdf

Also since you are essentially mastering the disc here you can take artistic authority, master 2 or 3 different versions and test them out, see wich one looks most correct (assuming you have a calibrated monitor to test with).

Some great questions. Hopefully I can answer one or two.


Perhaps. Since TV manufacturers are no longer tied to the limitations of analog CRT hardware, they can do pretty much what they want. I don't see a particular trend in one direction or another though with respect to average display gamma. So I don't think that's why the standards were changed.

IMO, the switch from ~.45 to ~.50 correction was probably done more to adjust to the different viewing conditions/habits of TV viewers. More people are watching big, bright TV screens in darker rooms than back in the 1950's and 60's when all you had was a small B&W box in your living room. So (assuming similar gamma on the displays) it probably makes sense to aim for a higher end-to-end exponent.


They're both approximations, since the new camera standard does not follow a pure power-law. Some will round the value down to .50, others will round it off to .52, and some will split the difference.


I don't know if telecine works the same for PAL. However you could certainly try contacting a post-house and ask.


I don't know about PAL, but I believe sotti is correct that there's no standard correction performed on NTSC DVD players (beyond what's inherently encoded in the MPEG-2 DVD-Video files.) Manufacturers are free to tinker with the picture as they like on their DVD players though. And many include gamma controls that the user can adjust (in addition to the usual Brightness and Contrast controls), so some variation in gamma is certainly possible from player to player.


Again, I can't really speak for PAL (and hopefully someone will correct me if I'm wrong about this), but I suspect that the newer camera transfer function is probably used for both ATSC/HD and NTSC/SD recordings, including probably DVD-Video. (Maybe someone like Mr. D could shed some more light on this.)

Much of the final adjustment is probably done by eye though. So you may want to try re-encoding with both ~.45 and ~.50 to see which looks best for your NTSC DVDs. (Edit: see this post below for an important caveat related to video levels.)

Agree.
 

 

Disagree.

I believe when you factor in the add'l math and linear slope at the lower end of the new transfer function, it changes the overall shape of the curve so the net result more closely resembles a power function of about .50 to .52. (That's the point this PNG document seems to be making anyway.) So on a 2.5 gamma display, the end-to-end result would be closer to 1.25 to 1.3.

A couple of quotes from the PNG spec which probably do a better job of explaining it than I can...

 

Quote:

The original NTSC video standard required cameras to have a transfer function with an exponent of 1/2.2, or about 0.45. Recently, a more complex two-part transfer function has been adopted [SMPTE-170M], but its behavior can be well approximated by a power function with an exponent of 0.52....
 

13.9. Video camera transfer functions

The original NTSC video standards specified a simple power-law camera transfer function with an exponent of 1/2.2 (about 0.45). This is not possible to implement exactly in analog hardware because the function has infinite slope at x=0, so all cameras deviated to some degree from this ideal. More recently, a new camera transfer function that is physically realizable has been accepted as a standard [SMPTE-170M]. It is

if (Vin < 0.018) Vout = 4.5 * Vin
if (Vin >= 0.018) Vout = 1.099 * (Vin^0.45) - 0.099

where Vin and Vout are measured on a scale of 0 to 1. Although the exponent remains 0.45, the multiplication and subtraction change the shape of the transfer function, so it is no longer a pure power function. It can be well approximated, however, by a power function with exponent 0.52.

 

This wikipedia article seems to suggest that the transfer function in Rec. 709 was designed with a 2.4 gamma display in mind though, which would probably yield an end-to-end result of about 1.20 to 1.25. I haven't looked at the specifications myself, but would not be surprised if this is the range the engineers were going for on the back end. Many displays fall short of that though because they have gamma lower than 2.4.

 

 

Sounds like good advice to me. Another thing you could do is compare your final product to some other well-authored DVDs.

Edited by ADU - 6/7/12 at 5:46pm

There's another potentially complicating factor to consider in this as well. The ~.50 to .52 correction that's applied in telecine and current video cameras would normally take place BEFORE the video content is compressed to 16-235 video levels.

If the video you're working with has already been compressed to 16-235 video levels, and you re-encode it with a different correction than the exact inverse of what you used to de-gamma the video content, then the black and white levels will be shifted in the final result, which is not something you want.

If the video is already compressed to 16-235 levels, and you're not going to expand those to 0-255 prior to undoing the gamma correction (and recompress back to 16-235 again later), then you should probably do as Mr. D originally suggested and re-encode with the exact inverse of the value you use to de-gamma the video, to keep the black and white levels as close to their original levels as possible.

That's not strictly true. The conversion to video from film these days is eyeballed subjectively by a colorist. Its not a simple gamma conversion.( telecine never was) They grade the film into video on a shot by shot basis. There isn't some hard map to 16-235 those are just reference points ( the actual color correction into video will be handled in float prior to downsampling to 10bit video anyway) and there isn't really any specific notion of gamma correction going on beyond what the end result looks like on an accurate video display.

Seriously though hit it with 2.2 do what you need and then hit it with 1/2.2 on the way back. You'll get the improvements in filtering and interpolation that linearisation brings . If you are unsure try it with 2.5 or 2.8 or whatever you like ( won't really matter as long as you hit it with the exact inverse on the way back out as I've said) and see if you can spot any improvement at any of the other gammas...I suspect not.

Fwiw, I agree with most of what you say above Mr.D (which is why I included caveats like: "much of the final adjustment is probably done by eye"), with the possible exception of this...


If you did not have a fairly good idea of the correction going on, then you probably would not be able to recommend specific values (like 2.2) to undo it. I'm guessing there are probably certain parameters you use as a starting point for color correction/grading as well, rather than working completely from scratch on each new project.

You might (or not) also be surprised how many people who work in the video field were completely unaware that there was any correction going on in their video projects, at least until recently when high-resolution, and higher bit depth film-scanning, DI, and linear encoding started to become more the vogue. (Many folks I've spoken to who work in post still do not understand this, unless their job explicitely involves translation between the linear and gamma-corrected material).

However, I understand exactly what you're driving at with statements like this. The whole question of how to author/adjust video for different displays and viewing conditions seems to be a pretty thorny one, especially with all the rapid developments taking place in display technology. So it's probably not something that can easily be boiled down into a simple set of numbers.

I also somewhat question the logic of including under-correction on the encoding end, as it seems like most of that should be handled more on the display end to accomodate different viewing conditions. And I suspect there are probably different ways of handling the capture and authoring of material for bright displays in dark surroundings than simply increasing the gamma in the content (which can often make colors look too saturated and contrasty). Most displays (and players for that matter) offered little or no user control over gamma when the new specs were put into place though. So maybe that was part of the rationale.

I can't completely discount the possibility that they might have been erring a bit in favor of displays with lower gamma as well, as absence speculated. By including some under-correction in the source they provide a bit of add'l depth/contrast for displays with lower gamma, and also provide a "dim surround" component for those who view in darker rooms. So there could be sort of an attempt to address two issues with one stone going on there.

To be honest, the possibility they might be trying to account for displays with lower gamma never even crossed my mind until absence mentioned it.

A couple other tidbits of info that might perhaps be of some use...

This sRGB document could be a source of some of the .51 estimates for the Rec. 709 transfer function. It uses a value of 1.0/1.956, which equals .5112.

You can get a general idea how closely the Rec 709 camera transfer function fits a .50 curve from this graph that was posted in another thread.

This article on picture rendering and perceptual uniformity as it relates to Rec.709 is also a very informative read. (This could also be the source of the 2.4 display value listed on wikipedia's Rec. 709 page.)

The bottom line, in answer to the original question, is that we have little control over much beyond the display, so no one is really correct with respect to identifying what gamma "number" to use to set up a display. Programming varies in how it is put together. The choice for display gamma is a judgement call that is affected by the capability of the display, the sources, and the light conditions. Note that Poynton tries to shift away from discussing "gamma" which is often characterized by a single number and refers to the transfer function. This is even more important on the display end, where many units behave very differently at various points in the response curve. Using a single number to characterize a more complex function can completely obscure the actual effect, and make the whole debate meaningless. We all need to become better versed in how luma response affects viewing quality, and the displays and processors need to become better at allowing adjustments throughout the range.

After doing some more reading and research I think I can give a somewhat more in depth reply on this.

As mentioned earlier, the main difficulty in assigning a specific exponent to the Rec. 709 camera transfer function is that the formula does not follow a pure power-law. It uses a two-part formula which looks something like this:

 

*If Input is < 0.018, then... 

Input (4.5) = Output

 

If Input is > or = 0.018, then...

1.099 (Input ^ 0.45) - .099 = Output

 

(I've also seen the 2nd part written like this: (1.099*Input)^0.45 - .099 = Output. However, I believe the interpretation above is correct because otherwise there's a jog between the linear tail and rest of the curve.)

If you graph these equations, it looks like this...



The values in green represent the equivalent power function at various points on the Rec. 709 curve.

These are the data points on the graph in case you want to verify my math...
 

Input	Output	Equivalent Power Function
.96026	.98013	.495
.63461	.79662	.500
.32419	.56300	.510
.19167	.42357	.520
.12530	.33259	.530
.08810	.26933	.540
.06547	.22327	.550
.02445	.10788	.600
*.01360	.06120	.650
*.00665	.02992	.700
0	0	1.000

The Output values should be approximately the same whether you use the formula above or a simple power function:

 

Input ^ Equivalent Power = Output

 

As you can see from the table and graph above, the effective power of the Rec. 709 transfer function varies from 1.0 at the bottom of the curve (black) to ~.5 near the top (white).

Folks I've spoken to in the industry tend to treat Rec. 709 as a square root though (or a 1/2.0 or .50 exponent), so here's how it compares to a .50 curve...



The green area represents the difference between the two functions.

The two curves are a fairly good match (much better than a .45 curve for example), but they still diverge by a noticeable amount toward the lower luminance range. Based on what I've heard and read from various experts on the subject, the higher effective exponent toward the darker end of the Rec. 709 curve is designed to minimize camera sensor noise in that range. Poynton in A Technical Introduction to Digital Video, (1996):

 

 

 

Apparently the sensors used to capture images in video cameras are sensitive to heat as well as light. And the noise generated by the heat-sensitivity shows up more in the darker color range. With the low luminance level noise factored in, perhaps Rec. 709 might behave more or less like a .50 power function across the entire signal range, sort of like the picture below.



(This illustration is hypothetical on my part btw, since I'm not an expert on video cameras, CCDs, or precisely how they work. Video cameras may not in fact work this way at all.)

The sRGB and PNG documents seem to have calculated an average value for the curve without factoring a noise component in though, which may be why their estimates for the effective power are slightly higher than .50.

If you calculate the effective power of the Rec. 709 curve (without noise) at regular intervals in a horizontal direction along the Input axis, the average of those samples is very close to the sRGB estimate of ~.51. If you sample the values in a vertical direction (Output axis), it's about .53. Average both directions together and you get ~.52, the PNG estimate. I'm not sayin that's how they arrived at their respective values of 1.0/1.956 (~.51) and .52, just pointing out how some simple averaging can yield similar results.

If you accept the scenario in the picture above, then it may be more correct to treat Rec. 709 simply as a square root for most post-capture purposes. If nothing else, treating Rec. 709 as a square root certainly seems to simplify some of the math...
 

Display Gamma	End-to-End Exponent with .50 Correction/Encoding Gamma
1.8	.90
1.9	.95
2.0	1.0
2.1	1.05
2.2	1.1
2.3	1.15
2.4	1.20
2.5	1.25
2.6	1.30

As Mr. D and others mention above though, the actual contrast/gamma will really depend on the operator and setup used for mastering. So the implicite ~.50 power of Rec. 709 is likely only a starting point for most video projects. And the reality is probably more complicated than the table above suggests.

(Note: you must be logged in to see the images in this post.)

Edited by ADU - 7/13/12 at 8:09pm

i was going to start a new thread with this question, but this may be appropriate

when talking gamma, what is the unit of measurement? 2.2 what? is it just a number that is the result of an equation involving luminence?

Gamma is simply an exponent. I think generally the calculation is done with relative luminance, so that when you normalize the luminance measurements there aren't any units involved at all. http://www.avsforum.com/avs-vb/showt...4#post14298274 or http://www.avsforum.com/avs-vb/showt...5#post10469695 are a couple ways to calculate a point gamma from measurements.

Sounds good to me. This is also not a bad place to start for a better understanding of gamma.

Just to expand a little on alluringreality's remarks, gamma (denoted by the symbol "γ") is a mathematical power law function describing a nonlinear relationship (or transformation) between image brightness values:

starting or input image value ^ γ = ending or output image value

The result of this function is an increase or decrease in overall contrast or brightness, with the black and white points in the image remaining unchanged.

By convention, the input and output values are usually normalized to the range 0 to 1, and the function is described just by the exponent γ (gamma).

Gamma values greater than 1 result in a darker or more contrasty image, while gamma values less than 1 result in a brighter or less contrasty image. (Some video players and software packages reverse this relationship though, and treat gamma as a 1/γ quantity instead, which can be a bit confusing.) A gamma of exactly 1 (aka "unity") leaves the image values unchanged, because any value raised to the first power equals itself.

Since gamma is a power function, different gamma operations are combined through multiplication rather than addition (and uncombined via division). For example, video recorded with a .50 camera encoding gamma viewed on a 2.4 gamma display would have an end-to-end or final image gamma on the screen of 1.2. (.50 * 2.4 = 1.2)

Gamma operations can be either physical/analog (as in a photographic print, motion picture film or the nonlinear response of a CRT), or digital (such as the gamma control on some DVD players, or the lookup tables on a PC or display).

In current video cameras, the transformation probably takes place in the digital RGB domain. The CCD sensors on video cameras are most likely linear with respect to measured luminance, so the "input" RGB values represent the relative luminance levels of the scene being captured. And the output or "gamma-corrected" RGB values (sometimes denoted R'G'B') are the ones recorded (unless the camera is designed to capture linear values as well).

We need gamma for a couple things: transmitting and storing images and video more efficiently, and adapting images/video to different viewing conditions.

Edited by ADU - 7/5/12 at 1:45pm

Another possibly easier way to think of gamma is as a contrast percentage (which is the way it behaves on a log-log plot). I think this is fairly common in the film/photography industry, and it could be applied to video as well.

A .50 encoding gamma could be thought of as reducing the contrast of the image to 50% or 1/2. And a 2.0 gamma display could be thought of as doubling the contrast, or increasing it to 200%. ("Contrast" in this context refers to the rate of change between light and dark, rather than the ratio of white to black.)

IMO these would be perfectly legitimate statements. Perhaps calibrators and videophiles do not refer to gamma this way though because it could be confused with % stimulus, or relative luminance.

Bumping this thread for a couple reasons, which I'll explain after the AVS changeover.

You may be interested in a recent related thread.

I won't add much to this thread, except to say Mr.D is correct.

Our CMS is used for a lot of professional post-production operations, and we also do a lot of physical calibration for such environments.
Almost all use a gamma 2.2 for their display set-up for tv work.
Film (theatrical grading via a print emulation LUT, or DCI P3) is different, but when remastering for tv the display used will almost certainly be a 2.2 gamma display.

And for info, just about all ctr displays we have ever measured come out very close to 2.2

This is a situation where what the 'specs' may say is not really relevant - in the real world 2.2 gamma is the majority standard.

Hope this helps.

Steve,

Are you advocating 2.2 for encoding gamma, or for decoding/display gamma (or both)?

Not sure if he mentions it above, but I believe Mr.D routinely checks his video output with both a 2.2 and 2.5 display gamma which, on average, would put him pretty close to the EBU and Rec. 1886 decoding specs of 2.35 and 2.40 respectively.

^ One of the reasons I bumped this thread. I've been following that discussion, and share some of the quandaries re current encoding/decoding paradigms expressed there as well.
Edited by ADU - 7/5/12 at 2:15pm

We are only ever interested in display gamma...

Cheers - Steve

Understood.

 

To be honest, I don't think it really matters exactly what display gamma is used for mastering or home viewing, as long as it's roughly the inverse of lightness in an average surround. What's more critical, in my opinion, is the relationship between the display's brightness and it's surroundings, which will vary depending on the application.

 

Display gamma is an important part of this equation though. If you're eyeballing the video correction in a room with gray walls and 16 lux illumination on a 100 cd/m^2 display, you won't get the same result with both a 2.20 and 2.40 display gamma. The correction/encode done with a 2.20 display gamma will almost certainly be darker.

 

To get around this problem, my suggestion is to use the display as a reference for adjusting either the display's white level or the surround levels, depending on which is easier to tweak in the setup. For home video content (ie DVD, Blu-ray, broadcast), IMO the surround reference should be close to the APL of video over time, and typical light levels used for home viewing (which may vary depending on the time of day). For "prime-time" video content in the US, I recommend a surround reference that's somewhere between a 12.5% to 20% stimulus gray, or nominally about 15% stimulus gray. Other parts of the world, and other video applications (such as web content) may differ.

 

In 8-bit consumer video, a 15% stimulus gray would be around R'G'B' or Y' = 49...

 

(49-16)/(235-16) = .15

 

^ Note this is in compressed 16 to 235 "studio swing" levels. In 0 to 255 "full swing" levels (before the palette is compressed to video levels), the value would be R'G'B' = 38.

 

38/255 = .15

 

 

IMO though, this is the essential paradigm that our current video system is based on...

 

• ENCODING GAMMA = roughly a square root, or ~.50
• DECODING/DISPLAY GAMMA = roughly the geometric mean of a cube and square, or ~2.45

 

All of the recent authoritative references I've run across on this subject support a display gamma for video production of ~2.40 (which is pretty close to the 2.45 value above). This includes Rec. 1886, EBU Tech 3320 (see Annex A: Gamma considerations), Poynton, and Motta. They all seem to agree that the value for display gamma should be around 2.40 (not 2.2).

 

This 2.40 reference value is based on both the gamma of legacy CRTs (per Motta and studies by the EBU/BBC), and on perception of lightness in an average surround (per Poynton), even though mastering and prime-time viewing is usually done in a dimmer surround. IMHO, using substantially lower values for mastering than this 2.40 value will result in poorer perceptual performance. (More about this issue here.)

 

Here's what I believe is actually happening though in current video mastering in the US...

 

• TYPICAL ENCODING GAMMA = .50 to .55 (nominally about .525)
• TYPICAL DECODING/DISPLAY GAMMA = 2.20 to 2.50 (nominally about 2.35)

 

For the record, I have no specific recommendation for display gamma on the users' end, except to stay in the neighborhood of the reference or typical values cited above, and to use whatever is most comfortable and appealing to your eyes in your typical veiwing arrangement.

 

For mastering US video content my recommendation would be to stay pretty close to the Rec. 1886 reference display value of 2.40 and/or the nominal value of 2.35 for best results. The only justification I can see for using lower values on the order of 2.2 would be ergonomic considerations (to reduce eye-strain).

 

For encoding prime-time US video content (for DVDs, Blu-rays, etc.) I suggest using whatever encoding value looks best with an ~15% stimulus surround reference as described above. IMO that should implicitely result in encodes close to the typical  range of .50 to .55 above, or about .520 to .525 on average.

 

If you look at the graph below, you'll notice that Rec. 709 has an effective exponent of .520-.525 right around 40% stimulus, which would be roughly the range of brightness for a perceptually middle gray in the dimmer surrounds used for mastering and home viewing. IMO this is no coincidence.

 

 

YMMV of course, on all of the above. And other applications may work better with different values.

Edited by ADU - 7/18/12 at 4:58pm