[Mr.D]

Quote:

Originally posted by Charles Black Mr.D, I have checked the ffdshow luma output exhaustively with a scope and test signals and it is linear as expected. I checked chroma ranges and they were as expected as well. My chroma experience is limited to a few samples though and would not have been able to see any chroma errors other than range. Charlie

I mean the actual pixel values rather than the output signal or data type. Effectively a colour correction to flatten the video curve prior to interpolating more pixels for the scale. If you think of a black pixel and a white pixel next to each other , the simplest interpolation is to generate a pixel between them thats 50% . If the pixel values themselves are nonlinear that 50% will be slightly skewed as a result.

 

[kromkamp]

To elaborate, video data is always processed through a gamma curve in the production studio to match well with a CRT display's inherent gamma curve. For example, NTSC is supposed to be displayed with a gamma of 2.2 . CRT monitors usually have an inherent gamma of around 2.5, so the video is preprocessed with a gamma curve of (IIRC) 1.8


So, the truely proper way to scale would be to undo that gamma preprocessing so that the space is again linear (Gamma of .55?), then scale, then reapply the desired final gamma curve.


Andy K.

 

[Charles Black]

Andy,


You are on the right track but a few of your numbers are incorrect. I'm no expert but I had just finished reading Charles Poynton's articles relating to gamma and thought it best to look up the relevant info again.

Quote:

To elaborate, video data is always processed through a gamma curve in the production studio to match well with a CRT display's inherent gamma curve.

The video camera is the usual place where the gamma function is applied.

Quote:

For example, NTSC is supposed to be displayed with a gamma of 2.2 .

For Rec. 709 television viewing the end-to-end exponent should be 1.25.

Quote:

CRT monitors usually have an inherent gamma of around 2.5, so the video is preprocessed with a gamma curve of (IIRC) 1.8

The video camera gamma correction is quite close to gamma 0.51.

Quote:

So, the truely proper way to scale would be to undo that gamma preprocessing so that the space is again linear (Gamma of .55?), then scale, then reapply the desired final gamma curve.

Why? Wouldn't that be much more computationally difficult than just adding two (or more) least significant bits for more precision like studio video. Also you would lose the relation between luma and perception for whatever thats worth.


Charlie

 

[Mr.D]

I hate saying video has a gamma. Video has a curve the straight line of which is often described as a gamma (similarly when film is described as having a gamma of 1.7 , its the straight line approximation of the curve(s)).


To properly inearise video it has to go through a more complex transfer function rather than a simple gamma.


Linearising the intensity curve if you like does help with more accurate pixel interpolation because the pixels generated have more meaningful relationship to the pixels they are derived from. Symptoms of not linearising prior to a scale are mushiness and softness because you tend to get pixels generated that are predominately too dark. Tends to make the image a bit crushed which can actually resolve as mushiness and lack of detail.


To be honest though it could well be too expensive computationally to bother with in a realtime system as the non-linearity of video is fairly slight in comparisson to other colourspaces

 

[Wilbert]

Quote:

Am I right in assuming ffdshow has no Catmull-Rom resizing?

It has BicubicResize. BicubicResize(x, x, 0, 0.5) is the Catmull-Rom spline.


At least, according to the avisynth docs ...

 

[kromkamp]

"To properly linearise video it has to go through a more complex transfer function rather than a simple gamma."


Please explain.

 

[Mr.D]

 http://people.ee.ethz.ch/~buc/brechb.../GammaFAQ.html


poyntons got a good graph that describes the transfer function for rec.709 anyway.

Its mostly gibberish to me ( I fly em I don't build em). It shows that the transfer function is not a straight easily expressed exponenet but for neatness sake everyone talks in terms of video's gamma ( exactly the same way in which film colourspace is referred to as having a gamma of 1.7 when in fact its an eyeball designed curve : I don't even think its expressable as a meaningful mathematical formula in its entirety)

 

[dmunsil]

For best results, scaling should in fact be done with linear-light values, not gamma-corrected values. And of course it should be done in RGB, not Y'CbCr, since Y'CbCr values aren't even gamma-corrected values, they're a linear combination of gamma-corrected values. Mathematically, processing in Y'CbCr is a complete abomination. Practically, it works sorta OK and requires a lot less CPU, so people go ahead and do it.


The best pipeline for scaling images in a Y'CbCr space is:


Y'CbCr->R'G'B'->RGB->[scale]->RGB->R'G'B'->Y'CbCr


Obviously the downsampling and upsampling of the chroma channel for 4:2:0 or 4:2:2 has an effect on the final image, but it can't be helped (other than by not using 4:2:0 or 4:4:4). There's no way to linearize Y', Cb, or Cr.


Don

 

[Charles Black]

Don,


Your post is a thought provoker.

Quote:

For best results, scaling should in fact be done with linear-light values, not gamma-corrected values.

Have you actually done a comparison between gamma corrected video and linear-light video scaling in a streaming system? If so what was the hardware you used? What would be the data rate of the linear-light stream. How much computing power is needed? How did you evaluate the results?

Quote:

And of course it should be done in RGB, not Y'CbCr, since Y'CbCr values aren't even gamma-corrected values, they're a linear combination of gamma-corrected values.

It is easy to transform Y'CbCr to R'G'B' and back again - I programmed my calculator to do this last June just to see if the Rec. 709 equations worked and it is trivial. Converting to R'G'B' to RGB (quantized) is just the inverse of the gamma encoding function but does have some computational overhead. One problem is that it is unknown what the gamma correction was for any particular video source but Rec. 709 and SMPTE 240 are similar enough to not matter (Ref. Poynton). If, on the other hand, you mean analog RGB then you will additionally need to have three 14 bit DACs running at about 25Mhz. For a 100 to 1 contrast ratio you will need 10,000 codes at a 0.01 spacing to get a smooth grayscale in linear-light (Ref. Poynton). Maybe it would be necessary to have more than 14 bit DACs if you want to preserve levels 1->15 and 236->254 since I don't think Poynton's 10,000 codes include them. The DAC speed is assuming a DVD 720x480 source and a 72Hz refresh rate.

Quote:

Mathematically, processing in Y'CbCr is a complete abomination. Practically, it works sorta OK and requires a lot less CPU, so people go ahead and do it.

Do you know of any specific problems processing Y'CbCr where the result is bad and not fixable with careful coding?

Quote:

The best pipeline for scaling images in a Y'CbCr space is: Y'CbCr->R'G'B'->RGB->[scale]->RGB->R'G'B'->Y'CbCr

Have you done a comparative experiment using this signal flow? If so what was the equipment that you used? How did you evaluate the results?

Quote:

Obviously the downsampling and upsampling of the chroma channel for 4:2:0 or 4:2:2 has an effect on the final image, but it can't be helped (other than by not using 4:2:0 or 4:4:4).

Do you have an issue with 4:4:4 other than data rate?

Quote:

There's no way to linearize Y', Cb, or Cr.

That's funny since you just recommended doing exactly that (Y'CbCr->R'G'B'->RGB...) in your best pipeline for scaling scenario.


Charlie

 

[Mr.D]

Linearising cineon film colourspace scans prior to filtering operations produces noticable improvements . Images looks sharper and they hold on to their fine intensity detail better . Although there are times when I won't lnearise prior to the filtering but this is mainly to do with getting a specific result rather than maintaining any improvement in accuracy.


linearising video on the other hand is something I tend to do , mainly because I've got an easy custom colour correct node that employs rec.709 sRGB or SMPTE 240M transfer functions. However I know many many people who don't bother ( some even call video colourspace images "linear" and will comp linear cgi on top of them without any real colourspace conversion and then monkey around with arbitrary colour corrections to try and get it looking somewhat nice)


This is on a visual effects workstation (10bit log rgb for film and 8bit rgb fo video : derived from D1 4:2:2 most likely, in either a 16bit linear or 32bit float workspace usually) and it sure isn't real time.


I suspect the advantage in linearising video prior to filtering isn't really all that significant an improvement in real terms but it is an improvement nonetheless.

I do suspect that on todays machines you could comfortably do it in real time but many developers either don't bother as no-one else does or simply don't know about it.

 

[JohnAd]

Quote:

Originally posted by Mr.D I do suspect that on todays machines you could comfortably do it in real time but many developers either don't bother as no-one else does or simply don't know about it.

I think you could probably just about delinearise and put things back again in real time in software but you wouldn't have any cycles left for anything else. The main issue would be the extra precision you would need in RGB linear space to be able to get back to where you started and at least obey "do no harm" to the video. More tricks may be possible using the graphics chips as floating point operations are faster but I think the killer would be the lookups required.


John

 

[Nima]

Quote:

Originally posted by Wilbert It has BicubicResize. BicubicResize(x, x, 0, 0.5) is the Catmull-Rom spline. At least, according to the avisynth docs ...

How does this translate into ffdshow settings ? Parameter, Luma and Chroma ?


Cheers

Nima

 

[VictorD]

dmunsil


"I adjust sharpness by combining a modified sharpening kernel with the basic scaling kernel."


Your info about scalling is VERY usefull, but can you provide some info about sharpening after resize ?

What is your recomendation from the "avisynth world" ?

Or maybe some info.

Thanx.

 

[Wilbert]

Nima,


I don't know how to do it by selecting the Resizer box (I don't think you can select the latter two parameters of BicubicResize). What you can do is the following:


You chould check the AviSynth box and add


BicubicResize(320,240,0,0.5)


in the text box (or whatever size you want).

 

[AndyIEG]

nice thread


atm im puzzling what i want do next, so lemme ask some question's.


Is the yuy2 to yv12 conversion done by ffdshow still a problem? Means is it importand to not do this conversion or is yv12 oki? If its a problem and we want to work on the yuy2 input many (better a small selection) of filters have to be changed to also work on yuy2 input direct.


For the resize algos... im still confused but i just looked at the Avisynth alpha 3.0 code wich has a clean and understandable implementation, the mplayer code is a mess if it comes to understanding.


Here is the code for calcing the coef's.


MitchellNetravali::MitchellNetravali(double b, double c)

{

p0 = ( 6. - 2. * b ) / 6.;

p2 = ( -18. + 12. * b + 6. * c ) / 6.;

p3 = ( 12. - 9. * b - 6. * c ) / 6.;

q0 = ( 8. * b + 24. * c ) / 6.;

q1 = ( - 12. * b - 48. * c ) / 6.;

q2 = ( 6. * b + 30. * c ) / 6.;

q3 = ( - b - 6. * c ) / 6.;


}


double MitchellNetravali:perator ()(double x) const

{

x = fabs(x);


return (x
: (x
: 0.0;

}



double Lanczos3:perator ()(double x) const

{

x = fabs(x);


static double const pi = 3.14159265358979323846;

static double const pi3 = pi / 3;


return (x
: 0.0;

}



So for lanczos2-10 its pretty clear, i just have to change the parameter for the sinc window.


If i understand the Catmull-Rom-Spline right its a normal Bicubic algo with b=0 and c=0.5?

While b=1/3 and c=1/3, are the org. Mitchell/Netravali values?


So u want lanczos2 to at least lanczos4 (maybe up to 10) and those 2 other settings for resize? If i also understand it right u want nearly 100% correct algo's since CPU time isnt a problem anymore?


Im still looking for other examples else than avisynth and mplayer for implementation of resize algos. Avisynth is the best to understand atm, but its pretty straight coded wich means not as tricky as the mplayer implementation. Is there any other project wich use resize algos and has opensource?


PS: are xvid rips encoded in yv12 or yuy2 or ..? How about DVD?

 

[Li On]

I think YV12 is fine. It's the fastest and 2 most popular decoders DScaler5 and NVDVD 4.0 (with NVPP) support YV12 out.


regards,


Li On

 

[karos]

andy


your ffdshow (in your sig) is earlier than the 10-12 release. is it more stable?

 

[AndyIEG]

Quote:

Originally posted by karos andy your ffdshow (in your sig) is earlier than the 10-12 release. is it more stable?

"more stable" ? im not aware that the actual versions are realy "unstable" but to answer your question, nope its not more stable.

 

[.....]

Quote:

Originally posted by AndyIEG PS: are xvid rips encoded in yv12 or yuy2 or ..? How about DVD?

AFAIK, both DVD, XviD and all other mpeg1, 2 and 4 use YV12.

 

[vairulez]

dvd uses both yuy2 or yv12 depending of the decoder