AVS Forum banner

Status
Not open for further replies.
41 - 60 of 111 Posts

·
Registered
Joined
·
3,510 Posts
Discussion Starter #41
The ringing is not added by Lanczos, Bjoern Roy has good evidence on his site.
 

·
Registered
Joined
·
9,884 Posts
Quote:
Not sure I understand. What is the scenario?


Personally, keep the image as big as possible for as long as possible. We recently started off with some 4096x1714 content. It was 16-bit per channel (4:4:4) or RGB48. The end results was 1280x536 8-bit 4:2:0.
Stacy -


I was actually talking about the funnel video typically goes through when compressed for delivery. We start with an oversampled video and then maybe make a nice processed compressed video for delivery on some medium.


But the story doesn't stop there as this thread shows. At display time it is often desirable to display at greater than the delivered resolution, if only to hide the raster or pixel structure.


The idea of having 3 separate capture, delivery, and display resolutions was actually best presented to me by a Microsoft paper. See section 2 of Converging Computer and Television Image Portrayal by John Watkinson .


But to me this means that as we choose or design our downscaling method we should probably be considering the eventual optimal upscaling when we evaluate the possible quality.


For instance Don above mentioned customizing the scaling a bit to add sharpening. But should that be done (if at all) during the downscaling, before compression? Or should something like that be best deferred to the later stage, sharpening while upscaling? (better compression but sharpened compression artifacts)


- Tom
 

·
Registered
Joined
·
335 Posts
Quote:
Originally posted by Jesse S
The ringing is not added by Lanczos, Bjoern Roy has good evidence on his site.
Please direct me there...
 

·
Registered
Joined
·
335 Posts
Quote:
Originally posted by gazzagazza
In another thread that I forget Charles Black posted some scope pictures of ffdshow scaling and the ringing on all Lanczos options was pretty obvious...
Actually its in the ffdshow thread page 161...

Filters that ring
 

·
Registered
Joined
·
1,005 Posts
Quote:
Originally posted by trbarry
...when working with integer arithmatic it is possible when you aren't careful to have a slight downward truncation bias. And when working in YUV color spaces this will appear as a greenish shift.
Exactly. It's really best to do all the calculations in floating point, even if the input and output samples are integers. Careful rounding on the conversion back to integer is also recommended.

Quote:
You've suggested, and I believe, that different algorithms may be appropriate depending upon whether you are upscaling or downscaling. Any further comment?
In a sense you can think of upsampling as interpolation and downsampling as lowpass filtering plus interpolation. For speed and convenience most scalers tend to use the same filter kernel for both, and honestly I think that works out fine.


When I first started working on scalers, a video expert told me that downsampling required very long (many lobe) filters in order to get reasonable frequency response. I went down that road for a long time, but long filters produce lousy-looking video, IMO. It took me a long time to really understand why. The key insight is this: frequency response in digital image scaling filters is irrelevant. Completely and totally irrelevant. You will find many "experts" who disagree, but they're wrong and I have the math (and the practical experience) to prove it.


The very best image scaling filters combine smooth interpolation with edge sharpening. The smooth interpolation is important so no false edges are added to the image. The edge sharpening is important because our visual system is very sensitive to edges, and smooth interpolation without sharpening reduces the slope of edges, making the image look soft.


The "perfect" scaling filter would produce a lovely smooth interpolation of objects in the scene while keeping the edges perfectly sharp. Such a filter does not (to date) exist, so we make do with compromises. Catmull-Rom is such a compromise, and it works pretty well. The one downside to it is that it's not quite sharp enough for most people. It's sharper than bilinear, and also a better interpolator, but many folks prefer a sharper image.


Thus the big practical issue with a scaling algorithm is how to get a sharper image while retaining the smooth interpolator. Many-lobed windowed sincs (like Lanczos3 and above) are one answer, but they really aren't appropriate for image scaling because they produce ripples around edges. Mathematically speaking, they don't at all reproduce the scaling of "reality."

Quote:
And if you are willing, any comments on downscaling when you expect to upscale again later? (very often the case, so it could be optimized with that intent)
Don't ever downsample if you don't have to. If you do have to downsample, use the best algorithm you've got. If you are resampling and you know you'll be resampling again later, don't do any extra sharpening on the first pass. Sharpening of any kind should only be applied at the very end, because sharpening must be tailored to the display environment. The appropriate amount of sharpening for a 32" display viewed at 8 feet is completely different from the amount of sharpening appropriate for a 96" screen viewed at 12 feet.


Don
 

·
Registered
Joined
·
1,638 Posts
Quote:
Originally posted by gazzagazza
Please direct me there...
"Again, the reason you see a bit more ringing for example in the Lanczos Unbreakable sample is NOT because it 'adds' that ringing. Its because the transfered frequency response is much better with that filter, thus those higher frequencies of which the ringing consists, are less subdued, thus more visible. So the bilinear filter 'blurrs away' some of that ringing. The problem is, that 'real detail' in that freqency range are blurred away as well."


http://www.videophile.info/Editorial...EE/Page_01.htm
 

·
Registered
Joined
·
1,005 Posts
Quote:
Originally posted by Jesse S
The ringing is not added by Lanczos, Bjoern Roy has good evidence on his site.
Bjoern is comparing bicubic and bilinear filters to 2-lobe Lanczos. Two-lobe Lanczos is fine. It's Lanczos (or any other sinc-based filter) with more than two lobes that gets problematic.


However, if you're going to use 2-lobe Lanczos, you might as well use Catmull-Rom since as mentioned previously the two are virtually identical. The reason you see any difference between the Lanczos filter and the "bicubic" filter in various image processing apps is that the bicubic they are using is generally not Catmull-Rom. Typically it's a Mitchell-Netravali filter with the coefficients tweaked to be "sharper."


Can you tell that I like Catmull-Rom a lot? I do. I like Lanczos2 as well, but again, the two are practically identical.


Don
 

·
Registered
Joined
·
1,005 Posts
Quote:
Originally posted by trbarry
For instance Don above mentioned customizing the scaling a bit to add sharpening. But should that be done (if at all) during the downscaling, before compression? Or should something like that be best deferred to the later stage, sharpening while upscaling? (better compression but sharpened compression artifacts)
In the best of all possible worlds (IMO), all consumer DVDs would be carefully scaled from a 1080p master using straight unadulterated Catmull-Rom with no additional sharpening at all. Then the displays would sharpen the image appropriately as a final step.


However, the reality is that people like sharp video, and thus the people making DVDs tend to sharpen the image so it "snaps" on a 20" monitor. That level of sharpening looks really excessive on a 80" screen, or even a 50" screen. And once the image has been sharpened, it's practically impossible to unsharpen it. In addition, the sharpening has been applied at the resolution of the DVD, not at the resolution of the display device. If you have a 1280x720 display, you can do a much better sharpen than the DVD maker can do at 720x480.


Sharpening is not a bad thing. Not a bad thing at all. But it really should be done as a final process, at the final resolution of the display device.


Don
 

·
Registered
Joined
·
9,884 Posts
Quote:
The key insight is this: frequency response in digital image scaling filters is irrelevant. Completely and totally irrelevant. You will find many "experts" who disagree, but they're wrong and I have the math (and the practical experience) to prove it.
Well, I'm certainly not an "expert" and I'm not sure I even disagree. But there is one part of this I at least worry about.


To me there seem to be at least 2 aspects of sharpness. One is the sharpness of edges. Our eyes like nice sharp edges and they likely help us a lot with eye motion tracking. Sharp edges certainly make the image look less soft and I think are the basis for your comments above. And sharpness of edges is probably not well reflected in frequency domain measurements.


But it does seem to me that at least my own eyes also measure sharpness by the amount of visible "texture". Textured objects also can look either sharp or soft and I don't think it is just an edge phenomena. Instead it does seem to me more a matter of frequency response, and MTF. That is, maybe texture is repeated higher frequency info over some area of the image, possibly perceived by the eye without much phase info. This is just my own perception and otherwise just quack science but if I'm at all right then preserving this texture info might be important, over and above sharp edges.


Comments?


- Tom
 

·
Registered
Joined
·
1,005 Posts
Quote:
Originally posted by trbarry
To me there seem to be at least 2 aspects of sharpness. One is the sharpness of edges. Our eyes like nice sharp edges and they likely help us a lot with eye motion tracking.


(snip)


But it does seem to me that at least my own eyes also measure sharpness by the amount of visible "texture". Textured objects also can look either sharp or soft and I don't think it is just an edge phenomena.
You're absolutely right about the sharpness of texture being important to perception of sharpness. It's of secondary importance to the big edges we tend to think of - the outlines of objects - but it's important.


However, texture is also an edge issue. Texture in reality is all about the juxtaposition of unlike-colored bits. Keeping those bits sharply contrasting up to the Nyquist limit (or as close as we can get) is what sharpness is all about.


However, it's worth noting that when talking about digital images the Nyquist limit is misleading. In audio, I really can represent a 75% amplitude sine wave that is just under (say, 15% under) the Nyquist limit and it will come out of the speakers sounding just fine assuming I've got a good D/A converter. In video, a pattern that has a frequency just under the sampling limit of the CCD in the camera will look absolutely awful.


One key insight is that audio is evaluated by our sensory apparatus in frequency space. The ear is a frequency analyzer. Vision is evaluated by our sensory apparatus in amplitude space. Our eyes are not sensitive to spacial frequency per se. Edges in audio are largely meaningless, wheras in video they are everything.


The other key insight is that the most frequency accurate image scaling filters have terrible ringing that does not represent any kind of real-world image artifact that exists. The blurring that happens as you move away from an object in real space is not analagous to a sinc function. If anything, it's closer to a gaussian, but it's not even that.


When you think about what "real" scaling represents, it becomes clear that spacial frequencies are not relevant at all. Take two possible examples of real scaling: look at a scene through a window at 3 meters, then walk to 6 meters and look at the same scene. Or imagine talking a digital photo of a scene with a 6 megapixel camera, then switching to a 3 megapixel camera and taking the same picture. Your goal with a scaling algorithm is to as closely as possible replicate those scenarios.


One thing becomes clear: you don't have enough information in the image to replicate either of those scenarios. The best you can do is make a guess and try to come up with an image that is a reasonable approximation of the image you'd really get if you were to shoot that same scene with a lower-resolution device.


Several researchers have attempted to develop scaling algorithms that try to use an iterative heuristic to actually physically model the lower-res CCD, but it's so far quite computationally expensive, and in the end you get just one of many possible image results, and the overall quality of the scaling isn't really significantly better than Catmull-Rom. It's still promising as an area of research, though.


For now, the three most important things to keep in mind for image scaling:


1. Preserve edges.

2. Preserve amplitude locality (i.e. don't distribute image information too far from the original pixel)

3. Interpolate smoothly.


- Bilinear is bad at 1, good at 2, and not great at 3.

- Straight Catmull-Rom is pretty good at 1, pretty good at 2, and great at 3.

- Many-lobed sincs are good at 1, terrible at 2, and great at 3.

- Catmull-Rom plus sharpening is great at 1, very good at 2, and great at 3.


Don
 

·
Registered
Joined
·
9,884 Posts
Okay, lots to think about. More later. ;)


- Tom
 

·
Registered
Joined
·
7,512 Posts
Quote:
Originally posted by Bob Sorel
I'm not. Science and religion don't mix well, and many people would rather continue believing on blind faith rather than be presented with facts. Facts just tend to cloud the issues and are not going to change anyone's religion anyway, so why discuss them?
It is amazing people will believe in atoms and molecules moving in a solid object like wood, yet they can't see it. Ask them about faith or belief in a being greater than themselves and...NOT!
 

·
Registered
Joined
·
529 Posts
Quote:
It is amazing people will believe in atoms and molecules moving in a solid object like wood, yet they can't see it. Ask them about faith or belief in a being greater than themselves and...NOT!
It is one thing to accept a postulate based on reports of multiple experiments (most designed to DISPROVE the theory) by multiple independent people. (Besides, a good tunnelling microscope can 'see' molecules.)


It is quite another to accept a postulate based on the premise that "it must be so" as defined by some particular 'interest group'.


You're far better off to start with Godel's Incompleteness Theorem.
 

·
Registered
Joined
·
1,005 Posts
Current video cards use a variety of scaling solutions, ranging from some form of bicubic to trilinear. What algorithm is used depends on whether you're using the overlay or the VMR, whether YUV mixing mode is turned on, what generation of the hardware you've got, the size of the image, and so forth.


I just talked to one of our experts on video cards here, and all the different caveats and exceptions made my head swim.


One thing to consider is that typically hardware image processors have a fixed number of available taps. So if the theoretically correct bicubic scaling algorithm requires 10 taps to do a particular scaling operation, and the hardware only has 6, they have to make do. Thus really large downscaling operations tend not to be "correct" bicubic, even if the card has the capability of doing bicubic.


Around here, we tend to prefer using VMR in YUV mixing mode, but even that won't guarantee what scaling algorithm you're going to get. Sorry I don't have a simple answer.


Don
 

·
Registered
Joined
·
2,164 Posts
Boy am I glad we have this thread. It was always assumed that upscaling requires the more accurate algo with the video cards downscaling being more than enough to handle the downscale to display device res. Our eyes tend to confirm this and we were happy. Now we have all these other proofs and theories to think about... which is always nice. I do hope we can come to some conclusions or at least some general guidelines... for instance if it is established that Lanczos2 is all that is needed for good upscaling then that would make many people with moderate hardware happy.


Cheers...

Duy-Khang Hoang
 

·
Registered
Joined
·
513 Posts
Quote:
for instance if it is established that Lanczos2 is all that is needed for good upscaling then that would make many people with moderate hardware happy.


Cheers...

Duy-Khang Hoang


if this is true, then I won`t have to spend $1000CAN for a new system I was planning to purchase!!!!
 

·
Registered
Joined
·
1,358 Posts
Quote:
Originally posted by hdkhang
... for instance if it is established that Lanczos2 is all that is needed for good upscaling then that would make many people with moderate hardware happy.


Cheers...

Duy-Khang Hoang
Right now the only Lanczos2 is in SSE2 optimized version of FFDShow and it definitely has issues. Lanczos3 as well. Once you get to Lanczos4 it looks good. Beyond Lanczos 4 it rings excessively. Avisynth has Lanczos3 and Lanczos4 and that's about it.


It would be great if Andy could implement at least current FFDShow SSE2 algos with 32 bit floating point precision. That would tighten up the rounding errors and potentially improve image quality. It would be even better to implement 32 bit floating point on a new and improved algo, but the obvious question is "what algo".


As far as what hardware will you need depends on your preferences. If you want to run filters after 720p resize, you will still need a P4 3.0, which is a moderate machine nowadays. I still like to run a filter after Lanczos4 resize and ASharp is IMO the best. It gives you ability to balance between texture sharpening and edge enhancement which is unique. Also can improve appearance of contrast in the image, which is great for a number of digital displays.


As far as the video card, if you are planning to use VMR9, you will need either a solid card like Nvidia FX 5700 Ultra, or a very good card like Nvidia GeForce 6600 or 6800 series. If you want the best, you will need to go for Quadro FX 3400+. Although Quadro cards are (almost) identical to their GeForce counterparts, Quadro drivers provide greater precision in image assembling and rendering. With VMR9 renderer Video card does much more than just scaling and driver support is much more important than with overlay. It's the entire 3D pipeline that assembles and renders the image, not a dedicated portion of the chip like in overlay mode.
 

·
Registered
Joined
·
491 Posts
Quote:
Originally posted by vpopovic
Right now the only Lanczos2 is in SSE2 optimized version of FFDShow and it definitely has issues. Lanczos3 as well.
What kind of issues Vlad?

Quote:
Originally posted by vpopovic
It would be great if Andy could implement at least current FFDShow SSE2 algos with 32 bit floating point precision. That would tighten up the rounding errors and potentially improve image quality.
How many bits precision today?

Quote:
Originally posted by vpopovic
It would be even better to implement 32 bit floating point on a new and improved algo, but the obvious question is "what algo".
What is ffdshow Lanczos today, Catmull-Rom or Spline? Obviously, we need Catmull-Rom, but is Andy really the right guy to develop this? Integrate the code, yes, but to write the algo itself?

Quote:
Originally posted by vpopovic
I still like to run a filter after Lanczos4 resize and ASharp is IMO the best. It gives you ability to balance between texture sharpening and edge enhancement which is unique. Also can improve appearance of contrast in the image, which is great for a number of digital displays.
What is aSharp really doing to the image, in technical terms, especially at the settings you prefer to run it at?


Have you played with LimitedSharpen as suggested by Socio?
 
41 - 60 of 111 Posts
Status
Not open for further replies.
Top