OK, if I keep waiting until I get a chance to build graphs I'm never going to get to this, so let's see if we can continue where we left off, sans visual aids.
Originally Posted by madshi
If you were right shouldn't Bilinear scaling produce the best results (apart from not doing sharpening)?
No, because the whole point is that the quality of the scaling is determined by how natural-looking the interpolation is. There's no particular reason to think that a straight-line interpolation will look good, and in fact multiple theoretical reasons to think it wouldn't look very good. And in fact it doesn't look good. It's better than nearest neighbor, but still not that great. Sharpening is only part of it.
You seem to think that using more than 2 pixels (more than Bilinear) only helps sharpness. I'm sorry to say, but I think you're wrong here. Using more than 2 pixels helps sharpness, but it also allows a smoother (less jaggied) interpolation of curves.
I absolutely grant you that in the specific samples you posted, some of the curves looked better using 4 lobe filters. But there's no specific theoretical reason to think that a 4-lobe would produce smoother curves across the board. They do in many of the cases in the specific image that you provided and in the lighthouse, but I'd want to see a lot more images before I'd make a blanket claim that 4-lobe in general produces smoother curves.
But even the ones that looked better, looked better because of a specific change in the filter curve at the edge - the specific curve interpolated in between a black pixel and a white pixel. Using energy gathered from 4 pixels away to decide what shape the interpolation curve should have makes no physical sense. It makes sense when you analyze interpolation in frequency space, but our eyes don't analyze edges in frequency space.
Moreover, if you decided you liked that interpolation curve better, there's no reason to go look at the pixels far away to get that curve. Just design a shorter filter kernel that produces that specific curve. Done.
The thing is, if we gather information from many taps, then we end up with energy migrating across the image in a completely non-physical way. No optical or imaging system causes the kinds of ripples that long sinc filters produce.
Here's a thought experiment. Imagine curved edge on a black field sitting on a white field. The interpolated curve is awesome-looking. I drop a black pixel 4 pixels away from the edge. Should the edge interpolation change? Of course not. In reality, I can stick a dot in the image, and that doesn't tell you anything about an edge 4 pixels away. What if the pixel is bright red? Should the edge interpolation incorporate some red? Or the complementary color? If I move the red pixel left or right, it changes whether the edge is slightly red or slightly cyan. Why? Does that make physical sense?
Say I'm interpolating in one dimension and I have as my original pixels this line (in a single channel):
1 1 1 1 0 0 0 0
The interpolation between the middle pixels (the 1 and 0) is going to be some curve. Depending on our algorithm, it'll be some shape which will look better or worse. If you believe the Lanczos4 is a good idea, then you're arguing that I should use a subtly different curve between that middle 1 and 0 if my original data is:
0 1 1 1 0 0 0 1
0 0 1 1 0 0 1 1
That makes no sense, physical or otherwise. Changing a value four pixels away cannot possibly have any physical effect on the local image area. If the local interpolation gets better, it's a coincidental effect of Lanczos4 happening to have an interpolation curve that is "smoother looking," but there's no particular need to gather all the extra data in order to have that smoother looking curve.
My "non-ringing" algorithm is based on allowing ringing in some areas of the image and not allowing it in others. So I think it could be tweaked a little to accommodate your likings, but I haven't tried that yet, because personally I can't stand any ringing, no matter how small it is. The only reason that my algorithm allows ringing in some parts of the image is that if I suppressed any and all ringing, the image would lose all its smooth curves for whatever reason.
I think your algorithm is a huge improvement, but I still think it's sub-optimal to use a ringing filter, then suppress the ringing. You can easily design a short filter that will produce the exact same curve on the edges you care about, but doesn't ring in the first place. Of course, maybe that's what you're doing, in which case: I approve.
But look at the specific curve generated between a black and white (or dark gray and light gray, etc.) by your favorite filter and ask yourself - how could I design a short filter that would produce exactly that curve on that same edge. And would that be the optimal curve across the board? The answers should be interesting.
Please check out especially the up/downscale. If you look at the ringing Lanczos4 up/downscale, you should notice that it looks almost identical to the original, while the Catmull-Rom result looks a lot softer.
Going up then back down to the same size isn't actually the best way to test a filter. Lousy filters do seriously degrade the image when doing it, but as you note, doing upsampling to a simple integer ratio followed by downsampling to the original size makes it seem like long sinc filters are optimal, because they preserve more of the original image. Since the filter lobes during the downscale exactly line up with the ringing in the image, the ringing you get on the upscale gets largely erased on the downscale. That doesn't tell you whether the upscale looks good or not, and I think we all agree that many-lobe filters have issues (though maybe not so much with your improvements).
Anyway, this came out sounding negative and I didn't intend that. I love what you're doing, and I hope you take my arguments as encouragement to continue. Keep it up! Dig deeper! Prove me wrong!