Video pre-processing - Page 6

I think that route would end up with a sharper image than doing a fractional resize. See the examples from Mr.D above of a 2k resized to 1920x1080 versus cropped, 4k to 1920x1080 would have similar issues although less pronounced.

John

Seems to me, for 1920X1080 you're always getting the full format resolution. It's just that for 'artistically' camera-filtered and selectively focused productions the higher frequencies/resolutions lack enough contrast to be clearly resolved; they merge into shades of gray (for luma or B&W). The Nyquist limitations of sampled signals (cameras, telecines/scans) limit this resolvability, too. By contrast, non-sampled signals, like 1920X1080 test patterns (computers, generators) do provide full resolvability; (discussed here in more detail).

Downconversion to 1920X1080 from 2k or 4k tends to make images appear sharper because the contrast of lower and mid-range frequencies/resolutions are boosted. Pictures can thus appear sharp without offering significant fine detail. Figure 8 in an Arri pdf paper contrasts the resolution-vs-contrast graphs of higher-resolution and 'sharper' camera images. Fig. 22 in this Arri paper shows how even with a 10k film scan, multiple stages of processing greatly reduces the final resolution (multiplication of the various MTFs).

Determining what resolutions survive onto home screens with 4k transfers/downconversions to Blu-ray , such as King Kong, seems to require spectrum analysis to distinguish between sharpness from boosted mid-range and perhaps slightly higher frequencies. For example, does it 'jump' from, say a 800--1000-line guesstimate for typical Blu-rays to only 1200? As the film-camera photos (Fig. 6) in the Arri paper show, sharpness created by the boosted contrast of coarse details can easily be misinterpreted as higher resolution. The author (below Fig. 6) draws an interesting evolution comparison involving monkeys. -- John

Proper resampling doesn't care at all about whether you scale by exactly 0.5 or maybe 0.500001. No difference at all in terms of image quality, as long as we're talking about true video content and not about computer type test patterns. What makes a difference is whether you scale once or twice, which scaling algorithm you're using, and whether you're scaling in linear light or gamma corrected light.

Which means: 2k resized to 1920x1080 is bad because 2k was already resized from the 4k master. So it's bad because you have scaled twice. If you go directly from 4k to 1920x1080, image quality should be identical to scaling 4k to 2k and then using cropping.

In theory maybe, in practice it does mattter, for practical algorithms scaling down by 1/2^n is more likely to produce an artifact free image.

That does not match my experience. Not with video type images when using reasonably good resampling algorithms.

Fair enough, which resampling methods are you thinking of? I usually notice a slight uptick in quality when going through powers of 2, say around branches, hair etc.

I like Lanczos4 with some custom tweaks to avoid ringing. Do you happen to have a sample image flying around which shows this power of 2 effect clearly? I'd like to see if I can reproduce it. Thanks!

I'll have another play tomorrow and get an example together.

John

Great, maybe you're right and I just missed it until now...

Downrezzing multiples prevents additional aliasing.

If you're working on an audio project with the intention of releasing a CD, it's the same theory. It's better to work at 44.1 kHz or 88.2 kHz rather than multiples of 48 kHz.

OK here is a quick test, this the original image, a crop from a Blu-ray



Here is same image resized by half using sinc3 in gimp not in linear light and then doubled with point scaling



now with slightly smaller than half



and a bit bigger than half



The bit more than a half and the exactly a half are quite close but I think the half one suffers from slightly less shapening and the detail degrades in more natural way. The less than half one is more obviously worse.

John

Interesting! Seems to me that Gimp is dropping the ball. Here's my own Lanczos4 implementation:

0.5:


0.5 - 1 pixel:


Your 0.5 image is sharper than any of mine. That's caused by the resampling filter you've chosen. "sinc" is even more agressive than "lanczos" (sharper, but also more ringing). So your 0.5 image is a tad sharper than my 0.5 image. However, my "less than half" image is noticeably sharper than your "less than half" image. I don't see any difference worth mentioning between my 2 images. Do you?

Please note that the resampling implementation I've used to produce these images is a simple one using gamma corrected light and integer math. My DirectShow video renderer uses float32 and (not yet, but soon) linear light scaling. So it should produce even better results, although the difference will probably be small...

It's small but say the shade of underside of the broom and the middle crane seem to change between the 2, again I find the half one closer to the original image.

Your downsampling looks pretty good though

John

Hmmmmm... I'm trying (really), but I don't see it. Here are 2 more images:

Lanczos4 x0.5 -> Lanczos4 x2.0


Lanczos4 x0.5 - 1 pixel -> Lanczos4 + 1 pixel x2.0


Load them both in separate MSPaint instances (or browser tabs) and switch back and forth. I can't see a difference, can you? The 2nd image was scaled twice with fractional scaling factors while the 1st image was scaled twice with very clean scaling factors. Still both look virtually identical to me. Try that with Gimp, I think the result of the fractional scaling will be ugly.

Or compare this one to the original Blu-Ray crop:

Lanczos4 x2.0 - 1 pixel -> Lanczos4 + 1 pixel x0.5


I can't see a difference between this and the original. Again, my image was scaled twice with fractional scaling factors.


Thanks! But it's really only a simple standard Lanczos4 implemention. No fancy special features, no tweaks, not even linear light scaling.

We are not talking about big effects, it is 1 or 2 in RGB terms.


Again it's very slight but I see some of the tones of the fine structures change slightly, with color picker it's only 1 or 2 different but still different. Also I see there is quite a lot of ringing with your upscale, say along the back of the sweeper and a line under the broom.


agree they look very similar but upscaling then downscaling isn't what we're talking about as there is no information loss


I just meant it's nice to see you having it working well in your renderer.

John

Yeah, there are minimal (very minimal) differences. And there have to be, after all the scaling operation is different. But who decides which image is better? I have a hard time seeing the difference at all. I'd have an even harder time deciding whether the 0.5 is better or worse than the fractional one. Furthermore, given how small the differences are, with one sample image you may prefer 0.5x, and with another sample image you might prefer fractional scaling (or not). Looking at my own sample images I'm not convinced that fractional scaling is any worse than non-fractional scaling.

Based on the Gimp images I'd definitely agree with you, though. It just seems to me that Gimp's resampling engine is broken.


True. Lanczos4 rings (a lot sometimes). I'm working on a tweaked implementation without ringing...

Good question, I judge it on closeness to original, particularly in terms of visual weight between the elements and the similarity of colours. Personally I think test patterns are very useful in this context for highlighting issue with certain angles, frequencies etc.

Looking back to the first page for Stacey's demo of linear light scaling, I'm looking for similar kinds of thing. I'll be on the lookout for a better torture test image, that one isn't that helpful.

John

The problem with test patterns is that resampling algorithms simply don't work well with computer type data (1 pixel on/off stuff). Real photo/video stuff never contains such extreme contrast jumps. That's like trying to resample a pulse train in audio land. It's useful to draw some specific conclusions. But if I e.g. chose an audio resampling algorithm based on pulse train resampling results, I'd end up with real bad sounding real world audio...


Good samples for resampling are always welcome/helpful!!

I'm not saying use only test patterns, I'm saying that it can be useful to have extreme data to make stuff that's only just visible with normal video really stand out.

John

From the examples posted it seems it's not worth cropping the image for such microscopic differences.

Thanks! Of course it would have been great to have the 4k version, too, cause that would have allowed us to compare a direct 4k -> 1920 resize. But 2k and 1920 crop/resize images should make good test material, too.

My original point was more that in the context of the current process, some implementations of fractional rescale are not that good. Those who have seen both the 2k and the final blu-ray say there is a noticable difference in quality on some titles, particularly a loss of resolution that is bigger than can be explained by 4:2:0 and compression. I think I've shown above that a badly implemented <0.5 resize can produce a noticable blurring, it would be similar to the effect of the 0.928 resize that would be required for 2k to 1920. Similarly madshi has shown that with a well implemented resize it is barely noticable on a random image sample.

Based on that I'd still say that if by some magic I could reconfigure all the HD production chains with one simple request, under my personal trade-offs my rule would be would be crop the 2k to avoid a potentially poor resize. This is because for me the chance of loss of resolution beats a few extra safe area pixels. If by further magic we had full 4k chain for all films I'd still vote for .5 resize then crop, if only because again there is some chance that the fractional resize may be bad.

I'm not in any way suggesting that it is impossible to do the resize well, I'm just saying it is sufficently hard to do resize well that a number of implementations seem to give poor results at fractional factors.

Alternatively we could easily want to magically fix every resize implemetation but I suppose I was just restricting myself to certain kinds of magic power.

John

Reasonable point of view, I can agree with that.

Intriguing images, not a big difference but definitely a little softer when resized, the wine bottle loses the slightest bit of HF, and you can see that more too in the way the light pinstripes in the two guys' suits darken in the resize.

But I would have to say that the more obvious difference is of course the AR change.

I don't really know how I feel about that, to be honest I think I am more concerned about the AR change frankly. I'm curious how others feel about that.

That sized crop is well within the tolerances of filmgates on lots of film equipment including the camera. Most films are cropped in slightly relative to the fullap film frame. I'm looking at a big budget film from one of the most respected directors working in film today.

The HD video rushes are a 1920x1080 crop from the 2k. The aspect ratio test slate for the film ( its rubbish and very innacurate as usual) is way more compromised than a 1920 crop from a plate.

Look at the Labrynth comparisson thread for more examples of varying film gate tolerances and HD extraction from 2k.
I for one am much happier wit the 1920 extraction from 2k rather than resize and that includes imagery I created.
I'd rather there was true 2ks worth of resolution than some mushed up image thats mistakenly trying to preserve some irrelevant bits of frame that the DP didn't see when he shot it, the editor didn't see when he cut it and no-one saw when it was projected in a theater.


Also bear in mind most people (including key personnel on the crew) never see the actual film scan in any meaningful way. They see it on video rushes or they see it written back to print and projected.

Thanks for the images, Stacey. I'm a bit torn. I think I prefer the cropping, but I'm not really sure.

One thing that I noticed is this in the 2K image:



Is this ringing coming from the 6K -> 2K downscale? Or is it even in the 6K master? Can't you use a downscaling algorithm which doesn't ring? E.g. Mitchell instead of Catmull-Rom?