Hi all
I'm forever seeing threads wander off into interesting territory, only for them to be closed or for people to say "Let's get back on topic".
This is entirely understandable, but I quite enjoy the organic nature of some of these threads and they often lead to some interesting thoughts.

So I thought I'd try a little experiment to see if there was any mileage in a kind of general discussion thread. It could fall flat but hey, why not...

Here goes...
Nothing is Off Topic as long as it relates to Video Processing.
Please try and keep the thread reasonably organic, so that people can follow it. (i.e. relate your post to something which has gone before).

I'll start the ball rolling in the next post so that people don't think that it's the thread's sole topic.

Motion Compensated Deinterlacing and inter frame creation (i.e. inserting newly created frames in between the existing 24 to smooth motion) are often talked about as being the 'next big thing'.

The only time I've seen anything like it is on PC solutions (both realtime and pre-rendered) and they have introduced as many distractions as they eliminate.

So has anyone actually seen an algorithm in action and liked it?
Do heavyweight players like S&W, Terranex, SiI etc have any super-powerful algs in the wings waiting until processors can cope with them realtime?
Has anyone seen any of these in action (even if not realtime)?

Anyone like the Meridian/Faroudja MCTi demo at CES?

Motion Compensated Deinterlacing and inter frame creation (i.e. inserting newly created frames in between the existing 24 to smooth motion) are often talked about as being the 'next big thing'.

I think this technique produces very subjective results. I personally don't care for the appearance of MCTi (or the Philips version), and I know many others who do not. In its current incaration, it doesn't look right.

Just my opinion:

De-interlacing whether or not motion compensated should be a separate step one from interframe creation being a distinct step two.

Interframe creation should work equally well with de-interlaced frames with motion compensation, de-interlaced frames without motion compensation, and frames progressive to begin with.

Video hints:
http://members.aol.com/ajaynejr/video.htm

An earlier thread that included motion-compensated deinterlacing discussion mentioned processors still weren't adequate cost-wise/power-wise for consumer applications. Couldn't help wondering if Intel's announcement earlier this year about a new multicore processor might change things. Might be confusing to mix the concepts of motion-compensated deinterlacing with new-frame insertion, despite some similarities. -- John

Sorry - didn't mean to suggest they were somehow linked - it's just that I've seen mocomp look really bad too - has anyone seen a good mocomp implementation?

Regarding inter-frame creation, here's what the guys at the Communications Research Centre up in Canada have cooked up: Advanced video frame rate conversion via object-based motion-compensated interpolation (http://www.videsignline.com/showArticle.jhtml?articleID=197008273).

I often push topics back on track but rarely close them here (lucky for us, we don't get into many heated debates anymore...).

My favorite topic regarding experimental VP is about super resolution. I would really love to see a realtime VP be able to extract superresolution.

De-interlacing whether or not motion compensated should be a separate step one from interframe creation being a distinct step two.

Interframe creation should work equally well with de-interlaced frames with motion compensation, de-interlaced frames without motion compensation, and frames progressive to begin with.http://members.aol.com/ajaynejr/video.htm
Absolutely. Most people think about interframe creation when motion compensation is discussed. De-interlacing with motion compensation however will produce more detail if done correctly. Broadcast-grade equipment has had this for some time.

I am thinking that the Realta processor could be used for this if other number crunching intensive functions like noise reduction is moved to either a separate box or to a Realta number 2.

My favorite topic regarding experimental VP is about super resolution. I would really love to see a realtime VP be able to extract superresolution.
Hi Ofer - I've not heard the term superresolution before - can you enlighten me?

he refers to upscaling 1080p to 4k or the like.

It will be needed very soon. I know several companies working on 4k chips for projection devices in more moderate price regions than the CineAlta 4k implementations.

And once the displays are there, there will be a need for better upscalers than the built-in ones.

The Sony SRX-R110 already has a very good upscaler that takes 1080i/p signals to 4k, but it is not quite a consumer level PJ :)

If I know Ofer, he's not talking about upscaling but rather using an iterative process to enhance and create additional resolution (making use of the information available in existing frames).

When this works, the results are very nice!

We need a better term for this since it will continually be confused with SuperResolution (4K pixel size).

If I know Ofer, he's not talking about upscaling but rather using an iterative process to enhance and create additional resolution (making use of the information available in existing frames).
So is this kind of process more akin to the vector algorithms used in interframe creation? Are there any photoshop plug-ins which do this on 2D images so we can take a look at it in action?

Here's an example of what is possible:

http://www.youtube.com/watch?v=6Naxs2jaXdI

I often push topics back on track but rarely close them here (lucky for us, we don't get into many heated debates anymore...).


Yes we do !!! (with monty python smile).

Daniel.

If I know Ofer, he's not talking about upscaling but rather using an iterative process to enhance and create additional resolution (making use of the information available in existing frames).

When this works, the results are very nice!

We need a better term for this since it will continually be confused with SuperResolution (4K pixel size).

vector based scaling is still scaling.
Hicon³² supports that, but I don't remember if it does more than 2k.

A solution that works very well w/o employing vectors is Algolith's Content Adaptive Scaling:
http://www.algolith.com/index.php?id=cas&L=0

It's available as a plug-in for Fusion, Shake and After Effects, and iirc can be used in Avid editing systems through Elastic Gasket

On the English forum, Gordon Fraser remembered that Snell & Willcox created new frames for the Matrix Bullet Time sequences. This wasn't in realtime, but I have to say, there are no visible artifacts on that. Is this commonly used?
Unless, the actual information created was only on faces and the rest was CGI. If so maybe the artifacts would be hard to spot.

Those Algolith CAS demo images are poorly created - not the same frame and not shown at the same size. What would be better would be a side-by-side of Bicubic vs CAS on the same frame.

I don't hear alot of complaints about the current levels of scaling performance. In fact, I hear lots of positive things about Lumagen's scaling solution. Am I missing something?

Well I guess we're only comparing these scaling solutions against one another. People like Lumagen's alg, because it doesn't add ringing. Which the DVDO scaling does. But I'm sure there is room for improvement.
Gordon mentioned a while ago that Lumagen's alg included MNR which was interesting. Made me wonder whether it is actually filtering the edges in order to eliminate ringing - rather than simply cleanly scaling (if there is such a thing).

Gordon mentioned a while ago that Lumagen's alg included MNR which was interesting. Made me wonder whether it is actually filtering the edges in order to eliminate ringing - rather than simply cleanly scaling (if there is such a thing).I think I remember Jim Peterson commenting that the MNR was an unintended byproduct of the scaling algorithm as opposed to something it was designed to so.

If you look at conventional scaling using multi-rate FIR filters, you typically see ringing around sharp transients or steps in the image. This is due to the negative lobes in the sin(x)/x function used by such filters.

One way to reduce or remove ringing is to identify the image features which cause these filters to ring (such as object edges) and to modify the behavior of the filter in those areas. For instance, if you substituted a filter which didn't ring in the areas where you'd otherwise get ringing, then you suppress the ringing artifacts. This would typically happen on both sides of an object edge, which is also one location where you tend to get mosquito noise due to the high spatial frequencies in the edge.

Now, most non-ringing filters I've seen also tend to have poor high frequency response, with the result that fine detail is filtered out. If you use such a filter in the area where other filters ring, which happens to be the same area where you get MN, then you remove the ringing and the MN at the same time. Of course if you don't have the MN in the first place, then you end up just blurring the image and losing detail, but you do suppress the FIR filter ringing.

- Dale Adams

On the English forum, Gordon Fraser remembered that Snell & Willcox created new frames for the Matrix Bullet Time sequences. This wasn't in realtime, but I have to say, there are no visible artifacts on that. Is this commonly used?
Unless, the actual information created was only on faces and the rest was CGI. If so maybe the artifacts would be hard to spot.

Those Algolith CAS demo images are poorly created - not the same frame and not shown at the same size. What would be better would be a side-by-side of Bicubic vs CAS on the same frame.


There is a filter in Avid Media Composer that does vector based timewarping.
It is by far the best implementation of such a feature that I have ever seen.
The filter is called "FluidMotion".

Here's an idea. Why doesn't the major HD broadcasters produce 1920x1080 frames regardless of what their source is and then send out an interlaced signal. This would leave us with a simple weave. Wouldn't it be better for 99% of the population to receive a pro-equipment processed signal?

There is a filter in Avid Media Composer that does vector based timewarping.
It is by far the best implementation of such a feature that I have ever seen.
The filter is called "FluidMotion".
Is there a demo video if it in action or something we can look at without having to install all the stuff?
It sounds interesting. How long does it take?

One of the problems with HICON32 is that it takes absolutely ages and produces an uncompressed AVI which then has to be put through another codec to get to something sustainable - my computer is used for business, so I didn't really like the fact that my PC was locked during both processes. (usually far longer than just an overnight stretch).

Is there a demo video if it in action or something we can look at without having to install all the stuff?
It sounds interesting. How long does it take?

One of the problems with HICON32 is that it takes absolutely ages and produces an uncompressed AVI which then has to be put through another codec to get to something sustainable - my computer is used for business, so I didn't really like the fact that my PC was locked during both processes. (usually far longer than just an overnight stretch).

I will see if I can get some FluidMotion demos.

Maybe some very challenging stuff like slowing down interlaced PAL DV from 100 percent to 25 percent or 10 percent.
It is challenging due to being 50i, and due to the not so stellar PQ of DV.

Then I will try to see if some progressive PAL DV is available, and possibly some HD samples.

Processing times are "acceptable" given the stellar results. A Core2Duo at 3.3 GHz takes approx. 10 times real-time for 50i sources.


As for Hicon³², it really is slow, but optimizing the code would certainly lead to better results. I don't even know if it is SMP-enhanced.

Are there any other PC filters/codecs around which can give a good indication of what's to come for realtime VPs?

AlgoSuite has excellent MPEG artifact and noise reduction, as well as awesome scaling.

We need to see screenshots of some of these things in action - possible?

yes, just need some time for that

We need to see screenshots of some of these things in action - possible?

http://cineramax.com/images/MotionAnalysis.jpg

http://cineramax.com/images/MCTi-in-Action.jpg

White paper:
Motion Compensated Temporal Interpolation (http://cineramax.com/images/Temporal_Interpolation.pdf)

I really liked Showscan 65 frame per second large format presentation, this promises to convert film based programme into a similar Virtual Reality Imaging. The purists of Nostalgia will complain. I believe that by using zoned based algorythms instead of full search that MCTi can be a viable alternative. The problem is finding a projector that will do 72hz. Not too many out there.

Hi mate!
I just found that Algolith has discontinued the AlgoSuite range. Just
wondering if you could recommend something comparable for upscaling and
noise removal. Or any other place that still may sell this software - perhaps on clearance..
Cheers!



AlgoSuite has excellent MPEG artifact and noise reduction, as well as awesome scaling.

RE: Super-resolution

Dale Adams of DVDO made a comment on another thread that got me thinking about the subject of video processing more. I think that video enhancement used in crime solving is the same technique being referred to.

Given several frames of the same subject, there may be slight differences in pixel straddling of subject matter details finer than the pixels. I now agree that under some conditions the resulting pixel content of consecutive frames can be analyzed and higher resolution can be delivered for parts of the picture compared with what the original pixel array ordinarily delivers.

If the subject is perfectly stationary, no additional detail would be gleaned. If subject matter has moved non-trivially, motion compensation techniques need to be applied first to identify the moving material in adjacent frames.

argh - you just brought up one of my pet hates - why is it in all crime films/ tv shows they just can't resist 'enhancing' images beyond believability! It really bugs me.

Anyway - back on topic. That sounds like a really interesting idea. But I guess we'd have to get good at frame interpolation first. I'd love to see the Sony VW200 MotionFLow implementation of that, as a lot of people seem to use it on video-based material. If so, it would be the first implementation I've seen where the introduced artifacts don't outweigh the benefits.

The Meridian unit seems to have disappeared since last CES 07.

Just in case someone finds out a place that still sells Algosuite licenses..:-(

CINERMAX QUOTE:
The problem is finding a projector that will do 72hz. Not too many out there.

Any good CRT front projector can do 72hz at 1080i and now some are pushing 1080p@72 just check out the screen shot thread on the CRT Front projector forum. I personally run 800p@72 on my Marquee 8500 so only the active video from scope movies(what i mostly watch) is only scaned by the CRT. This opens up more head room for using higher scan rates. I can push 96Hz this way but preffer the added clarity of 72Hz. This method is called active Area scanning (AAS) and can be done with a capable scaler and easily with a HTPC.


Athanasios

Like What nashou66 said, there are plenty of CRT projectors that can do 72hz @ 1080i. This is the beauty of multi scan CRT technology....