Possibility of software ever properly increasing resolution & keeping quality? - AVS Forum | Home Theater Discussions And Reviews
Forum Jump: 
 
Thread Tools
post #1 of 8 Old 02-05-2015, 05:01 PM - Thread Starter
Newbie
 
Welsh Jester's Avatar
 
Join Date: Jun 2011
Posts: 13
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 10
Question Possibility of software ever properly increasing resolution & keeping quality?

I wasn't sure where to post this question, bit of a difficult one.

Anyway, I've been thinking for a while now that there's a possibility of having software to basically rebuild footage. Something like turning a video file into vector video, where you could increase the resolution and not lose any quality. Now i know vector isn't perfect, but i figure this could in theory at least work well with animation. But i haven't heard anything like this being worked on. I have read that there's supposed to be a vector video codec of some sorts in the works, but will only work for new videos shot with cameras that can work that way in the first place.

So to cut a long story short, is there any way that DVD's etc.. will be able to be automatically reworked in software? I know there's upscaling in players but i'm talking about something a lot more advanced, wondering if anything like that is possible. I don't see why not.
Welsh Jester is offline  
Sponsored Links
Advertisement
 
post #2 of 8 Old 02-07-2015, 08:23 PM
AVS Forum Special Member
 
Joe Bloggs's Avatar
 
Join Date: Apr 2007
Location: UK
Posts: 3,114
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 446 Post(s)
Liked: 126
CGI Animation
Well with CGI animation, you wouldn't need to work back from a compressed video file to increase the resolution. The person who created it could/should have the original animation data files that the video was created from. That way they could then re-render at a higher resolution. Though I suppose it might need higher res textures incorporated sometimes or maybe higher polygon count models.

You could probably use something like an "auto-trace" function to convert it to editable (in 2D), but it would be much better to use the original animation data files (where you could have all the original 3d info).

Filmed content
With filmed content (including digital capture), the capture of the array of pixels is the highest amount of true info you can get. You can upscale it which is basically guessing. You could convert it to fractals - maybe that would be a good option. You could use super-resolution (I think that's the right term) and have it use info from multiple frames to make one frame - eg. a frame of a car in the distance could take picture info from the car when it was closer (or even at some point, re-create the scene in 3D space from multi-view cameras).

Another way is where I think some TVs have certain stored ways of upscaling certain content (eg. I think a set of common images and their upscaled versions), though I think you'd probably be better off with super-resolution.

Though mpeg type compression does create a set of vector info for the macroblock (or similarly named thing), it wouldn't give you enough for what you want.

I just looked and there does seem to be a vector codec in the works, but it didn't give much info apart from being "equal in quality to bmp" but whether that really gives better quality on upscaling I don't know. It would still only be encoding the amount of info in the given array of pixels it was given. It may be that creating the a video in vector form to the same quality as current mpeg type encoders could take up more bitrate, so it may be that quality-wise you may be better encoding it with codecs like H264 or H265 (HEVC).

Quote:
So to cut a long story short, is there any way that DVD's etc.. will be able to be automatically reworked in software?
You could re-encode a video (that you owned the copyright for) in software and it may give better quality than some hardware based video - you could have it take a long time to do the upscaling (and include whatever algorithm you had, including super resolution) - the output would be compressed though (and if not it wouldn't be easily playable). Or you could have a software based player do it - but it would need to do the upscaling in real time and may not be the best you could do.

Quote:
Possibility of software ever properly increasing resolution & keeping quality?
I think the answer is that you can't have it upscale normal DVD video and have true info for all of what would have been created had it been created at a higher res (other than by chance or because the lens or filters limited it to that res in the first place). Though when they remaster (eg. from a higher resolution) and put that on BD or UHD BD you'll have a higher res version than was on DVD.

Last edited by Joe Bloggs; 02-07-2015 at 08:46 PM.
Joe Bloggs is offline  
post #3 of 8 Old 02-08-2015, 12:05 AM
AVS Forum Special Member
 
tezster's Avatar
 
Join Date: Sep 2007
Location: Mississauga, Canada
Posts: 1,706
Mentioned: 5 Post(s)
Tagged: 0 Thread(s)
Quoted: 644 Post(s)
Liked: 542
Quote:
Originally Posted by Welsh Jester View Post
I wasn't sure where to post this question, bit of a difficult one.

Anyway, I've been thinking for a while now that there's a possibility of having software to basically rebuild footage. Something like turning a video file into vector video, where you could increase the resolution and not lose any quality. Now i know vector isn't perfect, but i figure this could in theory at least work well with animation. But i haven't heard anything like this being worked on. I have read that there's supposed to be a vector video codec of some sorts in the works, but will only work for new videos shot with cameras that can work that way in the first place.

So to cut a long story short, is there any way that DVD's etc.. will be able to be automatically reworked in software? I know there's upscaling in players but i'm talking about something a lot more advanced, wondering if anything like that is possible. I don't see why not.
Short answer: you can't. Any type of logic would only be "guessing" what the missing information is that you're trying to create. In other words, you're telling the upscaler to conjure something out of thin air.
tezster is offline  
Sponsored Links
Advertisement
 
post #4 of 8 Old 02-08-2015, 01:20 AM - Thread Starter
Newbie
 
Welsh Jester's Avatar
 
Join Date: Jun 2011
Posts: 13
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 10
Well rather than a scaler, maybe software that slowly converts video to vector, then back into video format?

Have you seen the software called Vector Magic? it converts pictures to full vector, and depending on what you input it can be pretty accurate.. but rather slow to convert 1 picture. Take a look at this https://vectormagic.com/home/comparisons

So if something like that was even more advanced and accurate i still don't see why it wouldn't be possible. But it would take a long time to convert even a 20 minute video.

Last edited by Welsh Jester; 02-08-2015 at 01:24 AM.
Welsh Jester is offline  
post #5 of 8 Old 02-08-2015, 01:53 AM
AVS Forum Special Member
 
Joe Bloggs's Avatar
 
Join Date: Apr 2007
Location: UK
Posts: 3,114
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 446 Post(s)
Liked: 126
Quote:
Originally Posted by Welsh Jester View Post
Well rather than a scaler, maybe software that slowly converts video to vector, then back into video format?

Have you seen the software called Vector Magic? it converts pictures to full vector, and depending on what you input it can be pretty accurate.. but rather slow to convert 1 picture. Take a look at this https://vectormagic.com/home/comparisons

So if something like that was even more advanced and accurate i still don't see why it wouldn't be possible. But it would take a long time to convert even a 20 minute video.
But that Vector Magic is only showing it working well for a 2D cartoon/artwork. There's no detail really between the lines of the cartoon/artwork. When it's done on realistic/photographic stuff it removes the detail and turns them into cartoons. You might be able to run an auto-trace in a video program but that too would remove detail, but it might be okay for 2D cartoons (with outlines) and upscaling them without jaggies (not sure though). But it would be no good for 3D polygon type animation or live action.

Last edited by Joe Bloggs; 02-08-2015 at 02:08 AM.
Joe Bloggs is offline  
post #6 of 8 Old 02-08-2015, 05:09 PM - Thread Starter
Newbie
 
Welsh Jester's Avatar
 
Join Date: Jun 2011
Posts: 13
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 10
Quote:
Originally Posted by Joe Bloggs View Post
But that Vector Magic is only showing it working well for a 2D cartoon/artwork. There's no detail really between the lines of the cartoon/artwork. When it's done on realistic/photographic stuff it removes the detail and turns them into cartoons. You might be able to run an auto-trace in a video program but that too would remove detail, but it might be okay for 2D cartoons (with outlines) and upscaling them without jaggies (not sure though). But it would be no good for 3D polygon type animation or live action.
Check out this news article from a couple of years ago, it's about the vector codec i mentioned - http://www.gizmag.com/vector-video-codec/25481/

I would be really interested to see the results of what happens if you record over regular video with that.
Welsh Jester is offline  
post #7 of 8 Old 03-08-2015, 12:36 AM
AVS Forum Special Member
 
Mr.D's Avatar
 
Join Date: Dec 2001
Location: UK
Posts: 3,314
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 26
When you have a certain amount of real information in any system you can only make an attempt at guessing when it comes to interpolating more "information".

If you have a black pixel next to a white pixel and you want to interpolate a pixel inbetween the two the most accurate /least innacurate compromise is to generate a pixel that is a 50% mix between the two either side.

Pretty much every scaling technique uses this process. There are ways of bringing more surrounding pixels into the averaging , there are ways of using temporal comparisson information in the preceding and following frames ( although this is normally used for vector based intraframe segmentation compression rather than upscaling).

However basically what you are doing in the case of scaling is biasing that interpolation average of those new pixels on the basis of some simple dumb rules.

The major trade offs in scaling are sharpness vs aliasing , You are generally better producing a smooth yet soft result and then adding sharpening back on top to give it some sharpness pop. (sharpness is merely high frequency contrast boosting ; too much gives you ringing on edges ; its an visual illusion , adding noise also fools the brain into seeing more detail and things looks sharper).

If you create aliasing its pretty much impossible to remove without blurring and then you are defeating the point of scaling in the first place.

Using sharper detail in one frame to create detail in others only works in a few scenarios of shots and even then needs a lot of manual intervention to produce clean enough results. Given the effort and the level of messing with the original photography a decent scaling job is the safest bet and likely will be for sometime give the laws of entropy in the universe.

Most people cannot tell the difference when viewing 2k upscaled to 4k and 4k original and that includes the director of photography. If you can only tell with any level of consistency through an AB comparisson (even then it often requires multiple viewings) then what is the point of worrying about it?

digital film janitor
Mr.D is offline  
post #8 of 8 Old 11-14-2015, 12:08 PM
AVS Forum Special Member
 
John Mason's Avatar
 
Join Date: Jul 2000
Location: New York, NY
Posts: 11,139
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 76 Post(s)
Liked: 125
Noticed an article in the Oct, 2015 SMPTE "Motion Imaging Journal" about super resolution (SR) mentioned above. The article is very similar to a 2014 original called "4K-to-8K TV Up-converter with Super Resolution" by a Japanese professor. See:
http://www.nl-superresolution.com/do...onverterSR.pdf
One of the minor differences: an image of a woman wearing a distinctly lined scarf that looks very 'crisp' compared to a blurry original image in Imaging Journal, while the linked 2014 paper only shows the upconverted scarf image (Fig. 1b). (If the pdf link doesn't work properly, run a Google search with the title/date etc.)
EDIT:See last pdf link below for a good before/after scarf-photo comparison
EDIT2: Here's another related but older pdf paper on the topic uncovered at scholar.google.com
http://www.nl-superresolution.com/do...sing_Video.pdf


The parallel processing technique uses a board slightly larger than a $1 bill to convert 4 full-HD images to a 4k output, then combines 4 4k images for an 8k ouput (Figs. 3, 4). Prof. Gohshi concludes his article (last labeled section), writing the technique differs from previous resolution improvement methods, including SR technologies, and "can create 8k images that have 8k resolution." Realtime processing is possible with the non-linear low-delay SR, he writes. His 2013 paper for the IEEE Computer Society (ref. 8) outlines some limitations of SR video image reconstruction. -- John


EDIT: Here's a 2015 paper by Prof. Gohshi (abstract only) proposing a related SR technique for enhancing the PQ of nighttime infrared images from CCTV cameras:
http://proceedings.spiedigitallibrar...icleid=2208595


Prof. Gohshi co-authored another paper in 2013, published by the Society for Information Display, about HD-to-4k SR conversion, (not up to 8k). Searching, or joining SID, should provide access, although I gave up after a few attempts. (Here's a Powerpoint pdf, though, with lots of HD-to-4k images.)

Last edited by John Mason; 02-27-2016 at 11:07 AM. Reason: add-on;typo
John Mason is offline  
Sponsored Links
Advertisement
 
Reply HDTV Software Media Discussion

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off