I'm surprised I don't see a sticky thread to debate the best available method of 2D to 3D conversion, it's spread out among various threads about different hardware and software that perform conversion. I have an idea I wanted to share that I think could be really exciting if a few talented software engineers got involved (and I'm sure there are some lurking around this forum).
The most important gap I see in 2D to 3D conversion is a way to correct errors on your favorite titles post conversion, because even the best automated software/hardware process is going to fail to convert some things. A human being is always going to pay special attention to scenes where really cool depth or popout effects would add emphasis to ehance the mood/experience of a movie, but obviously home users are not going to convert entire movies by hand.
Here is what I'm proposing: An automated converter that creates vector based depth maps and outputs these to a file, instead of just converting the video frames. With a good enough editor an average user could make adjustments to the depth maps and save the changes, then use the pre-calculated maps to playback the movie in realtime or to convert the video into the desired 3D video format.
The nature of this project leans towards open source and that's also where the real advantage comes in: Crowdsourcing the 2D-3D conversion process
. If there was a centralized repository where people could upload their depth maps, then other users could download the file and convert the movie with it. They could also make their own adjustments and then share their contributions with the rest of the user community. In the repository you would have different builds of the movie available. That way different users could watch whatever they enjoy most: pure depth, in your face popouts and of course something that is somewhere in between. A group of fans of a particular movie could get together to divide up the scenes they each want to work on fine-tuning. It should get around any legal concerns, since no one is sharing the actual video frames of the movie and it would be easy to prove that there is no way to reverse engineer the files to reproduce the movie in question.
The file format should store vectors and deltas of these vectors for each subsequent frame, until it reaches another frame where the shape or number of vectors changes. I'm essentially talking about keyframes and tweening: http://en.wikipedia.org/wiki/Inbetweening
. With a proper user interface users would be able to make adjustments and have these cascade from one frame to the next, so you don't have to edit each individual frame. This would also make the data smaller and easier to compress. I'm thinking a simple, human readable text format like XML or JSON that is zip compressed by the software. It would be sweet to be able to mux the depth map into an existing 2D video, then you could play back 2D and 3D from the same file. A format like this could easily become a defacto standard for upcoming software and hardware.
So the required software components for the project are:
- Converter software that will output the depth maps from a video source, which would also need to be able to import the depth maps and output converted videos in whatever desired 3D format (including anaglyph). There are probably already some open source 2D to 3D converter projects that could be leveraged to build this.
- A UI editor to edit the generated depth maps. It must allow you to edit the raw depth map and tweening in the main window and show the resulting 3D video frame in a preview window. It needs to have a timeline so you can jump around easily and should be able to playback the conversion in any format. Think about the user interfaces for Virtualdub, other video editing software and especially some popular commercial animation packages.
- Player software (or plugins/codecs for existing player software) that can output the conversion in realtime. A key feature would be the ability to playback from a DVD or Bluray without ripping the disc, this measure would be very beneficial towards proving that the software does not endorse piracy.
- A central repository to store and share the depth map files. This should have a good interface where users can leave comments and rate the available conversions on different parameters. Users should easily be able to find the most popular conversions of their favorite titles. There should be a robust version control system underneath the web interface, such as github or SVN where the files are properly backed up and protected from being overwritten by a bad upload.
I'm a software engineer and I could easily work on the repository and its web interface. Java/Web Development and source control are my primary skills. Some of the other components of this are outside my area of expertise. I know there are people on this forum who have the skills to contribute to the other components. Is anyone interested in building something like this?