[Dennis F]

VideoProcessor turns a computer into a 4k HDR capable live video processor by connecting a video capture card to a renderer and taking care of details such as conversion, timing and HDR metadata.

This allows advanced renderers to do things like 3D LUT, HDR tone mapping, scaling, deinterlacing and much more which can significantly improve image quality on most displays and beamers.

Screenshot

Action
I made a small video on the effects of a 3DLUT with VideoProcessor, which is why I got into this whole thing in the first place. (It turns out that making movies of movie screens a whole art in itself, one I clearly don't master. It looks significantly better in reality I can tell you )

How does it work?
You will need a Windows computer with a capture card and a video card. VideoProcessor will take video from your capture card, adjust, delay and processes them as required and then push them through a renderer and out of the video card.

An install with VideoProcessor would look something like the following:

Limitations
VP can only process what can be captured. You'll need an HDCP free source.

Links

• Manuals, guides and details are on videoprocessor.org
• The code, releases and instructions are on GitHub at defl/videoprocessor.

💾 Latest release: 1.0.0 (stable)

 

[Dennis F]

Reserved

 

[Dennis F]

Welcome back all

 

[markmon1]

View attachment 3149627

Video Processor: High-end video processing on live data for the rest of us

Video Processor is a Windows application which couples a capture card to a high quality video renderer (madVR) and takes care of all the plumbing in such a way the metadata stays intact. This allows the renderer to do things like 3d LUT, HDR tone mapping, scaling, deinterlacing and much more which can significantly improve image quality on the majority of displays. It is especially useful for accurate color-correction and HDR-like display on low lumen devices like projectors.

Get the latest release + instructions at defl/videoprocessor.

My suggestion here is to rewrite that whole wiki page to eliminate any and all discussion of madVR and simply explain that your app is able to feed video input into a directShow chain. It works with any renderer so let the user interpolate that using madVR is possible.

 

[Dennis F]

My suggestion here is to rewrite that whole wiki page to eliminate any and all discussion of madVR and simply explain that your app is able to feed video input into a directShow chain. It works with any renderer so let the user interpolate that using madVR is possible.

To be a bit more exact, it can indeed work with any DirectShow renderer but right now it is hardcoded to instantiate the system wide installed madVR renderer by just asking Windows to create a COM object with the madVR guid. There are a bunch of minor provisions for madVR but that's easy to get rid of so I guess one of these releases I'll add support for other renderers such as mpv as well.

I do take your advice, it indeed makes sense to start putting this in a broader context so I've removed madVR from the first paragraph.

I did add a rather large section on the situation in the README.md in return because I expect people to want to use madVR. The reason for this is to give users a clear understanding of the situation, so they can make an informed decision on it. I personally dislike it if things are left unsaid in such a way that people take the wrong choice accidentally/by default. VP is just one case, you can violate the new madVR license just as easily in pretty much anything that does DirectShow, this ranges from debugging tools such as graph-studio-next to big name players such as VLC and potplayer - they all have the ability to connect a DirectShow capture source to a renderer sink.

Just to clarify my thinking here, I have sympathy for @madshi's situation; he's in a bit of a bind between the business and the free lib, though in all fairness they are coupled; the community is what drives his business success . Still there is a difference between a free service for fun and earning a living, irrespective of what others would think of it, it only takes a single decision from him to end madVR for all. I can't stop people from doing anything they like, including breaking the law or legal terms, but I will point it out to them beforehand so they can make an informed choice. In the end of the day, people can always buy an Envy if they don't want any of the complexities associated with the situation but still want a recent madVR++, which they should if they can IMHO.

 

[markmon1]

To be a bit more exact, it can indeed work with any DirectShow renderer but right now it is hardcoded to instantiate the system wide installed madVR renderer by just asking Windows to create a COM object with the madVR guid. There are a bunch of minor provisions for madVR but that's easy to get rid of so I guess one of these releases I'll add support for other renderers such as mpv as well.

I do take your advice, it indeed makes sense to start putting this in a broader context so I've removed madVR from the first paragraph.

I did add a rather large section on the situation in the README.md in return because I expect people to want to use madVR. The reason for this is to give users a clear understanding of the situation, so they can make an informed decision on it. I personally dislike it if things are left unsaid in such a way that people take the wrong choice accidentally/by default. VP is just one case, you can violate the new madVR license just as easily in pretty much anything that does DirectShow, this ranges from debugging tools such as graph-studio-next to big name players such as VLC and potplayer - all have ability to connect a DirectShow capture source to a renderer sink.

Just to clarify my thinking here, I have sympathy for @madshi's situation; he's in a bit of a bind between the business and the free lib, though in all fairness they are coupled; the community is what drives his business success . Still there is a difference between a free service for fun and earning a living, irrespective of what others would think of it, it only takes a single decision from him to end madVR for all. I can't stop people from doing anything they like, including breaking the law or legal terms, but I will point it out to them beforehand so they can make an informed choice. In the end of the day, people can always buy an Envy if they don't want any of the complexities associated with the situation but still want a recent madVR++, which they should if they can IMHO.

I think anyone that uses this will likely use it with madVR. I just think that it's best (for you and the project's sake) to decouple itself with madVR specifically. Ideally, rather than hard code madVR's object, I would suggest some way to enumerate available renders and let the user pick one in a settings dialog etc. You could default to madVR, should it be present if you wish. People will know what to do with this. But when you hard code madVR and talk about madVR, you risk some potential legal problems that you may or may not want to fight. Then I'd suggest moving forward with your project full steam while never mentioning madVR again.

 

[Dennis F]

Another night, another experimental release you can get here.

This is another big one as we've become DirectShow render independent. VideoProcessor now can use any DirectShow renderer you have installed on your system, rather than solely MadVR. With this we're also saying goodbye to any madVR references in the UI or the code, things are now called "renderer". You can see this in action here:

The detection is by name, so there will be things calling themselves "Renderer" which won't work. If you know of valid renderers that have a specific name let me know and I'll add them to the filter.

 

[Dennis F]

I think anyone that uses this will likely use it with madVR. I just think that it's best (for you and the project's sake) to decouple itself with madVR specifically. Ideally, rather than hard code madVR's object, I would suggest some way to enumerate available renders and let the user pick one in a settings dialog etc. You could default to madVR, should it be present if you wish. People will know what to do with this. But when you hard code madVR and talk about madVR, you risk some potential legal problems that you may or may not want to fight. Then I'd suggest moving forward with your project full steam while never mentioning madVR again.

While I think madVR has exactly nothing they can start a fight over, there is simply nothing to win but only to lose. So as of tonight (the experimental-20210630 release) VP has grown support for any and all DirectShow renderers you have installed, I've also removed all references to madVR besides a single link in the readme to their commercial offering. See the announcement in #7.

Additionally there is a change to the license, commercial usage (=building computers with VideoProcessor on them as a video processor appliance and selling that) is now not allowed anymore, non-commercial is still under GPL3 open source rules.

 

[Miki1971]

Another night, another experimental release you can get here.

This is another big one as we've become totally render independent. VideoProcessor now can use any renderer you have installed on your system, rather than solely MadVR. With this we're also saying goodbye to any madVR references in the UI or the code, things are now called "renderer". You can see this in action here:
View attachment 3149829
The detection is by name, so there will be things calling themselves "Renderer" which won't work. If you know of valid renderers that have a specific name let me know and I'll add them to the filter.

MPV you mentioned in # 5 I am reading now that it is very good, if you add it I will try it

 

[Dr.OP]

Thank you for the great work!

Did anyone tried VP on a virtual machine running on a server? Back thinking is to have a server with GPU and VM configured for VP with a few other VMs running different "toys" (NAS, home automation, etc).

On paper it should look fine as there is an option to pass through both GPU and PCI/USB capture card.

Platform wise I would think of Linux/FreeBSD based PC to run the server.

 

[Dennis F]

Thank you for the great work!

Did anyone tried VP on a virtual machine running on a server? Back thinking is to have a server with GPU and VM configured for VP with a few other VMs running different "toys" (NAS, home automation, etc).

On paper it should look fine as there is an option to pass through both GPU and PCI/USB capture card.

Platform wise I would think of Linux/FreeBSD based PC to run the server.

No idea how feasible this is. Probably you can get it working but my bet would be that real-time and VM don't mix well. I'd expect a lot of stutter as the VM gets swapped in/out by the hypervisor. Then again, in hindsight some of the best things in life start out as "this is probably a really bad idea, but..." Let us know!

 

[buxtehude]

Hello Dennis,

thank you very much for your great work.
Version 20210629 works here without any problems together with 133. AVR lip sync delay set to about 200ms gives A/V in sync at my monitor.

 

[mcpherv]

Hey Dennis,
Running the last experimental (before current) release, everything is working really very well on my end. I have all queues set to 4 and getting a delay of about 185ms. Very odd dropped frame which happens mostly within first 30s of rendering but otherwise all VP default settings for clock, 9ms frame delay etc. This is the first release for me which runs better with your clock settings rather than clock set to none as per original setup. Even in its current form it’s a great piece of software and I’m looking forward to further delopment.

One thing I did discover and thought I’d bring up - I’m mostly using 4K24 to 1080p24. However, last night I set up the projector to watch TV - which is 60hz. I believe that this changed the audio delay going between 24hz and 60hz due to the difference in time per frame with the same number of queued frames. Not sure if it would be possible to configure VP to set up a calculated stable offset based upon a user assigned refresh? Ie if the user wanted to use 24hz as default with minimum VP queue size, then once changed to 60 hz then have a calculated longer queue within VP to accommodate the shorter frame time and keep the overall delay roughly stable?

Ie queue size of 4 in Madvr at 24hz is roughly 1/24=~42ms x 4 frames = 168ms assumed delay +whatever the rest of the setup for the user adds. But this should be roughly constant I’d think.

So then when you change to 60 Hz, then that same 4 frame queue is 1/60=16.6 x 4 = 66.4ms. So then you have roughly a 100ms difference, so VP could internally queue 100/16.6 = ~6 frames and the overall delay should be roughly stable?

Anyways just an idea. Thanks for your hard work!!

 

[Dennis F]

Hey Dennis,
Running the last experimental (before current) release, everything is working really very well on my end. I have all queues set to 4 and getting a delay of about 185ms. Very odd dropped frame which happens mostly within first 30s of rendering but otherwise all VP default settings for clock, 9ms frame delay etc. This is the first release for me which runs better with your clock settings rather than clock set to none as per original setup. Even in its current form it’s a great piece of software and I’m looking forward to further delopment.

One thing I did discover and thought I’d bring up - I’m mostly using 4K24 to 1080p24. However, last night I set up the projector to watch TV - which is 60hz. I believe that this changed the audio delay going between 24hz and 60hz due to the difference in time per frame with the same number of queued frames. Not sure if it would be possible to configure VP to set up a calculated stable offset based upon a user assigned refresh? Ie if the user wanted to use 24hz as default with minimum VP queue size, then once changed to 60 hz then have a calculated longer queue within VP to accommodate the shorter frame time and keep the overall delay roughly stable?

Ie queue size of 4 in Madvr at 24hz is roughly 1/24=~42ms x 4 frames = 168ms assumed delay +whatever the rest of the setup for the user adds. But this should be roughly constant I’d think.

So then when you change to 60 Hz, then that same 4 frame queue is 1/60=16.6 x 4 = 66.4ms. So then you have roughly a 100ms difference, so VP could internally queue 100/16.6 = ~6 frames and the overall delay should be roughly stable?

Anyways just an idea. Thanks for your hard work!!

Thanks, happy to hear you're enjoying it! I was thinking along the same lines to make a constant-time queue, that'll be version 1.1.0 (see TODO). Given that it takes a ton of bits and bobs to do a proper measurement, I've been slowly building up the required pile, sneak preview:

 

[Dennis F]

We're up to 0.3.0 beta1. This one features an auto-adjustment system for both the queue and the capture clock offset.

 

[buxtehude]

Maybe my projector setup is useful for anybody:

If your AVR is equipped with a Zone2 output like my Marantz no other HW beside splitter is needed to route the video. All audio formats run as usual.
Just switch the projector to Input2, set Zone2 to PC and your main input to your source (Shield TV here).

 

[Dennis F]

One thing that @hwf also noted in a PM was that VideoProcessor now also supports interlaced formats such as DVD.

My guess is that is a side-effect of becoming DirectShow neutral, VP first tries to connect with a special DirectShow mode which supports metadata such as HDR. Not many renderers understand this and so if the connection fails, it falls back to a more common connection to maximize compatibility, this allows DirectShow to inject any known filter it needs to make the connection. For interlaced formats that means that a filter such as LAV gets auto-added by DirectShow, this vastly extends the amount of formats VP can now play. (Obviously there is no HDR metadata in this mode)

(I still need to validate this myself but that's on the cards for the next release anyway)

 

[mcpherv]

Thanks, happy to hear you're enjoying it! I was thinking along the same lines to make a constant-time queue, that'll be version 1.1.0 (see TODO). Given that it takes a ton of bits and bobs to do a proper measurement, I've been slowly building up the required pile, sneak preview:
View attachment 3150050

That’s awesome! Can’t wait .

I see zoom is on the agenda as well - getting an A lens soon so I’ll put a plug in for this too . Anyways thank you again!

 

[Dennis F]

0.3.0 is out after finishing the TODO list from last night's beta 🎉

Changelog since 0.2.0:

• Always uses the best clock available (=card hardware clock)
• HDR info is now in CIE image
• Added final handler for throws which pops up a messagebox upon thrown exceptions, this allows the user to understand what's going on in many situations
• Removed all mutexes (outside of directshow), it now runs lock-free with a few atomics to ensure correctness
• Version number visible in window title
• Mouse not visible anymore in video render window, both windowed and full-screen
• Pressing return closed the window, now takes context dependant action with a default of restarting the renderer
• Our queue dropped frame counter (excluding renderer-dropped frames)
• Missing frame counter (never delivered by capture but found by timestamp-gap analysis)
• Process priority is now set to high on start
• User selectable frame-interval determination modes, clock-clock, clock-theo, theo and none
• Renderer reset button, which stops, clears the queue and starts playback again
• AVX required (requires < 10-year old CPU)
• Disable screen saver/power save when running
• VideoProcessor is now DirectShow renderer independent, you can use any installed DirectShow renderer.
• Optional auto-tune params based on renderer queue and exit latency.
• FIX: Deadlock situation with blackmagic callbacks
• FIX: frame timestamps now correct
• FIX: Correct process name in task list

Next up is 0.3.1, a minor release that will focus on more cleanups and more graceful error handling.

I would love to hear your experiences in this thread. Bug reports by PM or just in the github issue tracker please

 

[zombie10k]

@Dennis F - hi, thank you for the continuing efforts for this excellent project! Last night I spent several hours running various tests.

recommendation to others when posting feedback, please include details about your HTPC and AV path configuration, it will be helpful for comparisons and identifying issues.

HTPC - i9900k/16gb/Nvidia 3080FE

A/V - 2019 Shield (Netflix 4K HDR Stream) -> Denon 7200WA -> Vertex 2 -> HTPC (Decklink Mini Recorder 4K)

A/V - HTPC (Nvidia 3080FE) -> JVC RS600 & Sim2 Mico 50 projectors


I first tried the 'clock-clock' settings first. In both windowed mode and full screen, I noticed an intermittent visual jump between frames. This doesn't registered as a skipped/dropped frame in the ctrl-J stats, but I did noticed that 'clock deviation' from the hud info disappears and re-appears during that blip. rendering time is ~23ms when it's not happening, then it jumps in ~30ms range for a moment, recovers and back to ~23ms.

I then tried the 'clock-theo' and this was no longer an issue in my setup. no visual glitches, rendering stayed consistent @ ~23ms. queues were set to 4 for these tests. ~190ms audio delay on the Denon for correct lip sync. I will continue to run tests, this was preliminary feedback.

btw, if anyone is trying to track lip sync, check out 6 Underground on Netflix - 00:29:00 - 00:33:00 - this scene has close ups of all the actors and makes it easy to tell when the delay is correct. I have every word spoken in this 4 min scene memorized