AVS Forum banner

10481 - 10500 of 12825 Posts

·
Registered
Joined
·
958 Posts
I have a question… how long has the hue shift and saturation issue been going on with DTM? I have only been following this thread for a little more than a year now. Javs, first brought it to my attention a few months back. After seeing HDR comp on/off, I was convinced he was on to something. The D65 white point is visually impaired between the two. The red shift is apparent.

Why hasn’t this topic already today been discussed and resolved? Is it really that difficult? Aren’t there some underlying algo’s that can be adjusted? I just find this bizarre knowing madshi’s abilities…
 

·
Registered
Joined
·
9,177 Posts
I have a question… how long has the hue shift and saturation issue been going on with DTM? I have only been following this thread for a little more than a year now. Javs, first brought it to my attention a few months back. After seeing HDR comp on/off, I was convinced he was on to something. The D65 white point is visually impaired between the two. The red shift is apparent.

Why hasn’t this topic already today been discussed and resolved? Is it really that difficult? Aren’t there some underlying algo’s that can be adjusted? I just find this bizarre knowing madshi’s abilities…
I tracked it all the way back to beta 10. Thats where I could make it go away anyway. Back then we were using DPL target in the hundreds of nits, so it wasnt visible probably until later, but thats back when the saturation / hue correction factor got locked in, which was kinda just brought back (saturation).

Hue shift is not an issue, its a fix.

You say D65 white but I think you mean anything that appears white in film? Well its usually not exactly white, so more saturation on any of those will make the colour appear to shift but its actually not, its just more saturated.

DTM is not easy, yes its difficult clearly!
 

·
Registered
Joined
·
948 Posts
Then the ADPL can only go as high as 2xAvgHighlights (instead of the measured frame peak).
Cheers, that was really helpful :) (See below.)
Do you remember how "no compression limit" affects ADPL?

I quickly experimentid with it and it seems you can use values up to DPL (e.g. 150 in your case) and it doesn't change ADPL, but once you go over DPL (e.g. 151) it starts to make an effect.

Edit: That part of this thread is great! Many of these things I've been thinking about are being discussed. Thanks again, I'm gonna keep reading! :) I'm starting to feel confident that the tools we need are here and we just may need to tweak them for high-nit displays.
Just a note: there was a forum engine change here on avs (5 months ago?) that results in truncated posts (completely randomly). So if you read something that doesn't make sense then probably that's the reason :D
And as you will see there was a FALL Knee algo as well, but they dismissed the idea (don't remember the cause).

And take a look at this post about "disable avgHighlight ceiling".
 

·
Registered
Joined
·
948 Posts
Do you remember how "no compression limit" affects ADPL?

I quickly experimentid with it and it seems you can use values up to DPL (e.g. 150 in your case) and it doesn't change ADPL, but once you go over DPL (e.g. 151) it starts to make an effect.
I just found @Soulnight's post (RealNits means DPL, and TargetNits ADPL):

Code:
If FramePeak < Math.Max(RealNits, NoCompressionLimit):
  TargetNits = Math.Max(RealNits,FramePeak)
Else:
  TargetNits = Math.Min(framePeak, Math.Max(RealNits, NoCompressionLimit) + 2 * Fall / 50 * DynamicTuning)
That means using 100 NCL with 120 DPL (let alone 700 DPL) doesn't do anything :D
 

·
Registered
Joined
·
1,385 Posts
That means using 100 NCL with 120 DPL (let alone 700 DPL) doesn't do anything :D
Exactly, it forces a NCL at least equal to the DPL.

Removing the DPL from the formula to calculate the ADPL would allow you to have, for instance, a NCL of 100 with a DPL of 300.
For me, that means you get better control of the brightness (compression) of the image in the low end (not as dim as now on some scenes where the frame peak is very high and the FALL is low).

And if you want the current behavior back, you just have to set the NCL equal to the DPL (or higher).
 

·
Registered
Joined
·
20 Posts
Hello everyone, i've been reading this thread for some time now.

First, I want to thank madshi for the awesome work that he's doing and also thank everyone who contributed to this thread.
Now...I registered mainly to point out something that i have not seen anybody else mention.

You can adjust most monitor settings, and most importantly the backlight through the DDC/CI (Display Data Channel/Command Interface).
I don't know if this is possible on projectors.
I don't know if this should be included in madvr, because of its low-level nature.

There are some programs out there to try it, but these are low-level stuff and i don't think i should give any links...so for anyone who is very curious...wait for an explanation by someone who knows more about this.

I watch movies in a dark room and one possible use would be lowering the brightness during dark scenes while maintaining high saturation, which can help with light bleed and glow to improve human-perceptible contrast and it will make transitions to explosions or brighter scenes have higher impact.

So there could be an option to set 100% and 0% backlight nits and then madvr would adjust the backlight, which is essentially HDR without local dimming.

Thanks again madshi.
 

·
Registered
Joined
·
614 Posts
You can adjust most monitor settings, and most importantly the backlight through the DDC/CI (Display Data Channel/Command Interface).
I don't know if this is possible on projectors.
I don't know if this should be included in madvr, because of its low-level nature.

There are some programs out there to try it, but these are low-level stuff and i don't think i should give any links...so for anyone who is very curious...wait for an explanation by someone who knows more about this.

I watch movies in a dark room and one possible use would be lowering the brightness during dark scenes while maintaining high saturation, which can help with light bleed and glow to improve human-perceptible contrast and it will make transitions to explosions or brighter scenes have higher impact.

So there could be an option to set 100% and 0% backlight nits and then madvr would adjust the backlight, which is essentially HDR without local dimming.

Thanks again madshi.
A lot of projectors have an iris, sometimes two or a lamp or laser dimming function that is roughly equivalent but usually this is controlled by the projector directly, usually called dynamic contrast or something similar, with varying levels of success.

LCD flat panels that do HDR well, so not all of them, will already be modulating this themselves, there's no way an LCD can achieve a good black level in dark scenes any other way. Many will have FALD where individual areas of the display have their backlight dimmed independently.

I would expect a significant issue getting MadVR to control something like this in sync with the image and the display will likely already be controlling the backlight level anyway.
 

·
Registered
Joined
·
20 Posts
A lot of projectors have an iris, sometimes two or a lamp or laser dimming function that is roughly equivalent but usually this is controlled by the projector directly, usually called dynamic contrast or something similar, with varying levels of success.

LCD flat panels that do HDR well, so not all of them, will already be modulating this themselves, there's no way an LCD can achieve a good black level in dark scenes any other way. Many will have FALD where individual areas of the display have their backlight dimmed independently.

I would expect a significant issue getting MadVR to control something like this in sync with the image and the display will likely already be controlling the backlight level anyway.
You are right about dynamic contrast/brightness and HDR panels. If you have HDR on the monitor, it means it can process HDR metadata and adjust the backlight accordingly.
This is what cheap panels without local dimming do, they adjust their backlight level according to hdr metadata.

But SDR projectors and monitors, can't process that metadata and adjust accordingly.

If madvr could change the actual backlight to a known value, for example telling madvr that 10% backlight is 50 nits and 100% backlight is 350 nits and being able to dim the backlight without losing the display peak nits allows for greater range of details for both shadows, mid-tones and highlights.

The black floor is much lower and contrast ratio increases when you set 0% backlight and the full gamut is maintained but brightness steps become smaller, 50 nits divided by 255 vs 350nits divided by 255
 

·
Registered
Joined
·
958 Posts
I tracked it all the way back to beta 10. Thats where I could make it go away anyway. Back then we were using DPL target in the hundreds of nits, so it wasnt visible probably until later, but thats back when the saturation / hue correction factor got locked in, which was kinda just brought back (saturation).

Hue shift is not an issue, its a fix.

You say D65 white but I think you mean anything that appears white in film? Well its usually not exactly white, so more saturation on any of those will make the colour appear to shift but its actually not, its just more saturated.

DTM is not easy, yes its difficult clearly!
Thanks so much for your response Javs… Every time I interface with you I learn a little more… It sounds like DTM is much more involved (difficult) than my original thinking…exactly what “fix” caused red shift (over saturation) since beta 10 and was buried within the algo…wow that is really revealing… This is like a spy mystery (lol). My gut feeling is looking at hightlight recovery or fire tweaks as the culprits (which get deactivated with HDR comp on) ?

So soon we're suppose to evaluate madshi’s user destat controls and choose our favorite settings that he will then once again bury into the algo?
 

·
Registered
Joined
·
614 Posts
But SDR projectors and monitors, can't process that metadata and adjust accordingly.

If madvr could change the actual backlight to a known value, for example telling madvr that 10% backlight is 50 nits and 100% backlight is 350 nits and being able to dim the backlight without losing the display peak nits allows for greater range of details for both shadows, mid-tones and highlights.

The black floor is much lower and contrast ratio increases when you set 0% backlight and the full gamut is maintained but brightness steps become smaller, 50 nits divided by 255 vs 350nits divided by 255
These dynamic contrast tricks are not specific to HDR, my projector in SDR mode still does the same trick, my old SDR only projector does it too. They don't need any metadata, they just use the video level, it's not perfect, sometimes it goes nuts, commonly in movie credits.

You got me thinking about an issue it causes though: MadVR expects an SDR gamma curve, 2.2 usually and that at 100% the display will produce the specified nits, 60 in my case. I expect it scales that along the gamma curve such that it "knows" exactly how many nits should be produced at any point along the curve, now SDR is absolute, like HDR. Now MadVR can apply it's processing to tone map the HDR into this known SDR space, 10% luminance should be 6nits 50% should be 30nits etc after the gamma curve is reversed and taken in to account.

If the projector is applying dynamic contrast processing of it's own, MadVR doesn't know about this, so it outputs a 10% signal such that it expects 6nits but auto iris (or auto backlight) decides to increase the contrast by closing the iris, now that 6nits appears as maybe 1nit, MadVR's mapping is now wrong.

Apologies if this is what you've been getting at all along...

Of course we chose our tone mapping settings by looking at the end result of these systems interacting with each other but they are not properly defined, if they were known MadVR could take it into account. I'm certain that the same projector on the same auto2 setting will behave differently depending on the maximum iris setting, it's bad enough getting people to measure their peak white but producing an auto iris operating gamma curve would be interesting.

So what does everyone use, I left my JVC Projector on 0 iris as I only get 60 nits anyway (new lamp on Monday, hopefully ) and auto2. I'll run some comparisons but it seems I should turn off the auto iris if I want a predictable, well behaved SDR gamma. I think this was in the personal settings that everyone published early in this thread?

Does MadVR already factor in this iris behaviour? Intentionally or just indirectly?

So now I'm with you... if MadVR could control this iris or backlight, it could be used in a controlled predictable way. I still doubt it could be easily put in sync and potentially knowing how to control many different projectors and displays would be a huge job. that said MadVR already knows how to control JVC projectors at least, it does lens memories.

It would be taking over the auto iris function and it's probably a lot of work to end up at mostly the same place but with a bit more lag and probably huge jitter in the response due to network variability. Now we are in my specialist field, PDV (packet delay variation) in an Ethernet network is terrible, a wireless one especially so.
 

·
Registered
JVC RS500U, 2.35:1 125" screen, Marantz AV7705, 11.2 Def Tech speakers, Sunfire Theater Grand 400x7
Joined
·
3,051 Posts
If the projector is applying dynamic contrast processing of it's own, MadVR doesn't know about this, so it outputs a 10% signal such that it expects 6nits but auto iris (or auto backlight) decides to increase the contrast by closing the iris, now that 6nits appears as maybe 1nit, MadVR's mapping is now wrong.
AFAIK, most JVC folks disable the DI/auto-iris and set them manually to whatever closure they're comfortable with. Some prefer it darker, some want more light, but no one prefers pumping. Most owners that are using an HTPC have learned to tweak their displays before diving into MadVR. It asks if your display is calibrated with options for P3, BT.2020, gamma, and use of 3DLUT. There's really no excuse for users of MadVR to not define their setup correctly and to not (auto) calibrate their displays, and then expect great results. MadVR is not MagicVR.
 

·
Registered
Joined
·
20 Posts
These dynamic contrast tricks are not specific to HDR, my projector in SDR mode still does the same trick, my old SDR only projector does it too. They don't need any metadata, they just use the video level, it's not perfect, sometimes it goes nuts, commonly in movie credits.

You got me thinking about an issue it causes though: MadVR expects an SDR gamma curve, 2.2 usually and that at 100% the display will produce the specified nits, 60 in my case. I expect it scales that along the gamma curve such that it "knows" exactly how many nits should be produced at any point along the curve, now SDR is absolute, like HDR. Now MadVR can apply it's processing to tone map the HDR into this known SDR space, 10% luminance should be 6nits 50% should be 30nits etc after the gamma curve is reversed and taken in to account.

If the projector is applying dynamic contrast processing of it's own, MadVR doesn't know about this, so it outputs a 10% signal such that it expects 6nits but auto iris (or auto backlight) decides to increase the contrast by closing the iris, now that 6nits appears as maybe 1nit, MadVR's mapping is now wrong.

Apologies if this is what you've been getting at all along...

Of course we chose our tone mapping settings by looking at the end result of these systems interacting with each other but they are not properly defined, if they were known MadVR could take it into account. I'm certain that the same projector on the same auto2 setting will behave differently depending on the maximum iris setting, it's bad enough getting people to measure their peak white but producing an auto iris operating gamma curve would be interesting.

So what does everyone use, I left my JVC Projector on 0 iris as I only get 60 nits anyway (new lamp on Monday, hopefully ) and auto2. I'll run some comparisons but it seems I should turn off the auto iris if I want a predictable, well behaved SDR gamma. I think this was in the personal settings that everyone published early in this thread?

Does MadVR already factor in this iris behaviour? Intentionally or just indirectly?

So now I'm with you... if MadVR could control this iris or backlight, it could be used in a controlled predictable way. I still doubt it could be easily put in sync and potentially knowing how to control many different projectors and displays would be a huge job. that said MadVR already knows how to control JVC projectors at least, it does lens memories.

It would be taking over the auto iris function and it's probably a lot of work to end up at mostly the same place but with a bit more lag and probably huge jitter in the response due to network variability. Now we are in my specialist field, PDV (packet delay variation) in an Ethernet network is terrible, a wireless one especially so.

AFAIK, most JVC folks disable the DI/auto-iris and set them manually to whatever closure they're comfortable with. Some prefer it darker, some want more light, but no one prefers pumping. Most owners that are using an HTPC have learned to tweak their displays before diving into MadVR. It asks if your display is calibrated with options for P3, BT.2020, gamma, and use of 3DLUT. There's really no excuse for users of MadVR to not define their setup correctly and to not (auto) calibrate their displays, and then expect great results. MadVR is not MagicVR.
I'm sorry if i did not explain properly what im suggesting.
I was not talking about dynamic contrast or similar tech from TV firmware.
I was asking if allowing madVR to control the backlight power level of a display/projector, using the appropriate VESA standards and data links, with the API provided by Windows.
It is possible. I don't know whether it is worth it or safe.

A properly calibrated monitor/projector should have the proper colours for different backlight levels.

What you could theoretically do is this.
I am not familiar with projector terminology so i will use tv/monitors.
I'll use linear fake values for ease.

Let's say you have a TV capable of SDR and P3. No HDR support. Can't load LUTs. Calibrated. Dynamic options all disabled.
It's capable of 256 nits with black at 0.32nits when at 100% backlight. That's 800:1 contrast ratio.
At 50% backlight it's capable of 128nits with black at 0.08nits. That's 1600:1 contrast ratio.
At 0% backlight it's capable of 64nits with black at 0.04nits. That's 1600:1 contrast ratio.

You set the backlight on the monitor to your choice, for example 100%.
You tell madvr the peak nits of your display which is 256.
Those 256nits are then used by madVR according to some complex math where 1000nits are compressed to 256 nits to match your SDR monitor
They are distributed across 256 gray steps from black to white.
If it's linear there is 1nit for each step, this is like a "resolution" of your brightness.
It's not real HDR but it's good enough to watch HDR on SDR monitors.
A very dark scene shows with peak 64 nits and average of 15 nits, the scene is full of shadow detail.
On a proper HDR display, the backlight will dim to very low nits, but maintain the brightness steps.
On SDR displays, you have to use the lower brightness steps, otherwise the scene becomes brighter than it should be, due to the backlight being at max, blacks are elevated and you lose some contrast and a lot of detail

Now let's say madvr can control the backlight and you've configured it according to the display's measurements.

It now knows that the backlight can go from 64nits(0%) to 256nits(100%)
The dark scene comes along.
Instead of compressing all the detail, it dims the backlight to 0%.
Now it can tonemap the color range to 64 nits.

Another possible use would be matching backlight levels to the HDR brightness curve and have low and high backlight levels reserved for dark and bright(>600nits) scenes.
Achieving essentially HDR with global dimming.
 

·
Registered
Joined
·
958 Posts
Does anyone else actually have a negative experince toward the red shift issue with madVR… My understanding says Lumagen (or any other DTM solution) doesn’t exhibit this anomaly at lower nits (<100). That seems very strange to me. Why are the competitors so special to escape this condition? If they can do so, why can’t madVR?
 

·
Registered
Joined
·
2,770 Posts
Does anyone else actually have a negative experince toward the red shift issue with madVR… My understanding says Lumagen (or any other DTM solution) doesn’t exhibit this anomaly at lower nits (<100). That seems very strange to me. Why are the competitors so special to escape this condition? If they can do so, why can’t madVR?
Using Sep I could bump the dpl back up and no saturation issue

1.png

no issues with brightness in dark scenes which is surprising considering the actual measured illuminance, no issues with compressed looking bright scenes too.
 

·
Super Moderator
JVC RS4500 | ST130 G4 135" | MRX 720 | MC303 MC152 | 6.1.4: B&W 802D3, 805D3, 702S2 | 4x15 IB Subs
Joined
·
10,300 Posts
Oops nevermind I didnt realize what thread we are in and my response was off topic. (Cant delete)
 

·
Registered
Joined
·
948 Posts
Sky detection: sky detection
Brightness adjustment speeds: brightness adjustment speeds
Dynamic shadow recovery: dynamic shadow recovery
I had time to play with sky detection and I have the same issue with it as with dynamic clipping: it should not do anything when there's no need for it, but it does.
You use 150 DPL, find a dark scene with frame peak below DPL (e.g. 120 nits) where there's a wall (can be grey as well) behind. Sky algo modifies the FALL even if the original reported FALL is just 2 nits. Why?
Just like HSTM or shadow recovery, it should not be triggered with frame peak below DPL.

Although this setting can be used for fighting against ABL, since it reduces the FALL quite a lot (depending on the strength value), e.g. @00:00:05 keyframe with the clip you posted:
  • frame peak is 726 nits
  • with SS 0, FALL is ~284 nits
  • with SS 100, FALL becomes ~88 nits!

But if this ABL functionality needs to be included, then it needs to have another limiter, let's say 2/3 of the compressed ADPL/DPL, I don't know exactly. Because the frame above (that has 284 FALL) with:
  • 1000 DPL, it shouldn't do anything
  • 700 DPL, FALL is still arguably low enough to don't modify it
3068517
 

·
Registered
Joined
·
958 Posts
Is there any other areas of DTM we should be more focused on? The red shift (overstat) currently seems to be consuming the development cycle. I would like to see more emphasis on expanding the automation of madVR DTM and other algos. If we could wrap our minds around how wonderfully Envy was developed I think we would be in a much better place. You really do want the algos to control as much as possible. I love to sit back and watch a perfectly hands-off created picture. Just like the goal of Envy. Thanks, Madshi. What a beautiful creation.

I know we have been “trying” to manually control it all, and it has been fun. I have loved fine tuning it all. But, I see a more perfect future for madVR… we can still do our tweaking (low level profiles)… but most of the perfection should already be baked in with programmatic control… It could be called "madVR Pro".

btw: SVP is charging $19.99 for a life time sub... so madVR Pro having 10x more algos should be around $199 for a life time sub...
 

·
Registered
Joined
·
4,755 Posts
Is there any other areas of DTM we should be more focused on? The red shift (overstat) currently seems to be consuming the development cycle. I would like to see more emphasis on expanding the automation of madVR DTM and other algos. If we could wrap our minds around how wonderfully Envy was developed I think we would be in a much better place. You really do want the algos to control as much as possible. I love to sit back and watch a perfectly hands-off created picture. Just like the goal of Envy. Thanks, Madshi. What a beautiful creation.

I know we have been “trying” to manually control it all, and it has been fun. I have loved fine tuning it all. But, I see a more perfect future for madVR… we can still do our tweaking (low level profiles)… but most of the perfection should already be baked in with programmatic control… It could be called "madVR Pro".

btw: SVP is charging $19.99 for a life time sub... so madVR Pro having 10x more algos should be around $199 for a life time sub...
I have to agree. BUT it's us enthusiasts made it possible by tweaking testing and providing consant feedback to get this far and even for the Envy to be born. Of course with Madshi on the forefront. Being a free product and I think its the least we can do keep testing and reporting back as its a very important step in development IMO for Madshi to be able to further enhance the product.
Having said that I would love a stable release where I can just sit back and not having to think about "what if I changed this and that" catch 22 sort of thing. I am quite sure there will be a day when it will be licenced and turn int paid product though but then we will have a lot less "control" over development.
 

·
Registered
Joined
·
200 Posts
@madshi and others.

Q: Are the .measurements files a thing of the past?

I have around 50GB of .measurements files and are about to do some end of the year cleaning.




Best wishes,
//arcspin
 

·
Registered
Joined
·
1,050 Posts
I actually haven't checked in a number of betas, but awhile back measurement files and Soul night's tool returned only blue screens in madvr. It actually would be a nice feature to have them supported again since it would give madvr perfect foresight again, but I haven't heard anything about that coming back in about a year now, sadly.
 
10481 - 10500 of 12825 Posts
Top