[Soulnight]

Hmmmmm..., I do take more than just the first histogram column into account for my black bar exclusion logic. Maybe that explains why madVR sometimes produces higher brightness when using the FALL algorithm compared to Anna & Flo's tool? Maybe my logic does exclude black bars from Lucy while Anna & Flo's tool does not? FWIW, my black bar logic maxes out at 28% of the pixels of a frame, which is roughly what the black bars in a scope movie would be.

I checked with the nice analyser tool of @PandM for Lucy.

As we can see below, the black bars pixels are coded in the first colum of the histogram (~0nits).
Black bars represent here: 25,736% of the black bars.

And movie "black" is coded in the third column(>0,005nits mastering black luminance)

@madshi: so here you may exclude 28% of the pixels in Lucy, and we exclude only 25,736%.
How big the impact on the FALL value will change from frame to frame.

We have currently 3 logics in the code:
1) Detect black bars % to correct the FALL value of the complete movie
-->here we only look at the first colum % value (until now), and we take the 0,1% quantil from the bottom (to avoid cases where the movie as a few single full frame picture without black bars in the into or credits for example).
This works very well, and the correct FALL without black bars is in agreement with what madVR calculates as well.

2) We have the FALLnoBlack value recalculated for each frame that we use in the "FALL algo" where we calculate the FALL for every histogram colums except the 1st colum
--> If only the black bars are coded in the first colum as absolute black bars, we exclude them
--> If the movie black is also in the first colum, then we exclude it as well
--> If the black bars are not coded in the first colum, but in the 2nd or third, we do not exclude them (yet)

3) The FALLBT2390 recalculated for each frame introduced in V3.7.7.
In V3.7.7 we exclude only the first colum
In V3.7.8 we exclude the first 18colums to about ~1nits

How much should be excluded to be as close to the "on screen eye brightness perception" is tricky.

For example, if the movie has 25% of the pixels as black bars and a fully white frame, then it will appear very bright. The black bars (even 25% oif the picture) here do not play a role to how bright you perceive the picture. Probably the perceived brightness without black bars would be very similar.

An other example is if, the left half of a frame is black, and the right half is very bright. Very likely, we will perceive this picture as bright and ignore the black 50% of the picture.

But there is a limit to that I guess. If 80% of the picture is black, then your eyes may not focus only on the 20% bright this time. And we will consider this frame darker.

Even more true with 99% black, and 1% white. It will be mostly perceived as dark.

And if your picture is 50% black and 50% white, but with nicely mixed with 1 pixel black next to a white pixel, it will apear grey, and thus this time we have to consder ALL the black pixels.

So there are 2 questions:
1) How high in nits do we consider black "black" and can therefore potentially ignore it in the FALL calculation?
2) How much % of black picture area can/will we "most often" ignore in our " on screen brightness perception" and therefore focus on the rest of the picture/pixels which are not black?

For the future, we will adapt our FALLnoBlack definition to make sure it can also ignore black bars in the case there are not in the first colum.

And for FALLBT2390, maybe go less high than 1nits...and only exclude up to 50% of the pixels.

For those who are interested, here are the "histogram intervall centered nits value" for the first few colums of the histogram:

 

[Dexter Kane]

@Soulnight

In 2001 at around 1:57:00 there is a scene which is mostly black (space) but the parts which are bright are very bright. It might be a good scene to test how the tool should deal with black in a scene. I remember I had a quick look at this scene with 3.7.7 and didn't like it compared to FALL. With FALL the targets were fairly high but with 3.7.7 they were low and it makes a pretty big difference to the planets which get blown out at low targets. I haven't had time to test again with 3.7.8 but I imagine it would be better due to it ignoring more black.

 

[Manni01]

@Soulnight

In 2001 at around 1:57:00 there is a scene which is mostly black (space) but the parts which are bright are very bright. It might be a good scene to test how the tool should deal with black in a scene. I remember I had a quick look at this scene with 3.7.7 and didn't like it compared to FALL. With FALL the targets were fairly high but with 3.7.7 they were low and it makes a pretty big difference to the planets which get blown out at low targets. I haven't had time to test again with 3.7.8 but I imagine it would be better due to it ignoring more black.

I can confirm that the target is above 500nits in that scene with 3.7.8, which works better (this is with 115nits peak and 100 dynamic tuning).

I've looked at quite a few clips with 3.8.8 and am about to watch a first full film with it. I liked what I saw very much until now, and will report later when I've seen more.

Great work Anna and Flo

 

[Manni01]

OK I've watched a full film (Venom) with 3.7.8 / BT2390 and I have seen a lot of instability (DI disabled of course). Apart from this instability, I was happy with the tonemapping, which felt very natural.

It was the first time I was watching that film, so I was really not paying attention that much to tonemapping. It just caught my eye regularly and I went back to one of the most offending scene, the interrogation chapter starting around 01:14:50 to look into in more carefully after the end of the film.

It's a dark-ish scene, but there are lots of bright lights in it, so probably hard to deal with. Not sure if it's already part of Neo-XP and Fer15 list, but it probably should be if it isn't.

The target kept moving between 200 and 800 nits, and although I'm guessing here it looked like the fluctuation came from the fact that the BT2390 was trying to anticipate the next change. When the bad guy is talking, the target in the next scene is often too high, and as it's trying to anticipate it makes the shot go darker (noticeably) as we get closer to the scene change.

The scene should probably be seen at around 300-350 nits, without much variation, in an ideal world / with an ideal algo.

I played it again with the live algo, after disabling the measurements file, and I didn't have this issue. The target was also changing between 300-500nits, but there was no annoying shift in relative brightness.

This was with my usual 115nits peak / 100 dynamic tuning / 0 no comp with BT2390 and with 150nits no compression / 75 dynamic tuning with the live algo. The max was 10000 with BT2390 and 4000 with the live algo, but it doesn't matter in this example.

Hope this helps!

 

[Soulnight]

Thanks @Manni01 for the feedback!

I guess 3.7.8 is pretty unstable since it can exclude all the black up to 1nits and then only reacts to the remain bright spots in the picture.
@Dexter Kane reported something similar with Deadpool Bar scene. (btw. frame number would be usefull so I can check it out).

A few questions to Manni01
- Does the 3.7.7 Bt2390 algo does a better job in the sceen you observed with the instability?
- Does the FALL algo with measurements works similarly good / better /worse than the LIVE algo for this scene?
- What Chapters settings are you using?

Cheers,
Flo

 

[Manni01]

Hi Flo,

Makes sense.

I don't have a specific frame number because the whole scene does it. Look specifically for the shots of the bad guy (the host to the second symbiote, not the henchman that leaves the scene right away). I'll try to give timecodes later.

I used the default chapters setting, as I've been doing for a few versions now as I was quite happy with them.

I'll try to compare with 3.7.7 and with the FALL algo but time is hard to find at the moment... It's only the third film I watch on my rs2000, and there are 170 hours on it...

 

[Soulnight]

Hi Flo,

Makes sense.

I don't have a specific frame number because the whole scene does it. Look specifically for the shots of the bad guy (the host to the second symbiote, not the henchman that leaves the scene right away). I'll try to give timecodes later.

I used the default chapters setting, as I've been doing for a few versions now as I was quite happy with them.

I'll try to compare with 3.7.7 and with the FALL algo but time is hard to find at the moment... It's only the third film I watch on my rs2000, and there are 170 hours on it...

Venom
Minutes 1:15:01
Frame 107934

Here in both FALL algo and BT2390 algo, you see some pumping (catch-up) because the scene cut is not recognized from the previous scene AND the rolling avg time is set to agreesively with the default chapter settings (which we forgot to update...):
Rolling avg time: 120
Merge Scene: 100
Minimum (sandwich) duration: 60
Minimum Chapter duration: 50

If you try what we've been using:
Rolling avg time: 240
Merge Scene: 100
Minimum (sandwich) duration: 60
Minimum Chapter duration: 50

It's much more stable with those settings with both algo despite rolling above the sandwiches.
(Sandwich of 60 with 120 rolling avg is just too long a sandwich, or too short the rolling avg )

If you like nice clean cut instead of rolling over short "sandwich" scenes, just set:
Minimum (sandwich) duration: 0s
It will cut nice and clean at each new camera change similar to the LIVE algo (no pumping)

Next V3.7.9 will only remove black up to 0.1nits (first 8 colums) and maximum 50% black removed.
It also makes things a more stable.

But in any case, the FALL algo has much less variation in resulting target nits than BT2390 algo (at least at the moment).
This can be a good thing or a bad thing depending on the situation.

Cheers,
Flo

 

[Manni01]

@Soulnight

Great, thanks, I'll give these settings a try. Glad you found it. Yes around 01:15 was one of the worst offenders during the film.

Any ETA for 3.7.9? It sounds like it would be a good compromise between 3.7.7 and 3.7.8

 

[Manni01]

I did a test with your chapters settings and min chapter to zero: much better.

Very stable!

I used to have 240 frames too, I can’t remember when that default changed or why, but it’s definitely better like this.

I’ll watch a full movie with it and 2390 at the soonest opportunity.

Thanks!

Manni

 

[Soulnight]

madmeasuredynamicclipping Tool V3.7.9

@Soulnight

Great, thanks, I'll give these settings a try. Glad you found it. Yes around 01:15 was one of the worst offenders during the film.

Any ETA for 3.7.9? It sounds like it would be a good compromise between 3.7.7 and 3.7.8

So here it is:

V3.7.9 Download:
http://projectiondream.com/download/madmeasuredynamicclipping/

NEW:
- BT2390 algo:
only excluding black pixels below 0.1nits (8th histogram colum) but never more than 50% of total picture area
--> It should make the algo more stable for those scenes with a lot of dark/black than 3.7.8 and still better reacts than 3.7.7.
But this is only a slight tweak.

I'll have no access to my computer for the next 2 weeks so there will be no new version until then.

ps: I did try ( A LOT) to implement a variable "target brightness on screen" asking for a brighter target than 12.5nits for brighter scenes, and lower target brightness than 12.5nits for darker scenes.
This should (and di) bring the target nits of both kind of scenes closer together.
It has also the distinct advantage to make brighter scene brigher than darker scenes.
But I did not get to something (always) satisfactory even it the results were sometimes very interesting.
I'll look (probably) some more into that if I get some fresh ideas when I get back.

Enjoy,
Flo (&Anna)

 

[Manni01]

Thanks Flo (and Anna). Whether it's business or pleasure, have a good time away, and thanks for delivering this new build before your departure

 

[SirMaster]

Is there a way to see like line graph plot of what the target nits looks like for a measurement file like nits across time?

I think it would be neat to be able to see what the different overall plot looks like between the FALL and BT2390 algorithm. And also what changing parameters in the dynamic clipping tool does to the result nits.

 

[quietvoid]

Is there a way to see like line graph plot of what the target nits looks like for a measurement file like nits across time?

I think it would be neat to be able to see what the different overall plot looks like between the FALL and BT2390 algorithm. And also what changing parameters in the dynamic clipping tool does to the result nits.

There is: https://www.avsforum.com/forum/26-h...urehdr-optimizer-measurements-dynamic-clipping-target-nits-11.html#post57433788

 

[Fer15]

I've done some very brief testing, and I think I'm favouring the BT2390 algo now.


I quickly looked at the Harry Potter scenes again, the scenes from Venom and 2001, which Manni01 and Dexter Kane were referring to, and one from The Meg.

With 50% dynamic tuning strength, there’s not much difference in the BT2390 v.3.7.9. compared to v.3.7.8

I created an Excel doc and recorded the stats with two different settings for the Harry Potter scenes. The corresponding madVR live algo settings were also the same entries where applicable.



I also included the % of the dynamic target compared to the measured frame peak nits for some of the scenes. I know you don’t base the target on a %, but I think it’s quite good for quick illustrative purposes. It shows the rather large "% of" for the final Harry Potter test scene (72.6%) relative to the two other test scenes (42%, & 52%) with BT2390 at 50% despite the measured frame peak being lower


Here’s a comparison of the Harry Potter scene with the madMeasurementAnalyzer graph.







Number 1 is the untouched measurements file. Number 2 is FALL, number 3 is BT2390 (both at 50%). The three red Xs at the bottom roughly illustrate the position of the three scenes that I took. As you can see, the 3rd X has a higher dynamic target nits in comparison to the first two Xs (relative their corresponding position on the 1st graph), which is borne out by column H in the excel doc




With 50% dynamic tuning strength (which is on the lower end of the dynamic strength spectrum) for BT2390: there’s not much of a difference/increase in the dynamic target when comparing the 50% strength (column H) to 100% (column I), to the point where it doesn’t really matter if you set the strength to 50% or 100% for these scenes. In that respect, I quite like the increased flexibility in the dynamic tuning strength of the FALL algo. Compare 50% vs 100% strength (columns D vs E). There’s a much bigger gap for all three Harry Potter scenes, and 2001, which gives the user more flexibility in changing the strength to suit their needs. I do agree with Manni that the dynamic target for Venom might be a bit too high when the camera is on Tom Hardy (column H). Too much of his face is in shade (even when taking the fact that the light source is behind him into consideration). Also, the 2001 Space Odyssey scene has far too high a dynamic target relative to the measured frame peak for one of the scenes imo.



However, I think the dynamic tuning strength for BT2390 v.3.7.9 needs to be readjusted to make the results more comparable between BT2390 vs FALL

Graphs 2 and 3 of the Harry Potter madMeasurementAnalyzer bear that out i.e. the BT2390 chart has a higher dynamic target nits than the FALL graph when both are at 50%.


If I reduce the dynamic tuning strength with BT2390 v.3.7.9 to 30% and 35% for the Harry Potter sample, I get the following:


30%






35%






The dynamic target with 30/35% (BT2390) is now more comparable to FALL @ 50%
The Excel doc at 30% (column J looks better in that regard and is now more comparable to FALL @ 50 % strength column D)



I also looked at a sample from The Meg, which is quite interesting and it probably favours BT2390 (I think)


Here are four consecutive camera changes of the same scene







If you look at the excel data (columns D, H, & J), Bt2390 at 30% dynamic tuning strength looks more consistent imo



There’s a greater fluctuation in the measured frame peak (probably due to different reflection/light sources in the background) with FALL: with FALL the range is from 198 to 315. With BT2390 it is from 216 – 258 (which is more stable).


Also, what’s interesting to note is that shot 2 and 4 are the same shot albeit with a slightly different camera angle. There’s a bigger discrepancy with FALL than there is with BT2390. There’s a difference in the measured frame peak (it nearly doubles), but I don’t think that should change the APL level from a viewer’s perspective. The viewer should be expecting a similar brightness imo. Although in fairness, the discrepancy isn’t really that noticeable.


FALL cut 2 vs 4






BT2390 cut 2 vs 4











And if you look at this scene from Venom (which Manni raised previously) and columns D & J from the excel doc (the measured frame peak is consistent 915 vs 931), but the first shot might be too dark with FALL compared to BT2390. Also, I think BT2390 might be more consistent (i.e. dynamic target 204/215 vs FALL 385/194)

before/after cut


FALL






BT2390






Maybe this could be different again if I simply reduce the FALL dynamic tuning strength of FALL as well, but that could be a process of going back and forth between the two comparing 30 vs 40, 31 vs 39 etc




Of course, this is only a handful of scenes...

 

[Manni01]

@Fer15: great job, thanks for all the data. I'll try lowering dynamic tuning with BT2390 to see if I like it better. Can you remind me what your actual peak is?

 

[Neo-XP]

@Fer15 Thanks! it seems it might work for lower dynamic tuning values, but 30/35 for BT2390 is way too low, even for me

I get more consistent results with the FALL algo at higher dynamic tuning values:

BT2390@35 / BT2390@50 / FALL@75

 

[Fer15]

@Fer15 : great job, thanks for all the data. I'll try lowering dynamic tuning with BT2390 to see if I like it better. Can you remind me what your actual peak is?

130

@Fer15 Thanks! it seems it might work for lower dynamic tuning values, but 30/35 for BT2390 is way too low, even for me

I get more consistent results with the FALL algo at higher dynamic tuning values:

BT2390@35 / BT2390@50 / FALL@75

I don't think that I like anything higher than 50 dyn. strength (in the live algo at least (originally it was 75)).

Just looking quickly at your graphs, I wonder if something between 50-75 (i.e. 60-65) for bt2390 would be a higher, yet still hit a sweet spot for consistency?

I think it might be easier/quicker to test if/when Madshi implements this option into the live algo as we can test it on the fly

 

[Manni01]

@Fer15 thanks looks like we have a similar peak (130/115).

I was planning to do baby steps: try 75nits as in the live algo first, and take it from there. It sounds quite extreme to go down to 50, but I'm ready to experiment with an open mind

I'll let you know how I get on.

 

[Soulnight]

@Fer15 thank you for providing a feedback!

Bt2390 and Fall algo are completely different at core. So there is no simple way to make the dynamic tuning of both approach give you the same target nits.

As I wrote yesterday in the other topic:

Fall algo target nits =
Max(entered no compression limit , entered real nits ) + 2×Fall×dynTuning%

Most people use 150 no compression limit. But @Manni01 used to have 300 there. And @Dexter Kane has 200 real nits.
And if someone puts 100 no compression limit with 50 real nits, the start offset will be different again.

This offset is very important for any dark or middle average scene since for a no compression limit of 150, it will be at least 150nits.

If the FALL value is 1 nits.
You will get with 100 dynamic tuning: 150+2×1×100/100=152

If the FALL value is 2 nits.
You will get with 100 dynamic tuning: 150+2×2×100/100=154

If the FALL value is 4 nits.
You will get with 100 dynamic tuning: 150+2×4×100/100=158

If the FALL value is 8 nits.
You will get with 100 dynamic tuning: 150+2×8×100/100=166

If the FALL value is 16 nits.
You will get with 100 dynamic tuning: 150+2×16×100/100=182

If the FALL value is 32 nits.
You will get with 100 dynamic tuning: 150+2×32×100/100=214

So Fall algo close to the offset, will give very similar target nits even if the Fall gets multiplied by 16 or 32!!!

And if real nits is lower than the no compression limit, it does not play any role except as lower limiter.


-----
The bt2390 algo is set up to give exactly 12.5nits as average on screen brightness with 100% dynamic tuning.

(And if you have 200% dynamic tuning, it will target 12.5×100/200=6.25nits instead.

Anf if you have 50% dynamic tuning, target brightness will be:
12.5×100/50=25nits)

And what it does is try a target nits, calculate the corresponding Fall after bt2390 gamma manipulation, and see what it will give the user for a brightness on screen based on their inputed real nits on screen.

If it's more than 12.5nits, it will try a higher target nits. If it's less the opposite.

Point is, in the end a target will be found for which the 12.5nits on screen average brightness is achieved.

Bt2390 is linked to your real display nits. So let's take 100 real nits as an example.

Let's imagine the fall after bt2390 is not heavily linked to the chosen target nits in our example. (Every pixels are below the target nits).

Example 1:
If the FALL value after bt2390 is 1 nits.
With 100% dynamic tuning, we want 12.5 nits average screen brightness.

On screen brightness is: fallbt2390 × real nits / target nits

Here target nits= fallbt2390×realnits÷12.5=1×100÷12.5= 8nits
So here, if you want 12.5nits on screen average brightness , and your real nits is 100nits, you should choose a target nits of 8nits.
But since this is below your real nits, it will be 100nits instead and the average on screen brightness will be exactly 1nits (as intended by the director intend).

Example 2:
If the FALL value after bt2390 is 12.5 nits.
With 100% dynamic tuning, we want 12.5 nits average screen brightness.


On screen brightness is: fallbt2390 × real nits / target nits

Here target nits= fallbt2390×realnits÷12.5=12.5×100÷12.5= 100nits
So here, if you want 100nits on screen average brightness , and your real nits is 100nits, you should choose a target nits of 100nits.

Example 3:
If the FALL value after bt2390 is 16 nits.
With 100% dynamic tuning, we want 12.5 nits average screen brightness.


On screen brightness is: fallbt2390 × real nits / target nits

Here target nits= fallbt2390×realnits÷12.5=16×100÷12.5= 128nits
So here, if you want 100nits on screen average brightness , and your real nits is 100nits, you should choose a target nits of 128nits.


Example 4:
If the FALL value after bt2390 is 32 nits.
With 100% dynamic tuning, we want 12.5 nits average screen brightness.


On screen brightness is: fallbt2390 × real nits / target nits

Here target nits= fallbt2390×realnits÷12.5=32×100÷12.5= 256nits
So here, if you want 100nits on screen average brightness , and your real nits is 100nits, you should choose a target nits of 256nits.

-----

Now, in reality it will not be that linear since the fallbt2390 will change with your target nits.

Generally if you decrease your target nits, the bt2390 fall will also decrease. And the opposite is also true.

Let's take a fictive example of the meg:
Very bright picture:
Fall without bt2390= 1000nits
(Let's say with some pixels at 4000nits)
But...
Fall after bt2390 cannot be higher than the target nits.
So if you try a target nits of 500nits, your Fallbt2390 will maybe be 400nits.
This would give you an average brightness of: 400×100/500=80nits

if you try a target nits of 1000nits, your Fallbt2390 will maybe be 600nits.
This would give you an average brightness of: 600×100/1000=60nits

As you can see, it's not linear at all and depends on the histogramm distribution.


----

Conclusion, if someone tells me he would like a different value that 12.5nits for 100% dynamic tuning, I can of course change that, but Fall algo and bt2390 reacts completely differently to it anyway.

For the Fall algo, if Fall value are small, even it gets twice as big from 12.5 to 25 Fall, the target will almost not change anyway. Target will go from 175 to 200nits if you consider 150 as an offset.

With bt2390 looking for holding constant the brightness for the user, it will change a lot! Since the Fall is twice as big.

With the Fall algo, as the Fall value increases, so does the change since the offset (150) becomes small in comparison.

With bt2390, the logic does not change "sensitivity " based on the height of the Fall value like that.

---
For the future, I will continue to experiment with a variable brightness target for the bt2390 algo. Brighter scene would ask for example for 25nits and darker scene for 6nits.

---

Also please remember that we have limiter in place:
Bottom limiter: real/min target & knee100nits for the bt2390 algo

Top limiter: max target, 2×avgHL, frame peak

2×avgHL is very important for the Fall algo in order not to get target= peak most of the time in the Meg for example since tge Fall can be as high as 1000 to 2000nits even!


----
Hopes this help all of you to understand how different the fall algo and the bt2390 algo are.

Cheers,
Flo

 

[Manni01]

Thanks a lot @Soulnight for this detailed explanation.

To be fair, I started with 300nits for the no compression value when you first offered it, then I went down to 200nits fairly soon after that, and I settled for 150nits a while ago, as my peak brightness went down a bit. Now it's zero (well 115 internally) with BT2390