AVS Forum banner

1 - 20 of 1220 Posts

·
Registered
Joined
·
1,479 Posts
Discussion Starter #1 (Edited)
madVR Tool: MadmeasureHDR Optimizer (Measurements, Dynamic Clipping & Target Nits)

This Thread will be dedicated to our tool "MadmeasureHDR Optimizer" to help optimize measurements files generated by the process "MadmeasureHDR" from beta 38 onwards.


Please use madVR Beta V40 in combination with this tool:
http://madshi.net/madVRhdrMeasure40.zip

It was previously called madmeasure dynamic clipping tool since it all started with dynamic clipping.

Now the tool can also automatically choose a Target Nits in the measurement file for a movie.

More importantly, it can adjust dynamically the target nits DURING the movie in order to optimize the brightness and picture contrast at all time. :)


ps: goal is to offset the discussions about the tool here instead of the original madvr HDR Tone mapping developpement thread:
https://www.avsforum.com/forum/24-digital-hi-end-projectors-3-000-usd-msrp/2954506-improving-madvr-hdr-sdr-mapping-projector.html


The main focus of the measurement file and its optimization through our tool is to improve quality of madVR HDR tone mapping.

1 - madVR SETUP tutorial and explanations

2 - Madmeasurehdr Optimizer Tool Tutorial (German, please use google translate)

3 - Explanation / Tutorial for the dynamic tone mapping ala madVR (German, please use google translate)

Here a bit of explanation /history:

1) The measurement process was introcuded by Madshi to know in advance all the scenes cuts and all the frames peak nits.
Initial goal was there to get rid of ANY unwanted clipping (through knowledge of the future) while changing the peak value used in the tone mapping with a fixed speed to stay invisible.
-->The LIVE algo does not know what are the next frame which are coming. Therefore if it's currently tone mapping up to 500nits, but the next frame peak is 2000nits, it will NOT tone map to 2000nits because it would create visible flicker. Instead, it will maybe tone map slightly higher than 500nits, maybe 510nits, and everything between 510nits and 2000 will be clipped.

2) Then, madshi explained us the structure of the measurement file and we created in the tool something called "dynamic clipping".
For every frame, we look for a highlight knee to overwrite the real frame peak with an "effective peak" to gain back brightness and contrast/HDR in the picture at the cost of clipping only a few pixels above the said detected "highlight knee".
The algo was partially implemented in the LIVE algo, but because of 1), it can lead to visible clipping since it then combines the uncontrolled clipping of the LIVE algo + the controlled clipping from the dynamic clipping. That's why most people keep "dynamic clipping" deactivated in the LIVE algo but keep it activated at 100% with the optimizer tool to get an extra "punch in the picture".

3) Then we wanted for a long time to try out a "dynamic Target nits" per frame. For Christmas , madshi allowed the measurement file to hold such "Target Nits per frame" so we could play around but there is dynamic target nits present in the measurement file to begin with.
We then started a small journey with the madVR testing community here to develop a nice method and algo for a target nits per frame.
We came up with the FALL algo, chapter detection (containing mutiple camera cut) and centered rolling avg.

Since the the dynamic target nits idea and result was a success, it was implemented by Madshi in the LIVE algo. Only, once again, the LIVE algo does not know the future which means it can get surprised if the FALL gets higher or lower which can result in a too bright or too dark picture when this happen.
Also, to mitigate this, the LIVE algo logic is to reset the target nits at every camera cut, since it is mostly invisible when you do it there.
But, resetting at every camera cut can lead to strong brightness change between 2 following camera cut for 2 persons speaking in the same room or outside (Little girl intro scene in the MEG, Interrogation scene in Venom, are both good examples).

The optimizer tool however has access to the whole movie histogramm per frame saved in the measurement. So it knows in advance what's coming in the future.
Therefore, the tool can completely smooth out all the camera cut and just roll smartly over everything because it knows in advance what's coming and can prepare for it (same logic that was madshi implemented for the peak nits with the measurement).


Conclusion: the LIVE algo looks great in itself but since it cannot know the far future it has limitation that the optimizer tool together with the measurement file does not have.
So best quality, per frame but also smoothness wise will be adchieved with the measurement process introduced by madshi.
In itself, the LIVE algo still looks pretty awesome and does not need any measurement and can/will be integrated in the ENVY to get madVR awesome tone mapping for ANY source!

Like always, the last few percent of quality are always the hardest to achieve, and therefore that's not always for "everybody".
Here it means you have to perform a full measurement of the movie before hand and optimize it. So you loose the convenience of just starting a movie at anytime IF you did not do those steps yet.
But the tool makes those steps simple, even it measurement stays time consuming (~15min per movie with a GTX1080ti).


Hopes this clarify what our tool is for.
And my thanks to Madshi for making all this possible!

Flo
UPDATE 2019-08-25
V4.0.2
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-40.html#post58469164

UPDATE 2019-07-20
V4.0.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-36.html#post58319000



UPDATE 2019-05-27
V3.9.3
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-35.html#post58104732

UPDATE 2019-05-23
V3.9.2
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-35.html#post58089690

UPDATE 2019-05-19
V3.9.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-35.html#post58069964

UPDATE 2019-05-18
V3.8.8
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-35.html#post58065216

UPDATE 2019-05-12
V3.8.7
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-34.html#post58035720

UPDATE_ 2019-03-17
V3.7.9
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-31.html#post57761800


UPDATE: 2019-03-10
V3.7.7
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-29.html#post57723936


UPDATE: 2019-02-13
V3.6.2
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-26.html#post57597756

UPDATE: 2019-02-09
V3.6.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-25.html#post57575494

UPDATE: 2019-02-06
V3.5.8
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-24.html#post57561252

UPDATE: 2019-02-03
V3.5.6
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-22.html#post57544558

UPDATE: 2019-01-21
V3.5.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-17.html#post57475508

UPDATE: 2019-01-20
V3.4.1
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-16.html#post57468496

UPDATE: 2019-01-19
V3.4.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-15.html#post57464522

UPDATE: 2019-01-19
V3.3.3
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-15.html#post57463528

UPDATE: 2019-01-16
V3.3.2
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-13.html#post57448962


UPDATE: 2019-01-14
V3.3.1
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-12.html#post57437966

UPDATE: 2019-01-13
V3.3.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-11.html#post57431312

UPDATE: 2019-01-13
V3.2.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-10.html#post57428292

UPDATE: 2019-01-09
V3.1.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-8.html#post57408890


UPDATE: 2019-01-08:
V3.0.0
https://www.avsforum.com/forum/26-home-theater-computers/3040072-madvr-tool-madmeasurehdr-optimizer-measurements-dynamic-clipping-target-nits-7.html#post57401504


UPDATE: 2019-01-06
V2.9.9
http://projectiondream.com/wp-content/uploads/2019/01/madMeasureDynamicClipping-V2.9.9.zip

UPDATE: 2019-01-04
V2.9.8
http://projectiondream.com/wp-content/uploads/2019/01/madMeasureDynamicClipping-V2.9.8-Download.zip
:D
 

·
Premium Member
Joined
·
9,824 Posts
Good idea to create a new thread :)

Re the dynamic target nits (which sounds exciting!), will we be able to set a minimum target (or have a "keep content below diffuse white untouched" option), to make sure that 0-100nits in the content isn't fluctuating? For example, in my case, I'd specify 200nits as a minimal target.

Looking forward to testing this!
 

·
Premium Member
Joined
·
1,754 Posts
Hi Soulnight,

Thanks for starting this thread.

I've been attempting to use this tool, but could use a little info about how to use it properly. For example:

- at first glance it looks like you can either measure just one file or an entire folder but when you choose the 'Choose File' button it seems to be looking for a measurement file and not a .MKV file. Is the purpose of the CHOOSE FILE button to load a measurement file and modify it?

- When I used the CHOOSE PATH route it started measuring my MKV files. I noticed two things:
1) it reports that it might take up to 20 minutes when in fact it seems to be an hour plus for my files (using a 1080ti with files on a separate NAS). Looking at task manager performance I see the GPU is at 100% so I don't think it's network related?
EDIT: I missed this crucial tip: If you change LAV video filter to D3D11 (native) it works much faster.

2) The app seems to go into 'non-responding' mode (cursor is just a rotating circle instead of arrow) and I can't seem to click the STOP button.
EDIT: Tried this a second time and while the app still went white and seemed to not respond, it DID stop after the current movie and displayed a log file.


- I see that it creates 3 files: one in an ORIGINAL folder and then a measurement file + a 'details.txt' file. Having read various posts about whether to store the measurement files in the same folder or in a dedicated one, it seems that at this point it is safer to keep them all together? I ask because I'd prefer them to be separate but don't want to make your app unhappy. :)


- Once measurement files exist, in what way should we be modifying them to match the peak nits of our display?
EDIT: I now see we can click on either the Manni or Neo-XP button as an 'adjustment' and then when re-processing the folder it will revise the measurement files to have the corresponding target nits. Still whether manual means 'use profile' or one's own desired value...?

Thanks so much for all your (and Anna's) work on this!
 

·
Registered
Joined
·
5,351 Posts
Should there not be a link to the latest build somewhere in this thread? :p :D Thanks for your hard work!!
 
  • Like
Reactions: NoTechi

·
Registered
Joined
·
114 Posts
2) The app seems to go into 'non-responding' mode (cursor is just a rotating circle instead of arrow) and I can't seem to click the STOP button.

Same thing in my environment , GPU nVidia 1080Ti OC , CPU Intel I7 6700K and LAV video filter to D3D11 (native)
 

·
Registered
Joined
·
114 Posts
What it consists of deselect the setting (Apply dynamic Clipping) keeping only enabled (Apply target nits selection) that what changes for the purposes of measurement final?
 

·
Registered
Joined
·
127 Posts
Great to have this seperate thread for your optimizer :)

I was also thinking about the other tools which are being worked on like Bernds tool which is like a swiss knife with different functions like automatic JVC setting switching, Pandm1967 which has a great visualization of measurements to analyze the measurement files and even Manni still using a batch file to measure. To not confuse standard users it would be best to just have one tool. Or clearly devide the tasks between the tools.

A standard user use case would be to be able to start/stop measuing and optimizing new movies in specified paths to their pre-configured settings by pressing a button/starting a batch file and don't have to worry about anything else. I think this is also Mannies intention still using his batch file.

As of today I would run Bernds tool to measure since it is more reliable/responsive for me while measuring. Run MadmeasureHDR Optimizer to optimize. And on my todo list is to figure out how to get measurements working on a bdmv folder structure with Pandm1967 tool and the HardLink trick. Before doing all of this I have to check if there are updates to those tools and install them if required. Thats a bit cluttered workflow. Would be great if you tool developers would get together and combine the strength of all tools in one to come to the one click solution :)

Beside that really looking forward to the target nits adjustments during the movie!! :)

NoTechi
 

·
Premium Member
Joined
·
9,824 Posts
Great to have this seperate thread for your optimizer :)

I was also thinking about the other tools which are being worked on like Bernds tool which is like a swiss knife with different functions like automatic JVC setting switching, Pandm1967 which has a great visualization of measurements to analyze the measurement files and even Manni still using a batch file to measure. To not confuse standard users it would be best to just have one tool. Or clearly devide the tasks between the tools.

A standard user use case would be to be able to start/stop measuing and optimizing new movies in specified paths to their pre-configured settings by pressing a button/starting a batch file and don't have to worry about anything else. I think this is also Mannies intention still using his batch file.

As of today I would run Bernds tool to measure since it is more reliable/responsive for me while measuring. Run MadmeasureHDR Optimizer to optimize. And on my todo list is to figure out how to get measurements working on a bdmv folder structure with Pandm1967 tool and the HardLink trick. Before doing all of this I have to check if there are updates to those tools and install them if required. Thats a bit cluttered workflow. Would be great if you tool developers would get together and combine the strength of all tools in one to come to the one click solution :)

Beside that really looking forward to the target nits adjustments during the movie!! :)

NoTechi
Of course it'[s for Anna/Flo to decide, but if they provide a way to call the utility, both BerndFfm and myself will be able to call it as we please, so you'll be able to choose a single tool to use (Soulnight's, BerndFfm or my batch file) to do the whole work.

I can't wait to retire my batch file, but at the moment I have a loss of functionality or issues with either of the other tools, so I'd rather be able to call Soulnight's utility until this is resolved by one or the other. At the moment Soulnight's is the closest to my own needs as BerndFFm has evolved in a direction that doesn't work for me anymore (especially the measurements of undesired files).

One thing that I've added to my batch file is the option to force D3D11 native before the measurements for optimal performance, and an option to select D3D11 copyback afterwards if desired.

I'll upload a modified version when/if Soulnight offers a version that can be called from an outside batch/utility, so as to include all the changes in one update.
 

·
Premium Member
Joined
·
9,824 Posts
Tiny possible display bug when measuring a folder and skipping already optimized files:

it looks like the file name displayed is the last completed (skipped) one during the optimization of a non-skipped file, instead of the file currently optimized.

For example, if film1 is skipped because already optimized, the utility displays "film1" during the optimization of "film2", which is a bit confusing, because by the time the data for "film2" is displayed, "film3" (if skipped) is displayed during the optimization of film4, etc.

Not a big deal obviously as the optimisations are fine, it can just be confusing as the user might be thinking that already measured titles are unnecessarily re-measured, and titles that should be measured are skipped (as their title is never displayed during the actual measurements, when everything is working fine.

Hope this makes sense :)
 

·
Registered
Joined
·
1,479 Posts
Discussion Starter #14 (Edited)
madmeasuredynamicclipping Tool V2.9.8

Sooo... Update time!

madmeasure dynamic Tool V2.9.8

Download here:
http://projectiondream.com/wp-content/uploads/2019/01/madMeasureDynamicClipping-V2.9.8-Download.zip

NEW:

1) Mutiple path selection! :D :)
It also remembers:
a) that you had chosen "paths" last time you had the programm running
b) it remembers the paths you had chosen as well
--> So basically, you open the programm and if you want use the same settings as last time, just press Start!




2) Dynamic Target Nits during as movie! :cool:
--> To activate this, just check the "dynamic" checkbox next to "TARGET NITS Selection"

I advise to use "Flo" as a profile for both Static Target Nits and Dynamic Target Nits.
It is scalable with your real peak nits and works very well in my opinion.

For the dynamic target nits, you need to choose:
a) your real peak nits
b) Rolling avg time in frames--> I advise to keep it at the default value of 240 frames, it works very well like this.
c) Minimum Target Nits allowed --> Here I advise to peak your real nits to maximize dynamic advantages.
For @Manni01 of course, you can choose 200nits. ;)
d) Maximum Target Nits allowed --> 1000nits default.

I'll tell more about the dynamic target nits algo tomorrow. But from my testing, it works very well and provide a completely new experience.
No more compromise! :)

NEW GUI:



Example of the automatic target nits on the MEG with the above selected settings and 107 peak nits:

Target Nits: 971


Target Nits: 975


Target Nits: 624


Target Nits: 643


Target Nits: 520


@madshi
Thanks again for the new measurement file format. It works great for dynamic target nits.
However, as I told you per PM and that you can now see on the screenshot, the madVR only reads the dynamic target nits from the measurement file, when "dynamic target nits" flag is selected in the measurement file.

That's it!


Enjoy :cool:
Anna&Flo
 

Attachments

·
Registered
Joined
·
1,479 Posts
Discussion Starter #19 (Edited)
Yep, I'm going to give it a go now, can't wait to see how well it works.

If you don't mind me asking how does your method select the target nits?
For the static algo, I have included the description if you pass your mouse over "Target Nits recommended by Flo".

For dynamic, it's a bit more complicated...and I need now to go to sleep. :)
We'll discuss that later. ;)

Flo

Edit: but I am not sure how well it scales well above 100nits.
Between 50 and 107 nits, it scales very well.
If you have way more than 107nits, you may want to "lie" and write 107nits instead for a first try.
 

·
Registered
Joined
·
173 Posts
For the static algo, I have included the description if you pass your mouse over "Target Nits recommended by Flo".

For dynamic, it's a bit more complicated...and I need now to go to sleep. :)
We'll discuss that later. ;)

Flo

Edit: but I am not sure how well it scales well above 100nits.
Between 50 and 107 nits, it scales very well.
If you have way more than 107nits, you may want to "lie" and write 107nits instead for a first try.
I've tried it with my actual measured screen nits (212) and while the selected target nits values were very high, 2 or 3 times what I would normally use, the results when playing are very good. I'll try a few more titles but one thing is that the brightness change can be quite noticeable, I'll try with a longer rolling average value. I've also noticed that often the dynamic target nits value is quite a lot higher than the tone mapping value, is your dynamic target nits algorithm being applied to the original measurements or the clipped measurements? It looks good I'm just wondering if it's intended or not.

EDIT: Actually, yeah I see what you mean, I will try it with a lower display nits value.
 
1 - 20 of 1220 Posts
Top