Originally Posted by SirMaster
Hmm, I think the filter would fix the issue, but it's not worth it with your nits.
You could consider creating a rec709 3DLUT and outputting madVR as rec709. Yes you lose a little color gamut, but I think it's probably worth it to retain your brightness. You don't really reach P3 anyways without the filter so you aren't losing all that much. You might be surprised how good HDR movies still look converted to rec709.
If there is even a small but noticeable benefit to using P3 without the filter compared to using BT709, I think I still would prefer to stick with P3 for that marginal improvement (again, assuming it is noticeable, despite being marginal). That's because AFAIU ATM the clipping issue I'm seeing is on just a few rare and specific scenes. For instance just to have his red coat like perfect in Great Showman and to not have some clipping in Atomic Blonde (I haven't checked for clipping here yet) that's not enough motivation to move to BT709. Now as I pay more attention to the issue, if I start noticing it in more and more places, with a fair number of movies, and it's not as isolated as I think it is currently, then I may try BT709.
I ran into this banding and posterization effect when making 3DLUTs for my NX5 especially when trying to make one for BT2020. I did notice that using more patches lessened the effect, but I did not need very many patches. I was doing quick 3DLUTs like 115 or 175 patches and when I did an 485 and 895 patch 3DLUT, the posterization was pretty much gone.
After that I used P3 gamut profile rather than BT2020 and the posterization was even less with even 175 patches being enough to produce a result in which I did not notice any obvious posterization.
OK that may be a difference between my RS500 and your NX5 which is several (3 I think) versions newer.
Originally Posted by Manni01
As discussed already, using a small LUT is only possible if you 1) cover 100% of the target gamut and 2) use the JVC Autocal to get a good baseline (under 5 dE max, 2-3 dE average). Without the filter, you can't use a 3D LUT unless you do at least a 5,000 points LUT, simply because you are way too far from the targets as you only cover 90% of P3 at best.
Oh wow, I didn't realize we were talking about THAT much larger. I thought 1200 points was already pretty big.
I think 5,000 points would take CalMAN and DisplayCal maybe like 6-8 hours to complete. Could be worth a try, maybe just let it run overnight. If anything just to see out of curosity if it improves things.
Post calibration the CalMAN color checker showed exceptional results with the 1200 patches for all measures, except the ones at the edge of the gamut, which is what would be expected. But of course that doesn't sound like "good enough" for handling these particular "edge" cases in movies.
Your Calman LUT might not have shown the same issue on blue, but it shows one on red.
A bigger LUT might alleviate the problem, but it will still likely be there.
I agree that using a rec-709 3D LUT (if you cover 100% of rec-709 or close without the filter) would probably be a better solution than using the filter and drop 10-20% brightness.
You're not getting much more than rec-709 anyway without the filter, so would get better results that way likely.
As mentioned above, it's hard to know whether its worth moving to BT709 or not without knowing how much of a noticeable difference there is with BT709 vs DCI P3 without the filter AND without knowing to what extent I am having these edge cases like with the jacket in Greatest Showman. If BT709 vs P3 with no filter is virtually indistinguishable then sure, why not. But I assume there's still some benefit; otherwise it seems many people who use DCI P3 without the filter would just instead use BT709 in the first place? Unless those people are not using madVR so they don't have a choice to have excellent color space mapping...? And likewise as also mentioned, if these are just a few isolated cases, and there still is SOME benefit to DCI P3 without the filter, I'd probably want to just stick with that.
Also just to make sure - AFAIU, if I wanted to try BT709 I would just do a fresh calibration with the PJ set for BT709 and then remove the DCI P3 calibration from madVR, so the only calibration file it has is the newly created BT709 one. Then when I tell madVR to use the 3D LUT calibration it will see there is only the BT709 one, and will automatically map BT2020 to BT709 and apply that 3D LUT, yes?
Then again - I'm not sure it's even worth the trouble. See below where
says he has the issue with any 3D LUT including BT709.
Originally Posted by stefanelli73
The scenes of ATOMIC BLONDE concerning the clipping of the red are from the minute 36 ': 57 "to 40': 16" (bar scene) and from 50 ': 57 to 52': 40 "(disco scene) .... then the problem disappears only if I go to THIS DISPLAY IS ALREADY CALIBRATED, choose DCI-P3 and pure power curve 2.20, instead with any 3dlut (BT.709, SDR DCI-P3 and SDR BT.2020) the problem remains.
Thanks. Unfortunately this seems to confirm my findings as well, except I haven't tried with BT709 and probably won't go through that trouble considering you say it doesn't help. Just to be sure I understand - you are saying that even when tone mapping to BT709 and using a targetted BT709 calibration (and not a re-targeted on from a DCI P3 or BT2020 calibration) you have the same problem with the clipping and over-saturation?