[zoyd]

This thread is dedicated to the discussion of the usage of the ArgyllCMS tools to create 3D color look-up tables for the eeColor box . You may also be interested in using these tools for your HTPC with the MadVR rendering package.


For those DIYers who would like a free, very powerful cLUT generator for their eeColor box you can now use the ArgyllCMS tools to write out a properly formatted color look-up table (cLUT) for this device. Many thanks to Graeme Gill for implementing this into his software package! Also, a thank you to Florian Höch for developing dispcalGui and initial work on generating 3DLUT formatted output for the eeColor box.


I started out with a simple 2 pt. grayscale + brightness/contrast calibration and put the color control on native (it's worst setting). Using the ArgyllCMS tools I then calibrated the video card to the same white point, average gamma and peak white setting. Next I profiled the display using 3 different patch sets, 750 OFPS (Optimized Farthest Point Sampling), 1500 OFPS, and a 17x17x17 regular cube. Using the profile ArgyllCMS calculates the mapping required to match your source space (Rec709 for us) to your device and generates the 3DLUT in a format that can be uploaded by the eeColor TruVue application.


Here are some results using my Samsung D8000 display. All measurements were made with a Display Pro (D3) profiled against an i1Pro Rev E (2) spectrometer. After loading each 3DLUT to the eeColor box, I remeasured the same 750 OFPS patch set used to create the 1st 3DLUT.




The 750 patch profile set did an excellent job at reducing errors at all but 6% of the patches where errors actually increase (see plot note). Upping the density to 1500 smoothed out some of those bumps but the 17^3 profile had no statistical advantage over the 1500 OFPS.



Saturation runs at 75% stimulus through YCbCr chain (DVD player w/GCD disk)


Before cLUT



After with 750 OFPS based cLUT



After with 1500 OFPS based cLUT



After with 17^3 grid based cLUT



I have done one other cLUT test starting first with a calibration using the display internal CMS. The only advantage here was that I could reduce the number of patches in the profile to 500 and achieve the same performance as above.


There was some concern expressed in one of the other 3DLUT threads about the stability of the D3 using adaptive mode on Plasmas. I did many tests using the ArgyllCMS drivers for this device which have been specifically fine-tuned to optimize both speed and precision on refresh displays. I found absolutely no problems in repeatability or sensitivity with these measurements. The minimum integration time is set for 400 ms but you can manually override that if desired.


I will follow up in next post with a tutorial.


More data on the eeColor processor performance using various software packages can be found here .

 

[zoyd]

Required Hardware

1. An ArgyllCMS supported colorimeter or spectrophotometer
2. A commercial version of the eeColor processor

Required software for command line operation

1. ArgyllCMS - Download the latest version and follow the installation instructions and any probe specific installation notes. The D3 does not require any driver installation, the i1 pro Revs D and E do.
2. Source space .icm files(used in Step V)
3. TruVue Application software

Optional Software: Test pattern generator and GUI frontend

1. madTPG supplied in madVR package
2. dispcalGUI

Prior to using either the dispcalGUI example or the command line example, you should adjust your display (if possible) to D65 at the reference white level. Also, if available, choose a gamut setting larger than your desired source gamut.

Connect the eeColor box to your display and the PC/laptop to the eeColor box via HDMI. Turn the eeColor box processing off (eeC off on the remote) while profiling.


[dispcalGUI instructions]

-Borrowed heavily from the madVR thread -

This example will build a 3d-LUT for the eeColor processor assuming it will be used for video level processing. Calibration of the video card is not used in this example for simplicity and not recommended unless you know you have a specific need for it. The madTPG test pattern generator is also used as it has the ability to dither the test patterns to better than 8-bit resolution and allows for configuration of the pattern background for plasma profiling.

A. Install ArgyllCMS, DispcalGUI, and MadVR

1. Extract ArcgyllCMS to a folder of your choice (ie. C:\Argyll_Vx.x.x) == [ArgyllCMS_install_folder]
2. Install DispcalGUI to a folder of your choice == [DispcalGUI_install_folder]
3. Extract MadVR to a folder of your choice (ie. C:\MadVR)
4. Run 'install.bat' in the MadVR folder as Administrator

B. Create eeColor compatible 3DLUT from ArgyllCMS tools
1. Start MadVR Test Pattern Generator

1. Open madTPG.exe from the MadVR install folder, move the window to your display.
2. Enable "use fullscreen"
3. Enable "disable VideoLuts"
4. Enable "disable 3dlut"
5. Right click on madVR icon in system tray and select device->properties. Make sure display level settings are correct for what your display expects, either video 16-235 or PC 0-255.

2. Start DispcalGUI

3. Set the following settings in DispcalGUI

1. Select 'madVR' under Settings
2. Select 'White level drift compensation' under display device (Only select black level drift compensation if you have an older meter without internal black drift compensation)
3. Select 'XYZ LUT+matrix' for profile type
4. Select 'XXL testchart for LUT profiles, optimized for Gamma 2.2 with Rec. 709 (D65)' for testchart file
5. Click 'profile only' button. The duration of the process will depend on the speed of your meter.
6. Uncheck 'Embed calibration curves in profile'

4. After the process is complete click "don't install profile" and then go to Tools in the menu bar and select 'Create 3D LUT...'

5. Set the following settings for Create 3D LUT tool

1. Select 'eeColor Box (.txt)' for 3D LUT file format
2. For Source profile file, browse to the Rec709.icm file from the 'ref' sub-directory in the program path of ArgyllCMS
3. For Target profile file, browse to the .icm file created from the profile process or press use current
4. For Input Encoding Select 'TV RGB 16-235'
5. Enable 'Apply BT.1886 gamma mapping' and select '2.4 Absolute'
6. Select 'Absolute colorimetic with white point scaling' for Rendering intent
7. Click on 'Create 3D LUT...' button
8. Provide the filename and location to save the 3DLUT (It is convenient to store the file in the TruVue Application directory Data\3dtable\standard nonrgb led\General Media

C. Follow Buzz's guide to upload the LUT to the eeColor processor

1. There are 6 memory slots available on the eeColor Box. Rename the .txt file you created to 3DLUT_X.txt where X is the memory slot number.

D. Verify (proof) your 3DLUT using dispcalGUI

This can be done by measuring the display response directly with the eeColor processor on and your LUT active, or via virtual LUT (soft proofing) method.

1. (eeC On, LUT active) Set the following after clicking Tools->Measurement Report

1. Select 'verify_video_extended.ti1' for Testchart and RGB
2. Check Simulation profile and select 'Rec709.icm'
3. Check 'Use simulation profile as target profile'
4. Check 'Apply BT.1886 gamma mapping' and choose 2.4 + Absolute
5. Select the profile that was created during profile measurements as the Target profile
6. Click measure

After the process is complete, an HTML report will be generated and automatically loaded into your default browser. It should look like this.

2. (eeC Off, Virtual LUT) Set the following after clicking Tools->Measurement Report

*In order to soft proof you will need to create a device link profile with normal full range linking. This can be done by repeating steps B.5 above and selecting Full Range RGB 0-255 for both input and output encoding.

2. Select 'verify_video_extended.ti1' for Testchart and RGB
3. Check Simulation profile and select 'Rec709.icm'
4. Check 'Use simulation profile as target profile'
5. Check 'Apply BT.1886 gamma mapping' and choose 2.4 Absolute'
6. Check 'DeviceLink profile' and select the full range version of your device link profile*
7. Select the profile that was created during profile measurements as the Target profile
8. Click measure

After the process is complete, an HTML report will be generated and automatically loaded into your default browser.

[Command line tools example]

The example command line arguments below follows a similar process to the dispcalGUI flow above but uses a different set of target patches. Use of the -E switch in dispread is needed if your display expects Video levels and they are not provided by your video card.

I. Verify Levels

dispwin -v -d2 -c
[Do this first to load a linear calibration to the video card]

-v verbose
-d2 (which display you are working on in extended desktop, usually 1 is the pc and 2 is the eeColor box connected to your display [identified as SII Repeater])

To check that video card levels are matched to display levels:

dispcal -v -d2 -r -yr -X d3.ccmx
[This will give you a quick report of black and white level, color temperature and estimated gamma]

-d2 use second display found
-r Report on calibrated device
-yr refresh mode for probe
-X use the named correction matrix

II. Generate targets

This depends a bit on how large your color errors are and if they have any unusually steep gradients in one part of the gamut or another, but in general I would recommend starting with around 1000 patches to test the work flow. For complicated gamuts I've found you need no more than 2500 OFPS patches. For the sharp elites you may need as many as 4500, (see madVR thread)

targen -v -d3 -B4 -e4 -G -f1000 -g50 -s20 -c Rec709.icm display
[creates targets in file display.ti1]

-d3 Video RGB
-B4 4 black patches
-e4 4 white patches
-G good optimized OFPS rather than Fast
-g 50 add 50 neutral axis patches
-s 20 add 20 single channel patches
-c precondition using Rec709 color space, replace with previously measured native .icm for more efficient patch distribution

III. Measure targets

dispread -v -d2 -yr -E -X d3.ccmx -Iw display
[measures targets from .ti1 file and writes .ti3 file]

-E Output video levels (needed when video card is full range but display expects limited range) otherwise remove
-Iw White drift compensation (useful for plasma panel fatigue)

IV. Generate icm profile

colprof -v -qh -bl -ax display
[takes results from .ti3 file and creates .icm profile]

-qh high quality
-bl low quality B2A table (since B2A table is not used in cLUT creation)
-ax XYZ Lut algorithm

V. Link source and device profiles and output 3DLUT

collink -v -qh -G -IB -ia -3e -et -Et Rec709.icm display.icm display_link

-qh high quality
-G gamut mapping mode
-IB BT.1886 output offset (absolute, 2.4)
-ia gamut mapping mode intent=absolute colorimetric (generally gives best results when display has been pre-calibrated to D65)
-3e eeColor box output
-et input is 16-235 levels
-Et output is 16-235 levels

*note - if you can't pre-calibrate the display to D65 (and that is your desired post-LUT white point) use the switch -iaw instead of -ia. This will scale the white point luminance to avoid clipping near white. This will also have the side effect of lowering total luminance somewhat depending on how far away your pre-calibrated white point is from it's target.

The 3DLUT file to upload to the eeColor box will be created as display_link.txt

note: The default -3e output when not using the -Et switch will be a 3DLUT for use with 0-255 RGB input/output. This option will also generate 6 files that will be needed to properly treat 255 code inputs.

VI. Upload 3DLUTs to eeColor box

Follow Buzz's guide (2nd post) if you are using YCbCr flow.

For full-range RGB in/out 3DLUTs you will also need to replace the following 6 files in the directory truvue\Data\default:

first1dred.txt
first1dgreen.txt
first1dblue.txt
second1dred.txt
second1dgreen.txt
second1dblue.txt

with the equivalents produced by collink. Also, check the box "Calibration Values" in the Download selection. You can load the calibration values at the same time as the 3DLUTs or by themselves.



If you wish to restore linear calibration values for YCbCr use:



I have attached some OSD images that can be used instead of the stock ones.

 

[buzzard767]

Welcome to the fray.

Your work with Argyll has been a fascinating watch. It won't be long before everyone will be using 3D LUTs.

Quote:
Originally Posted by zoyd  /t/1464890/eecolor-processor-argyllcms#post_23118579


I started out with a simple 2 pt. grayscale + brightness/contrast calibration and put the color control on native (it's worst setting).

The Samsung Color Space does indeed need to be set to Native as it turns off the processing. Custom and Auto play havoc with LUTs. FWIW I have a new F8000 arriving tomorrow so I'll soon be able to verify the same applies to the 2013 models.

Quote:
Originally Posted by zoyd  /t/1464890/eecolor-processor-argyllcms#post_23118579


The 750 patch profile set did an excellent job at reducing errors at all but 6% of the patches where errors actually increase (see plot note). Upping the density to 1500 smoothed out some of those bumps but the 17^3 profile had no statistical advantage over the 1500 OFPS.

Schmastistically speaking, for example, LightSpace 4913 profile LUTs vs Calman 771 point, graph about the same. The difference is in the viewing of content. It is especially evident in paused video where color nuance between LUTs is almost like turning Darblet processing on and off. Moving video is another case though - very difficult to see any difference.

 

[N3W813]

Now we just need madshi to implement a custom pixel shader into MadVR that can take one of these 3dluts!!

 

[zoyd]



Quote:
Originally Posted by buzzard767  /t/1464890/eecolor-processor-argyllcms#post_23118935


Schmastistically speaking, for example, LightSpace 4913 profile LUTs vs Calman 771 point, graph about the same. The difference is in the viewing of content. It is especially evident in paused video where color nuance between LUTs is almost like turning Darblet processing on and off. Moving video is another case though - very difficult to see any difference.

In addition to the statistical data, I have not seen any visual difference between the two cLUTs (1500 OFPS vs. 17^3) in any material moving or not and I spent considerable time looking for it. It's simply a waste of time measuring the additional ~3413 patches, at least for this display. It won't hurt of course but if someone claims it's a must to get the best result I won't believe it. It's also no mystery why this happens, the brute-force grid spacing is just not an efficient way to fill the volume.


@N3W813: madshi already has the shader working, you just need to find someone to translate the eeColor cLUT into whatever format MadVR wants.

 

[buzzard767]



Quote:
Originally Posted by zoyd  /t/1464890/eecolor-processor-argyllcms#post_23119976


In addition to the statistical data, I have not seen any visual difference between the two cLUTs (1500 OFPS vs. 17^3) in any material moving or not and I spent considerable time looking for it. It's simply a waste of time measuring the additional ~3413 patches, at least for this display. It won't hurt of course but if someone claims it's a must to get the best result I won't believe it. It's also no mystery why this happens, the brute-force grid spacing is just not an efficient way to fill the volume.

LightSpace has several tools for cleaning up LUTs.

Quote:
Originally Posted by zoyd  /t/1464890/eecolor-processor-argyllcms#post_23119976


at least for this display.

For sure. I have a DLP (going into the garage tomorrow) that is completely useless.

 

[fahrenheit]

Very cool zoyd.


$600 for the eeColor box is still a sizeable investment for me at this stage, but its certainly getting to the point where I can consider it.

 

[N3W813]



Quote:
Originally Posted by zoyd  /t/1464890/eecolor-processor-argyllcms#post_23119976


@N3W813: madshi already has the shader working, you just need to find someone to translate the eeColor cLUT into whatever format MadVR wants.

In the current version 0.86.1?

 

[zoyd]



Quote:
Originally Posted by N3W813  /t/1464890/eecolor-processor-argyllcms#post_23120300


In the current version 0.86.1?

well the last time I tried it (don't know what version) you could load 3DLUTs. Someone had written a tool (ti3parser I think) to take ArgyllCMS output and write a madVR formatted LUT. When I tried it the luminances were all off but I'm now thinking that was because I hadn't matched my display card peak white to the display peak white.

 

[N3W813]



Quote:
Originally Posted by zoyd  /t/1464890/eecolor-processor-argyllcms#post_23120364


well the last time I tried it (don't know what version) you could load 3DLUTs. Someone had written a tool (ti3parser I think) to take ArgyllCMS output and write a madVR formatted LUT. When I tried it the luminances were all off but I'm now thinking that was because I hadn't matched my display card peak white to the display peak white.

So how do you match a video card peak white to a display peak white using ArgyllCMS (or DispcalGUI)?

 

[zoyd]

sorry, didn't say that right. I think the peak white (or video card range) was not set the same between profiling in dispcalGUI vs. displaying and verifying through MPC-HC+madVR with the cLUT since the chromaticities looked good but all the luminances were off.

 

[gwgill]



Quote:
Originally Posted by N3W813  /t/1464890/eecolor-processor-argyllcms#post_23119920


Now we just need madshi to implement a custom pixel shader into MadVR that can take one of these 3dluts!!

The "3dlut" is (essentially) an ICC device link. I've suggested this approach in the past.

See http://forum.doom9.org/showthread.php?p=1526114#post1526114

 

[Leondavid]

Zoyd I can use with my probe eecolor i1 display pro or need anything else? thanks

 

[DigitalFilm]

I've tried to use ArgyllCMS a number of times, but have never got acceptable results.

I'M not totally sure why, but the need to go via an ICC (ICM) profile has always bothered me.

ICC profiles are notoriously poor at accurate calibration, specifically in shadow, which is why there are 'Black Compensation' features in programs such as Photoshop.


I've also tested by converting known accurate LUTs into ICC profile and compared the results, and the ICC profiles are always 'different'...

Converting ICC profiles into 3D LUTs always produced matching results.


Worse, ICC profiles often produce different results in different systems - applied differently.


Other than differences in LUTs with different cube sizes, and so some variation in the interpolation, I have not seen such issues with real 3D LUTs.


I also struggle to understand the various required software 'pre-sets' in ArgyllCMS. With true independent profiling I can't see a need for such settings, which leads me to think the profiling is not all it suggest it is - but that's just a personal feeling.


However, as an alternative to other calibration systems such as CalMan, it is interesting.

 

[buzzard767]



Quote:
Originally Posted by Leondavid  /t/1464890/eecolor-processor-argyllcms#post_23122114


Zoyd I can use with my probe eecolor i1 display pro or need anything else? thanks

That will work fine but will always be more accurate if profiled from a spectro.


AVS will probably kick my a$$ for this but the eeBoxes are available from me for $650 including shipping, handling, insurance, and tracking. International shipping is $30 more and includes testing of the Box, power supply, and remote control to avoid the possibility of any immediate warranty returns due to inoperative units. [email protected]

 

[Leondavid]



Quote:
Originally Posted by buzzard767  /t/1464890/eecolor-processor-argyllcms#post_23122157


That will work fine but will always be more accurate if profiled from a spectro.


AVS will probably kick my a$$ for this but the eeBoxes are available from me for $650 including shipping, handling, insurance, and tracking. International shipping is $30 more and includes testing of the Box, power supply, and remote control to avoid the possibility of any immediate warranty returns due to inoperative units. [email protected]

thanks buzz! x now try to handle me with what I have, because I'm not a lot of money! I'm out of work and I find it difficult when I have the money, I'll order. thanks for everything

 

[10k]



Quote:
Originally Posted by DigitalFilm  /t/1464890/eecolor-processor-argyllcms#post_23122145


I've tried to use ArgyllCMS a number of times, but have never got acceptable results.

I'M not totally sure why, but the need to go via an ICC (ICM) profile has always bothered me.

ICC profiles are notoriously poor at accurate calibration, specifically in shadow, which is why there are 'Black Compensation' features in programs such as Photoshop.


I've also tested by converting known accurate LUTs into ICC profile and compared the results, and the ICC profiles are always 'different'...

Converting ICC profiles into 3D LUTs always produced matching results.


Worse, ICC profiles often produce different results in different systems - applied differently.


Other than differences in LUTs with different cube sizes, and so some variation in the interpolation, I have not seen such issues with real 3D LUTs.


I also struggle to understand the various required software 'pre-sets' in ArgyllCMS. With true independent profiling I can't see a need for such settings, which leads me to think the profiling is not all it suggest it is - but that's just a personal feeling.


However, as an alternative to other calibration systems such as CalMan, it is interesting.

I tried the above a couple of times over the weekend, only changing the last step to use Ti3parser to create a MadVR compatible 3dlut. The result measured very well (avg dE

 

[gwgill]



Quote:
Originally Posted by DigitalFilm  /t/1464890/eecolor-processor-argyllcms#post_23122145


I've tried to use ArgyllCMS a number of times, but have never got acceptable results.

I'M not totally sure why, but the need to go via an ICC (ICM) profile has always bothered me.

ICC profiles are notoriously poor at accurate calibration, specifically in shadow, which is why there are 'Black Compensation' features in programs such as Photoshop.

I think you need to distinguish between the accuracy of a device profile, as pertaining to how well it captures the behavior of a display, and how accurate an overall color transformation is when two device profiles are used by a CMM (Color Management Module). The latter is subject to the details of how the device profiles are linked. Default CMM's make use of the built in device profile inversion tables (B2A tables), and there are some fundamental limitations to the idea of mixing and matching profiles that use pre-computed tables, leading to the sort of black problems that require workarounds such as BPC (Black Point compensation), and other issues regarding gamut mapping.


But with the right tools you can bypass these weaker parts of using ICC profiles. A smart linker (such as ArgyllCMS collink) can properly and accurately create a device link profile, which is as super-set of typical 3DLuts. In fact, using a profiling approach opens up a number of possibilities that are quite hard do using pure calibration, such as computing multiple different emulations from a single set of display measurements, or properly adjusting for viewing conditions, etc.

Quote:
Worse, ICC profiles often produce different results in different systems - applied differently.

Yes, with device profiles that's a possibility. But not with device links. They are relatively unambiguous, as you've observed about 3DLuts.

Quote:
I also struggle to understand the various required software 'pre-sets' in ArgyllCMS. With true independent profiling I can't see a need for such settings, which leads me to think the profiling is not all it suggest it is - but that's just a personal feeling.

ArgyllCMS is a toolkit. It gets used for a wide variety of color management tasks, hence a bunch of options. One of the things it can do, is to create a device link that allows one device (say the display you've got) to accurately emulate a device you haven't got (say an idealized video display). In the print world, that's called "proofing".

 

[zoyd]



Quote:
Originally Posted by 10k  /t/1464890/eecolor-processor-argyllcms#post_23122198


I tried the above a couple of times over the weekend, only changing the last step to use Ti3parser to create a MadVR compatible 3dlut. The result measured very well (avg dE

 

[zoyd]



Quote:
Originally Posted by DigitalFilm  /t/1464890/eecolor-processor-argyllcms#post_23122145


I've tried to use ArgyllCMS a number of times, but have never got acceptable results.

I'M not totally sure why, but the need to go via an ICC (ICM) profile has always bothered me.

ICC profiles are notoriously poor at accurate calibration, specifically in shadow, which is why there are 'Black Compensation' features in programs such as Photoshop.

As Graeme explained, this work flow is different than the example you cite. I have independently calculated the 3DLUTs via direct transformation of the measurements to RGB space, calculating the look-up values to match Rec709 colors, and then interpolating these with a modified Shepards's algorithm to the 65^3 grid. Testing with these LUTs produced results that matched the performance measured with the Argyll LUTs shown in the first post. So as a baseline Argyll produces very accurate LUTs, but will also have the ability to do look matching and environmental compensation as well.