A comparison of 3DLUT solutions for the eeColor box - AVS Forum
Forum Jump: 
 1Likes
Reply
 
Thread Tools
post #1 of 656 Old 02-14-2014, 12:55 PM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
I recently had an opportunity to compare the performance of currently available 3DLUT generating software for the eeColor box. I believe this is the first systematic and rigorous look at the relative results of these software packages in a typical home theater set-up. I will describe the initial conditions first, then stability checks and assessment of precision for the comparisons, and then discuss the results.

Display initial conditions: Samsung D8000
  • Native gamut selected, larger than target gamut.
  • Internal CMS off
  • 2pt. grayscale set for D65 calibration at 20% and 100% video levels
  • 100% video level = 136 cd/m^2
  • 109% video level = 165 cd/m^2
  • Black level: 0.028 cd/m^2
  • No visual clipping
  • Brightness set via normal methods

Software set-up: Source space used for all transforms was Rec.709. The EOTF (gamma) target was BT.1886

ArgyllCMS [ver. 1.6.3]
Cube parameters
20 single axis, 40 grayscale, 4 white, 8 black, 2393 Optimized Farthest Point Sampling (OFPS)
Dark region emphasis (1.3), Neutral axis emphasis (0.75), preconditioned with previously measured native gamut.

Command lines:
Code:
targen -v -d3 -G -e4 -B8 -g40 -s20 -f2500 -V1.3 -N0.75 -c native.icm argyll_patch_set
dispread -v -d2 -X d3oem_jeti.ccmx -yr  -P0.5,0.5,2.2,1.3 argyll_patch_set
colprof -v -qh -bl -ax argyll_patch_set
collink -v -qh -3e -et -Et -G -IB -ia -w  rec709.icm argyll_patch_set.icm 3DLUT_1.icm

Probe parameters
0.4 second integration, auto measure delay, sync, adaptive integration time
JETI-1211 profiled Display Pro-OEM

LightSpace CMS [ver. 6.5.0.1820]
Cube parameters
17x17x17

Probe parameters
0.5 second integration, LLH enabled
JETI-1211 profiled Display Pro-OEM

CalMAN 5 [ver. 5.2.3.1416]
Cube parameters
17x9 (optimized), 109% included

Note: Limit to 100% is broken

Probe parameters
0.5 second integration, LLM 2 seconds, 5 cd/m^2 trigger, sync
JETI-1211 profiled Display Pro-OEM

Pattern geometry used: 11% windows on a black background.
Pattern generator: PC (intel HD3000/HDMI out - RGB Full, driver ver. 9.17.10.3040).

The most critical thing in a comparison like this is to know how reproducible the measurements are so that you can assess the significance of the measured color difference between one set of results and another separated in time, this will define our precision in being able to discriminate between performance results. The test was designed so that ideally nothing should change except the software used to calculate the 3DLUT transforms from their respective test patch characterizations. In this scenario the stability of the display and probe are the limiting factors. To ensure the best precision, all the profiles were measured in one session over an eight hour period without disturbing the hardware set-up. Additionally a baseline monitor measurement was performed after each profile and at the end of the verification steps. The measurement of offsets between the JETI-1211 and Display Pro was measured once and incorporated in each software package.

Baseline test set (BTS)



This is a 1000 patch quasi-random perceptual space-filling volume chosen so that it does not overlap with the device characterization test sets used by any of the software packages. I will measure the display native response several times bracketing the profile measurements to test system stability. It will also be used when LUTs are loaded and tested for performance against each other. Measurement time for the BTS is ~15 minutes.

Test sequence
  1. BTS - Unity lut, Native
  2. ArgyllCMS profile
  3. BTS - Unity lut, Native
  4. LightSpace profile
  5. BTS - Unity lut, Native
  6. CalMAN profile
  7. BTS - Unity lut, Native
  8. BTS - ArgyllCMS LUT
  9. BTS - LightSpace LUT
  10. BTS - CalMAN LUT
  11. CalMAN SG - All LUTs
  12. HCFR Grayscale, Saturations, Color Checker - All LUTs
  13. BTS - Unity lut, Native

Stability Results

Comparing steps 1,3,5,7,13 I find:

dE2000 color difference
3 to 1 - avg. = 0.15, max = 0.71
5 to 3 - avg. = 0.12, max = 0.67
7 to 5 - avg. = 0.14, max = 0.83
13 to 7 - avg. = 0.21, max = 0.96

So in this hardware configuration there is a dE floor of ~0.16 due to a combination of probe precision/display repeatability or bit noise.

Steps 8-12 were used to assess performance.

Tabular Results:




Histograms:


Expanded view


The ArgyllCMS LUT is significantly more peaked than either CalMAN or LightSpace and is able to drive 93% of the test patches below 1 dE. This is consistent with previous measurements of ArgyllCMS performance. Note that the CalMAN distribution is skewed so that it's peak is about halfway between the ArgyllCMS and LS peaks. This is due to the way it iteratively optimizes dE compared to the other 2 software approaches. This yields about the same average performance as LightSpace but does better at pushing more points below 1 dE (72% vs. 67%) at the expense of a slightly higher maximum error (4.3 vs 3.6, Colorchecker SG 3.8 vs. 2.9)

In all the verification measurements I ran the Argyll generated LUT outperformed both LightSpace and CalMAN in a statistically significant manner. I do not consider the differences between CalMAN and LightSpace statistically significant except in the case of gray scale calibration where CalMAN did a bit better than LS (0.8 vs. 1.3) and the skin tone tests where LS did better than CalMAN in both the SG (0.8 vs. 1.4) and Pantone (1.0 vs. 1.8) sets. But are these differences perceptually significant? In an average sense, no, all three solutions produce an average dE at or below the level of human perception when viewing side by side patches in environmentally controlled test conditions! These differences will be even less noticeable in isolated moving images. I've poured over a lot of source material and cycled through these three LUTs using the eeColor box, and there simply is no way to tell them apart on a still image, let alone a moving one. I would be happy to use any of them long term.

You'll also notice that the internal CMS for this device is not too bad (>95% of errors are < 3 dE). You might ask if this is perceptually noticeable. The answer is yes but it's subtle and more noticeable over time, such that it's easier now to tell when I switch from a LUTed playback back to the display's CMS then it was when I first started using the eeColor box. Is it a $700+ difference? Hard to say as people value things differently so I'll leave that to you. The box has other benefits that I enjoy (aside from this type of testing) in being able to easily switch amongst different transfer functions and/or HD/SD color spaces depending on the material I'm viewing.

Other Items of note:
  • ArgyllCMS achieved a better match to the source color space in a significantly shorter time.
  • LightSpace was the easiest to use and they have fixed their BT.1886 target EOTF (thanks to me smile.gif ).
  • LightSpace is computationally very fast and easy to retarget to other color spaces and EOTFs
  • CalMAN does not seem to have a retargeting capability (must remeasure)
  • CalMAN loads the 3DLUT directly to the eeColor box which is a nice feature
  • CalMAN was the only solution that found and resolved a minor amount of clipping at codes 235 and above
  • LightSpace has no verification metrics other than gray scale dE because this vendor does not "believe" in perceptually based calibration.
  • ArgyllCMS is free

I'm not going to declare a "winner" because as I have stated in the past from a calibration standpoint (meaning minimization of color difference errors to below perceptual limits) all three solutions will get you there. Each has it's own advantages/disadvantages in terms of available features, ease of use, cost, etc. but none has an an accuracy advantage if used and evaluated properly.


The following were summarized in average/max form in the table above.

CalMAN SG:

Native

Internal

CalMAN LUT

LightSpace LUT

Argyll LUT


HCFR gamma:

Native

CalMAN

LightSpace

Argyll

HCFR gray scale:

Native

LightSpace

CalMAN

Argyll



ArgyllCMS .ti3 measured patch files for verification and stability
verify.zip 175k .zip file

.chc files for HCFR verification
3dlut_hcfr.zip 16k .zip file

*Updated* - Additional algorithm comparisons and analysis

This post demonstrates that given the same optimized patch set that LS comes very close to matching Argyll performance.


A factor of at least 2 in patch number reduction compared to standard cube sampling can be achieved while maintaining color correction performance with both programs by utilizing Farthest Point Sampling (FPS), in conjunction with perceptual patch placement and display specific preconditioning profiles.
zoyds_ultimate_pak_3k.zip 27k .zip file

Series of Rec709 optimized sets including grayscale and single channel sweeps, sizes range from 2000 to 10000 patches.
ofps_series_2000_10000.zip 328k .zip file

A sparse sampling test was performed to test algorithm robustness. LS fails to generate a usable LUT under these conditions, indicative of a more simplistic gamut mapping and interpolation approach than what Argyll currently uses.


Some further investigation of the topology of the 3D LUTs created by the three solutions reveals that given the same patch set the LS algorithm produces noisier corrections than Argyll.

Attached Files
File Type: zip verify.zip (175.1 KB, 26 views)
File Type: zip 3dlut_hcfr.zip (16.0 KB, 16 views)
File Type: zip zoyds_ultimate_pak_3k.zip (26.7 KB, 39 views)
File Type: zip ofps_series_2000_10000.zip (328.2 KB, 22 views)
ceru, Chad B, 10k and 8 others like this.

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
Sponsored Links
Advertisement
 
post #2 of 656 Old 02-14-2014, 01:44 PM
Member
 
1forsnow's Avatar
 
Join Date: Mar 2010
Location: Chicago
Posts: 174
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 3 Post(s)
Liked: 38
Fantastic write up Zoyd, it was very informative, thank you!

I see the calman cube parameters was only a 17x9. That is only 785 points. If this is correct, the results came out pretty good considering only a 785 pointer. It would have been nice to see what a calman 4913 point cube could have done, though the Argyll LUT looks impressive.
1forsnow is offline  
post #3 of 656 Old 02-14-2014, 01:47 PM
10k
Advanced Member
 
10k's Avatar
 
Join Date: Sep 2005
Posts: 688
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 38
Thank you for taking the time to do this Zoyd. Would you mind editing your post to include the argyll command line parameters you used? I see from your description it could probably be figured out but I wanted to confirm that I read your parameter selection correctly.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
(Last update 5/20/13)

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
10k is online now  
post #4 of 656 Old 02-14-2014, 02:35 PM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by 1forsnow View Post

Fantastic write up Zoyd, it was very informative, thank you!

I see the calman cube parameters was only a 17x9. That is only 785 points. If this is correct, the results came out pretty good considering only a 785 pointer. It would have been nice to see what a calman 4913 point cube could have done, though the Argyll LUT looks impressive.

Each point in the calMAN sequence is read multiple times to optimize on dE. This has the effect of pulling the whole gamut error surface down but because of the sparser coverage you get the small skewed effect.

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #5 of 656 Old 02-14-2014, 03:59 PM
AVS Special Member
 
sillysally's Avatar
 
Join Date: Jun 2006
Location: Chicago
Posts: 3,718
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 76 Post(s)
Liked: 263
Did you sync the 1211 for LS as you did for CM and ArgyllCMS ?

LightSpace CMS [ver. 6.5.0.1820]
Cube parameters
17x17x17

Probe parameters
0.5 second integration, LLM (???)
JETI-1211 profiled Display Pro-OEM

It looks like ArgvII may use a different cLUT or mapping for eecolor, is it the same as LS and CM use.?

ss

"Don't worry be happy"

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
sillysally is offline  
post #6 of 656 Old 02-14-2014, 04:01 PM
Advanced Member
 
Iron Mike's Avatar
 
Join Date: Oct 2006
Posts: 794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 358
Quote:
Originally Posted by zoyd View Post

To ensure the best precision, all the profiles were measured in one session over an eight hour period without disturbing the hardware set-up.

nice write up... so to ensure I understand this correctly, u say u finished all 3 profiling sessions within ONE eight hour session using an i1D3 ?

Did u have the Avg LLH enabled in LS ?

calibration & profiling solutions: Lightspace, Spaceman, Calman, Argyll, ColorNavigator, basICColor
profiling & calibration workflow tools:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

meter: Klein K-10 A, i1Pro, i1D3
AVS thread:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
Iron Mike is offline  
post #7 of 656 Old 02-14-2014, 04:39 PM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by sillysally View Post

Did you sync the 1211 for LS as you did for CM and ArgyllCMS ?

LightSpace CMS [ver. 6.5.0.1820]
Cube parameters
17x17x17

Probe parameters
0.5 second integration, LLM (???)
JETI-1211 profiled Display Pro-OEM

It looks like ArgvII may use a different cLUT or mapping for eecolor, is it the same as LS and CM use.?

ss

To minimize variables I only used one profile measurement of WRGB values for the JETI and D3 using HCFR (synced) and used those to build profiles for the other software.

I'm not sure what you mean by different cLUTs, each software solution has it's own method of mapping source to device. The format of the LUT itself is the same, 65^3 but the 65th point (1.0) is not adjustable.
Quote:
Originally Posted by Iron Mike View Post

nice write up... so to ensure I understand this correctly, u say u finished all 3 profiling sessions within ONE eight hour session using an i1D3 ?

Did u have the Avg LLH enabled in LS ?

Yes, LLH enabled, I tested the measurement time and stability for a 10% white patch (~7 seconds) and a 5% white patch (~15 seconds). This is very similar to the internal adaptive mode that ArgyllCMS uses and both were stable in x,y to the 4th decimal place. I don't know where the trigger is but probably in the 5-10 cd/m^2 range, above that it ran at 0.5 second integration times. As stated in the first post, LS took the longest at 120 138 minutes. And the reason it takes more than twice as long to obtain similar results is that regular grids are an inefficient way to fill the volume you are trying to map. It does however allow for faster interpolation. Once measured the LUT creation takes a few seconds to compute in LS. With ArgllCMS, the profile creation and linking used in the first post takes 10-15 minutes.

Here is the LS 3D and collapsed to 1D LUT (prior to video level scaling)



-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #8 of 656 Old 02-14-2014, 04:49 PM
Advanced Member
 
Iron Mike's Avatar
 
Join Date: Oct 2006
Posts: 794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 358
Quote:
Originally Posted by zoyd View Post

Yes, LLH enabled... As stated in the first post, LS took the longest at 120 minutes.

so u ran a full LS 17^3 with 4913 patches with Avg LLH enabled using an i1D3 in 2 hours ?

the last time I used an i1D3 in LS, it took quite a bit longer for a full 17^3 mainly because the Avg LLH adds on a lot of time, but it is really worth it as it greatly improves accuracy in low light reads...

so they must have improved the i1D3 handling in low light dramatically....

calibration & profiling solutions: Lightspace, Spaceman, Calman, Argyll, ColorNavigator, basICColor
profiling & calibration workflow tools:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

meter: Klein K-10 A, i1Pro, i1D3
AVS thread:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
Iron Mike is offline  
post #9 of 656 Old 02-14-2014, 05:06 PM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by Iron Mike View Post

so u ran a full LS 17^3 with 4913 patches with Avg LLH enabled using an i1D3 in 2 hours ?

the last time I used an i1D3 in LS, it took quite a bit longer for a full 17^3 mainly because the Avg LLH adds on a lot of time, but it is really worth it as it greatly improves accuracy in low light reads...

so they must have improved the i1D3 handling in low light dramatically....

Sorry, I just check my notes, it was 138 minutes.

I don't know how it performed before but it is comparable to how ArgyllCMS handles it in terms of stability for the patches I tested and average read time for large patch sets.

ArgyllCMS: Average 1.44 seconds per read includes 250 ms patch change delay
LS: Average 1.7 seconds per read

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #10 of 656 Old 02-14-2014, 07:47 PM
Advanced Member
 
spacediver's Avatar
 
Join Date: May 2013
Location: Toronto
Posts: 840
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 71 Post(s)
Liked: 70
Quote:
Originally Posted by zoyd View Post

.


Test sequence
  1. BTS - Unity lut, Native
  2. ArgyllCMS profile
  3. BTS - Unity lut, Native
  4. LightSpace profile
  5. BTS - Unity lut, Native
  6. CalMAN profile
  7. BTS - Unity lut, Native
  8. BTS - ArgyllCMS LUT
  9. BTS - LightSpace LUT
  10. BTS - CalMAN LUT
  11. CalMAN SG - All LUTs
  12. HCFR Grayscale, Saturations, Color Checker - All LUTs
  13. BTS - LightSpace LUT

Stability Results

Comparing steps 1,3,5,7,9, 13 I find:

dE2000 color difference
3 to 1 - avg. = 0.15, max = 0.71
5 to 3 - avg. = 0.12, max = 0.67
7 to 5 - avg. = 0.14, max = 0.83
13 to 9 - avg. = 0.21, max = 0.96

Should step 13 read "BTS - Unity lut, Native"?

Excellent work Zoyd - it's stuff like this that motivates me to learn more smile.gif
spacediver is offline  
post #11 of 656 Old 02-15-2014, 01:58 AM
Advanced Member
 
realzven's Avatar
 
Join Date: Mar 2010
Posts: 519
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 84
Thank's zoyd for the great work smile.gif
realzven is offline  
post #12 of 656 Old 02-15-2014, 02:39 AM
Advanced Member
 
Light Illusion's Avatar
 
Join Date: Aug 2010
Posts: 531
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 216
Why not use the same patch sequence with LightSpace as used with Argyll?
LightSpace is not limited to a grid based sequence, although that is the in-built standard.
That would change the time taken significantly for LS.

With a non-plasma display we'd also use the Hybrid profiling mode by default.
And a Quick Profile test would have been interesting to see...

Actually, what you could do is take the sequence as read by Argyll and format it for LightSpace, and vice versa.
That would be an interesting check.

But, for the best results a 21^3 LightSpace profile would be our suggestion for profiling, as Silly Sally for one would agree biggrin.gif
That does obviously take longer, but the results are definitely worth the time cool.gif

Also, as LS uses a different colour level when syncing one probe to another you may have introduced an error by not doing that directly within LightSpace?
Worth checking.

And I can't see post calibration min/max (black/white) levels - can you point me at them?

Steve

Steve Shaw
LIGHT ILLUSION

Light Illusion is offline  
post #13 of 656 Old 02-15-2014, 02:42 AM
Advanced Member
 
Iron Mike's Avatar
 
Join Date: Oct 2006
Posts: 794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 358
Zoyd,

did u happen to do a LS verification profile of the LS LUT ? Like a typical 141 point LS QP ?

If so, could you post the resulting profile here in this thread ?

This is a nice write up and comparison, not a picture perfect comparison as u ran 3 different color patch sets in 3 different profiles.... that could easily explain the minor discrepancy in dE...

I know it's not possible with CM, but did u happen to run the same color patch set from the Argyll profile within LS ? LS added the ability to import custom patch sequences a while ago...


- M

calibration & profiling solutions: Lightspace, Spaceman, Calman, Argyll, ColorNavigator, basICColor
profiling & calibration workflow tools:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

meter: Klein K-10 A, i1Pro, i1D3
AVS thread:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
Iron Mike is offline  
post #14 of 656 Old 02-15-2014, 02:49 AM
AVS Special Member
 
ConnecTEDDD's Avatar
 
Join Date: Sep 2010
Location: Athens, Greece
Posts: 2,321
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 97 Post(s)
Liked: 476
Zoyd,

Please upload a Zipped file with the 3 3D-LUT TXT files you used.

Also export from LightSpace the measurement file you took, also export the 2 saved meter profiling tables also.

BTW did you used any delay before each meter read for each software?

Upload also the CalMAN Saved Session Files with these ColorChecker measurements.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

S/W: LightSpace CMS, SpaceMan ICC, SpaceMatch DCM, CalMAN 5, CalMAN RGB, ChromaPure, CalPC, ControlCAL
Meters: JETI Specbos 1211, Klein K-10A, i1PRO2, i1PRO, SpectraCAL C6, i1D3, C5
ConnecTEDDD is online now  
post #15 of 656 Old 02-15-2014, 05:03 AM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by spacediver View Post

Should step 13 read "BTS - Unity lut, Native"?

Excellent work Zoyd - it's stuff like this that motivates me to learn more smile.gif

Thanks, I enjoy exploring the outer limits of calibration. Yes, that step is mislabled, I'll update the post. The repeat LS measurement was actually done the next day as I was curious how repeatable the result was after turning the display off, warm-up of equipment, etc.

Quote:
Originally Posted by ConnecTEDDD View Post

Zoyd,

Please upload a Zipped file with the 3 3D-LUT TXT files you used.

Also export from LightSpace the measurement file you took, also export the 2 saved meter profiling tables also.

BTW did you used any delay before each meter read for each software?

Upload also the CalMAN Saved Session Files with these ColorChecker measurements.

I did put an extra delay of 0.2 seconds on the LS reads just to be cautious because I was unsure what the nominal delay was, so that added about 15 minutes. You can find the files you requested here.

I know that LS can import an arbitrary patch set but the reason I chose the 17^3 grid for LS was to run all the programs in a typical default mode as a baseline. Also when using the D3 these modes are roughly equivalent in measurement time. If I were to choose to try and improve the LS results to match Argyll I would not run a 21^3 grid because I don't like this method and don't want to wait 5 hours for the D3 to do it, but importing the Argyll patch set would be interesting.

@steve


-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #16 of 656 Old 02-15-2014, 05:29 AM
Advanced Member
 
Iron Mike's Avatar
 
Join Date: Oct 2006
Posts: 794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 358
thanks for the files... did u happen to run a verification / validation profile in LS after you uploaded the LS LUT to the eeColor box ?

calibration & profiling solutions: Lightspace, Spaceman, Calman, Argyll, ColorNavigator, basICColor
profiling & calibration workflow tools:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

meter: Klein K-10 A, i1Pro, i1D3
AVS thread:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
Iron Mike is offline  
post #17 of 656 Old 02-15-2014, 06:13 AM
Member
 
DrFaxe's Avatar
 
Join Date: Nov 2012
Posts: 192
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 76
Quote:
Originally Posted by zoyd View Post

I know that LS can import an arbitrary patch set but the reason I chose the 17^3 grid for LS was to run all the programs in a typical default mode as a baseline. Also when using the D3 these modes are roughly equivalent in measurement time. If I were to choose to try and improve the LS results to match Argyll I would not run a 21^3 grid because I don't like this method and don't want to wait 5 hours for the D3 to do it, but importing the Argyll patch set would be interesting.

Zoyd, thanks a lot for this comparison! You provided some very interesting data, as always. smile.gif

If I understand right, the patch sequence in Argyll is based on the cube parameters You defined and non grid based respectively random.
As You said You don't like a grid based sequence, what advantage do You see in the Argyll sequence over a grid based sequence?

Do You have any plans to run a new profile with the Argyll patch set in LS? If so I would be interested in how to export the patch sequence from Argyll into LS.

I use
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
DrFaxe is offline  
post #18 of 656 Old 02-15-2014, 07:04 AM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by Iron Mike View Post

thanks for the files... did u happen to run a verification / validation profile in LS after you uploaded the LS LUT to the eeColor box ?

I did not at the time because there does not appear to be any quantifiable output to use for verification or validation purposes. That is one thing lacking in LS and why the comparisons used tools that do provide this data directly. The other thing I'd like to see added to LS (unless I missed it) is the ability to run profiles at video levels since all my eeColor LUTs use video levels and I need to load a separate full range one just to run LS through the final LUT. I also noticed that exporting a full range eeColor LUT does not include the per channel input curves needed to take care of the 65th node point which is anchored at 1.0

Here is a recent 141 point qp using the LS built eeColor LUT



qpwlut.zip 4k .zip file [
Attached Files
File Type: zip qpwlut.zip (3.8 KB, 13 views)

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #19 of 656 Old 02-15-2014, 07:34 AM
Advanced Member
 
Light Illusion's Avatar
 
Join Date: Aug 2010
Posts: 531
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 216
I was after the black/white values for all post calibrations.

My experience has been some very unexpected changes in black especially post calibration with other systems.

Cheers, Steve
ConnecTEDDD likes this.

Steve Shaw
LIGHT ILLUSION

Light Illusion is offline  
post #20 of 656 Old 02-15-2014, 07:38 AM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by DrFaxe View Post

Zoyd, thanks a lot for this comparison! You provided some very interesting data, as always. smile.gif

If I understand right, the patch sequence in Argyll is based on the cube parameters You defined and non grid based respectively random.
As You said You don't like a grid based sequence, what advantage do You see in the Argyll sequence over a grid based sequence?

Do You have any plans to run a new profile with the Argyll patch set in LS? If so I would be interested in how to export the patch sequence from Argyll into LS.

The stability and verification test set I used was random (so as not to overlap any of the characterization test sets) but the profiling set for Argyll is not random, it's based on OFPS algorithms.

I'm running the Argyll patch set in LS right now, will post results later. The argyll patches are defined in a text file as normalized RGB triplets, you just have to convert that to PC levels in a comma separated list to import into LS.

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #21 of 656 Old 02-15-2014, 07:46 AM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by Light Illusion View Post

I was after the black/white values for all post calibrations.

My experience has been some very unexpected changes in black especially post calibration with other systems.

Cheers, Steve

yes, black level shift has been an issue for Argyll in the past (don't know about CalMAN) but Graeme has dealt with that. CalMAN did lower the white level a bit based on luminance measurements of the cyan channel but that did not seem to be necessary based on the other results.

post cal as measured by HCFR [cd/m^2]:

LS: 0.028 black, 133 white
Argyll: 0.028 black, 136.2 white
CalMAN: 0.028 black, 127.1 white (note, CalMAN starting at 109% picked up some clipping and fixed it)

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #22 of 656 Old 02-15-2014, 07:54 AM
Advanced Member
 
Light Illusion's Avatar
 
Join Date: Aug 2010
Posts: 531
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 216
With the eeColor used in a video workflow the 65th point is not relevant.
For data workflows it could be an issue, but if the colour temperature of the display is pre-set to be accurate there is little problem.
And per-channel input curve will not effect the fixed point.

Using closed-loop mode, there is no need for any additional delay.
(Unless the display screen has 'memory' as with some plasmas)

A for 'verification' with LS you simply run a profile with the LUT 'active' (either within LightSpace, or in the LUT box) and compare that to the desired target.
The Cube display shows the accuracy - the closer to a perfect cube you have the better the accuracy.
We will be adding some unique tools for accuracy checking in a future upgrade as delta-e is not really viable as it fails to understand the true volumetric accuracy of a given calibration.

A good and very accurate verification of the real final accuracy between all the calibrations would be to run a large cube profile and compare the final results.
Now that would be interesting!

As for video level output, use an 'Active LUT' to re-scale as needed.
But, that questions the image path you are using, as the path should take care of levels automatically?
If you require a re-scale of the patch levels I would question the calibration path used...

And finally, the RGB separation graph shows the display has clipping issues in the peak values.
That is probably a display setup inaccuracy.

Steve
ConnecTEDDD likes this.

Steve Shaw
LIGHT ILLUSION

Light Illusion is offline  
post #23 of 656 Old 02-15-2014, 07:56 AM
Member
 
DrFaxe's Avatar
 
Join Date: Nov 2012
Posts: 192
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 76
Quote:
Originally Posted by zoyd View Post

The argyll patches are defined in a text file as normalized RGB triplets, you just have to convert that to PC levels in a comma separated list to import into LS.

Thanks!

Could You please provide a guide for that and upload the file You imported in LS. I have never used Argyll before and to be honest it looks a bit complicated to me with all the command line switches.
JimP likes this.

I use
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
DrFaxe is offline  
post #24 of 656 Old 02-15-2014, 08:15 AM
Advanced Member
 
CalWldLif's Avatar
 
Join Date: Feb 2011
Location: South Bay'Los Angeles California.
Posts: 506
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 64 Post(s)
Liked: 29
nice.
thanks for all the info and work.

Loving D65
CalWldLif is online now  
post #25 of 656 Old 02-15-2014, 09:39 AM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Some quick results using the Argyll Patch set, results are now almost identical to ArgyllCMS

Measure time: 88 minutes.
87% dE < 1

best 90% stats
avg: 0.55, Max: 1.2



It also cleaned up the lower portion of the gray scale and better matched the desired EOTF (compare to OP)


Vitalii427 likes this.

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #26 of 656 Old 02-15-2014, 09:53 AM
Advanced Member
 
Light Illusion's Avatar
 
Join Date: Aug 2010
Posts: 531
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 216
Kool - that's what we expected cool.gif

It is interesting how a different patch sequence can produce such different results.
Would be worth doubling the patch sequence for better granularity.

But - we would still prefer to use a cubic verification check, as delta-e really is a limited method when it comes to checking final accuracy.
We would expect the use of a non-regular patch sequence to cause inaccuracies within the volumetric space, which the delta-e checks are not picking up.

This is why the 21^3 profile approach produces such noticeable improvements in final calibration.
It has the granularity of the non-grid based patch set, but with far more volumetric data measurements.

It is an example where taking more time really will produce better results.

biggrin.gif

Steve
ConnecTEDDD likes this.

Steve Shaw
LIGHT ILLUSION

Light Illusion is offline  
post #27 of 656 Old 02-15-2014, 01:26 PM - Thread Starter
AVS Special Member
 
zoyd's Avatar
 
Join Date: Sep 2006
Location: Planet Dog
Posts: 4,565
Mentioned: 25 Post(s)
Tagged: 0 Thread(s)
Quoted: 145 Post(s)
Liked: 329
Quote:
Originally Posted by DrFaxe View Post

Thanks!

Could You please provide a guide for that and upload the file You imported in LS. I have never used Argyll before and to be honest it looks a bit complicated to me with all the command line switches.

I've attached the optimized patch set that I use, however it's just a starting point. The degree of granularity (aka total number of patches) that you will need depends on the degree of non-linearity of your display, you may need more or you may need less. Only test and evaluation can answer that question. The smarter patch distribution will reduce your measurement time by roughly a factor of 2 compared to a regular grid approach.

Unzip and import using the csv tab in LS display characterization set-up
argyll_patch_set.zip 16k .zip file
Attached Files
File Type: zip argyll_patch_set.zip (15.7 KB, 52 views)
sillysally and Vitalii427 like this.

-----

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
zoyd is online now  
post #28 of 656 Old 02-15-2014, 02:19 PM
 
SCProCalibrator's Avatar
 
Join Date: Feb 2014
Posts: 78
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 15

Brilliant comparison, Zoyd! 

 

I, myself, am a proud user of ArgyllCMS when it comes to 3D LUT calibration. Using a custom 2677 testchart, I produced a LUT with a maximum dE 1.5 (1 ColorChecker instance), followed by a dE 1.3 (1 ColorChecker instance) and the rest were all under dE 1.0. It took me about an hour to do calibration and profiling, but only one single attempt.

 

ArgyllCMS takes more grayscale measurements than CalMAN and LightSpace, which resulted in a more accurate grayscale and gamma. ArgyllCMS intelligently takes measurements using many passes - 8pts, 48pts, 84pts, 96pts or something close to improve accuracy and do less guess-work/interpolation.

 

One thing that you haven't compared or it may be hard to compare is whether there is any grayscale banding, but it is not likely to happen with 3D LUTs.

SCProCalibrator is offline  
post #29 of 656 Old 02-15-2014, 03:13 PM
Advanced Member
 
Iron Mike's Avatar
 
Join Date: Oct 2006
Posts: 794
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 23 Post(s)
Liked: 358
Quote:
Originally Posted by SCProCalibrator View Post

ArgyllCMS takes more grayscale measurements than CalMAN and LightSpace, which resulted in a more accurate grayscale and gamma.

not true, I do 100 point custom Greyscale with Lightspace... Anything goes, u can use any custom patch sequence...
ConnecTEDDD likes this.

calibration & profiling solutions: Lightspace, Spaceman, Calman, Argyll, ColorNavigator, basICColor
profiling & calibration workflow tools:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

meter: Klein K-10 A, i1Pro, i1D3
AVS thread:
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
Iron Mike is offline  
post #30 of 656 Old 02-15-2014, 07:56 PM
AVS Special Member
 
TomHuffman's Avatar
 
Join Date: Jan 2003
Location: Springfield, MO
Posts: 6,419
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 52 Post(s)
Liked: 216
Quote:
Originally Posted by zoyd View Post

I've attached the optimized patch set that I use, however it's just a starting point. The degree of granularity (aka total number of patches) that you will need depends on the degree of non-linearity of your display, you may need more or you may need less. Only test and evaluation can answer that question. The smarter patch distribution will reduce your measurement time by roughly a factor of 2 compared to a regular grid approach.

Unzip and import using the csv tab in LS display characterization set-up
argyll_patch_set.zip 16k .zip file
I'm curious, what is the advantage--other than saving time by using fewer colors--to using a custom set of test patterns, such as this as opposed to a regular matrix? Also, once you decide to NOT use a matrix by what criteria are the colors defined? Maybe I missed it, but these colors seem random to me. How is this specific sequence "optimized"? Optimized relative to what criteria?

Tom Huffman
ChromaPure Software/AccuPel Video Signal Generators
ISF/THX Calibrations
Springfield, MO

TomHuffman is offline  
Reply Display Calibration

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off