AVS Forum banner
1 - 20 of 26 Posts

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #1 · (Edited)
test.full.band - open source video testing/calibration patterns

This is a thread for community discussion of open source pattern set from the test.full.band project (look also for the sources on test-full-band/tfb-video project at GitHub).

This is a young project and doesn't yet contain all the common useful patterns, and initially was focused on providing something unique, not yet existing elsewhere. But this is changing with time and transforms into complete set of video patterns for the purposes of testing video reproduction equipment such as TVs and media players and for calibrating them.

The latest released version of the patterns can be always found at:
Releases · test-full-band/tfb-video

Features

Modern picture formats


  • H.264 (AVC) and H.265 (HEVC)
  • FullHD (1080p) and UltraHD 4K (2160p)
  • BT.709 and BT.2020 color
  • 4:2:0 YCbCr
  • Standard dynamic range and HDR (HDR10, HLG10)

Setup & Tests

  • PLUGE pattern for precisely setting black level
  • BT.2111 color bar test pattern for HDR10 and HLG10
  • Checker board test patterns of different amplitudes
  • Quantization uniformity / local non-linearity detection:
    • 2D Quantization grid
    • 3D Quantization cross pattern (dynamic sweeps)

Calibration patterns

  • Full screen fills and 5%, 10%, 20%, 50% window calibration patches
  • Grayscale patterns in 5% steps
    • additional 'Whiter than White' patterns
    • additional precision near-black patters (1% steps)
  • Miscellaneous 10 step Saturation Sweep patterns (CalMAN compatible)

Special for LG OLED

  • HDR10 10% window calibration patterns for LG OLED 2016 and 2017

We are welcoming contributions. If you can do something to the project - please join. We need help with documentation, contributions of code for new patterns (it is mostly Java), feedback about features and bugs, and of course discussions.

Currently 'We' consists of only me, but I hope it is not for long and together we can do something big. May the force be with us!
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #2 · (Edited)
The site dedicated for the project documentation is up and running at test.full.band. As well as the code it is hosted on GitHub (GitHub Pages) and is open for contributions. There is not so much useful yet except just one page documenting Quants2D pattern and providing reference examples of good and bad renderings of the pattern.

Besides that while there is silence here for a few months there are a lot of developments and new release going to happen soon extending both to support of different formats/codecs (AVC/HEVC, Full/Ultra HD, BT.709/BT.2020, HDR10/HLG, etc.) and some new/unique patterns as well, besides a ton of fixes and pleasing visual improvements on the existing ones.
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #3 · (Edited)
I would like to announce a new release of the project (version 0.4). A lot were added (already totaling to 3825 files!), new formats, new patterns, enormous amount of calibration patches. Please download.

So, to highlight the most important. On the HDR side support for HLG (Hybrid-Log Gamma) is added. As well as good old SDR and FullHD is supported too (and there are plans to go to ICtCp, ideas how to get into DV and other dynamic meta formats). For HDR/HLG there is BT.2111 color bars test pattern (simple, but as far as I know not yet available in any test file set).

There is now PLUGE test pattern, so the most important task of setting black level is covered by the test set for all supported formats.

Calibration patches now go in 5%, 10%, 20%, 50% window and fullscreen sizes. Among them 5% step grayscales, 1% near-black steps plus whiter-than-white (105% and 109%) steps. Also 10% step saturation sweeps for primaries and secondaries (supposedly CalMAN compatible, but I've no chance to verify, so please let me know if they are correct distributions or not).

There is also checkerboard patterns (and it's 4K versions reveal a serious bug on my Nvidia SHIELD TV)!

So, please download, test, give feedback (I really need one, at least let me know this work is needed for someone!), ideas, needs, and better yet - contribute. I am trying to update documentation but I really need help from someone with native English to review, correct, make it pleasant for readers, and of course writing some articles from scratch for the project is always welcome. ;)
 

·
Registered
Joined
·
1,110 Posts
I've tried downloading the zip file several times and it always stops after a hour or so (about half-way done) with a "failed-network error" message. (I only have 1M down where I live!)

I guess I keep trying and hope that it eventually works!
 

·
Registered
Joined
·
2,009 Posts
I missed this topic before but I'll download and give them a look. But I'm without a meter (yet, just a money thing holding me back currently), but I'll check these out. I've been thinking about doing some perceptual comparisons of a few displays I have out of pure interest, comparing the displays and their different settings. And I'm always interested in different sources of near black patterns for my OLED. I'll also see if I can help with the grammar, diction, etc. on the documentation :)
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #6 ·
I've tried downloading the zip file several times and it always stops after a hour or so (about half-way done) with a "failed-network error" message. (I only have 1M down where I live!)

I guess I keep trying and hope that it eventually works!
Yes, the set size is 'exploded' in the last version due to calibration patterns. I am planning to separate them from other test patterns to reduce download size (and also for those who doesn't have a meter and so doesn't need most of them).

But in meantime you can use some download manager that can recover broken download without starting it from the beginning... On a slow internet connection it could be useful for other downloads as well.
 

·
Registered
Joined
·
1,110 Posts
Yes, the set size is 'exploded' in the last version due to calibration patterns. I am planning to separate them from other test patterns to reduce download size (and also for those who doesn't have a meter and so doesn't need most of them).

But in meantime you can use some download manager that can recover broken download without starting it from the beginning... On a slow internet connection it could be useful for other downloads as well.

That would be very helpful. I'm just looking for a good set at the basic consumer level.

When I try "resume" on the download manager, I just get a "Failed-Forbidden" message for some reason...
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #8 ·
When I try "resume" on the download manager, I just get a "Failed-Forbidden" message for some reason...
You may try uGet, I haven't tried it myself (because never had problems with interrupted downloads) but people report that it works well with GitHub releases.
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #10 ·
It's been a long time this thread is silent. But it isn't dead yet. A very small pre-announce.
BT2111.U4K_DVp5.mp4
You might guess from the file name of what is inside. What does it mean... a whole set of patterns for testing Dolby Vision capable gadgets is very close!
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #11 · (Edited)
kryptonite5 requested on a project issue tracker a Black PLUGE pattern and as it is already made I've decided to publish it before the whole DV set is ready.

So it is here: BlackPLUGE.U4K_DVp5.mp4

Please use, test, and report if there is (or isn't) any problem with it.

Also he requested a White PLUGE (or White Clipping) patterns. And in light of Dolby Vision with its dynamic metadata and predictable tone-mapping it is an interesting topic for discussion - what those patterns should be. This is a copy-paste of my answer from the GitHub:

Concerning the white clipping - it is an interesting problem on what those patterns should be. I definitely have those patterns in mind (for all the SDR/HDR/DV sets) but dynamic metadata adds a new dimension to them. What actually would you want to test with them in DV? Is it that DV correctly maps all light range of the scene to the TV capabilities given the metadata correctly presented? Is it finding the point where it actually clips given the metadata is at the TV capabilities or below them? For HDR it is also an interesting problem (that would logically only depend on Max Content Light Level meta, but in practice sometimes depend on Mastering Display Max Luminance meta)...

For SDR it is simple - you are just finding at what code point it clips. For HDR/DV I would like to be discussed what patterns do we really need (re. Mastering Monitor meta, HDR10 Content (global) meta and DV Content (per frame) meta).
I encourage anyone to express their thinking on this topic.
 

·
Registered
Joined
·
11,120 Posts
Also he requested a White PLUGE (or White Clipping) patterns. And in light of Dolby Vision with its dynamic metadata and predictable tone-mapping it is an interesting topic for discussion - what those patterns should be. This is a copy-paste of my answer from the GitHub:
Even without getting into the complexities of dynamic metadata, I've been wondering about white clipping patterns for HDR10.

It's simple for SDR - every display should properly display up to 100% white and maybe beyond, with whatever luminance levels the TV is capable of, or what the calibrator is targetting.

For HDR using the ST.2084 PQ curve, every single display will clip at a different level - from ~100 nits for projectors, up to the theoretical 10,000 nits. At what level should the white clipping patterns clip? And, assuming the display applies some tone mapping to avoid hard clippling, how visible would those white flashing patterns be?
 

·
Registered
Joined
·
10,210 Posts
kryptonite5 requested on a project issue tracker a Black PLUGE pattern and as it is already made I've decided to publish it before the whole DV set is ready.

So it is here: BlackPLUGE.U4K_DVp5.mp4

Please use, test, and report if there is (or isn't) any problem with it.
Thank-you for sharing this, @IgorZep! You asked for feedback. I can only see two stripes on the right hand side of this pattern, even if I turn Brightness up to 100 :(. I find it difficult to use with only two stripes, as there is less "precision".

I would find it easier to adjust if it contained the same boxes in the left top/bottom corners, and the extra stripes on the left as in the SDR version of the pattern. I think the same thing about the HDR10 and HLG10 pluge patterns.
 

·
Registered
Joined
·
1,032 Posts
Discussion Starter · #14 ·
I can only see two stripes on the right hand side of this pattern, even if I turn Brightness up to 100 :(. I find it difficult to use with only two stripes, as there is less "precision".

I would find it easier to adjust if it contained the same boxes in the left top/bottom corners, and the extra stripes on the left as in the SDR version of the pattern. I think the same thing about the HDR10 and HLG10 pluge patterns.
Thank you for the feedback. The right side boxes and additional stripes (representing blacker-than-black) are not present in DV file as it is coded full-range as specified by DV profile 5. But the left side boxes are there in the content, so it is interesting why your setup is loosing precision there.

I discussed the problem with a colleague and we came to conclusion that it might be due to using 10K nit max in metadata (as copied from previous BT.2111 pattern), so the TV is trying to compress the pattern and loosing the precision due to poor math or lack of dithering during re-quantisation. Definitely this pattern should not have such a high max brightness in the metadata to avoid any compression / tone mapping due to limited TV max brightness.

I will prepare new file soon with more appropriate DV metadata.
 

·
Registered
Joined
·
10,210 Posts
Thank you for the feedback. The right side boxes and additional stripes (representing blacker-than-black) are not present in DV file as it is coded full-range as specified by DV profile 5.
Right! I understand.

But the left side boxes are there in the content, so it is interesting why your setup is losing precision there.
I did actually see them later after I posted, when it got dark outside :)

I discussed the problem with a colleague and we came to conclusion that it might be due to using 10K nit max in metadata (as copied from previous BT.2111 pattern), so the TV is trying to compress the pattern and loosing the precision due to poor math or lack of dithering during re-quantisation. Definitely this pattern should not have such a high max brightness in the metadata to avoid any compression / tone mapping due to limited TV max brightness.

I will prepare new file soon with more appropriate DV metadata.
Wonderful idea! In fact if that is the cause would it be possible to have both 1,000 and 4,000 nit versions of the DV pluge and BT.2111 pattern, and maybe 1,000 and 4,000 nit versions of the HDR10 pluge and BT.2111 pattern?
 

·
Registered
Joined
·
24 Posts
I observe something similar to mrtickleuk.

On my C8 panel (dark room, TV warmed up for hours)

On the HDR pluge pattern, at brightness of 50, the 2 vertical bars and all 3 boxes on the left are all visible.
if I reduce brightness to 49, the 3 boxes on the left, and the first of the 2 visible vertical bars disappear, leaving only the rightmost visible vertical bar visible.

On the DV pluge pattern, at brightness of 50, I can only see the 2 vertical bars. I don't see the 3 boxes on the left at all. Increasing brightness makes no change.
If I reduce brightness to 49, both vertical bars completely disappear.

I presumed (maybe incorrectly) that this could be LG's way to trying to fix elevated blacks in DV.
 

·
Registered
Joined
·
10,486 Posts
I observe something similar to mrtickleuk.

On my C8 panel (dark room, TV warmed up for hours)

On the HDR pluge pattern, at brightness of 50, the 2 vertical bars and all 3 boxes on the left are all visible.
if I reduce brightness to 49, the 3 boxes on the left, and the first of the 2 visible vertical bars disappear, leaving only the rightmost visible vertical bar visible.

On the DV pluge pattern, at brightness of 50, I can only see the 2 vertical bars. I don't see the 3 boxes on the left at all. Increasing brightness makes no change.
If I reduce brightness to 49, both vertical bars completely disappear.

I presumed (maybe incorrectly) that this could be LG's way to trying to fix elevated blacks in DV.
Hi, the DV or HDR10 patterns for contrast/brightness are only for evaluation, to check how different metadata affect your display tone mapping maths. You will not use them for adjustments.

OLED Light/Contrast/Brightness settings of your LG should be left to default positions, any adjustment of these controls will increase your errors, because based to these default positions the internal TV tone mapping is working.

While for SDR calibration, you can adjust these controls to the values it will be better for your setup.
 

·
Registered
Joined
·
10,210 Posts
Hi, the DV or HDR10 patterns for contrast/brightness are only for evaluation, to check how different metadata affect your display tone mapping maths. You will not use them for adjustments.

OLED Light/Contrast/Brightness settings of your LG should be left to default positions, any adjustment of these controls will increase your errors, because based to these default positions the internal TV tone mapping is working.
But since my panel has some black crush: in Dolby Vision mode I would far rather fix that by adjusting the Brightness than lose all of my shadow detail.

I am being forced to choose between
  • slight errors in tone mapping (not very visible)
  • massive errors with crushed shadow (VERY visible)

I will not change Contrast.
 

·
Registered
Joined
·
11,120 Posts
Hi, the DV or HDR10 patterns for contrast/brightness are only for evaluation, to check how different metadata affect your display tone mapping maths. You will not use them for adjustments.

OLED Light/Contrast/Brightness settings of your LG should be left to default positions, any adjustment of these controls will increase your errors, because based to these default positions the internal TV tone mapping is working.
Hi Ted,
If the 5-60% readings are consistently higher than the PQ curve, would you still not recommend lowering Contrast to better match the PQ curve? (A general question, not specific to OLED).
 

·
Registered
Joined
·
10,486 Posts
Hi Ted,
If the 5-60% readings are consistently higher than the PQ curve, would you still not recommend lowering Contrast to better match the PQ curve? (A general question, not specific to OLED).
If you lower the contrast, you will compress the data, but with multiple measurements with different metadata, if this will help and not introduce any problem to the gamut also...then you can try it.

But very rare a contrast slider can affect tracking of PQ or gamma etc. (almost nowhere with a display)
 
1 - 20 of 26 Posts
Top