AVS Forum banner

1 - 20 of 650 Posts

·
Registered
Joined
·
970 Posts
Discussion Starter #1 (Edited)
Finally I could satisfy my (and I hope your) curiosity to know which test pattern generator (or TPG) in the low price range is really accurate. To do this I bought a medium-high end TPG: DVDO AvLab TPG, which has inside a signal analyzer whose accuracy has been verified and confirmed with an Accupel DGA-6000 by the legendary Maciej Koper.

Let's start! Here are the contenders:


Apple TV 3rd Gen (Model A1427)


Apple Media Center of two generations ago. Can be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace). The limit of this object is the adjustable refresh rate, which is limited to 50 / 60hz, no 24hz.

Apple TV 4K


The latest Apple release. Can also be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace).
On the spec sheet a bomb...

Amazon FireTV Stick


Very economic and portable object, with a more than decent video quality. Can also be used with MobileForge (CalMAN) and LightSpace Connect (LightSpace). Here the limit is that the output is only one: RGB limited (16-235), no YCrCb

Raspberry Pi 3 B + with PGenerator by Riccardo Biasiotto


A tool that can really do everything! The only limit is the imagination!

Is my laptop bit accurate? Dell Inspiron 17-7779

...and my Blu-ray disc player?!? Panasonic DMP-BDT260


Before moving on to the test results and their commentary, I would like to define what the reference TPG should be... easy, it has a name: Accupel DGA-6000! It is perfectly accurate in every color space (supports up to Rec2020), with every color depth (8/10/12 bit), at every resolution (up to 4K) and frequency (24/25/30/50/60), both in SDR and HDR. It has one single flaw: it lacks of Dolby Vision support, but we know whose fault it is!
Limiting the test to SDR, I can state that a reference TPG must be bit accurate in YCbCr at 1080p resolution with all refresh rates, in this case the reference color space is Rec709. It is therefore in this "enclosure" that I have carried out my tests. Where a device resulted not accurate in YCbCr (or the YCbCr output setting is not present) I have tested its accuracy in RGB Full or Limited but with 16-235 videoscale. The display used is a TV Monitor Samsung SyncMaster P2370HD which, despite obsolescence, it has played its irrelevant task very well.

How I did the tests: DVDO AVLab TPG powered via USB from the laptop, TPG under analysis connected (one at a time) to the HDMI input of the DVDO with the HDMI output connected to the display, CalMAN 2018 Ethusiast R3 Beta Build 27, LightSpace CMS HTP 9.0.0.2448.

TPG for CalMAN:

Ted's Disc Media Files with third generation Apple TV
YCbCr 444 8 bit


MobileForge with Apple TV third generation
YCbCr 444 8 bit


RGB High


...and RGB Low


MobileForge with Apple TV 4K
YCbCr 444 8 bit


MobileForge with Amazon FireTV Stick
RGB Limited 8 bit


RPi PGENERATOR by Riccardo Biasiotto
RGB Full 8 bit


11.21.2018 UPDATE: As requested by @ebr9999 I tested RGB limited output of RPi PGenerator. As previously stated, the not so bad results still prove that RPi is bit accurate only with RGB Full output with LightSpace or CalMAN videoscale set to 16-235.

Here it is:


06.24.2019 UPDATE: As requested by @ebr9999 I tested RPi PGenerator YCbCr limited output, videoscale set to 0-255. Again, as stated above, the results still prove that RPi is completely bit accurate only with RGB Full output with LightSpace or CalMAN videoscale set to 16-235. But, what is better for you, it depends on your TV RGB ---> YCbCr rounding errors.

Here it is:

END OF 06.24.2019 UPDATE

Dell Inspiron 17-7779 with CalMAN built-in TPG
RGB 8 bit


Ted's bluray disc with Panasonic DMP-BDT260
YcbCr 444 8 bit


Conclusions: As shown by the graphs, at the moment, MobileForge is not bit accurate with every device even if the output of the latter is (as in the case of Apple TV third generation, as proven by the test with the media files of Ted's disk). MAYBE wrong triplets are sent. With Apple TV 4K the situation seems to improve a bit but it's just an illusion. The Apple TV 4K, in fact, with output YCbCr and RGB seems to process the image continuously returning fluctuating values with digital errors ranging from a theoretical minimum of 1 to a theoretical maximum of 6 per triplet (as shown in the chart in which Ted patterns are used). For this reason, even if I reported the minimum values (by looking for them pixel by pixel) I did not calculate the relative dE for a simple reason: Apple TV 4K is not effectively usable as a TPG. That's why I will not proceed to test it further.

11.13.2018 UPDATE: Obviously, if ATV 4K is the only TPGable device you own you have no choice than use it. My only recommendation is to NOT measure the patches produced by it in contact mode, the more patch area you measure the more the digital errors will be reduced... forewarned is forearmed.
Since there are still some doubts about ATV 4K behavior with YCbCr/RGB high output, I filmed two videos, both displaying Ted's Color Checker Classic Media file via Infuse 5 Pro App with:
1) ATV 4K at 1080p 24 Hz YCbCr output
2) ATV 4K at 1080p 24 Hz RGB high output

09.19.2019 UPDATE: I've checked again AppleTV 4K YCbCr/RGB High with tvOS 13... it still keep the same behavior filmed here for YCbCr and here for RGB High

That confirms my belief that it's not a software issue.

02.28.2020 UPDATE:

it'll be a quick update! I did some testing on an Apple TV FullHD (4th Generation) and an Amazon Fire TV Stick 4K:

- Apple TV Full HD is bit accurate only with RGB output. YCC is not accurate with YCC content or if used as TPG.

- Amazon Fire TV Stick 4K has YCC output bit accurate but can't be used as TPG because I tried with LS Connect and I didn't get bit accurate results.

So, we can use Apple TV Full HD as TPG only with RGB output, while Amazon Fire TV Stick 4K can't be used as an accurate TPG at all.

NOTE: Except Rpi, I tested all devices with Ted's Color Checker Classic media files (or BD for my BD player) and all of them showed solid triplets except ATV 4K.

A very nice request to analyze CalMAN built-in with my laptop HDMI set to YCbCr has been made by baller02, I didn't think about it when I ran all the tests so I've analyzed Calman built-in TPG only with my laptop HDMI set to RGB and it turned out to be bit accurate. So prior to analyze CalMAN internal TPG in YCBCr, I have to do it with Ted's patterns and... Read this post for more.

The Raspberry Pi (RPi) PGENERATOR by Riccardo Biasiotto is bit accurate and very stable (in RGB Full, not in YCbCr). We're talking about a DIY solution made by a NON PROFESSIONAL. Really outstanding, especially because it is 360° configurable, it connects via Ethernet, Wi-Fi, Bluetooth and it works with every calibration software I know!

Surprisingly, the HDMI output of my laptop resulted bit accurate!!!

TPG for LightSpace

LightSpace Connect with Apple TV third generation
YCbCr 444 8 bit


LightSpace Connect with Amazon FireTV Stick
RGB Limited 8 bit


RPi PGENERATOR by Riccardo Biasiotto
RGB Full 8 bit


Conclusions: Apple TV 4K has been excluded for the reasons mentioned above. All TPG solutions for LightSpace are bit accurate. The choice here could fall on the third generation Apple TV which is also perfect in YCbCr. On the same level, the RPi PGENERATOR by Riccardo Biasiotto.

Final Conclusions and tips: For those who only use CalMAN or even LightSpace, the only affordable and accurate bit solution is the RPi PGENERATOR by Riccardo Biasiotto.

For those who only use LightSpace and own a third-generation Apple TV, they can download and purchase LightSpace Connect but it is strongly suggested to use it connected via an Ethernet cable. Wi-Fi, during the generation of a color cube, could disconnect making you lose happy hours of real life. The RPi is also highly recommended with a super stable wireless Bluetooth connection.

Please, let me know your opinions, suggestions and (why not?!) your own tests!
 

·
Registered
Joined
·
970 Posts
Discussion Starter #2
Reserved
 

·
Registered
Joined
·
970 Posts
Discussion Starter #3
Reserved 2
 

·
Registered
Joined
·
629 Posts
Great post Miki!

If you have time, can you try Ted patterns inside the very latest Infuse on ATV 4K with RGB HIGH?
 

·
Registered
Joined
·
970 Posts
Discussion Starter #6
Great post Miki!

If you have time, can you try Ted patterns inside the very latest Infuse on ATV 4K with RGB HIGH?
Will do. Probably tomorrow morning. But I suspect I will get the same floating results.
 
  • Like
Reactions: MrRobotoPlus

·
Registered
Joined
·
970 Posts
Discussion Starter #7
Great work Miki thanks for your efforts thought you're were a lazy Italian like I am :)
:D
I am! You don't know how many test I could have done and how much I could have written but my laziness made me desist almost immediately! lol
 

·
Registered
Joined
·
970 Posts
Discussion Starter #9
Sure Steve, I will write you my address so you can send me those various devices to test. Oh and a Klein K-10A and a CR-300, thanks! :D Just kidding ;)

What other devices would you like me to test?
 
  • Like
Reactions: L30Z3N and jrref

·
Registered
Joined
·
6,577 Posts
Can the Rpi send HDR patterns or is another device needed?
 

·
Registered
Joined
·
970 Posts
Discussion Starter #11 (Edited)
It needs another device (HDFury Integral/Vertex).

What did emerge once again from my testing it is how Ted's disc is a MUST in a calibrator's "library". It was and still it is a necessary tool in manual (enthusiast) calibration but it is also needed when you don't know if one of your device is bit accurate. I'm thinking about laptops HDMI output for example.
 

·
Registered
Joined
·
233 Posts
It would be interesting to put together a list of PC output solutions that work reliably...

Currently I use the Masciola HDR10 patterns which are very nice.

Anyone try using an Intel NUC7 to run CALMAN, and to generate 4K HDR patterns via HDMI 2.0 output?

ERic
 

·
Registered
Joined
·
233 Posts
Miki,

Please test the RPi PGENERATOR by Riccardo Biasiotti in YCC, it will help debug the current YCC output problem...

E
 

·
Registered
Joined
·
233 Posts
In any case, we really need to find a low-cost 4K HDR solution
 

·
Registered
Joined
·
970 Posts
Discussion Starter #15 (Edited)
I tested it and I already spoke about it with Riccardo. It is technically easy to fix but it requires a lot of time because a analysis color by color is needed as a subsequent triplets compensation... we are talking about at least 9,261 triplets (21^3 cube points). It's a six months job: 50 colors per day during free time.
 

·
Registered
Joined
·
970 Posts
Discussion Starter #16

·
Registered
Joined
·
233 Posts
Worry that color would be off...
Need easy reliable solution that puts TV in correct mode. 4K HDR10
Own integral, but too many boxes too much setup for std calibration

E
 

·
Registered
Joined
·
970 Posts
Discussion Starter #18
I wouldn't worry so much about that. First because HDR10/DV calibration is all about grayscale as touching the CMS for HDR10 introduces artifacts while Color Gamut calibration for DV (eg. DV file tweak) gets color even more off, second because you and I don't have reference meters, so we have at least a dE of 2 already from the "Ready? Go!". This hobby is fun and educational but when it comes to have the job really done a good pro calibrator is the only really CHEAP solution.
 

·
Registered
Joined
·
6,577 Posts
It needs another device (HDFury Integral/Vertex).

What did emerge once again from my testing it is how Ted's disc is a MUST in a calibrator's "library". It was and still it is a necessary tool in manual (enthusiast) calibration but it is also needed when you don't know if one of your device is bit accurate. I'm thinking about laptops HDMI output for example.
If you added the HDFury Integral/Vertex to the Rpi would the HDR Test patterns be accurate or does that still need to be tested?
 

·
Registered
Joined
·
970 Posts
Discussion Starter #20
It would need to be tested. I might do that for 1080p SDR but then we should introduce a new dogma in our lives "with HDFury Integral what is bit accurate for SDR , it will also be for HDR" and trust it until someone with a 2k/4k HDR signal analyzer will confirm or destroy our dogma. I admit my ignorance here, I don't know if DVDO is able to analyze a HDR signal, I guess it can not, but @ConnecTEDDD surely might clear this thing up.
 
  • Like
Reactions: jrref
1 - 20 of 650 Posts
Top