AVS Forum | Home Theater Discussions And Reviews (https://www.avsforum.com/forum/)
-   AVS Forum® Podcasts (https://www.avsforum.com/forum/138-avs-foruma-podcasts/)
-   -   Value Electronics TV Shootout 2016 (https://www.avsforum.com/forum/138-avs-foruma-podcasts/2501826-value-electronics-tv-shootout-2016-a.html)

Scott Wilkinson 06-30-2016 07:51 PM

Value Electronics TV Shootout 2016
 
Joel Silver, founder and president of the Imaging Science Foundation, and Robert Zohn, president of Value Electronics, talk about the Value Electronics TV Shootout last week, including a bit of the history of the event, which TVs were included and why, the entire evaluation system (including the new Panasonic UHD Blu-ray player, a new HDR pattern generator, and a new HDMI matrix switcher), the SDR calibration of each one, the challenges of evaluating HDR performance, how the proceedings were conducted, the evaluation criteria, the results, some problems encountered during the event, plans for next year, answers to chat-room questions, and more.


Here are the SDR/BT.709 calibration results from David Mackenzie; the Vizio RS65 arrived too late for David to calibrate, so Joel Silver simply matched it to the other calibrations as closely as possible.

LG G6

https://www.avsforum.com/forum/imageh...6b33deca1c.jpg

https://www.avsforum.com/forum/imageh...6b33e68cc2.jpg

Samsung KS9800

https://www.avsforum.com/forum/imageh...6b33e8a3e2.jpg

https://www.avsforum.com/forum/imageh...6b33ec159a.jpg

Sony X940D

https://www.avsforum.com/forum/imageh...6b33edf68c.jpg

https://www.avsforum.com/forum/imageh...6b33f0270f.jpg

ray0414 06-30-2016 08:46 PM

Do we have native black level measurements of the lcds?

imagic 07-01-2016 02:24 AM

Quote:

Originally Posted by ray0414 (Post 45086554)
Do we have native black level measurements of the lcds?

And how about the calibration report from that Vizio Reference?

Also, if the Vizio arrived too late for David Mackenzie to calibrate, is it fair to say the Reference had very few hours on it when calibrated? Was there even time to find optimum settings? Last year the Panasonic CX850 also arrived late and came in fourth place, which could be pure coincidence.

It would be nice to have at least some calibration info for the TV that came in last place in every single category except off-axis viewing, especially given what Joel Silver said in response to Scott's final question...

"I think this is probably the closest group that Robert has ever put together as far all the TVs making a nice picture, there's wasn't a dog in the room... they were all good." says Joel. He continues... "Minute differences were magnified. We put up checker-board patterns and they were all good in SDR, and they all popped in HDR. So this was a tough thing to judge, so minute differences became magnified... they had to hunt for differences more than before." - Joel Silver

Mattopotamus 07-01-2016 04:46 AM

Quote:

Originally Posted by imagic (Post 45088802)
And how about the calibration report from that Vizio? Also, if it arrived too late for David Mackenzie to calibrate, is it fair to say the Reference had very few hours on it when calibrated? Was there even time to find optimum settings? Last year the Panasonic CX850 also arrived late and came in fourth place, which could be pure coincidence.

Anyhow, it would be nice to have at least some calibration info for the TV that came in last place in every single category, especially given what Joel SIlver said in response to Scott's final question...

"I think this is probably the closest group that Robert has ever put together as far all the TVs making a nice picture, there's wasn't a dog in the room... they were all good." says Joel. He continues... "Minute differences were magnified. We put up checker-board patterns and they were all good in SDR, and they all popped in HDR. So this was a tough thing to judge, so minute differences became magnified... they had to hunt for differences more than before." - Joel Silver

These shootouts are becoming less and less of a comparison or shootout and more of just a showroom. The Vizio doesn't support HDR10, yet DV was not displayed. That doesn't even make sense to me. They need to do their best to make the content on all displays the exact same. Of course HDR content is going to look better than no HDR content, but to compare them as if they are equal :confused:

I guess in the end it is all about selling expensive displays, but a poll where the forum users pick the displays would be really nice and probably generate even more interest.

jrref 07-01-2016 05:48 AM

Quote:

Originally Posted by Mattopotamus (Post 45089722)
These shootouts are becoming less and less of a comparison or shootout and more of just a showroom. The Vizio doesn't support HDR10, yet DV was not displayed. That doesn't even make sense to me. They need to do their best to make the content on all displays the exact same. Of course HDR content is going to look better than no HDR content, but to compare them as if they are equal :confused:

I guess in the end it is all about selling expensive displays, but a poll where the forum users pick the displays would be really nice and probably generate even more interest.

They did play marco polo from Netflix on all the TVs. The LG and Vizio displayed them in DV and the Sony and Samsung displayed them in HDR10. In my opinion the Vizio showed well playing DV and regular HD content but not as good as the others. The screen uniformity, in my opinion was terrible using the test patterns.

imagic 07-01-2016 05:59 AM

So here's an idea... Make the arc of the TVs optimal for critical viewing by aiming all the screens at one spot in the room. Forget trying to position them optimally for the seated audience, that will never work (under the current configuration, members of the audience will always be off-axis relative to some of the TVs).

If all the TVs were aimed at a spot, and that spot was a typical viewing distance from the screens (10-15 feet), then when it came time to stand up and evaluate the TVs, the attendees could line up to stand in that spot. Have them make their determinations while looking at ALL the TVs from an optimal angle. Sure it would take a bit longer to get through the voting, but the front end of the shootout had quite a bit of chit-chat that could be abbreviated if need be. It's far better than asking people to rely on their memory as they move from one TV to the next.

tjcinnamon 07-01-2016 07:01 AM

Given the proliferation of streaming and the variability of internet speed. A capped 5Mbps internet stream of a service would be appreciated. I'd like to see how the upscalers perform. The UFC Fight Pass has a shoddy 720p stream; YouTube has questionable quality content; HBOGo and Netflix can have slightly diminished quality. This will even become a greater factor with HDR streams because there is potential to have stream quality vacillate (avg 13Mbps but 15Mbps for 20 minutes, 5Mbps for 5 minutes...)

Even 480p DVD content is not aliased. Some aliased 1080p content or highly compressed crushed blacks content is becoming a more prevalent fact of life.

I've heard things about Vizio and LG upscalers not being the greatest.

JWhip 07-01-2016 07:15 AM

Quote:

Originally Posted by imagic (Post 45090634)
So here's an idea... Make the arc of the TVs optimal for critical viewing by aiming all the screens at one spot in the room. Forget trying to position them optimally for the seated audience, that will never work (under the current configuration, members of the audience will always be off-axis relative to some of the TVs).

If all the TVs were aimed at a spot, and that spot was a typical viewing distance from the screens (10-15 feet), then when it came time to stand up and evaluate the TVs, the attendees could line up to stand in that spot. Have them make their determinations while looking at ALL the TVs from an optimal angle. Sure it would take a bit longer to get through the voting, but the front end of the shootout had quite a bit of chit-chat that could be abbreviated. It's far better than asking people to rely on their memory as they move from one TV to the next.

I think this is a great idea. I would also hope that next year in addition to rec 709 calibration, they will be able to calibrate for WCG as well as DV and HDR-10. Doing HDR evaluations with out of the box settings doesn't make sense to me. The sets should all be on the same playing field. I would also like to see the chit chat and presentations be cut down. Deal with each voting category, briefly explaining what it is and show some content that illustrates it. I know there are some out there that loved Joel Silver's presentation but being there it went on and on and on. Most attendees and those viewing on line don't need a three hour plus demo to judge the sets. Robert should also have a time limit as to when he has the sets. If the manufacturer can't get the set to him two weeks before the event, it should be out. Also, given that the calibrations will likely be more involved next year, I would suggest that David not go it alone and that they have two calibrators there to split the work.

lyris 07-01-2016 01:54 PM

Quote:

Originally Posted by JWhip
If the manufacturer can't get the set to him two weeks before the event, it should be out.

Agreed 100%. Of course an event like this is hard to put on, but the last minute nature can only be a bad thing.

tjcinnamon 07-02-2016 06:44 AM

man, I thought this thread would be more popping!?!?

Deja Vu 07-02-2016 10:38 AM

I would like to have heard some details about each individual T.V. and how it stood up against the competition. Overall scores really don't tell me much, especially when we're told all these T.V.s looked very close to each other. Here's a side by side comparison and none of the attendees are saying -- I could live with these two particular T.V.s, etc. and why. Somewhat disappointing.

ray0414 07-02-2016 11:54 AM

Quote:

Originally Posted by tjcinnamon (Post 45110658)
man, I thought this thread would be more popping!?!?


I think the other 2 threads pretty much wore people out.


Quote:

Originally Posted by Deja Vu (Post 45114346)
I would like to have heard some details about each individual T.V. and how it stood up against the competition. Overall scores really don't tell me much, especially when we're told all these T.V.s looked very close to each other. Here's a side by side comparison and none of the attendees are saying -- I could live with these two particular T.V.s, etc. and why. Somewhat disappointing.


I agree. Hard numbers would he great. There's still a chance we could get them:) I'd be really interested to see the black level numbers between the ks9800 and 940d, because the 940d scored higher in votes, but all the reviews has the ks9800 measuring lower than the sony models.

tjcinnamon 07-02-2016 01:13 PM

Quote:

Originally Posted by ray0414 (Post 45115634)
I think the other 2 threads pretty much wore people out.





I agree. Hard numbers would he great. There's still a chance we could get them:) I'd be really interested to see the black level numbers between the ks9800 and 940d, because the 940d scored higher in votes, but all the reviews has the ks9800 measuring lower than the sony models.


I still think my streaming/compressed HD content category should be considered in the next one.

jocedeg 07-02-2016 04:16 PM

The more I read about this, the more it seems that they were trying to compare apples, oranges, pears, bananas...

The tvs all seem to be in different classes. The feature set is all over the place. Some of them weren't calibrated...

I think this whole "shootout" is the wonkiest yet. It kinda tarnishes Value Electronics' credibility.

tjcinnamon 07-02-2016 04:21 PM

Quote:

Originally Posted by jocedeg (Post 45119362)
The more I read about this, the more it seems that they were trying to compare apples, oranges, pears, bananas...

The tvs all seem to be in different classes. The feature set is all over the place. Some of them weren't calibrated...

I think this whole "shootout" is the wonkiest yet. It kinda tarnishes Value Electronics' credibility.

How so? I think the only real difference was the OLED vs LED/LCD. The TV's were all "Flagship" TV's of the brand.

jocedeg 07-02-2016 04:36 PM

Not all tvs support the same "kind" of HDR.

imagic 07-03-2016 05:09 AM

Quote:

Originally Posted by jocedeg (Post 45119626)
Not all tvs support the same "kind" of HDR.

It's more like Vizio does not yet have firmware for HDR10 support, even though the company promised it would. HDR10 is the standard as far as these things go since it's required on Ultra HD Blu-ray. Dolby Vision is the cherry of top, except for Vizio owners who can only watch Dolby Vision content. All the other TVs work with HDR10.

However, another issue at the shootout was that the Samsung had issues with the HDR10 signal. I noticed it happen repeatedly during the session I attended. As Chris Boylan from Big Picture Big Sound put in his article: "...the Samsung set had some issues locking to an HDR-enabled 4K signal, which may (or may not) have been related to the 4K signal being split out and fed to multiple devices. Both the Sony and the LG sets identified the HDR 10 signal quickly and set themselves into HDR-optimized picture modes. In many cases, the signal shown on the Samsung was not exploiting the set's HDR capabilities." - read the full article here.

ray0414 07-03-2016 07:03 AM

Quote:

Originally Posted by imagic (Post 45124786)
It's more like Vizio does not yet have firmware for HDR10 support, even though the company promised it would. HDR10 is the standard as far as these things go since it's required on Ultra HD Blu-ray. Dolby Vision is the cherry of top, except for Vizio owners who can only watch Dolby Vision content. All the other TVs work with HDR10.

However, another issue at the shootout was that the Samsung had issues with the HDR10 signal. I noticed it happen repeatedly during the session I attended. As Chris Boylan from Big Picture Big Sound put in his article: "...the Samsung set had some issues locking to an HDR-enabled 4K signal, which may (or may not) have been related to the 4K signal being split out and fed to multiple devices. Both the Sony and the LG sets identified the HDR 10 signal quickly and set themselves into HDR-optimized picture modes. In many cases, the signal shown on the Samsung was not exploiting the set's HDR capabilities." - read the full article here.


You don't say.

ray0414 07-03-2016 07:11 AM

Quote:

Originally Posted by imagic (Post 45124786)
It's more like Vizio does not yet have firmware for HDR10 support, even though the company promised it would. HDR10 is the standard as far as these things go since it's required on Ultra HD Blu-ray. Dolby Vision is the cherry of top, except for Vizio owners who can only watch Dolby Vision content. All the other TVs work with HDR10.

However, another issue at the shootout was that the Samsung had issues with the HDR10 signal. I noticed it happen repeatedly during the session I attended. As Chris Boylan from Big Picture Big Sound put in his article: "...the Samsung set had some issues locking to an HDR-enabled 4K signal, which may (or may not) have been related to the 4K signal being split out and fed to multiple devices. Both the Sony and the LG sets identified the HDR 10 signal quickly and set themselves into HDR-optimized picture modes. In many cases, the signal shown on the Samsung was not exploiting the set's HDR capabilities." - read the full article here.



By the way, Lee Newkirk, who was a speaker at the shootout, currently has his ks9800 in for review for reviewed.com, and is taking questions via Twitter. I asked him to comment on how his review sample compares to the ks9800 at the shootout. He might be able to shed some light as well.

losservatore 07-03-2016 09:56 PM

The Samsung have the highest peak brightness at full screen and small screen ,but the Oled score was higher than the Samsung? could be that the higher contrast and image depth (depth perception) was giving the illusion that it was brighter?


I know that even without the brightness score the Oled will still have won.

helvetica bold 07-04-2016 06:58 AM

What was the name of the HDMI cables they used for the shootout? Did anyone catch that?

HockeyoAJB 07-04-2016 05:41 PM

Quote:

Originally Posted by losservatore (Post 45138042)
The Samsung have the highest peak brightness at full screen and small screen ,but the Oled score was higher than the Samsung? could be that the higher contrast and image depth (depth perception) was giving the illusion that it was brighter?


I know that even without the brightness score the Oled will still have won.

There wasn't a "brightness" category/score in the event. Many people seem to be confusing "day time viewing" with brightness. If all they cared about was how many nits the TV was putting out, they wouldn't ask the audience to vote on it. They would just measure it. While the Samsung is the brightest TV of the four (and possibly the brightest consumer display ever made), it does not provide the best day time picture quality because A) the other displays are all plenty bright enough for day time viewing and B) the Samsung had issues that impacted its PQ including poor uniformity, inaccurate color (particularly red), and more visible blooming than the Sony.

JWhip 07-05-2016 03:50 PM

I also found the Sammy to be way too bright period. In the dark it just seared the eyes. It was hard to watch with the lights on as well.

imagic 07-06-2016 04:06 AM

Quote:

Originally Posted by JWhip (Post 45170305)
I also found the Sammy to be way too bright period. In the dark it just seared the eyes. It was hard to watch with the lights on as well.

At this point it is unclear what the deal was with the Samsung, but except for the HDR demos it should not have been any brighter than the other TVs when showing BT.709 content... if it was calibrated and functioning properly.

imagic 07-06-2016 04:12 AM

Quote:

Originally Posted by HockeyoAJB (Post 45152449)
There wasn't a "brightness" category/score in the event. Many people seem to be confusing "day time viewing" with brightness. If all they cared about was how many nits the TV was putting out, they wouldn't ask the audience to vote on it. They would just measure it. While the Samsung is the brightest TV of the four (and possibly the brightest consumer display ever made), it does not provide the best day time picture quality because A) the other displays are all plenty bright enough for day time viewing and B) the Samsung had issues that impacted its PQ including poor uniformity, inaccurate color (particularly red), and more visible blooming than the Sony.

Two things...

A. It's not clear that turning on the fluorescent lights in that basement room is an accurate simulation of a room with ambient daylight coming from Windows.

B. Potentially something was wrong with that KS9800 sample used in the shootout... it's just not clear what. The uniformity issue was identified as damage or defect. There were issues with the KS9800 displaying colors properly that popped up throughout the second full session.

lyris 07-06-2016 08:40 AM

Quote:

Originally Posted by imagic (Post 45179025)
At this point it is unclear what the deal was with the Samsung, but except for the HDR demos it should not have been any brighter than the other TVs when showing BT.709 content... if it was calibrated and functioning properly.

Was it brighter during the Rec709 portion, or just during the Powerpoint presentations?

The couple of times I went downstairs to take a look at how things were going, the Samsung had kicked itself into PC Mode, a problem I pointed out to the shootout guys. (PC mode is enabled on some displays when the TV receives a 60hz signal in a PC resolution, of which 1920x1080 is one). In that configuration, the calibrated modes aren't used and are replaced by "Standard" and "Entertain", which are very bright and don't use a D65 white point, among other things.

Edit: Robert just contacted me to assure me the calibrated modes were indeed used for critical viewing, and checks were done after slides were shown to ensure it wasn't in PC Mode. Therefore, there's no reason why the Samsung should have been brighter with content, but the above was my best guess.

JWhip 07-07-2016 09:42 AM

Quote:

Originally Posted by imagic (Post 45179025)
At this point it is unclear what the deal was with the Samsung, but except for the HDR demos it should not have been any brighter than the other TVs when showing BT.709 content... if it was calibrated and functioning properly.

I don't doubt that. I just found the set difficult to watch.

DDailey 07-07-2016 12:36 PM

Could you guys maybe come up with something other than "shootout" - so completely tone deaf in light of America's gun violence! I guess I should expect nothing less from a bunch of old white guys!

HockeyoAJB 07-07-2016 12:55 PM

Quote:

Originally Posted by DDailey (Post 45213337)
Could you guys maybe come up with something other than "shootout" - so completely tone deaf in light of America's gun violence! I guess I should expect nothing less from a bunch of old white guys!

OMG! Are you also petitioning the NHL to rename the thing that takes place during the regular season when a game is still tied after OT? Should we rename the stat that tracks attempts on goal/net in hockey, basketball, and soccer? We'll call them flings in hockey, throws in basketball, and kicks/heads/knees/volleys in soccer. Sound good? No? Sorry, I gave it my best shot!

[Irishman] 07-07-2016 02:33 PM

Quote:

Originally Posted by JWhip (Post 45091954)
I think this is a great idea. I would also hope that next year in addition to rec 709 calibration, they will be able to calibrate for WCG as well as DV and HDR-10. Doing HDR evaluations with out of the box settings doesn't make sense to me. The sets should all be on the same playing field. I would also like to see the chit chat and presentations be cut down. Deal with each voting category, briefly explaining what it is and show some content that illustrates it. I know there are some out there that loved Joel Silver's presentation but being there it went on and on and on. Most attendees and those viewing on line don't need a three hour plus demo to judge the sets. Robert should also have a time limit as to when he has the sets. If the manufacturer can't get the set to him two weeks before the event, it should be out. Also, given that the calibrations will likely be more involved next year, I would suggest that David not go it alone and that they have two calibrators there to split the work.

I agree. That's too much for one calibrator. In fact, this might be the first year I'm aware of that multiple calibrators weren't involved (and present to discuss their process)?


All times are GMT -7. The time now is 01:39 PM.

Powered by vBulletin® Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
vBulletin Security provided by vBSecurity (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.

vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.