A bit over a month ago I was in New York for a press event, where I ran into David Katzmaier, the Senior Editor/HDTV at CNET. I'm a fan of his writing, and I was curious about how he goes about reviewing televisions. Fortunately, David was very gracious and extended an invitation to visit his TV-testing facility located in the heart of Manhattan.
If you want to test TVs, it helps to have a room like this one.
The facility itself is a home-theater geek's dream. Considering the midtown-Manhattan location, there is a generous amount of floor space for the dual home-theater arrangement—back-to-back couches with a heavy black drape that can separate them. When I arrived, David had arranged one comparison—a pair of UHD televisions and an HD plasma of the same size. On the other side of the curtain, he had set up a comparison of some televisions that I chose beforehand. We will get back to that topic in a little bit.
In this panorama you can see how a heavy black drape splits the room
The first thing to know about Katzmaier's work is that his reviews are always comparative—if he is reviewing a 65-inch LED-edgelit LCD, he will also have a similar model from a competitor, or another model from the same manufacturer, and perhaps a similarly priced plasma panel, all set up next to the review unit. This really highlights any differences in performance between the panels, something I could see quite clearly during the demonstration.
Before we sat down, David showed me a few of the cool gadgets he has lying around in his lab. I got a kick out of a Konica-Minolta LS-100 luminance meter—a single-purpose device that cost $2500 and allows you to measure brightness levels of any surface by simply pointing and pulling a trigger. It feels a bit like a superpower—who would have ever thought that taking black-level readings could be addictive. For seriously accurate measurements, the lab also features a Konica-Minolta CS-2000 spectroradiometer, which costs close to $30,000 and is an industry-standard tool.
Konica-Minolta LS-100 luminance meter
Konica-Minolta CS-2000 spectroradiometer
Another interesting device was a Gefen HDMI Detective Plus EDID (Extended Display Identification Data) emulator, which turns out to be a crucial tool—if you test televisions for a living. EDID information passes between the source and the display, and sometimes with new devices the connection cannot be completed. With an EDID emulator, you can copy the information from one device, and then plug in another device and use the information from the first device.
Gefen HDMI Detective Plus EDID emulator
Prior to my visit, David asked if there any comparisons I would like to see. I decide to shoot for the moon and mentioned that I would love to see a UHD television as part of a comparison. Since CNET had not published any reviews of UHD televisions up to that time, I hedged my bet and asked to compare a couple of mainstream LCD televisions with a mainstream plasma.
For the LCD units, I requested a Vizio M551D and a Samsung UN55F8000. For the plasma, I requested a Panasonic TC-P60ST60—a TV with the distinction of having earned the highest rating ever from CNET. The Panasonic plasma received its top ranking because of the overall value it represents, as opposed to being the ultimate in producing a reference-quality picture. At the last minute, we decided to add a fourth TV—Sony's KDL-55W900A LCD TV. Then, temptation struck again and we decided to toss in 2013's best-performing plasma panel, Panasonic's TC-P60ZT60. There were dozens of TVs to choose from, even a Kuro of some sort that was sitting in a dark corner waiting for another chance to shine.
Katzmaier had dozens of TVs to choose from
A Kuro sits in a dark corner
I was thrilled that David had all those options available—since I own a Vizio M-series TV, I was curious how it would hold up in such a lineup. However, as it turned out, David had a surprise in store—a pair of UHDTVs. There was a Samsung UN65F9000 and a Panasonic TC-L65WT600. That setup, which was on the other side of the black drapes that bisected the room, also included a third television—a Panasonic TC-P65S64 plasma, which is a special model for sale at membership-based warehouse-type stores. It features image quality that comes close to the ST60, but has fewer features—for example; the S64 does not have 3D capability.
David preps a pair of UHDTVs and a 1080p plasma for a comparative review
We decided to have a look at the UHDTVs first. David explained the difficulties involved in testing these TVs—namely the lack of good native-4K content. Sony's UHDTVs work with that company's FMP-X1 Ultra HD media player, but the player only works with Sony's own UHDTVs. When I arrived, David was still waiting for Panasonic to drop off a 4K player, so I missed the opportunity to see native UHD footage in that comparison.
Instead, we compared 1080p Blu-ray clips on the three panels to gauge the effectiveness of their upscaling 1080p to 2160p. It was immediately obvious why Katzmaier prefers to review a TV by comparing it to other TVs—it levels the playing field. In this case, the modestly priced 1080p plasma panel was never any worse at rendering video than the UHD panels—which cost three times as much for the same size screen. There are numerous image-quality parameters where the S64 plasma actually had the two LCD-based UHD panels beat, including screen uniformity and viewing angles. While the F9000 did exhibit very deep blacks, it only beat the S64 plasma's black levels during very dark scenes.
Comparing image quality between two UHDTVs and a 1080p plasma
When it came to rendering detail, I saw no advantage to upscaling on either the Panasonic or Samsung UHD units. In fact, the Samsung appeared to lose a little bit of detail versus the other two TVs. In the recent past, I witnessed Sony's XBR-65X900A outperform top-tier native 1080p plasma panels in terms of detail rendition, so seeing this mediocre performance was a bit of a letdown.
Based on that experience alone—even if price were no object—I would choose the plasma panel over the UHD units. In the real world, I would gladly purchase a reference-level plasma versus buying one of these first-generation UHDTVs. Samsung did build the full-array local-dimming 85" S9, a technological tour de force that offers a glimpse of the format's potential, but sadly the 65" units offered no such reward for the early adopter. Without true 2160p content, the new UHD panels are stuck in idle.
Getting UHD to work properly can be a hassle and content is hard to find
I wish I could have stayed for the 2160p content comparisons, but time was somewhat limited. David gave me a full three hours of his time, which is very generous considering the cost of running a facility like CNET's. We moved to the other side of the room and began our comparison of 1080p panels.
David prepares five HDTVs for comparison
Since I enjoy 3D movies, I wanted to see how the five panels handled that format. I was mostly interested in the differences between active and passive technology, and the difference between plasma and LCD when it comes to active-shutter 3D performance. We chose Hugo because it has a lot of depth to the 3D, which can result in excessive crosstalk on systems that do not perform well.
The results confirmed what I have read and personally experienced. Active-shutter LCD TVs are intrinsically flawed—in fact, I found that both the Sony KDL-55W900A and the Samsung UN55F8000 were unwatchable in 3D mode because of the excessive crosstalk. The 3D was much cleaner on the two Panasonic plasma panels. The Vizio performed as I expected, providing a bright and crosstalk-free 3D image, but the resolution loss that results from the use of passive technology was far more obvious to me than it is at home, when I am only looking at my own TV. That was the experience I sought—and there is no way to get it, aside from vising CNET's facility.
David proceeded to run tests for black level, color rendition, gamma, motion resolution, viewing angle, and so forth. I suspect that many AVS readers will not be surprised to learn that the plasma panels outperformed the LCD panels in just about every single image-quality parameter. The bigger surprise was how close the ST60 gets to the ZT60 in terms of image quality—the main visible difference being how deep the blacks were. It costs a full $1000 extra to get that last bit of depth, but on certain scenes, the effect could be mesmerizing. That said, both plasmas outperformed the LED-LCDs in numerous other areas, including motion resolution and screen uniformity.
Katzmaier checks each TV for color and grayscale accuracy
After the comparison ended, I left with a new degree of respect for 1080p plasma panels. My own HDTV purchase (Vizio M-series) resulted from my preference for passive 3D. However, over the last year I have become keenly aware of the deficiencies of LCD TVs, and especially LED-edgelit models with their pseudo local dimming. My visit to CNET's Manhattan laboratory cemented these views.
Screen uniformity and black levels compared—edgelit LCD on the left, plasma on the right
I cannot wait for OLED to become affordable. I tried out active glasses with OLED, and that combination is essentially perfect—very bright with zero crosstalk. After experiencing David's method of comparing TVs, I feel no impatience about UHD OLED coming to market, because 1080p really is more than adequate. I would love to own a 70-inch flat OLED one of these days, but I'm in no rush.
The one application where I have found that LCD technology offers a compelling advantage is glasses-free 3D, a technology capable of taking advantage of UHD resolution and providing a unique experience. I recently witnessed a demo of a technology called Ultra-D that convinced me there is an image quality-driven application for LED-LCD panels. I would love to see one of those TVs in a comparative evaluation versus conventional active and passive 3D panels, Katzmaier style.
One thing is clear—the TV industry is pursuing more than one path to the consumer's wallet. Comparing TVs in a store is not the same thing as comparing TVs under controlled lighting, with each calibrated TV oriented to face the viewer. David Katzmaier deserves a lot of credit for going through the effort to review TVs under such rigorous conditions. Considering all that, it is no surprise that Panasonic plasmas dominate the ratings when it comes to CNET's highest recommendations for flat-panel TVs. Yet it appears that Panasonic is getting out of the plasma business. I wonder what sort of TVs will top CNET's list next year?
All photos ©2013 by Mark Henninger
Edited by imagic - 11/2/13 at 6:41am