I would not be inclined to trust magazine sponsored "tests". The methods, standards and scope are not only subject to bias; but, full procedural documentation is not available to allow verification of the results or peer review. The degree to which "controlled conditions" is applied in these "tests" is not at all without fault. Further multiple tests are not conducted to allow correlation of the resultant data sets.
While you are exactly correct, THX will test a product to determine compliance to a set of established criteria (criteria known to the submitting party prior to submission), it is the genesis of the criteria and consistency of the testing process which comes into play. The criteria is developed, and pass/fail metrics, are established prior to any testing and without regard to any manufacturer or technical approach or, in your case, material. The audio and video criteria to pass are based upon what is required to achieve a desired result or meet a predefined standard. As an individual with industry experience, you are certainly aware that meeting the height of that bar is not an easy nor straight forward process...even knowing the criteria in advance. Many, many have gone back to the bench for redesign to pass.
With THX, you must meet some predefined standard which has been designed to assure a specific level of performance. In an independent third party lab, there is no such standard to meet ... simply tests with data flowing from those tests. With standardized testing, products can be compared without bias. That's a good thing; but, in the absence of a minimum set of criteria, who's to know if the differences between two products make a twits bit of difference in the intended application. In the world of AT screens all tests have been conducted using different equipment, in different environments under different testing conditions. Secondly, I know of no such testing which has not been "encouraged by" or sponsored by someone with no vested interest in the outcome. In the case of THX, youse pays your testing fees whether or pass or don't. Clearly both parties want a "pass" but a pass is never assured...you meet the mark or you don't and just because you paid a fee, doesn't mean you get to slide under the bar.
Gary and his crew do some credible work; however, we must recognize our testing equipment can measure events way beyond the ability of humans to detect or perceive. The ability to test beyond perception is helpful if we're trying to determine a cause behind a problem. What needs to be first established is exactly what is the desired baseline of performance. Then, determine from the testing whether or not these baselines have been achieved. For example, if one screen exhibits a 1 dB roll off between 19kHz and 20kHz, is that relevant? It certainly is to the marketing lads; but, for the material's intended purpose, not at all relevant.
I am reasonably certain there are products on the market today which were submitted to THX for certification and failed. The manufacturer, for any number of reasons simply made the decision to perfume the pig and move on.
