This is an e-mail response (it was forwarded to me) from Dave Schulte as to how he thinks about testing and why/how it should be accomplished:
All SE upgrade clients who have Onkyo's and Integra's have heard dramatic improvements on both on Pure Direct, Direct, Stereo, all surround modes, Audessey on or off, all settings, all software, wether it's via HDMI, RCA or XLR analog, or the SPDIF coax or toslink inputs. We've never had one returned. The RFI & EMI shielding is the key to our success. When you guys compared the Onkyo's running through a switcher, doing so reduced the "Signal to noise ratio" and the upgrades down literally to the quality of the switcher, and down to the "signal to noise ratio" of the switcher. In effect, you erased the upgrades. Far more so then if an SE upgraded Onkyo is running into a stock amplifier. This was due to all of the additional HDMI cabling which operates in the "radio frequency domain" collecting a ton of RFI out of the airwaves, as did the HDMI switcher and it's parts and circuit boards inside it. The HDMI switcher also contaminated the audio and video signals with EMI from the power used to operate the relays. Less then millimeters away inside the relays. Bad news, no comparison should never have been setup that way. The Stereophile reviewer who loaned you the HDMI switcher should be shown this. Please forward this to him and direct him to our website so that he may discover the root cause of less then faithful electronic reproduction of audio and video signals. Consumer electronics are built "For Profit" and as such are not effectively shielded from internal nor external RFI or EMI.
The best method I have found to compare is:
1) Afford each unit at least a 2+hr warmup time since the last time it was powered off.
2) Never shut it down during comparisons.
3) Use the exact same HDMI cable and the same RCA or XLR analog output and input cables
4) Use identical power cords plugged into the same outlet on a good powerline conditioner.
5) Both units should be set upon the same resonance support, the stock feet directly upon the same
shelving. Never stack units on top of each other or atop another component, this changes the sound
for the worse.
6) Ideally both the stock and upgraded units should have close to identical hours of use since new.
There is a direct correlation between hours of use and perceived performance in double blind studies.
In short, here's what's going on with all stock audio or video equipment:
Take a look inside any brand. The wiring, circuitry, and parts on the circuit boards are almost always all unshielded, left open and exposed to collect the radio frequency radiation emitted by the numerous clocks inside the unit and collected out of the airwaves that are all around us. Some manufacturers have started employing wrapping on flat ribbon conductors and sleeves over wiring, but it's not done correctly nor enough. This "trade secret upgrade approach" was pioneered in consumer electronics by David J. Schulte nearly 30 years ago. It is a theft of our trade secret in fact.
Stock consumer electronic equipment is bombarded with high amplitude radio frequency contamination constantly, millions of times per second, right out of the airwaves which are all around us, and from 1 or more "clocks" inside the unit. The Oppo 95 and Onkyo Pro have 5-6 clocks inside spewing RFI millions of times per second into anything metallic.
Airborne RFI collection by all associated cabling as well as the circuitry and parts themselves has been proven to ruin both the measured and performance of both digital and analog equipment alike.
Take an Oppo 93 for example. Installing high end parts and a dac produces small improvements, but it has never lifted the modest 93's performance to State of the Art. Leaving the entire Oppo 93's wiring, circuitry, dac, transport, DSP, video, analog stages etc everything inside exposed to heavy internally generated radio frequency contamination ruins it's performance potential. The 93 has many "clocks" which each spew high amplitude RF inside the unit into all circuitry, all wiring, all parts, ruining it's potential by constantly contaminating the actual signals containing the audio and video information, millions of times per second.
All digital devices have at least 1 clock, which contaminates the entire unit inside. In all stock players, none of the parts or wiring or circuit boards or chip sets or DAC's are shielded. Audio & video signals actually become corrupted with the out of bandwidth signals emitted from the clock(s) which then gets reproduced right along with the actual audio or video information. The DAC & video chipsets are typically located just millimeters to inches away from the actual dac/video scaling chipsets trying to faithfully reproduce the content. What's really alarming is the fact that the volume of radio frequency noise is typically the same volume as the actual audio or video signals themselves. Most signals operate in the millivolt range in digital or analog sources, digital or analog preamplifiers, and the first stage of all power amplifiers. It is no wonder
stock digital has never reached the performance of an excellent vinyl playback setup.
The "out of band" radio frequency noise being emitted from the digital clocks places extra heavy demand upon all, chipsets, transport, DSP, HDMI, and audio and video amplifying devices in both audio and video circuits. All sections must work overtime reproducing the radio frequency noise millions of times per second right along with the actual audio or video signals. This dramatically harms any brand or model's ability to faithfully reproduce the actual audio and video signals, regardless of cost or prestige.
In addition, in both analog and digital gear, the power supplies get drained reproducing millions of pulses of RFI per second. This places extra heavy demands on both the main and local power supplies, draining the power storage capacitors very fast, leaving greatly reduced power on tap to reproduce the actual audio and video content.
Surprisingly, analog tubed and solid state power amplifiers & tubed & solid state preamplifiers exhibit tremendously improved performance on both measurements and in comparison tests, not just digital units. Radio frequencies are in the airwaves all around us. All internal wiring/circuitry/parts and external power cords and interconnect cabling collect RFI noise millions of times per second. This all gets piled on top of the RFI & EMI the unit itself generates. Lastly, solder joints and the entire underside of circuit boards not just the circuit traces themselves, all collect RFI like crazy right out of the air from the clocks inside the unit and out of the airwaves all around us.
As you can see, RFI & EMI contamination is the root cause of less then perfect signal reproduction.