Originally Posted by BiGBADDABOOM
I find bosso's suggested method for these GTGs or sub shootout/reviews in general to be the best of both worlds.
Use SpecLab waterfalls with a digital reference that many are familiar with in the sticky thread here. Cut down on the number of subs tested to the serious contenders, or at least plan enough time to do this right. With calibrated measurement gear, setup each sub in the same room properly and measure reference playback of the best scenes to capture a waterfall.
Distortion of any kind will be as visible as anything could be, and you can truly see which subs offer an accurate reproduction.
Bosso can expound, I'm sure I didn't do it justice. But it's always made sense to me as the best combination of objective data gathering, but using real-world material and practicality at the same time.
Actually, I nicked the idea from Keith Yates' standard-setting Way Down Deep series in Ultimate AV mag. I just didn't get around to actually doing the tests until much later.
SpecLab lets you 'see' what you're hearing in real time and as a comparison to the exact translation of the ones and zeroes through screen capture after the fact.
HTTYD, one of the scenes used at the GTG and one of the most difficult scenes of all time to reproduce at reference level, digital vs mic'd at the LP:
Irene is an excellent example because, in addition to frequency response linearity, bandwidth, compression (the usual suspects) it clearly exposes any and all even order harmonic distortion from all of the fundamentals of the effect, and you can even fairly approximate the total even order harmonic distortion. This, IMO, goes a lot longer toward identifying nuance differences in sonic signature than torturing a sub with a slow-burning sine sweep in a parking lot:
Looking at the Hulk Car bashing scene, here is a full BW, 10 Hz roll off and 20 Hz roll off, which is a close approximation to what the results would have been at the GTG. The middle graph would approximate the SM and the bottom graph, the CHT and Cap stuff. But, the dip in the teens seen in the in-room response would have shown up vs the digital:
It's data. It goes further to explaining subjective comments. It shows a lot of things in a single graph. Yates is cool as it gets and the nod goes to him. But, I've always believed that in-room data is more important than outdoor torturing, which only reveals how much torture it takes to get a confession.