Originally Posted by SyntheticShrimp
My intuition is that if there is any effect at all it would be small compared to the time domain effects from modal ringing in the room, but I'm sure Mr. Jones could give you a more complete answer.
The idea of small subwoofer drivers being "faster" than large ones is often stated, but is not true.
If by fast we mean greater acceleration, then this is certainly not true. Acceleration is governed by force and mass. Higher mass means more force needs to be applied to achieve a certain acceleration. This is not rocket science (well, actually this principle is used in rocket science, but...!). It simply reflects the obvious issue that higher mass drivers are less efficient in their passband.
The frequency response in the passband of a driver, before cone breakup, is governed only by force and mass, so with sufficient force you can get identical response. The transient or impulse response is simply a reflection of the frequency response, related by the Hilbert transform. A fast impulse response simply means that you have an extended frequency response. Now it may be easier to engineer a smaller woofer to have a better high frequency response, but this is irrelevant, because the high frequency response is of no concern, it's a subwoofer!! It will get filtered so that it does not reproduce high frequencies.
By definition then, the system defined as subwoofer and filter, will have a transient response limited by the bandwidth of the applied filter.
Of course, there is a further factor that effects "perceived" transient response, and why bigger woofers appear to be slower. Its because they nearly always go down lower in frequency, as it is often easier to get extended low frequency response out of a large woofer in a larger box, than from a small woofer in a small box.
The additional low frequencies will alter your perception of bass, and often make it appear to sound slower. This is a simple psychoacoustic effect, and is nothing to do with the subwoofer per se.
Music signals contain both fundamentals and harmonics in certain relationships. However, the relationships are not static with time. The fundamental may take longer to reach it's maximum level once excited than will some of the higher harmonics. The tonality will change in a complex manner with time. Now, if you reduce the level of the fundamental with respect to the harmonics, you will change this characteristic tonality signature, and often the music will sound faster and more upbeat as a result. You can do the same by filtering off the bass signals.
In fact, musicians recognize this phenomenal, of bass notes taking longer to "explode" relative to the plucking of the string on a string bass for example, and will alter their timing to compensate, the alteration changing as they go down to lower notes.
Add to all of this the effect of the room modes, and "fast bass" is not something to put at the top of your worry list!