I've been around the finest, very high output systems in the world. Systems of this ilk, operating full tilt at a great outdoor venue, are glorius to experience. No ingress of smeared reflected early energy, no troublesome resonant build-up. Only the coherent, direct sound, and maybe some spacious late reflections that only add to the goodness ....
There is a physical point whereby our hering mechanism can take no more, that's not what were discussing here.
Aside from that physical limit, our tolerance of loudness is primarily based on distortions and non-linearities entering the equation.
Momentary peaks are extra-ordinarily demanding, sapping power supplies, and output stages, easily clipping amps in an manner most audio enthusiasts aren't even familiar with. These drive signals elicit magnetic non-linearities, compression and distortions and driver break-up. All types of distortions can occur prior to the diffraction on the face of the speaker and it's immeadiate surroundings ..... because aesthetic concerns trump the distorted nearfield acoustic of the front stage. Thus, instead of an ideal launch of energy toward the LP, we get multiple exposures ... smeared in time of the aforementioned upstream non-linearities.
In and around proaudio, it's recognized that a clean system, in the right environment, can approach phenomenal SPLs without one fully realizing it and the inherent warnings that typically we encounter in the home. These distortions and non-linearities, are what's greatly responsible for what forms our tolerance for how loud we lik it on the big end. It doesn't matter if these distortions occur before, or after the energy leaves the loudspeakers.
Most AVS'er systems seem pretty capable, but most rooms/set-up are garbage. Aesthetic concerns trump acoustic needs for many, .. I get it,.. a balance must be had. But I see more "Audio 101" fundamental errors made around here than I care to even think. These diffracted, smeared, soundstages (mostly with killer bass

) aren't as tolerable psychoacoustically, as a more ideal acoustic environment. Treatment as well, but I'm not talking treatment here, this is set up. But I do understand.
I'm not immune either, my room is quite modest, warts and all. My system doesn't encounter limitations often, I've got (4)18s, (4)15s, and Seaton Catalysts. The room is an ongoing acoustic experiment. With an open kitchen to the rear, it's anything but ideal

Thanks