I have both. In my opinion it makes no sense to get the X if you have no interest in playing games. I have used my S almost exclusively for 4K and Atmos, and have had no issues with any noise. I don't usually play games on the S, but even when I do, I still don't hear the machine. Kids have been playing Destiny and Fortnite, NBA 2K periodically when their friends come over, and I have played Horizon 2, and Killer Instinct on the S, but never for extended times, so I cannot comment on if the fan turns on with stress, but I don't think the system is stressed at all by playing 4K. The machine is literally silent.
If you have any inclination to play games, and want to play on a 4K device to perhaps 'showcase' your system, then consider the X, but otherwise skip the X it is not worth it as a dedicated 4K player over the S. As far as I know there is no difference between S and X for 4K, HDR, Atmos, and DV for movies and streaming.
The S is an absolute steal as a 4K player and gaming machine, but the whole point of the X is for 4K gaming. Black Friday deals can probably get you an S for under $200. The X will be twice the price for no difference in 4K movies.
Xbox and PS3 Gamertag = Xerostomia
Wii = I'm too lazy to get the number