I could upgrade my NAS and run Plex Media Server off of that, but that would be expensive and less future-proof.
Asustor AS3 series, much cheaper x86 NAS then any of the competition runs Plex. The AS2 series are even cheaper but slower CPU, go with AS3.
That will give you transcoding support in Plex, but it wont be full transcoding as it is still a low power CPU, Plex can do some remuxing tricks to make mkv work better in iOS .
I could add a Mac Mini to run Plex Media Server and have it stream from my NAS. Could I have the Mac (running Windows) access the NAS via Ethernet? Or would this not allow transcoding..
Transcoding performance will be severely affected if pulling the media from a NAS, it needs to be on the local storage unit of whatever is transcoding. A Mac Mini has the CPU power for full transcoding, a DAS (NAS without network) like Lacie BigQuadra or Drobo would provide storage to the Mac.
I could build HTPCs to play my media from my NAS on XBMC.
If you can do it then this is the most flexible approach. No server upgrades required, get an Intel nuc or wait for the Lenovo Beacon announced at CES a commercial mini PC with XBMC (linux) pre installed like a media player.
Get the A410 and give up the tablet streaming. (Popcorn Hour also has a Plex app)
What you can do with the existing NAS is run the mkv files through MKVtoMP4
, this will repackage the video/audio into MP4 container making the media compatible with iOS native media player,.
So get the Arm version of Plex (no transcoding) available in the QNAP app center and see if the MP4 files work better on iOS or turn on DLNA server, install Skifta on iOS and play the MP4 that way.
The Arm version of Plex is not as fast as the x86 version so expect some delays in loading artwork on your Plex client.