I have a mid level yamaha connected to my apple tv 3 via optical, and I have most of my music encoded in either apple lossless or 128kb mp3's, so some of the older recordings or lower bit rate songs sound a bit on the harsh side, something I was not getting with my sony es sacd player via the analog connections. So I'm wondering if I get a DAC that would perhaps solve my problem, or is it the old "garbage in garbage" out scenario
You don't mention which Yamaha you have, but I would assume that it has a very respectable DAC, as I don't know of a Yamaha that doesn't. A separate DAC would likely be a step down. How do your apple-lossless tracks sound?
128 kbps is awfully lossy for MP3, so the only real solution might be to replace them.