I did some testing this week and thought I would provide my conclusions.
The bottom line is, unless you have a very compelling reason to use ATSC go with QAM. QAM works great with little or no issues. Of course, your mileage may very as some of this could be TV specific.
I ordered my two modulators as ATSC versions because I have 1 old set that doesn't have a QAM tuner, I thought channel changes might be smoother if every channel was of the same type (OTA and my modulated channels), and I did not think there would be an appreciable difference.
When I first set up my ATSC versions out of the box I drove them with a 1080p signal from a Blu-Ray concert DVD ... lots of motion, scene changes, camera panning, strobe lights, etc.
The manual said to set the bitrate to 16Mbit/sec, so I did. I started out with the Lowdelay on Mode 1. The TVs found the channel without issue and tuned to it. I had the original signal playing on one TV and the modulated signal playing on an identical 60" LG plasma a foot away.
The latency for mode 1 is around a second (maybe less). The concert was playing, but it looked like it had 80's music video special effects added to it... jerky motion that would slow down and then speed back up to catch up. Attempting to watch a basketball game was comical with that happening. I moved the delay to mode2 and things were better, but not fixed. So, finally I set the delay to normal. That fixed the motion issues. The latency however increased to around 3 seconds.
In this configuration (and in with the delay set to the other modes) I had random audio dropouts. So, I contacted PVI. I described the issue and also asked for updated software/firmware for ATSC as well as QAM.
The rep responded the same day... within about 2.5 to 3 hours and suggested that I set the bitrate to 14Mbps. I did. That seemed to fix the audio dropouts.
Comparing the original video to the encoded/decoded version showed what I thought was a significant quality loss. I could see a lot of compression artifacts in scenes with a lot of motion - the concert bluray was a sort of torture test for that.
PVI also provided me with updated software/firmware for both ATSC and QAM.
I first installed updated to the latest ATSC version. After doing so, I was able to increase the bitrate to 15 and then 16Mbps without the audio dropouts returning. That improved the compression artifacts, but the output still didn't really excite me.
So I installed QAM onto the other modulator... boy was that better. The manual indicates that the bitrate should be set to 19Mbps, so I did. The lowdelay mode1 worked much better with QAM. The overall video quality is much better as well - either due to the higher bitrate or superior processing in the TV for QAM. Comparing the two signals, the difference is best described as basically the difference between 1080p and 1080i.
There is more color/warmer color in the original - perhaps due to a wider color gamut. Otherwise, it's hard to see a difference - especially if you are feeding the box 1080i versus 1080p.
Note that the latest firmware includes the ability to set Lowdelay to "Manual" and then choose an incremental delay between 1 and 500. I don't think the firmware that my box came with had that option.
So... customer support is responsive, and the product works as advertised. If you need to distribute HD to a bunch of TVs, it's a workable solution that might be the right one depending on your circumstances.