Never thought twice about it. My ported DIY subs use good old pink R-19, but I've never noticed any fibers floating around the room or building up on furniture in a dedicated theater room.
From the photos I've seen, I'm pretty sure SVS uses batt insulation for stuffing/lining. I would expect most other mfgs do, too. Dickason's testing in the LDC showed fiberglass insulation to be as good or better than other options, including some of the supposedly purpose made products. On a bulk mfg basis, I imagine it's also much more cost effective.