You'll never do a 137" x 108" wall area with 2 quarts.
With light not being an issue, rotating the mix toward RS-MaxxMudd can gain you over 1.5 Quarts additional Volume, and that will be enough.
I did not see any RustOleum Matte Polyurethane, so that too will bulk up the mix. (...I wonder if that was the "24 oz. item you listed...? )
I've recently painted up two entire walls as Screens, both at / over 15' x 9', corner to corner...edge to edge. I can provide instructions as to how to apply paint "edge cut" deep into the corners and ceiling/wall edge without runs.
It's quite telling, you new preference for a edge less, adjustable size / format Screen. Why, even my Wife, who herself has seen literally dozens of my Screens, most all with Velvet Trim, clearly stated last night when looking at a 174" diag 2.39:1 image on a 15' x 9'-6" Edge less painted Screen wall, that she REALLY
preferred it over one with a border.
Of course choosing the correct color scheme is always an asset. In the Theater shown, the side walls and rear walls are Elephant Gray, the Screen is Black Flame X1I , and the Ceiling is color matched to the BFX1I. All Trim forward of the Bar is pained with the Ceiling Gray, only in a Oil based Satin.
The carpet is a extremely dark Charcoal Gray with Light Gray fibers, that mates up with the medium Gray Theater Chairs.
All Pictures taken sitting down, film in motion, from a distance of 10' from Screen Wall.