With 8 bit precision there will be inevitably some small rounding errors.Here's a concrete example based on your settings, assuming Rec 709, 2.4 gamma, and a measured White of 100 nits. Your 60% gray, RGB triplet 153,153,153, within the 0-255 range, will have a target Y of 29.357 nits. Under the covers, the graphics card is rescaling that pattern to RGB triplet 147,147,147 (most likely) within the 16-235 range, so that's what's actually displayed and measured. That pattern would actually have a target Y of 29.133 nits, if you were to configure your calibration software for range 16-235 (and were using a pattern generator path that passed it unaltered to the display). So with 0-255 there's a mismatch between what the calibration software is calculating for targets, and the patterns that are actually being displayed and measured.
As mentioned previously, (Measuring Greyscale and Primary/Secondary Colours Using...) you will likely get more accurate results by having the software perform the scaling, but that involves overriding the EDID and is something I do not believe a “beginner” (the intended reader if the guide) should get into.
Setting the software to 16-235 can potentially lead to range mismatch errors which are far more serious than rounding errors. That is the default of HCFR and many new users have run into that error.
For the ultimate accuracy you will need to use a disc player with bit-perfect patterns (Ted’s Calibration Disc), or a pattern generator that’s verified to be “bit perfect”:
Finally I could satisfy my (and I hope your) curiosity to know which test pattern generator (or TPG) in the low price range is really accurate. To do this I bought a medium-high end TPG: DVDO AvLab TPG, which has inside a signal analyzer whose accuracy has been verified and confirmed with an...