H = non overclocking.
Z = overclocking possible, and marketed for gaming.
Just a quick note that many manufacturers enabled overclocking on H87 and H97 boards as well, Intel's specifications notwithstanding. Therefore the only practical difference for a lot of boards out there, is the PCIe configuration.
Starting with Wolfdale, Celerons and Pentiums are actually pretty darned capable. If not for motherboard failures, I'd still be happily running Celeron E3300s and Pentium E5200s on the HTPCs. Granted, for the extra $5-10, I usually opt for the Pentium.
I'd actually say any C2 processor, Conroe included, is fine in terms of compute-power. However using one of these is mainly only considered if the CPU (and RAM) is on-hand already. Building one by acquiring used parts is going a little too far back, IMO. In the C2 days compared to now, there's many board features missing (USB3, SATA 6Gbps, etc.) as well as the lack of usable built-in video/iGPUs, and the use of DDR2 memory.
Lynnfield & Clarkdale are probably as far back as anyone would want to go, building a machine from scratch, even using used parts. Of course those are pretty old too, so most people will probably instead opt for new components like Haswell-based Pentiums and i3s, with H81 or H87 boards.
The main advantage of using newer Core iX, etc. parts like Haswells are the fact that power consumption is a lot less; accordingly, there's less heat generated (and lesser cooling requirements).
However, for me Atom-based stuff are too little or just skating right on the edge of acceptable performance so I tend to steer clear of those.
I agree, and that goes for Celeron Js and Pentium Js as well, which Intel has managed to go and rename from Atom

However my needs are a lot more than Atom can deliver as well. For someone just doing video playback, BayTrail processors can certainly be capable.
Mind, peace of mind is exactly why I would go for i5 instead of i3. For a build that's gonna cost me $500 or so, I'd rather spend the extra $50 on a true quad than regret skimping on an i3. Intel's got pretty good power gating now anyway that at idle, both i3 and i5 should have similar power consumption.
Again, it depends on what you're using it for though. While I have an Lynnfield i7 HTPC myself, I currently have it dismantled, and my Clarkdale i3 machine is filling in and not really missing a beat. It can't game as well because it has a 7770 in it (opposed to the 7950 in the other one) but other than that I see no differences in "general computing". I'm quite sure a Haswell i3 will have
more than enough performance for a good lot of HTPC users.