Unless you have a structured media panel somewhere there is likely a multi-port splitter buried somewhere in a wall or the attic that is feeding to each of the F-connector wallplates in your bedrooms and living spaces.
That wall wart power plug could be any number of things from a step down transformer for some other piece of equipment to a power injector for a powered antenna somewhere or even a power tap for a satellite dish LNB. It may also be powering a splitter/amplifier somewhere in a wall or the attic.
Trace wires where you can. If in *any* doubt, pull your own wires.
Since you have already stated that you are pulling new Ethernet (please tell us it's CAT6 or better) you will want to have 2x RG6 drops per location anyway. Depending on the builder you might only have RG59 in the walls anyway.