The BalUn is to match the impedance of the video source to the cable, and to match the cable to the load.
Video is designed to used a coax cable, with a 75 Ohm characteristic impedance.
Cat5 is not coax, it's twisted pair, and it has a characteristic impedance of 100 Ohms...resulting in a mismatch. An impedance mismatch at high frequencies results in signal reflections, and the inherent loss of 'quality'.
Because Cat5 is used for high speed data, some people, who don't know any better, think it would be ideal for video frequencies as well....they're mistaken.