Assuming you hooked everything up correctly...
Youtube doesn't (unless it changed recently) do 5.1. There are a number of tests on youtube for 5.1 surround, but that doesn't mean that youtube actually streams in 5.1. Youtube streams in stereo.
I'm also running an optical cable from my TV to my receiver (smart TV, I can play youtube, stream netflix, amazon instant video, play movies from my server, etc.). I can stream [compressed] audio in 5.1 through an optical cable with no problems (NOT from youtube. From Amazon instant video, or my file server through DLNA)
Back to the TV /settings/ -- For example if I'm playing a movie from my server, each time the source sound encoding changes (different movies, different encoding), I need to manually select whether it's DD or DTS (both 5.1), otherwise it defaults to 2-channel PCM. So in your TV audio, you need to check and make sure that you're sound output is /not/ defaulting to 2-channel PCM.
Assuming you have a cable box, you need to connect it with either HDMI or optical to your TV/receiver, otherwise those red/white RCA connectors will only do stereo. If you don't have a cable box, try watching something on TV that has 5.1 sound (I don't know if OTA can do 5.1), and set your TV to the proper sound output mode and see if the rear speakers work.
Also, how is the bluray connected to your receiver?
Originally Posted by davehvos
I tested this setup with blu-ray movie files on my pc, as well as youtube surround sound test vids, however I am not getting any sound played from these extra 3 speakers.
Opps, just noticed this. I think the culprit is the way the TV audio is being send to your receiver. With your equipment can you send the bluray movie from the computer straight to your receiver and see if you have 5.1 sound?