Way off base. In the outfield actually. Read up on lowest common denominator.
OK, here's the one thing you must remember about HDMI to do what you want to do. Just one thing... HDMI only allows for one video and one audio stream to be sent at any time over the HDMI cable.
Now, why is that so important? It sounds simple but it adds complications with your setup. If your source component can only send one audio and one video signal at a time and it is going to your projector/TV and your audio/video receiver, then the source can only send either stereo or multichannel. It can't send both types of audio at the same time because it can only send one audio stream at a time. So, what to choose?
Well, the HDMI spec requires the source to only send a signal that is compatible with all sinks that are connected. The only type of signal that would be compatible with every sink (the TVs, the projector, the AVR in your proposed case) is a stereo signal. It can never send multichannel because only one device of many devices that are hooked-up is multichannel capable. This is the lowest common denominator for HDMI and it prevents you from sending multichannel for compatibility reasons.
This is why people hook up their AVR first and then let the AVR send the video out. In that case, the AVR sends its information instead of the TV's information to the source and the source says, OK I see a multichannel device only.
There are matrix switchers, including Joe's excellent Octava models, that will allow to you "trick" the sources into thinking that only multichannel devices are present, but those types are more costly. In that case the receiver will get multichannel audio but the TVs will get no compatible audio.
So, I think your first change is to find a way to send the HDMI signal to your AVR first, assuming you don't need any audio with the TVs. Or, do it exactly like you wrote and accept that all sinks will only receive stereo. Depending upon your source components, there might be other ways, too. It becomes much easier if you have an OPPO BDP-93/95/103/105 Blu-Ray player.