I usually just use my bluray player or TV for that but Vudu is the only way to stream if you cant get or dont want to wait for the bluray. I tried it once on the 360 just to see how it compared and all seemed fine to me.
I don't know why it would be different on yours, but I've rented Ted and Brave both this weekend in HDX on my 360. I think we're planning on watching The Dark Knight Rises today and it'll be in HDX.
I previously used my ps3 for Vudu but sold it since I wasn't using it for anything else. When I did I made sure I could get Vudu on my 360 the same way and did read some vague stuff about recent problems for some people with it. When I first tried downloading the app I kept getting error messages that day. I tried again a few days later and it downloaded fine and I've watched several movies using it since then.
Maybe try deleting the app and redownloading it?
here's a crummy camera phone pic I just took of my 360 giving me the HDX option.
I can confirm I can get HDX movies through my XBOX 360. You have to go to the app store and download the app. Once you have the app you can pick SD, HD or HDX for almsot all the movies (all the new releases offer HDX). I really don't find the quality to be better or worse than the Microsoft HD movies in 1080P (ZUNE HD). On my 110 inch screen Blu Ray is still king but these 1080P downloads are pretty high quality and are impressive.
I can also confirm that the Xbox can stream movies from Vudu in HDX. I have a library of over 100 movies in HDX and an currently watching one now. The only advise I can suggest is either checking the video output on the console itself or possibly switching out your hdmi cable.