

Thread Tools 
Senior Member
Join Date: Mar 2009
Posts: 327
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 11
I recently rated one movie as 1 star on Amazon ("Inception"). Average rating for this movie is 4 stars.
My review generated several responses after which I started pondering about movie ratings on Amazon, Netflix, etc.
I found it very interesting that I couldn't find any movie with more than a several dozen reviews with average rating of 1 star. I don't even think I saw anything below 3 but I haven't looked all that much.
Isn't it interesting? Wouldn't you agree that some movies are just REALLY BAD? And there has to be some with 1 star rating.
Possible the explanation is that for any crappy movie there is enough of an audience that likes this particular one, they will watch it and rate it highly.
Another explanation is that some people just can't critically judge anything.
What do you think?
My review generated several responses after which I started pondering about movie ratings on Amazon, Netflix, etc.
I found it very interesting that I couldn't find any movie with more than a several dozen reviews with average rating of 1 star. I don't even think I saw anything below 3 but I haven't looked all that much.
Isn't it interesting? Wouldn't you agree that some movies are just REALLY BAD? And there has to be some with 1 star rating.
Possible the explanation is that for any crappy movie there is enough of an audience that likes this particular one, they will watch it and rate it highly.
Another explanation is that some people just can't critically judge anything.
What do you think?
Sponsored Links  
Advertisement


AVS Special Member
Join Date: Aug 2009
Location: Northwest
Posts: 1,701
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 12
Most of this is true. Movies, I feel should follow the normal distribution. That means if you were to pool all movies ever made into a big box and begin to draw them out separately, on average (on a 10 point scale), most movies you grab will be around a 5 or "average." Very few movies would be a 1, just as very few movies should be a 10. Imagine a bellshapped curve. There is also the concept of sample sizes which deals with the central limit theorem, but that may be getting over many heads here, but I'll explain if there's any interest.
Likelwise, I like how you brought up critical judging. Some people exhibit floor effects (rating things constantly low, such as giving a "1" to everything because of some personal or expectation reasoning) just as some people exhibit ceiling effects (rating things constantly high, giving "10" to everything). These effects cancel themselves out as the sample size increases, but I agree that people should be more analytical during rating, rather then rational. This doesn't mean you shouldn't give "10" or "1" to a movie, but you shouldn't be giving those ratings very often at all, otherwise you have an issue with your rating metrics.
It really isn't the manner of WHAT you rate something, but the manner of your internal consistancy of how you rate something. For instance, is "Inception" really a 1star as you've rated? That seems a bit low when taking into account what the movie offers and what the movie is, especially when compared to other "1" star movies. However, getting people to rate this way would lend us more accurate ratings, but then that's relying on people to be analytical and stable rather than overlyrational and irregular, which is definitely optimism. But, atleast there's some hope that people should be aware of metric inconsistencies and how they may be perhaps rating suboptimally.
To answer your question in a basic sense — there are plenty of low rated stinkers (23/10) movies at IMDB, just as there are plenty of highly rated epics (9+/10) at IMDB. IMDB I find very accurate, most of the time. For the times that I don't find it accurate, it's because of low sample sizes, which is indicative of the central limit theorem — The higher the sample size, the more it should resemble the population distribution (or the "theoretical" rating of what the movie should be). Sometimes, the issue can also be time, as movies that are just released typically have much higher ratings than what they will receive (due to the "buzz") given the variable of time. If you are confused, I can break this down further.
For instance, there is no way that this movie is a 5.2/10 (which denotes just about bone average): http://www.imdb.com/title/tt0082801/
Notice how there's only 3,479 votes, which is very little. Increase the votes to 200,000, and you may have something that resembles the population distribution, which I would bet to be MUCH higher than 5.2/10 (guessing in the mid 6s at least).
Your question can be answered by understanding normal distribution principles and sample sizes. So essentially, the prevalence of extremely low rated movies should be the same as the prevalence of extremely high rated movies, which is probably pretty accurate to what you'll find.
Likelwise, I like how you brought up critical judging. Some people exhibit floor effects (rating things constantly low, such as giving a "1" to everything because of some personal or expectation reasoning) just as some people exhibit ceiling effects (rating things constantly high, giving "10" to everything). These effects cancel themselves out as the sample size increases, but I agree that people should be more analytical during rating, rather then rational. This doesn't mean you shouldn't give "10" or "1" to a movie, but you shouldn't be giving those ratings very often at all, otherwise you have an issue with your rating metrics.
It really isn't the manner of WHAT you rate something, but the manner of your internal consistancy of how you rate something. For instance, is "Inception" really a 1star as you've rated? That seems a bit low when taking into account what the movie offers and what the movie is, especially when compared to other "1" star movies. However, getting people to rate this way would lend us more accurate ratings, but then that's relying on people to be analytical and stable rather than overlyrational and irregular, which is definitely optimism. But, atleast there's some hope that people should be aware of metric inconsistencies and how they may be perhaps rating suboptimally.
To answer your question in a basic sense — there are plenty of low rated stinkers (23/10) movies at IMDB, just as there are plenty of highly rated epics (9+/10) at IMDB. IMDB I find very accurate, most of the time. For the times that I don't find it accurate, it's because of low sample sizes, which is indicative of the central limit theorem — The higher the sample size, the more it should resemble the population distribution (or the "theoretical" rating of what the movie should be). Sometimes, the issue can also be time, as movies that are just released typically have much higher ratings than what they will receive (due to the "buzz") given the variable of time. If you are confused, I can break this down further.
For instance, there is no way that this movie is a 5.2/10 (which denotes just about bone average): http://www.imdb.com/title/tt0082801/
Notice how there's only 3,479 votes, which is very little. Increase the votes to 200,000, and you may have something that resembles the population distribution, which I would bet to be MUCH higher than 5.2/10 (guessing in the mid 6s at least).
Your question can be answered by understanding normal distribution principles and sample sizes. So essentially, the prevalence of extremely low rated movies should be the same as the prevalence of extremely high rated movies, which is probably pretty accurate to what you'll find.
AVS Addicted Member
Join Date: Oct 2006
Location: Illinois, East of St Louis
Posts: 11,419
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 656 Post(s)
Liked: 943
Different strokes for different folks ..
Personally, I prefer the reviews at rottentomatoes ..
Personally, I prefer the reviews at rottentomatoes ..
Uncle Willie
Oddball: Why don't you knock it off with them negative waves? Why don't you dig how beautiful it is out here? Why don't you say something righteous and hopeful for a change?
Senior Member
Join Date: Mar 2009
Posts: 327
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 11
Quote:
Originally Posted by FendersRule
Most of this is true. Movies, I feel should follow the normal distribution. That means if you were to pool all movies ever made into a big box and begin to draw them out separately, on average (on a 10 point scale), most movies you grab will be around a 5 or "average." Very few movies would be a 1, just as very few movies should be a 10. Imagine a bellshapped curve. There is also the concept of sample sizes which deals with the central limit theorem, but that may be getting over many heads here, but I'll explain if there's any interest.
Likelwise, I like how you brought up critical judging. Some people exhibit floor effects (rating things constantly low, such as giving a "1" to everything because of some personal or expectation reasoning) just as some people exhibit ceiling effects (rating things constantly high, giving "10" to everything). These effects cancel themselves out as the sample size increases, but I agree that people should be more analytical during rating, rather then rational. This doesn't mean you shouldn't give "10" or "1" to a movie, but you shouldn't be giving those ratings very often at all, otherwise you have an issue with your rating metrics.
It really isn't the manner of WHAT you rate something, but the manner of your internal consistancy of how you rate something. For instance, is "Inception" really a 1star as you've rated? That seems a bit low when taking into account what the movie offers and what the movie is, especially when compared to other "1" star movies. However, getting people to rate this way would lend us more accurate ratings, but then that's relying on people to be analytical and stable rather than overlyrational and irregular, which is definitely optimism. But, atleast there's some hope that people should be aware of metric inconsistencies and how they may be perhaps rating suboptimally.
To answer your question in a basic sense — there are plenty of low rated stinkers (23/10) movies at IMDB, just as there are plenty of highly rated epics (9+/10) at IMDB. IMDB I find very accurate, most of the time. For the times that I don't find it accurate, it's because of low sample sizes, which is indicative of the central limit theorem — The higher the sample size, the more it should resemble the population distribution (or the "theoretical" rating of what the movie should be). Sometimes, the issue can also be time, as movies that are just released typically have much higher ratings than what they will receive (due to the "buzz") given the variable of time. If you are confused, I can break this down further.
For instance, there is no way that this movie is a 5.2/10 (which denotes just about bone average): http://www.imdb.com/title/tt0082801/
Notice how there's only 3,479 votes, which is very little. Increase the votes to 200,000, and you may have something that resembles the population distribution, which I would bet to be MUCH higher than 5.2/10 (guessing in the mid 6s at least).
Your question can be answered by understanding normal distribution principles and sample sizes. So essentially, the prevalence of extremely low rated movies should be the same as the prevalence of extremely high rated movies, which is probably pretty accurate to what you'll find.
Most of this is true. Movies, I feel should follow the normal distribution. That means if you were to pool all movies ever made into a big box and begin to draw them out separately, on average (on a 10 point scale), most movies you grab will be around a 5 or "average." Very few movies would be a 1, just as very few movies should be a 10. Imagine a bellshapped curve. There is also the concept of sample sizes which deals with the central limit theorem, but that may be getting over many heads here, but I'll explain if there's any interest.
Likelwise, I like how you brought up critical judging. Some people exhibit floor effects (rating things constantly low, such as giving a "1" to everything because of some personal or expectation reasoning) just as some people exhibit ceiling effects (rating things constantly high, giving "10" to everything). These effects cancel themselves out as the sample size increases, but I agree that people should be more analytical during rating, rather then rational. This doesn't mean you shouldn't give "10" or "1" to a movie, but you shouldn't be giving those ratings very often at all, otherwise you have an issue with your rating metrics.
It really isn't the manner of WHAT you rate something, but the manner of your internal consistancy of how you rate something. For instance, is "Inception" really a 1star as you've rated? That seems a bit low when taking into account what the movie offers and what the movie is, especially when compared to other "1" star movies. However, getting people to rate this way would lend us more accurate ratings, but then that's relying on people to be analytical and stable rather than overlyrational and irregular, which is definitely optimism. But, atleast there's some hope that people should be aware of metric inconsistencies and how they may be perhaps rating suboptimally.
To answer your question in a basic sense — there are plenty of low rated stinkers (23/10) movies at IMDB, just as there are plenty of highly rated epics (9+/10) at IMDB. IMDB I find very accurate, most of the time. For the times that I don't find it accurate, it's because of low sample sizes, which is indicative of the central limit theorem — The higher the sample size, the more it should resemble the population distribution (or the "theoretical" rating of what the movie should be). Sometimes, the issue can also be time, as movies that are just released typically have much higher ratings than what they will receive (due to the "buzz") given the variable of time. If you are confused, I can break this down further.
For instance, there is no way that this movie is a 5.2/10 (which denotes just about bone average): http://www.imdb.com/title/tt0082801/
Notice how there's only 3,479 votes, which is very little. Increase the votes to 200,000, and you may have something that resembles the population distribution, which I would bet to be MUCH higher than 5.2/10 (guessing in the mid 6s at least).
Your question can be answered by understanding normal distribution principles and sample sizes. So essentially, the prevalence of extremely low rated movies should be the same as the prevalence of extremely high rated movies, which is probably pretty accurate to what you'll find.
You bring very good points, especially higher ratings for recently released movies. To clarify: ratings in my original post are from 1 to 5.
Regarding the movie Neighbours: do you mean it's worth more than 5.2 out of 10? I haven't seen it. Seems it is not available from Netfilx at all, which is surprising. It is, however, available for streaming from Amazon.
Member
Join Date: Dec 2008
Location: Toronto, Canada
Posts: 75
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Neighbors at 5.2 on IMDB? Too high.
Bill
AVS Special Member
Join Date: Aug 2009
Location: Northwest
Posts: 1,701
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 12
You mean too low. Inception at "9" is too high, which is why it will continue to fall, especially from the "9.3" it was at a few months ago.
And while "Inception" falls, "Neighbors" and other rarities will continue to grow as more people see them (sample size increases).
Joeschmoe Neighbors is kinda a rare treat in dark comedy. It's purely a fantasy movie, almost like a nightmarishcartoon right down to the cartoonish soundtrack. It's about a 24hour journey that is filled with practical jokes that involves an imposing "wild" family that seems to live strictly in the moment, and a laidback conservative family. Fantastic roles by Belushi and Aykroyd, with some highly funny lines (they swapped roles right before filming). There is NO other movie out there like Neighbors, and everyone that I personally know that has seen it raves about it (except my friend's girlfriend).
As long as you are expecting a liveaction cartoon set in a moody surreal world defined by John G. Avildsen, then you should be enjoying yourself. I think people that don't like it simply didn't understand what it was trying to do being a live action cartoon. Anyhow, this isn't a thread about "Neighbors," that thread can be found here: http://www.avsforum.com/avsvb/showt...php?p=18959137
The DVD is only available in the UK. The Amazon or iTunes offering though, is pretty good.
AVS Special Member
Join Date: Jan 2006
Location: Charlotte
Posts: 1,401
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
The higher the sample size, the more it should resemble the population distribution (or the "theoretical" rating of what the movie should be)
The higher the sample size, the closer the mean of the sample should be to the mean of the population distribution (the standard deviation is smaller). The central limit theorem says that the sample mean is distributed normally in regards to the population mean if the sample is large enough, and the chi square distribution describes the distribution of the sample standard deviation.
Basically, you are incorrect when you say that the sample more accurately describes the population distribution, it just better describes the mean of the population distribution. Also, the central limit theorem doesn't say anything about accurately describing the mean, that fact follows from the sample mean being an unbiased estimator for the population mean.
AVS Special Member
Join Date: Aug 2009
Location: Northwest
Posts: 1,701
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 12
Quote:
Originally Posted by sound dropouts
The higher the sample size, the closer the mean of the sample should be to the mean of the population distribution (the standard deviation is smaller). The central limit theorem says that the sample mean is distributed normally in regards to the population mean if the sample is large enough, and the chi square distribution describes the distribution of the sample standard deviation.
The higher the sample size, the closer the mean of the sample should be to the mean of the population distribution (the standard deviation is smaller). The central limit theorem says that the sample mean is distributed normally in regards to the population mean if the sample is large enough, and the chi square distribution describes the distribution of the sample standard deviation.
This is exactly what I said. It isn't just the mean either, it's the entire distribution which takes into account the mean (average), range, and standard deviation. The Chisquare distribution and Zscore distribution (which is a standardized function) describe the deviation distribution as well
What this means to people reading: If you expect every single "production" movie ever made to follow a normal distribution (which I believe) in that most of the density would fall around average, and very little density would be "perfect" or "terrible bad" then:
If you had chosen 10 movies at random from the entire pool of production movies (hypothetical)
vs
If you had chosen 1,000 movies at random from the entire pool of production movies (hypothetical)
Then the distribution of the 1,000 randomly selected movies would more closely resemble the hypothetical distribution. All this simply means in Layman's terms: Have large sample sizes, and include everyone's opinion and random.
Remember Sound Dropouts, that the theory of central limit theorem resides on the basis of random sampling as well. Without random sampling, then it's meaningless.
Quote:
Basically, you are incorrect when you say that the sample more accurately describes the population distribution, it just better describes the mean of the population distribution. Also, the central limit theorem doesn't say anything about accurately describing the mean, that fact follows from the sample mean being an unbiased estimator for the population mean.
When you have a small sample size, then your distribution is less likely to resemble the hypothetical distribution, which increases the likelihood of making a type I/II errors (also creating low power). I see no problem calling extremely small sample sizes an inaccurate depiction of a the hypothetical distribution. Just how Neighbor's 5.2/10 rating with barely over 1,000 votes is probably not an accurate depiction of what people probably think of this title, signifying a pretty small effect size. Maybe it is? Maybe it isn't? We really don't know until there's a higher sample. Most likely, the score would increase, especially as it builds cult status. "Nightbreed" would probably do the same, as well.
What all this means for the people listening:
Movies at review sites that have low samples (vote counts) may not be an accurate representation of what they should look like with higher votes. Then, we can move on the topic of whether or not these votes are following random selection principles. Does everyone use one site to cast their vote (no)? What about people that don't have internet access (no)? So no, review sites do not follow random sampling procedures, and they are inaccurate in themselves. However, we can still use the framework to explain why it is that some votes seem rather lower/higher than others, sample sizes, and how much "grain of salt" you should take with all this.
AVS Special Member
Join Date: Jan 2006
Location: Charlotte
Posts: 1,401
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
then the mean of the 1,000 randomly selected movies would more closely resemble the hypothetical population distribution in theory (which is what Central Limit Theorem is — a theory). All this simply means in Layman's terms: Have large sample sizes, and include everyone's opinion.
this is where I disagree with you...though we might just have different definitions of the central limit theorem. The result is the same, but the central limit theorem ONLY says that the mean of the sample will be normally distributed NOT that the mean will necessarily be better with higher sample sizes. The mean WILL be better, but because of different theorems. (Unbiased estimators, etc).
Quote:
All this simply means in Layman's terms: Have large sample sizes, and include everyone's opinion.
Quote:
Movies at review sides that have low samples may not be an accurate representation of what they should be with higher votes.
This I agree with.
Newbie
Join Date: Jan 2003
Posts: 9
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
The problem is that internet ratings are skewed by the overly excitable individuals. They tend to be young, but not always.
So you'll get lots of "1: this movie sucks! It's the worse movie of all time! I hate it hate it hate it!" and even more "5: this movie is the greatest movie of all time! It should win 200 Oscars!" Those that actually rate movies in a reasonable manner get lost in the noise. On top of that, the people who vote are not really representative of the population at large. Lots of people don't vote in these ratings. Those that vote are mainly in the overly excitable group.
As far as critical judging, do you really think Inception deserves a 1? So you didn't like it, that's fine. But a 1 rating is the lowest rating, so it means that it's one of the worst movie that you've seen. Is it really? It does not deserve a 1 just as it does not deserve a five.
So you'll get lots of "1: this movie sucks! It's the worse movie of all time! I hate it hate it hate it!" and even more "5: this movie is the greatest movie of all time! It should win 200 Oscars!" Those that actually rate movies in a reasonable manner get lost in the noise. On top of that, the people who vote are not really representative of the population at large. Lots of people don't vote in these ratings. Those that vote are mainly in the overly excitable group.
As far as critical judging, do you really think Inception deserves a 1? So you didn't like it, that's fine. But a 1 rating is the lowest rating, so it means that it's one of the worst movie that you've seen. Is it really? It does not deserve a 1 just as it does not deserve a five.
Senior Member
Join Date: Mar 2009
Posts: 327
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 11
Quote:
Originally Posted by Cedric A
...
As far as critical judging, do you really think Inception deserves a 1? So you didn't like it, that's fine. But a 1 rating is the lowest rating, so it means that it's one of the worst movie that you've seen. Is it really? It does not deserve a 1 just as it does not deserve a five.
...
As far as critical judging, do you really think Inception deserves a 1? So you didn't like it, that's fine. But a 1 rating is the lowest rating, so it means that it's one of the worst movie that you've seen. Is it really? It does not deserve a 1 just as it does not deserve a five.
No, it's probably not the worst. 1 means that it wasn't worth watching for me. I'd rather stare at the wall for 2 hours.
Senior Member
Join Date: Oct 2006
Location: Farmington Hills MI
Posts: 313
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 19 Post(s)
Liked: 19
There has been more than one generation of general grade inflation in society as a whole. Many, if not most, of the raters were educated in a system that handed out top grades and awards for mediocre performance.
Also there has been a war against discrimination in the broad sense (see a dictionary) by a large segment of the population.
Getting back to the topic the ratings are semiworthless unless you habitually follow the herd. My best course of action is to check 1 or 2 1star and 5star ratings then a few in the 23 range to see what I might think of the movie. Yes it is time consuming but not as much as getting then watching 515 minutes before abandoning it.
Also there has been a war against discrimination in the broad sense (see a dictionary) by a large segment of the population.
Getting back to the topic the ratings are semiworthless unless you habitually follow the herd. My best course of action is to check 1 or 2 1star and 5star ratings then a few in the 23 range to see what I might think of the movie. Yes it is time consuming but not as much as getting then watching 515 minutes before abandoning it.
"Most people would die sooner than think, in fact they do so." Bertrand Russell The ABC of Relativity, 1925
Sponsored Links  
Advertisement


Thread Tools  
Show Printable Version Show Printable Version
Email this Page Email this Page


Posting Rules  