Stupid Questions

They say there are no stupid questions, yet this is a story about a stupid question.

My friends and I were talking about Rotten Tomatoes, which is a review aggregator for movies. They read all the reviews and decide if they're positive or negative, then report the percentage of positive reviews. If it's over 65% or so they certify the movie as "fresh."

I was explaining that the number RT reports is biased if it's meant to predict which movies are "good." Now I didn't specify what I mean by good but I think the intuitive definition is something like "most (American) people liked it" and you could estimate the percentage p that do/did by forcing N random people to watch the movie and vote thumbs up or down. Critics aren't like typical movie-goers, they tend to systemically have different tastes, so the RT number is a biased estimator of p.

The counterpoint is that, although it's rarely clearly stated, is that what makes things good or bad are some objective structural properties, things like complexity, pacing, lighting, or narrative structure. Critics are important because they're good at understanding these things, and the Academy Awards isn't done by a poll of the country because only the Academy actually knows good from bad. If that doesn't sound stupid then try to articulate some structural properties and then explain why they're the right ones (I'll return with an analogy later).

One of my friends asked "well, isn't there just good and bad music, so why not with films?" I quickly noted that "no, there isn't just good and bad music." He didn't that was a good enough answer, because I "don't know enough about music." (Implicitly I think the assumption here is that if you study music for long enough you'll understand, by divine revaluation, "the" structural properties and, well presumably then craft the perfect song possible. No, that is not meant to have a mocking tone.) A friend who played in the high school band explained that . . . well he didn't, he just talked about what kind of music he likes. I guess God just revealed to him one day that whatever he likes is also the right preference for everyone else.

After a while a light-bulb went on and everyone admitted that RT is a bad predictor (relative to say Yahoo! Movies, Netflix or Flixster) of what movies everyone likes because of selection bias. But no one informed Flixster as, ironically, they print only the RT rating when you search movies on their app, and you have to click each one-by-one to see the Flixster ratings (which is a somewhat biased version of the poll described above).

Now we return to the structural properties question. You actually can articulate a lot of properties that make any kind of art good. You can do the same for properties that make people attractive, which is illustrative. We know height makes men more attractive (up until about 6'2'' in the U.S.) and high cheek bones make women attractive (obviously likewise up to a point), and symmetric faces make both sexes more attractive. But how do we know that? Because we asked people to rate faces and those traits are correlated with higher ratings. You see, the structural approach gets things backwards.