Here’s the image that inspired this post:
Now, there’s an awful lot of political catnip in that post, but I’m actually going to ignore it. So, if you want to hate on Captain Marvel or defend Captain Marvel: this is not the post for you. I want to talk about an apolitical disagreement I have with this perspective.
The underlying idea of this argument is that you should rate a movie based on how good or bad it is in some objective, cosmic sense. Or at least based on something other than how you felt about the movie. In this particular case, you should rate the movie based on some political ideal or in such a way as to promote the common good. Or something. No, you shouldn’t. ALl of these approaches are bad ideas.
The correct way to rate a movie–or a book, or a restaurant, etc.–is to just give the rating that best reflects how much joy it brought you. That’s it!
Let’s see if I can convince you.
To begin with, I’m not saying that such a thing as objective quality doesn’t exist. I think it probably does. No one can really tell where subjective taste ends and objective quality begins, but I’m pretty sure that “chocolate or vanilla” is a matter of purely personal preference but “gives you food poisoning or does not” is a matter of objective quality.
So I’m not trying to tell you that you should use your subjective reactions because that’s all there is to go on. I think it’s quite possible to watch a movie and think to yourself, “This wasn’t for me because I don’t like period romances (personal taste), but I can recognize that the script, directing, and acting were all excellent (objective quality) so I’m going to give it 5-stars.”
It’s possible. A lot of people even think there’s some ethical obligation to do just that. As though personal preferences and biases were always something to hide and be ashamed of. None of that is true.
The superficial reason I think it’s a bad idea has to do with what I think ratings are for. The purpose of a rating–and by a rating I mean a single, numeric score that you give to a movie or a book, like 8 out of 10 or 5 stars–is to help other people find works that they will enjoy and avoid works that they won’t enjoy. Or, because you can do this, to help people specifically look for works that will challenge them and that they might not like, and maybe pass up a book that will be too familiar. You can do all kinds of things with ratings. But only if the ratings are simple and honest. Only if the ratings encode good data.
The ideal scenario is a bunch of people leaving simple, numeric ratings for a bunch of works. This isn’t Utopia, it’s Goodreads. (Or any of a number of similar sites.) What you can then do is load up your list of works that you’ve liked / disliked / not cared about and find other people out there who have similar tastes. They’ve liked a lot of the books you’ve liked, they’ve disliked a lot of the books you’ve disliked, and they’ve felt meh about a lot of the books you’ve felt meh about. Now, if this person has read a book you haven’t read and they gave it 5-stars: BAM! You’re quite possibly found your next great read.
You can do this manually yourself. In fact, it’s what all of us instinctively do when we start talking to people about movies. We compare notes. If we have a lot in common, we ask that person for recommendation. It’s what we do in face-to-face interactions. When we use big data sets and machine learning algorithms to automate the process, we call them recommender systems. (What I’m describing is the collaborative filtering approach as opposed to content-based filtering, which also has it’s place.)
This matters a lot to me for the simple reason that I don’t like much of what I read. So, it’s kind of a topic that’s near and dear to my heart. 5-star books are rare for me. Most of what I read is probably 3-stars. A lot of it is 1-star or 2-star. In a sea of entertainment, I’m thirsty. I don’t have any show that I enjoy watching right now. I’m reading a few really solid series, but they come out at a rate of 1 or 2 books a year, and I read more like 120 books a year. The promise of really deep collaborative filtering is really appealing if it means I can find is valuable.
But if you try to be a good citizen and rate books based on what you think they’re objective quality is, the whole system breaks down.
Imagine a bunch of sci-fi fans and a bunch of mystery fans that each read a mix of both genres. The sci-fi fans enjoy the sci-fi books better (and the mystery fans enjoy the mystery books more), but they try to be objective in their ratings. The result of this is that the two groups disappear from the data. You can no longer go in and find the group that aligns with your interests and then weight their recommendations more heavily. Instead of having a clear population that gives high marks to the sci-fi stuff and high-marks to the mystery stuff, you just have one, amorphous group that gives high (or maybe medium) marks to everything.
How is this helpful? It is not. Not as much as it could be, anyway.
In theoretical terms, you have to understand that your subjective reaction to a work is complex. It incorporates the objective quality of the work, your subjective taste, and then an entire universe of random chance. Maybe you were angry going into the theater, and so the comedy didn’t work for you the way it would normally have worked. Maybe you just found out you got a raise, and everything was ten times funnier than it might otherwise have been. This is statistical noise, but it’s unbiased noise. This means that it basically goes away if you have a high enough sample.
On the other hand, if you try to fish out the objective components of a work from the stew of subjective and circumstantial components, you’re almost guaranteed to get it wrong. You don’t know yourself very well. You don’t know for yourself where you objective assessment ends and your subjective taste begins. You don’t know for yourself what unconscious factors were at play when you read that book at that time of your life. You can’t disentangle the objective from the subjective, and if you try you’re just going to end up introducing error into the equation that is biased. (In the Captain Marvel example above, you’re explicitly introducing political assessments into your judgment of the movie. That’s silly, regardless of whether your politics make you inclined to like it or hate it.)
What does this all mean? It means that it’s not important to rate things objectively (you can’t, and you’ll just mess it up), but it is helpful to rate thing frequently. The more people we have rating things in a way that can be sorted and organized, the more use everyone can get from those ratings. In this sense, ratings have positive externalities.
Now, some caveats:
Ratings vs. Reviews
A rating (in my terminology, I don’t claim this is the Absolute True Definition) is a single, numeric score. A review is a mini-essay where you get to explain your rating. The review is the place where you should try to disentangle the objective from the subjective. You’ll still fail, of course, but (1) it won’t dirty the data and (2) your failure to be objective can still be interesting and even illuminating. Reviews–the poor man’s version of criticism–is a different beast and it plays by different rules.
So: don’t think hard about your ratings. Just give a number and move on.
Do think hard about your reviews (if you have time!) Make them thoughtful and introspective and personal.
Misuse of the Data
There is a peril to everyone giving simplistic ratings, which is that publishers (movie studios, book publishers, whatever) will be tempted to try and reverse-engineer guaranteed money makers.
Yeah, that’s a problem, but it’s not like they’re not doing that anyway. The reason that movie studios keep making sequels, reboots, and remakes is that they are already over-relying on ratings. But they don’t rely on Goodreads or Rotten Tomatoes. They rely on money.
This is imperfect, too, given the different timing of digital vs. physical media channels, etc. but the point is that adding your honest ratings to Goodreads isn’t going to make traditional publishing any more likely to try and republish last years cult hit. They’re doing to do that anyway, and they already have better data (for their purposes) than you can give them.
Ratings vs. Journalism
My advice applies to entertainment. I’m not saying that you should just rate everything without worrying about objectivity. This should go without saying but, just in case, I said it.
You shouldn’t apply this reasoning to journalism because one vital function of journalism for society is to provide a common pool of facts that everyone can then debate about. One reason our society is so sadly warped and full of hatred is that we’ve lost that kind of journalism.
Of it’s probably impossible to be perfectly objective. The term is meaningless. Human beings do not passively receive input from our senses. Every aspect of learning–from decoding sounds into speech to the way vision works–is an active endeavor that depends on biases and assumptions.
When we say we want journalists to be objective, what we really mean is that (1) we want them to stick to objectively verifiable facts (or at least not do violence to them) and (2) we would like them to embody, insofar as possible, the common biases of the society they’re reporting to. There was a time when we, as Americans, knew that we had certain values in common. I believe that for the most part we still do. We’re suckers for underdogs, we value individualism, we revere hard work, and we are optimistic and energetic. A journalistic establishment that embraces those values is probably one that will serve us well (although I haven’t thought about it that hard, and it still has to follow rule #1 about getting the facts right). That’s bias, but it’s a bias that is positive: a bias towards truth, justice, and the American way.
What we can’t afford, but we unfortunately have to live with, is journalism that takes sides within the boundaries of our society.
Strategic Voting
There are some places other than entertainment where this logic does hold, however, and one of them is voting. One of the problems of American voting is that we go with majority-take-all voting, which is like the horse-and-buggy era of voting technology. Majority-take-all voting is probably much worse for us than a 2-party system, because it encourages strategic voting.
Just like rating Captain Marvel higher or lower because your politics make you want it to succeed or fail, strategic voting is where you vote for the candidate that you think can win rather than the candidate that you actually like the most.
There are alternatives that (mostly) eliminate this problem, the most well-known of which is instant-runoff voting. Instead of voting for just one candidate, you rank the candidates in the order that you prefer them. This means that you can vote for your favorite candidate first even if he or she is a longshot. If they don’t win, no problem. Your vote isn’t thrown away. In essence, it’s automatically moved to your second-favorite candidate. You don’t actually need to have multiple run-off elections. You just vote once with your full list of preferences and then it’s as if you were having a bunch of runoffs.
There are other important reasons why I think it’s better to vote for simple, subjective evaluations of the state of the country instead of trying to figure out who has the best policy choices, but I’ll leave that discussion for another day.
Limitations
The idea of simple, subjective ratings is not a cure-all. As I noted above, it’s not appropriate for all scenarios (like journalism). It’s also not infinitely powerful. The more people you have and the more things they rate (especially when lots of diverse people are rating the same thing), the better. If you have 1,000 people, maybe you can detect who likes what genre. If you have 10,000 people, maybe you can also detect sub-genres. If you have 100,000 people, maybe you can detect sub-genres and other characteristics, like literary style.
But no matter how many people you have, you’re never going to be able to pick up every possible relevant factor in the data because there are too many and we don’t even know what they are. And, even if you could, that still wouldn’t make predictions perfect because people are weird. Our tastes aren’t just a list of items (spaceships: yes, dragons: no). They are interactive. You might really like spaceships in the context of gritty action movies and hate spaceships in your romance movies. And you might be the only person with that tick. (OK, that tick would probably be pretty common, but you can think of others that are less so.)
This is a feature, not a bug. If it were possible to build a perfect recommendation it would also be possible to build (at least in theory) an algorithm to generate optimal content. I can’t think of anything more hideous or dystopian. At least, not as far as artistic content goes.
I’d like a better set of data because I know that there are an awful lot of books out there right now that I would love to read. And I can’t find them. I’d like better guidance.
But I wouldn’t ever want to turn over my reading entirely to a prediction algorithm, no matter how good it is. Or at least, not a deterministic one. I prefer my search algorithms to have some randomness built in, like simulated annealing.
I’d say about 1/3rd of what I read is fiction I expect to like, about 1/3rd is non-fiction I expect to like, and 1/3rd is random stuff. That random stuff is so important. It helps me find stuff that no prediction algorithm could ever help me find.
It also helps the system over all, because it means I’m not trapped in a little clique with other people who are all reading the same books. Reading outside your comfort zone–and rating them–is a way to build bridges between fandom.
So, yeah. This approach is limited. And that’s OK. The solution is to periodically shake things up a bit. So those are my rules: read a lot, rate everything you read as simply and subjectively as you can, and make sure that you’re reading some random stuff every now and then to keep yourself out of a rut and to build bridges to people with different tastes then your own.
Blink 182 – Enema of the State: I went through a huge Blink 182 phase through middle school and into my freshman year of high school. Aside from some radio play (“
Incubus – Make Yourself: During one of our family vacations, my older sister Tori let me listen to her copy of Make Yourself by Incubus. I’d heard some of their hits on the radio, but this was the first time going through the entire album. I fell in love with it to the point that my sister just gave it to me. Brandon Boyd’s vocals and the somewhat unique twist on 90s alternative rock stood out to me as did its ability to capture my various teenage emotions, from angst to puppy love to a desire for self-direction. Their follow-up albums during my high school years–Morning View and A Crow Left of the Murder–received even more constant rotation than Make Yourself, but it was this album that began my still ongoing love affair with Incubus and their talent for both capturing my emotional states and transporting me to new ones. Below is the video for their song “Stellar.”
Metallica – Ride the Lightning: I had heard Metallica growing up. Who hasn’t heard “
Pink Floyd – Dark Side of the Moon: My brother-in-law JC has been a guitar player since he was a teenager (at least). Whenever we would visit my sister, he would always go through his ginormous digital collection of music in hopes of educating me out of my Blink 182 phase and moving me beyond Metallica. He first used David Gilmour’s 
Led Zeppelin – Led Zeppelin II: I originally bought Led Zeppelin II for my dad for his birthday(?) one year at the suggestion of my mom. I wasn’t very familiar with Led Zeppelin at the time and even though I thought “Whole Lotta Love” was pretty cool, it didn’t peak my interest all that much. However, after picking up the guitar and shifting away from pop punk bands, I started “borrowing” (i.e., making a permanent part of my personal collection) Led Zeppelin II from my dad. While I acquired The Best of Led Zeppelin: Early & Latter Days, Vol. 1 & 2, it was this album that made me really appreciate the bluesy elements of rock. It felt like a bridge between the old and the new, between traditional blues and modern rock. And everyone was amazing: Plant’s vocals, Page’s guitar, Jones’ bass, and Bonham’s drums. Hard to find a band in which every member is of the highest caliber. And yes: I still prefer it to Led Zeppelin IV. You can see them performing “Whole Lotta Love” live from the Led Zeppelin DVD I watched consistently in my later years of high school.
Pearl Jam – Ten: I used to hate Pearl Jam. I remember revealing my dislike of them to a bass player friend my freshman/sophomore year and he was flabbergasted that a guitar player would not like them. “But they’re such good musicians!” he protested. I don’t know what it was about them. Maybe it was Eddie Vedder’s voice (a co-worker of mine once described him as sounding like a man singing in a freezer). Maybe it was the flannel. My suspicion is that I just had not been properly exposed to them beyond “
Rush – Moving Pictures: My first introduction to Rush was their video for “Time Stand Still” early one weekday morning on VH1. The 
Alice in Chains – Dirt: Lucky for me, my YM/Scout leader for the longest time was also one of my best friend’s dad. While I was listening to Blink 182, my friend (due to his dad’s influence) was listening to the likes of Led Zeppelin, Pink Floyd, and even the 
Fleetwood Mac – Rumours: For Christmas one year I received a year-long subscription to Guitar World magazine. In one of the issues, it featured a kind of boxing bracket for guitarists, rating them on a 0-5 scale on things like chops, influence, creativity, etc. Unfortunately, my mother trashed all of my Guitar World issues while I was on my mission (I’m still not sure why), so I’m unable to reference it properly. But at the time, I used the bracket to learn about guitarists I had never heard of before. At one point, I came across the name Lindsey Buckingham with something like a 3.7 in chops. When I discovered that he was the guitarist/co-vocalist of Fleetwood Mac, I remembered that my mom had Rumours in her van. I promptly borrowed the album and began soaking in Buckingham’s fingerpicked style. The album remained in constant rotation along with their (then) new album Say You Will. The mix of male and female vocals gave it a more diverse sound than I was used to and my enjoyment of Rumour‘s more pop-oriented style helped expand my musical palate. You can see their performance of “The Chain” from their live album The Dance (which I also spent of fair amount of time listening to) below.[ref]The Dance‘s renditions of “
Steve Ray Vaughan and Double Trouble – Texas Flood: In my first year of guitar playing (and therefore still in my pop punk phase), I had a Sunday School teacher who recognized that my fellow Blinkophile friend and I were “big into music.” One day, she held us after class and gave each of us a copy of SRV’s Texas Flood. Because we were guitar players, she knew we would appreciate SRV’s skills. In actuality, we both had a bit of an aversion to the album: it was straight blues and that just wasn’t us. We were “punk rockers” and SRV was definitely not that. Fast-forward a year or so. I was going through my CD collection and pulled our Texas Flood to give it another listen. I was floored: the tone, the bends, the precision. It was beautiful. While other artists (see Zeppelin and Floyd above) opened the door to a bluesier style, it was this album that solidified the blues in my book. It paved the way for my embrace of other blues guitarists like B.B. King, Albert King, Buddy Guy, Joe Bonamassa, Robben Ford, and others. Texas Flood is the reason that I earned the name “Blues Man” from a co-worker due to my Pandora picks. You can see SRV & Double Trouble performing my favorite track off the album–“Lenny”–below.
One of the most surprising things about Cixin Liu’s award-winning science fiction novel 
) that are even more haunting. The way that an author who is so willing to stare the political doublethink of the Cultural Revolution directly in the face has no qualms about positively describing the important role that political officers would play in a modern (presumably less totalitarian) Chinese military shows, to me, how deeply embedded some of the assumptions that led to the Cultural Revolution still remain. So many things that do not seem political to use are that way purely because nobody has bothered to politicize them. But physics, like anything else, can be and has been politicized.