On the 5-star rating system

and how it hurts more than it works.

Ratings and reviews are a frequent topic of conversation in my household. My partner (a triple Virgo, I tell people by way of explanation, even though I don’t really believe in all that) is obsessed with tracking every video game he plays, every movie he watches, and every book he reads, often by way of popular apps like Letterboxd and StoryGraph (but also through his own bespoke spreadsheets, because he is a nerd). Even if he’s not dishing out stars or long-winded summaries passed off as analyses—why do people do this?!—it’s inevitable that he’ll come across other people’s ratings or reviews as we step out of a movie theater or into a bookstore, and then we’ll talk about it, even though we almost always conclude that random strangers’ opinions about art should be taken with a grain of salt.

I’ve always been fairly ambivalent about reviews. I fancy myself a relatively independent thinker who can decide for herself what she enjoys, thank you very much, and unless a restaurant or book has something like 1 star out of 412 ratings (perhaps indicating that the restaurant has roaches or the author was blatantly racist), most scores fall somewhere in the middle, making everything look more or less the same. Still, I never really understood the tirades against the 5-star rating system. Sure, how we experience art is subjective, but how else are we supposed to share our opinions about quality?

I expected my feelings to change after publishing my first book; after all, it’s easy to feel indifferent about ratings when they have nothing to do with you or what you’ve made. Even now, though—stars having been gifted in my favor and wielded against it—I have less of a distaste for the 5-star system than a fascination in it. Because if there’s anything I’ve learned since my debut, it’s that people’s rating and review frameworks vary wildly from one individual to the next, rendering the whole thing pretty damn useless.

Yet we march on, tossing stars at everything from electric toothbrushes to other people’s magnum opuses.

I began this essay in early January. In the weeks since, I’ve gone from feeling relatively confident that I could defend the 5-star rating system from its haters to surprised that we insist on continuing to use it. At this point, I hope I can provide a little food for thought on how rating and review culture impacts how we engage with the world.

At the start of this year, I made a deal with myself: I will not check my own book’s ratings and reviews for six months.

“It’s tempting to be hawkishly online during the publication of your debut in order to gain some semblance of insight into the opaque, rarefied process that is mainstream publishing,” novelist Sharlene Teo writes in her essay “On Reception and Resilience” in Letters to a Writer of Color¹ (my favorite craft manual of all time). “I didn’t have the willpower to avoid my own reviews; hunting for an ego stroke, instead stumbling across some thorny critique or snarky comment that lingered in my consciousness for days afterward because it confirmed some insecurity I harbored about my own writing.”

When I read Teo’s essay a month after my authorial debut, so much resonated that I found myself highlighting what must have been half of the text. Even more delicious was the fact that this was something I wasn’t “supposed” to talk to people about (especially following the recent Elyse Myers drama) and therefore didn’t, instead chewing on my own aches and anxieties in private.

Because if promoting your own work is embarrassing, seeing it scored on a highly subjective scale is downright mortifying.

Actually, the most embarrassing thing was customizing this meme in the library while a staff member walked behind me

Tao continues:

“Surprisingly, it was not the one-star zingers that stung the most but the three-star ones, the readers who spent time with the book and at the end of it thought, ‘meh.’...I couldn’t separate readerly experience from the maelstrom of feelings and effort that went into writing that novel. Dismissals felt like not being seen.”

I share many of Tao’s feelings, though I’m not sure which is worse: not being seen, or being seen and the other person not liking what they’ve found. But even an ostensibly low star rating says little about whether someone likes or dislikes what they see, because everyone’s internal scale is so different.

My partner and I are not stingy with our stars. Why would we be? Those stars cost us nothing—we could give everything 5 stars, if we wanted, and some say we should—and if the point of the 5-star rating system is to reflect our personal experiences, we’re comfortable allowing it to fulfill that purpose. Meanwhile, throughout my years of using Goodreads and now StoryGraph, I’ve seen too many reviews to count that say something along the lines of Exactly what I was looking for! or Just what I needed, only to stick up a 3- or 4-star rating.

The prevalence of these types of reviews reminds me of videos I keep seeing about how the 5-star rating system works outside of my American norm. In Japan, for example, it’s apparently normal for people to rate what I’d consider a 5-star dining experience—great ambiance, a clean space, delicious food, friendly servers—3 or 3.5 stars. Frank Striegl, a Japan-born food blogger who runs ramen tours in Tokyo, says this is owed to Japanese culture’s emphasis on humility and indifference toward excessive praise. If someone rates a Japanese restaurant 3 stars, it generally means the dining experience went the way they wanted or expected it to go.

“A Japanese person on Google might write something along the lines of ‘I’ve been eating at this restaurant for 10 years and love it’...only to give them a 3 star review.” - Frank Striegl

So, cultural differences add another layer of complexity to the ostensibly “standard” or “simple” 5-star review system. Add personal perspectives on rating, and you could potentially meet 5 different people who all dish out their 5 stars in very different ways, even if they all somehow have an identical experience with a restaurant or a work of art.

But can we have identical experiences? Probably not. One of my favorite books, John Green’s The Anthropocene Reviewed, emphasizes how fundamentally personal star ratings are by attaching them to both concrete topics (Dr. Pepper, Super Mario Kart, air conditioning, etc) and nebulous concepts (including Googling strangers, plague, and our capacity for wonder). Every multi-page essay-turned-memoir weaves Green’s personal experiences in with the relevant topic before he declares his rating. And though this premise may sound a tad ridiculous, that’s by design; Green acknowledges that rating anything on a numerical scale is both an exercise in futility and a relatively new phenomenon.

Green cites his experience as a book reviewer at Booklist in the early 2000s, noting in his introduction that review culture has shifted dramatically over the past half-century:

“The five-star scale has only been used in critical analysis for the past few decades. While it was occasionally applied to film criticism as early as the 1950s, the five-star scale wasn’t used to rate hotels until 1979, and it wasn’t widely used to rate books until Amazon introduced user reviews. The five-star scale doesn’t really exist for humans; it exists for data aggregation systems, which is why it did not become standard until the internet era.”

Maybe aggregated star ratings were useful once upon a time (I’m not old enough to argue with Green on that point), but today, if you look at any given book, movie, restaurant, hotel, or consumer product, it likely features a score somewhere between 3 and 4.5 stars—AKA “good enough.” Because almost nothing is universally liked or hated, it’s rare to see anything with an aggregated star rating so high or low that it guarantees a particular experience.

Essayist Emily Mester, whose book American Bulk features an entire essay about why she no longer rates or reviews things online, notes the same:

“Anytime I considered a new bar or restaurant, even if I was standing right in front of it, I’d dutifully open [Yelp], squinting down at my little screen to determine whether the experience was worth having…But rarely did the experience match the aggregated rating on the restaurant’s Yelp page, which somehow always seemed to be four stars. Many people gave five stars indiscriminately…People don’t care about things in the same amounts.”

But, oh, do they care. A friend once told me that she was hesitant to join Goodreads or StoryGraph because she didn’t want to evaluate books on a 5-star scale as she read; she worried it would alter her perception. Not only do I suspect such a shift has happened within myself—though it’s unflattering for me to share, I’ll sometimes find myself thinking, Hmm, this one’s a 3-star for me long before a book is over—but Mester describes her dad following into a similar mentality on Yelp:

“...restaurants became his favorite pastime…He always knew when a new spot was opening, was eager to dissect the crispness of its side potatoes, the layout of its menu, the visual interest of its light fixtures. In the middle of a lackluster meal, my mom might comment on how the tables felt too crowded together, or the AC was cranked too high, or she’d note, during a good one, how expertly the steak had been seared, how warm and attentive the server had been, and she would turn to my dad and tell him he should put that in the review. They crafted their critique as they chewed.”

If there’s anything I resent about modern review culture, it’s that it’s now a sport to craft critiques for apps and websites that aggregate star ratings. I once felt that star ratings were fairly self-serving: I, for example, rate books on StoryGraph mostly so that I can observe trends in my reading tastes and see how I felt about a given book long after I’ve read it. But anyone who attaches a written review to their rating—myself included—knows they’re being watched. It’s why I don’t dare join Letterboxd; based on my limited experience, the app is an arena in which users compete to see who can leave the snarkiest one-line review, and that’s annoying as hell.

Like, this isn’t even funny

Unfortunately for me (an author who would like to think reviews are primarily a way for people to thoughtfully share their experiences with a work or product), similar priorities can be found among the users on other platforms, which are numerous, to say the least.

Green writes:

In the years since I’d been a book reviewer, everyone had become a reviewer, and everything had become a subject for reviews. The five-star scale was applied not just to books and films but to public restrooms and wedding photographers. The medication I take to treat my obsessive-compulsive disorder has more than 1,000 ratings at Drugs.com, with an average score of 3.8. A scene in the movie adaptation of my book The Fault in Our Stars was filmed on a bench in Amsterdam; that bench now has hundreds of Google reviews. (My favorite, a three-star review, reads in its entirety: “It is a bench.”)

If they wanted to, a person could spend the rest of their life reading ratings and reviews on virtually everything under the sun. The existence of those ratings and reviews isn’t necessarily a bad thing; we all have the right to share our thoughts and experiences. (This is the relatively weak argument I’d always come back to when I heard someone griping about the 5-star system.) But it’s difficult to trust that those data points mean anything at all when the scale is so subjective and reviews are often written for comedic purposes. And don’t even get me started on incentivized ratings and reviews, which are really just ads.

Maybe that isn’t the worst thing in the world, though. If star ratings and other quantitative measurements of quality were truly objective, there would be yet another avenue through which people could game the system by producing art (and restaurants and so on) in repetitive and frankly boring ways, all for the purpose of fulfilling some unshifting list of criteria. And who would get to decide what goes on that list, anyway?

I’m both comforted and left a little bit wanting by Tao’s conclusion to her essay, which reads:

“I don’t think I will ever write a book that everyone loves. And I’ve made peace with the fact that I can’t please everyone and that the story-shaped impulses and ideas I harbor will never live up to their reality.”

It is true that none of us will ever make anything that is universally loved. But I wish that, faced with a genuinely endless quantity of spaces in which to share our opinions, the best-known channel responsible for communicating those opinions was a little bit clearer, a little more earnest. Faced with a cold and unaccommodating alternative, however, I suppose this is the best we’ve got.

What’s been inspiring me lately:

✰ People in Chicago and Minnesota who are using their car alarms to alert their neighbors that ICE is nearby.

✰ Learning that author Charlotte McConaghy wrote speculative fiction before moving into literary fiction, thus reminding me that it’s okay for our interests to shift!

✰ The Verge’s very bold choice to feature a “Best gas masks” round-up by features editor and lawyer Sarah Jeong on its homepage.

1. If you buy any of the books I mention using my links, Bookshop.org will give me and my favorite indie bookstore a small commission at no extra cost to you!