The Bias of MyAnimeList Scores

Not to be confused with “the bias of my anime list’s scores”, which would mean that it’s all my opinion anyway.

One reason why I’ve never signed up for a MyAnimeList (henceforth “MAL”) is the way users score. The reviews, the fact the site hasn’t gone back to full functionality after waiting for weeks and the other problems are a topic for another time, but for now, let’s go through why I have such problems with MAL scoring.

As I was checking up information about the upcoming SSSS.Gridman (note this was in August 2018), it already had a 6.7 out of 10. Unless people evaluated the premiere at Anime Expo, I don’t see how anyone could score it. Plus, even if people had done that, it wouldn’t be a proper evaluation of the series, since a large portion of the show hasn’t come out yet to any part of the world!

That’s just the tip of the iceberg though – MAL apparently counts 5 as “average”, whereas people based on MAL tend to score higher than that. As in, they won’t score under 6 unless they consider something “terrible”. It could be said that this is from school scoring, where 70+ out of 100 is seen as “acceptable or great”, but even then, having a small spectrum of scores, even if they are more heavily weighted towards one side of the scale, would be more suitable for the three or 5 step systems (i.e. smiley-face ratings, five star ratings or their equivalents). MAL only operates under a number out of 10 system, which means people should use the nuances of the in-between values to their advantage – something that’s been argued long before this post came out (in internet time, if not real time).

However, people use MAL scores as a “trustworthy” metric as to whether a show is “good”. Scores over 7 are seen as “healthy” scores which require at least checking out if the show is up your alley, while 6 and below are seen as “polarising” or just plain “bad”. However, when they’re assigned to things happening in real time (e.g. Satsuriku no Tenshi’s MAL score of 7.1 as of the 18th of August 2018), they’re subject to mass fluctuation, which is hardly a great way to check if a show is “good”. Furthermore, even if the community doesn’t like a show, it’s possible you might still like it – hatewatches, for instance.

Sidebar: This brings into question what anything from 6.1 to 6.9 stands for – it would presumably be that these decimals are where “good” is really filtered from “bad”.

Seasonal Prattle has some nice discussion on both that very topic and scores in general in regards to Planet With (which admittedly were part of the inspiration and fact-checking of this post, since the about page, a.k.a. “where the site’s definitions of the scores are meant to be”, was out of commission as of the time of posting), so if you want to read more about this topic, feel free to hop on over there…Make sure to finish reading this post first, though.

There is admittedly some grain of truth when it comes to scores of really horrible anime – Mars of Destruction, often cited in quests for horrible anime, has a score of 2.3 out of 10 (as of August 18th 2018) for one thing – but after a certain score threshold around 4 or 5 out of 10, these scores no longer hold any water.

Even with the mass exodus after the site became less active, these rating problems persist within MAL’s webpages. Thus, unless your challenge is to watch anime under/over/exactly at a specific MAL score, the advice would be to not just rely on one score for your entire PTW – make sure you get some recommendations outside MAL scores, use multiple sources for checking whether you’ll like a show if you’re unsure about it (and only want a score metric) and bring a healthy dose of critical thinking when you parse score information about shows that haven’t run their course yet. It sounds simple, but not everyone does it.


So what do you think about MAL scores? Are they completely untrustworthy and is there a better substitute for them, or are they only true up to a certain score?

 

8 thoughts on “The Bias of MyAnimeList Scores

Add yours

  1. I don’t pay any attention to scores as a number tells you nothing (given everyone has a different understanding of what the scale means anyway and MAL allows shows to be scored while airing and before they are finished). I read reviews instead and like to see reasons why someone liked something or didn’t. If a show is new and not finished, I tend to focus on genre, plot, or I just try an episode and see if I like it.

    Liked by 2 people

    1. Generally, unless the show is a comedy (which means any reading ahead might make jokes lose their element of surprise), I’ve found personal experience of the show tends to be a better indication of “like factor” than any given review. Comedy is rather hit and miss as a genre, though.

      Liked by 1 person

  2. I think with any aggregate sites, the numbers don’t mean much unless there’s opinions backing them up. It can be indicative of quality but it helps read reviews or try the work itself. I also think the number can only be indicative when the show actually finishes. You can’t fully judge something when it isn’t entirely available.

    Now, with regards to MAL specifically, I agree that there’s aspects of the scoring system that makes no sense to me. I get why it’s turned on while the show is airing but for an advanced screening? I agree, it’s too limiting and there’ll be too much bias when scoring is allowed at that time. The admins really should keep it turned off until the show actually premieres.

    Liked by 1 person

  3. I generally don’t rely on scores… Text reviews are more my thing. (Also I use Anime-Planet.)

    I post scores, but mostly to gather together shows of more-or-less the same quality. (While admitting that quality is a highly subjective thing, one man’s trash and all that.) If you’ve ever seen one of my season review posts, I almost always supplement the score with a comment. That lets me separate out the average-but-watchable from the average-but-never-again. At the upper end, for example, it lets me be more specific about why a show falls short of greatness while still being outstanding.

    Like

    1. That sounds like a good compromise.

      I use AniList, which isn’t perfect with its scores but tries to offset that by having a weighted average as well as a standard average, plus you can consult the notes of any public profile with the work you’re looking at if you want text opinions, so it’s pretty handy while making the same kinds of compromise.

      Liked by 1 person

      1. All that fiddling about with various averages doesn’t change the basic issue – GIGO.

        I don’t see any actual reviews on Ani-list (though I don’t use it so I may be missing something)… Just what look like tweets.

        Like

What do you think about this?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Start a Blog at WordPress.com.

Up ↑

%d bloggers like this: