Invalid characterset or character set not supported What do you all think of a review scoring system change?





What do you all think of a review scoring system change?
April 25, 2015

I'm thinking it might be time to adjust how we rate games. We have had the 1-10 scale since the site's inception. Initially, it allowed for decimals. So you would see scores like 7.3 or whatever. A number of years ago, we did away with the decimals and only allowed whole numbers.

Now, I'm thinking we would go to a five-star system, so you would see a little image of five stars next to one another, with a one-word description to the right, like so:

***** = Terrible
***** = Poor
***** = Unremarkable
***** = Good
***** = Awesome

It would look a bit prettier than that, maybe, but you get the idea.

If implemented, the new scale would go into place immediately, and would impact all future reviews. I'm thinking I would also remove the option to edit scores on all reviews.

So, why am I proposing a 5-star system? Mostly, it's because I'm as sick of the score debate as anyone. While I think a general score can be useful, worrying too much about the numbers 1 through 10 is a distraction. Basically, games fall into one of the five categories outlined above, and if you worry much beyond that, you're just trying to establish a pecking order. That's what Top 10 lists are for, right?

If I'm thinking about buying a game, I want to know if it's good, great, mediocre, crummy, or just plain awful. I don't care if it got a 7 or an 8. I want to know if I'll enjoy it. And I think the proposed revision to our scoring system accomplishes that.

However, if the grading system changes, I'd like to think that it won't change again for years to come, if ever. So it's worth making sure this system is one the site's community can get behind. Do you like the proposed change? Let me know!

Most recent blog posts from Jason Venter...

Feedback
EmP EmP - April 25, 2015 (02:52 PM)
I'll give this some thought. I've long been a member of the 'nuke the scores all together' brigade but understand why that remains unrealistic. I'd love to never again have the argument that a seven doesn't magically become a five just because it's posted on the end of a videogame review, but I'm not sure stars will serve any better and I strongly dislike the one word summery. I liked it best when we could write our own custom summery to accompany and may never fully forgive you for nuking that without warning.
Germ Germ - April 25, 2015 (03:03 PM)
I for one, hate the 10 point scale. It always gives me trouble when reviewing. I strongly prefer play it/skip it (like the thumbs up or down movie review method) over anything else, but anything that reduces the degrees of differentiation is a step in the right direction. Explaining how good or bad a game is is the purpose of the text,in my opinion.
honestgamer honestgamer - April 25, 2015 (03:12 PM)
I think a difference between "good" and "great" (or whatever words a person might use internally) is worth recognizing as part of the scoring system, and that extends to "bad" and "awful" (or whatever) on the lower end of the spectrum. Besides that, we've all played games that left us lukewarm.

So it seems to me like really, there are about five useful degrees. After that, numbers translate to "but how good is it?" or "how great is it?" I can see value in arguing between good and great, but not between good+1 and good+2, if that makes any sense. That's my motivation to move away from the 10-point scale that seems to do that.

Besides allowing for perhaps too much precision, a 10-point scale tends to mean one of two distinct things to different people. It either mean you're secretly following the school grading system, where (in the United States, at least) 7 out of 10 translates to average and anything lower pretty much means you're a drooling idiot, or you're following common sense where 5/10 is average, middle-of-the-road. Devotees from the two groups LOVE battling it out, and review discussion quickly becomes an exercise in pedantry that benefits no one.

So anyway, I'm not married to the 5-star system if we implement a change, but I like it a lot. The one-word descriptions nicely summarize the experience a person is likely to have with a game, and even "unremarkable" is a warning (since so many of us have less time than we would like to spend with entertainment) without being an indictment. But a five-star system without the one-word descriptions could certainly also work. I'm open to suggestions, for sure, and I hope that people who have any suggestions to make will feel welcome to continue offering them here for my consideration and to contribute to the discussion.
Roto13 Roto13 - April 25, 2015 (03:28 PM)
I'm ok with either scale. Just never bring back decimal points.
EmP EmP - April 25, 2015 (03:28 PM)
where (in the United States, at least) 7 out of 10 translates to average and anything lower pretty much means you're a drooling idiot

I did not know that. Man, that suddenly makes so much make sense.

You lot are idiots.
overdrive overdrive - April 25, 2015 (04:32 PM)
To me, the idiots are the people who take the "well, that's how it is in school, so that's how I look at it for games" approach. Congrats, morons, you just created a scale where 60% of the grades are basically used to determine just how badly a company/designer failed and you only have four scores that can be used in order to rate a game from good to excellent. Nothing more than brain-damaged logic, that.

As for this, I think I personally favor the 10-point scale, but because so many people are stupid about how to do a 10-point scale, I wouldn't have any issue with going to a 1-5 star thing.
JoeTheDestroyer JoeTheDestroyer - April 25, 2015 (04:46 PM)
They set the average for school higher for a reason. Unfortunately, most American readers don't realize this and think 7/10 translates to average for everything. That would be why I've occasionally responded to some of our Disqus and Facebook comments from a few years ago with, "This isn't high school."
honestgamer honestgamer - April 25, 2015 (04:56 PM)
Yeah, basically. So what we can do is pretend we live in a world where most casual readers don't have that misguided expectation, or we can recognize the world we ACTUALLY live in and try to find a scoring solution that works more efficiently.

One approach sites have taken is letter grades, but to my mind, that converts too easily back into the 1/10 scale and removes none of the ambiguity. Some would argue that a 1-5 scale does the same thing, since you can easily multiply any number by 2 and suddenly you have a 10-point scale, but I still think it works better overall and a lot of people probably aren't going to bother with that math. The one-word explanations would also have helped make the score's meaning clear, but I can do without if that's what people want.
jerec jerec - April 25, 2015 (05:19 PM)
I usually just tack on a score at the end of my reviews because I have to, so this simplifies things.
joseph_valencia joseph_valencia - April 25, 2015 (06:59 PM)
I prefer a four star system. Letter grades are worse than a 1-10 scale, because then you open up a can of worms with the whole +/- thing.
JoeTheDestroyer JoeTheDestroyer - April 25, 2015 (11:42 PM)
I used to view ratings as a way of indicating what kind of review you're about to read. Lately, though, I've been of the opinion that they're more trouble than they're worth, especially when knuckle-draggers fret over a game's Metacritic or Gamerankings score. The worst is when people bellyache over a single damn point. I recall someone at N4G whining about Jason's Ocarina of Time 3D review because he gave it a 9 and not a 10. I'm still trying to wrap my mind around why a single point matters so much that people feel they have to be indignant pricks and completely ignore the body of text. And I won't even get into the developers' end of it. I think I remember reading about PiranhaWorks complaining about receiving "only an 8" for Gothic 3. ONLY an 8.

That having been said, I think this is a step in the right direction. It shows you whether or not you're reading a positive, negative, or apathetic piece, and it establishes unambiguous terms for each rating without getting needlessly detailed. Just don't do half-stars. Those bug me.
dementedhut dementedhut - April 27, 2015 (08:53 PM)
While I'd be lying if I said the 10-point scale has never caused me problems, I kinda got used to it over the last few years. However, as many here has stated, I wouldn't mind a more simpler rating system, one where it wouldn't leave readers with an ambiguous meaning. I have faced situations in the past where, no matter how far I went when describing in great detail the pluses/minuses of a game, I still got comments that were simply hung up on the rating at the bottom of the page.

Like Roto said, though, as long as decimals don't make a return. I'm not a fan of the letter grading, either; readers would just get as hung up on looking at a B or a C as if they were looking at a 7/10 or 8/10.
dementedhut dementedhut - April 27, 2015 (09:00 PM)
I'm thinking I would also remove the option to edit scores on all reviews.

As someone who goes back and readjusts ratings, I'm not really a fan of that, either. xD
Suskie Suskie - April 28, 2015 (01:33 AM)
No strong feelings towards either the 10-point or 5-point system, other than that I really don't like the idea of those one-word explanations. I feel like putting the score into words should be the review's job, you know? And anyone upset over their own definitions of what each rating means won't be turned by a single word anyway.
wolfqueen001 wolfqueen001 - May 01, 2015 (06:15 PM)
Hm... I've often wondered if a five-point system would work here, but on the other hand, I always kind of thought the 10 scale made this place a little more unique. While I (and, frankly the rest of the main community on this forum) was actually born with intelligence in this country, I understood that 5 meant average, but I can definitely understand the rationale behind wanting to minimize the debate that comes out the ravings of idiots.

Really the only downsides to the five-star system that I can think of, and some of these make me kind of hesitant, have to do with the following:

1) The one-word summary idea kind of dumbs the whole thing down... While this may be the intention, it also may actually motivate readers to read less of the review because of the score. For example, they may just look at the score, shrug, then go away and take whatever their interpretation is of it with them. Now, that's not to say some people do this anyway, but for me, having the five-star system there almost makes it more likely, in my opinion.

2) This is more of a minor thing, but I know with places like Netflix, which uses the exact same system proposed here, I'll sometimes have the problem of deciding whether the movie should be one score or the other because sometimes the five-star system feels too restricted. Whereas, on a 10 scale, the choice would have been easy - e.g. a 9 - on the five star, I may struggle between a four and a five. Granted, this kind of thing isn't terribly common, but it's especially noticeable when trying to decide whether the movie should be a two or three, or a three or four.

Now, I actually kind of like the idea of not being able to change the score once you grade it because, well, as some have said, the scores are kind of arbitrary anyway, and a change in score may change the way readers actually read the review (that is, what tone they interpret from the review). Of course, we all know that the review itself should speak for itself, but even I will go into a review sometimes knowing the score ahead of time and then look for how the person wrote the review to reflect that opinion. The only problem I can see with not being able to change the score would be cases where someone actually clicked the wrong button when submitting it, or, if their opinion of the review changed completely and so substituted the review with another. Granted, I'm sure provisions would be set up for these kinds of eventualities, and in the case of the latter, the writer might just want to delete the review entirely and resubmit (like Suskie did with Dark Souls), or submit the new one under a different platform.

Anyway, as long as this post is, I ultimately feel indifferent about this as well because I really can see both sides to the issue here and so can understand both the pros and cons to either system. I may be biased from using the old system as long, but that might just be the part of me that doesn't like change. *shrug*

Oh, I do have one other question, about all this. If you do change the system, what happens to all the old reviews? Will they convert as well or will they continue to show the 10 scale?

eXTReMe Tracker
© 1998-2024 HonestGamers
None of the material contained within this site may be reproduced in any conceivable fashion without permission from the author(s) of said material. This site is not sponsored or endorsed by Nintendo, Sega, Sony, Microsoft, or any other such party. Opinions expressed on this site do not necessarily represent the opinion of site staff or sponsors.