Re: Alternatives to the 5-star system?

#1
Note: This isn't meant as any sort of agitation for change here on RR. This is just meant to be a discussion of alternative rating systems, and their pros and cons. 

So, RR has a 5-star rating system (Really, a 10-star rating system, since you can give half-stars), where a novel's overall rating is the average value of all the reviews, and it's tied to the novel's rank in the best-ranked system. 

Now this seems like a pretty typical review system, but it does come with some cons:

-Everyone has different opinions on what a star rating means. For some people, a 3-star review means "average", for others, it means "Glaring flaws." For some, a 5-star review means "perfect", for others, it just means "I enjoy this."

-Since the average rating is inherently tied to your rank, and your rank gets you tons of free views, at some point, any rating less than your average actually "hurts" your story, causing you to drop in the ratings. I'm not a ratings fiend myself, but it would be a lie to say I wouldn't enjoy more readers, and your rank has a huge impact on that. To their credit, RR tries to come up with alternative systems to give people views that are not tied so much to your average rating (such as the trending list, although I think that comes with its own problems.) I think this is what ends up inflating the ratings, since a lot of people don't want to *hurt* a story they like, even if they think it's imperefect. 

-Vulnerability to troll reviews

So I was thinking - what if, rather than an average point system, there was a sum point system?

For example: Each review you could give 5 *positive* stars, and 5 *negative* stars. So let's say that I found a story I liked, and thought it deserved 4 positive stars.  But I thought it had some grammar issues, so I gave it 1 negative star. Overall, my review would have 3 positive stars. 

Then you could have two metrics. 

1. One, you could just take (overall positive stars - overall negative stars) as a measure of rank. This would eliminate the problem of taking an average, where a story with, say, a 4.6 average score is "hurt" by anything less than a 5 star rating under the current system. So long as some individual review has more overall positive stars than overall negative stars, even if it's just by 1, it is "helpful" to the story. And maybe this fact would help combat inflated review scores.

One con of this is that it doesn't actually give an idea of a story's *quality*. But for that, you could use another metric:

2. Another possible metric is a ratio: (overall positive stars)/(overall negative stars + 1). (The +1 is there to avoid divide by zero errors.) This could be an indication of overall story quality. With this metric, you would still hurt a story if your ratio in your individual review is less than the story's current ratio. However, you could make it so this metric is not actually tied to the ranking system, and simply serves as an overall guide to the story's quality. 

Anyway, this is just something I was thinking about. What about you, have you thought up of alternative rating systems? Are there rating systems other sites use that you think are interesting? 

Re: Alternatives to the 5-star system?

#4

ArDeeBurger Wrote: I think Up Votes and Down Votes ought to come into play.

I like this idea. The alternate is to have no star system at all, and have reviews that will only have the text. Do away with the rating system, because the highest rated list overlap heavily with the most popular/highest view count list anyway. Any story with 10 5-star review will still be lower in the highest rated list than a story with 100 ratings with half of them 0.5. Another is to keep the rating but only on the story page, and not have it pop up in the trending or homepage lists. Hope these ideas help.

peolove peoapproval

Re: Alternatives to the 5-star system?

#5
There are two rating systems I would consider if I were creating a site like this one. 

The first and simplest would be a simple three-point system. Rate the story as good, bad, or neutral. Upvote, downvote, novote. This is because while upvotes and downvotes are useful, they don't play into averages very well; what if your reaction was "needs work" and don't think it deserves an up or a down? Pick a middle option and let that factor into the averages. 

The other option is to actually label what the levels mean. It wouldn't eliminate troll reviews, but it would mean that everyone would understand what the levels represent on this one hypothetical site.
  1. "I didn't like it because of issues that go beyond my personal tastes." 
  2. "This did not meet my expectations for the entertainment promised."
  3. "This met my expectations." 
  4. "I would recommend this to others." 
  5. "I intend to read this again." 

Re: Alternatives to the 5-star system?

#6
I think as long as the ratings, are tied to the stories visibility, the system is going to be gamed. Reviewers (and Authors since it seems like most reviewers are swaps around these parts) are going to feel bad handing out an honest three, and going to feel bad for receiving one. No matter what personal standard that three star means. Lots of the really popular stories can really, only go down because they're like a net 4.8 something or other, and anything other than 5's across the board is a down trend.

Divorcing them from trending and the like might help the five stars be more accurate, but then it just ends up abandoned. People would just move to game whatever the new setup is that trending cares about. Because of this gaming of the system, it doesn't matter if its a five star, upvotes, or whatever. The flaw exists meta to the system.  The flaw, is a 'I want my work to get discovered and have an adoring fanbase' and the best way to go about that is gaming whatever system exists and enables that.

And why are you here if that's not what you want? Every writer on here craves feedback I think, so I'll take it in whatever form it comes.

Re: Alternatives to the 5-star system?

#7
First, reviewing reviews is just stupid. It's someones opinion of the work that is not your own opinion. Just post your own review if its different, and state why. Can't be bothered? Then pike off. You have no strong opinion. In either case a reviewer opinion shouldn't affect rank. Reads, or if commercial, sales, decide rank. The most read is by definition the most popular if ranked for a particular time period, say a year. It does little good to note that someone has acquired more reads in ten years than another has in one year. Duh. If its not part of the rank process, it wont be trolled, as it does not affect rank. Also if someone wastes their day trolling reviews, it wont mean anything. Trolls are trolls. 

Second, ratings. There is nothing wrong with such specious, thumbnail critiques, save that they are not critiques, and the "Overall"score should not be filled in by the reader. it should be an average based on the average of the persons specific categorical scores. Its a function of those.

Other than that, since I am not opinionated, I have nothing to suggest.  Heh.

Re: Alternatives to the 5-star system?

#8
I personally would rather have no rating system of any kind to be honest.

The problem with a "like"system is that it brings in a whole new set of issues. It seems that sort of system is more prone to people becoming even more fixated in accumulating the best possible ranking because it "like" or a "+" is easy to do. A simple tap or click. It becomes just as meaningless as giving every story as many 5 stars because anything lower than that people interpret as bad. In this case, people are tying to inflate their likes to get noticed anyway possible. It says nothing about the quality of the stories or that all these people found it enjoyable. You can beg for likes

If you have ever been on a platform that became a culture of begging for likes, it's not fun. It becomes a popularity contest and a race of who can get the most likes. I'm looking at you Wattpad and Figment(when it existed, and  I think it was worse) And if you add downvoting in to the mix, well that is easier to abuse unless there is some way to protect a user from anonymous donwnvoting bombs.

One of the reason I liked DeviantArt was there no rating system. Views was the thing that mattered. Sure you could like/favorite a piece, but it didn't affect your exposure at the time. I have no idea how it works now. And yes there were flaws in the system where people thought that a rating system would be nice to improve visibility of quality works, but DA would say no. When they were a young community before I joined, they had one. The only reason they got rid of it because it was being abused. People would purposely rate something low they didn't like and were targeting specific kinds of art.

So just using view, sound a lot better, right? Well there is a flaw in that as well. A big one. This craps on shorter works. You can accumulate a lot of views if you write a lot of chapters. 

I suppose followers could work splitting it for not only just followed stories, but showcasing writers with a large following. I don't know, I'm just throwing out my thoughts.

Re: Alternatives to the 5-star system?

#9

L.J.Anders Wrote: If you have ever been on a platform that became a culture of begging for likes, it's not fun. It becomes a popularity contest and a race of who can get the most likes. I'm looking at you Wattpad and Figment(when it existed, and  I think it was worse)


Figment was SO bad I can't even. There was no verifiable way to tell if someone even read your story when they gave it a like because they'd give a like just to get one

Re: Alternatives to the 5-star system?

#10
Eh.  Any rating system I can think of can be gamed and abused.  Stars are not the worst way it could be handled.  Reviews are what I tend to go for, and even there, more for what the reviewer wrote than the stars.  When buying stuff on Amazon, some of the negative reviews have actually made the decision for me to buy a product (the ones that were detailed enough that I knew what they were down checking the product for was something more to my taste).  

Re: Alternatives to the 5-star system?

#15
I mean, the system I proposed was essentially a like-dislike system, except instead of being able to give either like or dislike, you are able to give up to 5 likes, and up to 5 dislikes (and can do some mixture of the two: four likes, 2 dislikes.)

I agree that any rating system that's tied to popularity is going to be gamed, but you can design systems that are more robust to gaming than others. I think a problem with having a system based ENTIRELY off of popularity (which to be clear: RR's current one is not, though popularity is a big factor) is that it has a "Rich get richer" problem: By being more popular, you get more exposure, which means you become more popular, etc. I think this is going to be the case to SOME extent in any system, but it shouldn't be the dominating factor - you want to be able to provide mobility, so that the stories you are providing exposure to don't become stagnant and dominated by those who least need it. Some sort of time limit (which is incorporated into the trending system) might work well. 

I thought up the system I outlined because I thought that a review system should balance three things:

1. A reviewer should be able to always *help* a story they like, even if they think it's imperfect, without having to hand out all 5s
2. There should be mobility on the ranking lists not entirely determined by a story's existing popularity
3. Popularity should come into play somehow, though to showcase those stories which have attracted a large audience

Re: Alternatives to the 5-star system?

#16
Hmm … I’ll admit, a part of me does like the idea of a simple like/dislike system as opposed to the stars. It’s definitely frustrating to watch your rating go down when someone gives you a 4 or a 4.5. 

That's not the readers’ fault, of course. They’re probably thinking, “Yeah, it’s a good story, but I save my 5-star ratings for my all-time favorites.” And that would make perfect sense if rank didn’t affect visibility. To them, 4 stars might be good even though it might hurt the author’s rank. They don’t realize that the majority of readers use 5-stars as a like button, and half-stars as a dislike. And if they’re not authors themselves, they wouldn’t know that you basically need an average rating of 4.5+ to play in the big leagues.

This is also why I’d have to disagree with the idea of labeling the different stars. If we encourage more “medium” ratings, that will hurt people who have a few hundred followers, but it probably wouldn’t make a dent in something like Beware of Chicken which undoubtedly has thousands of five-stars to balance it out.

However, one big downside to any +/- system is that it might actually make the trolls feel even more justified when they bombard a story with low ratings. At least the 5-star rating system implies that you should put some thought behind your rating. It implies that you've actually read the story long enough to judge its quality. A binary system could easily lead to people using the negative option as an “I disagree” button. We know this happens on YouTube and reddit already.

And then if you do just “likes” without any dislike button, it’s basically no different from Popular This Week.

It almost seems like the 5-star system is the worst way to handle this, except for all the other ways.

Re: Alternatives to the 5-star system?

#17
The only flaw I see in the system is the anonymous ratings. Only written reviews should affect the score of a fiction and users should be encouraged to write more reviews. Even a low quality opinion in writing tells everyone infinitely more than a one-click ninja rating. The admins have already lowered the ratings' importance in calculating the overall ranking (more than once, I think?), but I don't really see any reason why they should be kept at all.