A More Beneficial Ratings System that also Fights Trolls

#1
Summary:
  1. Change the star Rating to Technical Rating and reduce its visibility and how much it affects rankings 
  2. Replace Ratings with Favourite as the primary visual distinction between stories
  3. Rankings affected, in order, by favorites > follows > views > technical reviews, as this is more reflective of popularity
  4. All Technical Ratings have reviews so that they can be appealed 
  5. Emoticon feedback on chapters
  6. Replace Technical Ratings on the front page of a fiction to Comments or Endorsements not tied to technical feedback
First off, let me say that I’m relatively new here and I haven’t been party to a lot of previous conversations about RR and its systems. I don’t know what’s been considered and tried in the past. It’s just that, as a newcomer, interacting with the system has gotten me thinking and maybe there’s something I can add to the conversation. 

I think RoyalRoad is a fantastic platform. I’m very thankful that it exists and for all the people who have put so much time and effort into it. As a member, I’d like to add my perspective on how it might evolve.

While there are many things that work on RR, I think the rating and review system, as on most websites, has significant challenges. As such, the current rating system does not always accurately reflect a story’s popularity or quality. The system is also open to abuse. Also, completed works don’t get enough visibility. 

Issues:
  1. people are able to review bomb stories and cause a lot of unfair damage; trolls do it to hurt others; underhanded authors do it to hurt their ‘competition’
  2. the rating system doesn’t really work as intended across a 5* spectrum; most people only use 4-5 stars 
  3. ratings and likes are used as the same thing, but whether people like/dislike a story, and how popular it is, has nothing to do with technical quality, nor are most people able to accurately rate their own like from 1-5*s, making the 5* system confusing and innaccurate
  4. Most people leaving ratings on the technical aspects of the story are not pro editors or English teachers, and therefore aren’t really qualified to do so
Example Change
(image)
Above: The star Rating system is the most prominent aspect. That’s what people see (and judge) first. But it’s somewhat innaccurate. 
https%3A%2F%2Fwww.timothybaril.com%2Fwp-content%2F...atings.png
(image)
Above: Technical Ratings (stars) are hidden and only viewed once you go to a fiction’s page and open the Stats, or click on the Plus button on the right. By being hidden, they don’t have a chance to mislead. More valuable ways of deciding if a story is worth reading are prominent: Faves, Followers, Emotions. These are all more reflective of how reader's feel than a star rating.

Ratings vs Likes (Favourite)
We currently rate a story based on how much we like it. If we hate it, we give it 1* and if we love it 5*s. But what if it's written pro quality and just isn't to our personal taste? How do we compare something on RR with published fiction? 

This is too complex. The human brain doesn’t rank and compare things very well. The most we can confidently ask someone is whether or not they like it. 

In addition, emotion is heavily involved in the rating process. Everyone hates getting anything less than 5*s. People feel guilty about giving out low ratings, especially in review swaps. So there is a consistent bias towards one end of the scale. Thus, ratings are not spread out and they’re not always accurate. Especially when some dumps a 0.5* rating on a story just because they hate female leads or LGB characters or harems. This isn't fair to the story. 

Instead of asking readers to compare a story to every other story they’ve ever read and rank it against them, we should just be asking something simple: Did you like it?  

We already have a tool in place for this: Favourite. By making this tool the primary source of feedback instead of Ratings, we make the system simpler and more accurate regarding popularity. 


Favourite Prevents Trolls and Abuse
As author’s we’ve all had someone leave a blind 0.5* Rating. Or someone trashed us in an Advanced review. And sometimes it really hurts. It’s emotionally abusive. It makes us want to stop writing and sharing. And it reduces our rankings on lists, so we get less visibility. The current system gives power to the bad people trying to hurt us. We must find a way to stop that. 

We can fight back two ways: making the primary feedback Favourite (Like), and changing Ratings to Technical Ratings. 

By making Favourites primary, there’s no way to hurt anyone. You can’t leave a bad Favourite. You can’t drag down someone else’s ranking. You either like something or do nothing. So only good can result. 

By changing Ratings to Technical Ratings and reducing their importance, and allowing them all to be appealed, trolls and bad authors can’t come along and destroy someone’s ratings. If they try, their damage can be undone. 


Favourites Help Completed Stories
Currently, completed stories struggle hard to gain visibility because the system heavily favours works that are constantly being updated. 

By making favourites the primary (and most visible) tool for reader feedback, it matter less if a story is ongoing or completed. A like is a like. As long as the story can attract readers, it can gain more likes. If featured well, Faves will be far easier to get than Reviews, the same way people are used to Liking stuff on social media. This gives completed works a better chance against constantly updated stories.


Emoticon Feedback
Anyone using Facebook will recognize the emoticon feedback they introduced: hearts, smiley faces, angry faces, sad faces, vomit face and so forth. This works really well because it’s emotional. It’s very easy for people to understand how they feel when they read something. Facebook actually weights emoticon feedback more than Likes and views because they take more effort and are more specific.

Emoticons on RR would be easier to use and understand than trying to give star ratings. In addition to marking a story as a favourite (a like), we could allow readers to give emotional feedback as well. At the end of a chapter, they could give a heart because it was so romantic, or a scared face for a great horror. 

You could get very creative with emoticons for things like thrill (action), cool dungeon-crawling, fun slice of life, or shocking twist. These are very easy to understand, easy to use, and will likely encourage reader feedback for those very reasons. Asking a reader to comment on grammar and style and characters is hard. Asking them to rank a story against every other story they’ve read is impossible. But they’ll jump at the chance to push the dungeon button on a chapter with a really cool dungeon crawling scene. Or a shocked face on a chapter with a huge twist they never saw coming. 

These emotional reactions might not be a part of current lists, but you could make new ones. Want to find the scariest story? Which one has the most scary faces given? How people felt when reading a story is going to be much more valuable in discovering new stories than how good grammar or style is. 

Emoticon feedback gives more visibility to completed stories. Yes, a horror story with 1000 chapters could earn more total scary faces than one with 25 chapters. But we could rank stories with more complexity than that. Perhaps it’s not just total emoticons recieved of that type, but also factors in how many faves and views it has. Wattpad uses a similar ranking system for tags. It's not perfect, but with tens of thousands of stories to choose from, anything to help discovery is a good thing. It should also stimulate authors to write better in order to achieve that emotional feedback.


Technical Ratings more Accurate
If the importance of ratings is reduced, then Technical Ratings would become less used. But if they are weighted less in the overall value and ranking of a story, they should also become more accurate. We should get more honest feedback than we currently do. We won’t be reliant on begging others to exagerate aspects of our work and leave glowing comments. We won’t get as mad when someone gives our grammer score a 2 because it won’t hurt us, allowing us to accept it as more valid criticism of something we can improve.

The more accurate Tech Ratings become, the more they reflect quality of writing and story craft, regardless of popularity. And if we can get more accurate technical feedback, it should help us improve the quality of our work.

I would also consider utilizing the same give and take method Scribophile uses, so that people have to give technical reviews in order to get them. This would discourage casual troll reviews.


Front Page Endorsements
We’re currently using Advanced Ratings as endorsements. This reduces their accuracy and quality. Technical feedback should not be a sales pitch.

Allow readers to make a comment or endorsement on the story that shows up on the first page instead. 



Thanks for reading.

Re: A More Beneficial Ratings System that also Fights Trolls

#11
Existing system works. Your suggestions would not, in my opinion affect this site positively over the long term.

Someone disliking a work enough to leave a low rating does not make them a troll, it merely means they do not like a work...  On the other hand if someone, repeatedly harasses a single person or makes real life threats of any kind, that is a troll. 

RR is a gladiatorial attention arena. Diminish that and you no longer have Royalroad, you have something else.

I encourage you to make your own site and implement your suggestions, if they are as good of an idea as you posit, then your success will bring change and other sites will implement similar measures as a result of popular demand.

Re: A More Beneficial Ratings System that also Fights Trolls

#12
Idk. Over 99% of the site's users are readers. Not writers. This would bias things too much towards the writers, I think. Which would result in there being no negative reviews, no negative ratings. All removed because they're "bad people" and "trolls" who "make us sad". That's how it sounds like to me, at least. A bit ridiculous. Getting negative feedback is perfectly normal on the internet. I'm not sure you could, or even should, try to avoid that. 

And the give-and-take system assumes that the RR user is a writer. That's absolutely ridiculous. It would prevent the only-readers (almost everybody) from leaving reviews at all. When they are the audience, they are the consumers. It's THEIR opinion that matters to the other readers, when they are looking for something new to read, and glancing at the reviews section. I don't understand your perspective on this, not even a little bit. 

Re: A More Beneficial Ratings System that also Fights Trolls

#13

Winged Wrote: I encourage you to make your own site and implement your suggestions


That doesn't make any sense at all. That would be completely innefficient. "Go try something, invest tons of time in it, and if it works, we'll steal the idea."

Or I could just mention the ideas here and if someone was interested, they could experiement, as they are already doing in other ways. Good websites evolve. 

Why shut down people trying to contribute and make the site better? Are you scared of change? Of course the existing system works, but that doesn't mean we can't improve it.


Ararara Wrote: negative reviews, no negative ratings


I'm not sure what you're getting at. Why are these necessary? I'm suggesting that we only give positive votes (likes) for things. If something is popular, it will rise in visibility. Bad stories will get less exposure because they won't get likes. You don't need to downvote something, you can just ignore it. 

There are still Favourites, views, followers, Comments/Endorsements, and Technical Ratings in the above that would boost stories. Most of which is already in place, I'm just suggesting we tinker with it. 


Ararara Wrote: give-and-take system assumes that the RR user is a writer


They can leave comments, endorsements. That would be their primary written feedback. And by separating it from technical reviews, we don't confuse someone having something good or bad to say with feedback on the writing. 

We don't necessarily have to implement the Scribophile system, it was a suggestion. If you want to let anyone leave a technical review at any time, fine. The most important thing I mentioned was separating how someone feels, their personal taste, from the technical aspects of the writing found in Advanced Reviews. Leaving a 0.5* rating because you personally didn't like the story has absolutely nothing to do with whether that story was well written. And that's a major flaw in the system. 

Re: A More Beneficial Ratings System that also Fights Trolls

#14
And I'm saying that people don't want to read stories "because they're techically well written". They want to read them because they want to like and enjoy the story and the characters. A movie can have horrible cinematography but be entertaining and have a great story. It can still be a 9/10 movie, not a 1/10 because of the "technical aspects".

Real reviews are way more useful than technical reviews. RR wants to judge and sort by likeableness and entertainment value, I think. And that's what's important to the people who click on a story: to see the authentic feedback. Not some dissertations on the technical quality.

Re: A More Beneficial Ratings System that also Fights Trolls

#15
Your proposed system will only work to make stories written by frequent forum users who advertise their stories frequently get even more attention, while stories by busy authors who don't spend time advertising their writings on the internet will get left even further behind in the dust. There is no way in the world that such a situation could possibly be considered better.

Also... This part seems especially ridiculous. "Or someone trashed us in an Advanced review. And sometimes it really hurts. It’s emotionally abusive." You... Let me just clarify something. Did you mean someone trashed you personally? Because personal insults are already forbidden in reviews, as they should be- which leaves me to assume you're saying someone trashing a story as badly written is 'emotionally abusive'. Which... If you can't tell the difference between a personal insult and somebody criticizing your story, and if you think the latter is emotionally abusive, you probably shouldn't be writing. I've seen other people on Royalroad forums explicitly say they stopped reviewing because of toxic authors who couldn't handle the slightest criticism, and this seems to be the sort of mentality that causes that.

I'll just emphasize something another commenter said related to this.

Ararara Wrote: Idk. Over 99% of the site's users are readers. Not writers. This would bias things too much towards the writers, I think. Which would result in there being no negative reviews, no negative ratings. All removed because they're "bad people" and "trolls" who "make us sad". That's how it sounds like to me, at least. A bit ridiculous. Getting negative feedback is perfectly normal on the internet. I'm not sure you could, or even should, try to avoid that. 

And the give-and-take system assumes that the RR user is a writer. That's absolutely ridiculous. It would prevent the only-readers (almost everybody) from leaving reviews at all. When they are the audience, they are the consumers. It's THEIR opinion that matters to the other readers, when they are looking for something new to read, and glancing at the reviews section. I don't understand your perspective on this, not even a little bit.

Re: A More Beneficial Ratings System that also Fights Trolls

#16

Ararara Wrote: And I'm saying that people don't want to read stories "because they're techically well written". They want to read them because they want to like and enjoy the story and the characters. A movie can have horrible cinematography but be entertaining and have a great story. It can still be a 9/10 movie, not a 1/10 because of the "technical aspects".

Real reviews are way more useful than technical reviews. RR wants to judge and sort by likeableness and entertainment value, I think. And that's what's important to the people who click on a story: to see the authentic feedback. Not some dissertations on the technical quality.


Look at what I wrote above. A story would gain more visibility regardless of the bad "technical review". If you also want some written comment about the story in general there could be another section with "comments about the story". No need for the stars in that, since it's just a comment on how you find the story. Maybe instead of the technical review, put the like and dislike buttons there, or in both.

Besides, there's nothing wrong with sharing ideas. That way we can find out how worthwhile they are, and find deficiencies we wouldn't see otherwise. If it's not liked, or unfeasible, then that's it. Am I wrong?

Re: A More Beneficial Ratings System that also Fights Trolls

#17
Most reviews and ratings are not done by trolls. Someone likes or dislikes the story, they rate it what they feel. It frequently gets the max of 5 or 0.5 because monkey brains are simple. On that note, SenescentSoul posted a very in depth and interesting post on the merits of changing ratings from 0.5-5 stars to a binary thumbs up, thumbs down system. That was a pretty decent suggestion that has its own merits. Most of the changes you're suggesting, not so much. You can add a filter to list stories by favorites without needing to eliminate the other choices, and you can add emotive responses without removing anything (even if I dislike the idea).

Plus the actual overall star rating of the story is not relevant to 99% of readers. Most people look at what's popular in rising stars or trending and pick stories from that based on the story tags and synopsis. Even stories with an overall mediocre rating of 2.5 stars will be no less popular for it thanks to how the rising stars system works. Getting reviews, even ones that are 0.5 stars and absolutely thrash your story with the utmost relish, will make the story higher ranked on those lists and infinitely more visible to potential readers. And being on the rising stars list will get you far more new readers in a month than being ranked a little bit higher overall will get you in years. So even if there are trolls reviewing your fiction? That would make it more popular and be better for the story. Which is about the best possible way to handle it.

But none of that matters. Because the reason you're suggesting this isn't about logic, it's about how you feel.

Timothy Wrote: As author’s we’ve all had someone leave a blind 0.5* Rating. Or someone trashed us in an Advanced review. And sometimes it really hurts. It’s emotionally abusive. It makes us want to stop writing and sharing. And it reduces our rankings on lists, so we get less visibility. The current system gives power to the bad people trying to hurt us. We must find a way to stop that.


Unless someone is personally insulting you, no, it is not emotionally abusive. And if they are, those reviews get rightfully removed when you report them. If the review is just critiquing your writing, either decide you disagree and move on, or take it as constructive criticism and do better. This goes from poorly written reviews where they just say your writing is bad, to well written critiques that last for paragraphs but point out every single flaw in your writing.

Any producer of content will face criticism. Whether it's writing, music, painting, or so on. Trying to limit the criticism from your audience will almost always backfire and have you be seen even more poorly by your audience for the effort. And you'll fail in the attempt. You might as well try to blot out the sun.

The only thing you can control is if you pay the ratings and reviews any attention, and how it affects you. Efforts to improve your emotional resilience or ignore reviews you disagree with will provide much better results than trying to get anonymous people on the internet to stop giving their opinions.

Re: A More Beneficial Ratings System that also Fights Trolls

#18

Ararara Wrote: And I'm saying that people don't want to read stories "because they're techically well written". They want to read them because they want to like and enjoy the story and the characters. A movie can have horrible cinematography but be entertaining and have a great story. It can still be a 9/10 movie, not a 1/10 because of the "technical aspects".

Real reviews are way more useful than technical reviews. RR wants to judge and sort by likeableness and entertainment value, I think. And that's what's important to the people who click on a story: to see the authentic feedback. Not some dissertations on the technical quality.


Yes. This is all exactly what I argued in the original post. The current rating system is confusing. We can change it so that it reflects enjoyment instead using Favourites. And they can leave feedback as Endorsements. You're literally agreeing with me. 


sharoncorsola Wrote: Your proposed system will only work to make stories written by frequent forum users who advertise their stories frequently get even more attention, while stories by busy authors who don't spend time advertising their writings on the internet will get left even further behind in the dust


Where are you getting this from? None of the changes I mentioned have anything to do with forums or ads at all. I am suggesting making Favorites the primary way to like stories, and make the Ratings part more effective at what it's supposed to do. 

Ziggy Wrote: binary thumbs up, thumbs down system

I'm suggesting the same, but without a thumbs down part because it gets abused and there's no need for it at all. And the changes to Ratings are to make them more effective. Whether or not you believe 99% of readers don't care about Ratings, it's the most visible recommendation and if there is a way to improve it, there's no reason not to. If 99% truly don't care, then you should scrap it entirely.

Re: A More Beneficial Ratings System that also Fights Trolls

#19
I agree to cut away the stars, and only have the number of favorites, follows, etc. display the 'quality' of a story. 
It works well enough for other sites, and it takes away one of the biggest antagonists an author can face on this site - the 0,5 star! 

It's my primary reason I don't post my story on RR. I'm honestly not interested in getting 0,5 ratings that don't give me a reason why I deserved it.
I don't mind negative critique since I don't expect everybody to agree with me or like what I write, etc. But I'm sensitive to this anonymous weapon of motivation destruction that trolls can use and abuse to their heart's content. 

My experience is that as soon you enter the trending list, 0,5/1/2 stars start raining. Like, as if everyone suddenly starts hating your work  peocrying

So, away with the rating system! 
Burn it with wildfire! - Aerys Targaryen 

Re: A More Beneficial Ratings System that also Fights Trolls

#20
Points 1 and 3 (reducing the importance of ratings on rankings) seem strange to me. Not sure a compelling case can be made for these points as the best rated stories are NOT the same as the popular this week stories. Some of those stories actually have a relatively average overall rating; such as The Legend of Randidly Ghosthound which has been on the front-page list for at least two years as far as I am aware. Possibly longer. It definitely doesn't seem to be hurting some stories. I also recall when the Arcane Path story was in the most popular list with an overall rating of less than 3 stars. And it was on that list for a decent period of time. 
In actuality, of all the ranking lists, the best rated list is actually at the very bottom of the site. Even currently popular reviews are featured above this. And even that bottom-of-the-front-page rating ranking list doesn’t seem to be solely based on overall rating. I would guess it takes into account number of ratings too as stories exist that only have a single 5 star rating that is definitely higher than those currently at the top of the list. So think these points are based on a flawed premise of overall ratings playing a bigger role than they really do in terms of determining a story's visibility and position on lists. 

Point 2 (Replace Ratings with Favourite as the primary visual distinction between stories) looks interesting. Like the argument about how there can be no 'hurtful' favourite. However, what would be the downsides of such a change? One I can think of is that it will unfairly be biased towards already popular and visible stories and will disadvantage new stories by new writers. Something that is already difficult enough for new writers.

Point 4 (All Technical Ratings have reviews so that they can be appealed) might also mean there are less ratings overall. Some people won’t even leave ratings, so ratings will become a lot rarer if they require much more than a click of a button. Not something that I think will benefit this site. And the truly 'bad' actors (those who want to hurt a story and are motivated enough to do so with actions) can get around 'rules' if they are really intent on leaving a bad rating. They can just edit repealed ratings in a reworded way to make sure the rating sticks. Humans are generally pretty capable beings at getting around rules. And there are plenty of negative reviews that are fully in keeping with the rules. Some of them are even valid, much as it might hurt a writer to admit.
Not backed by any strong evidence, but I feel that a larger number of ratings will give a more accurate star rating and dilute the ‘bad’ actors. Not reducing the number of ratings even further to those bothered enough to write a mini-essay on why they rated the way they did. Especially not if this exclusion only applies to low star ratings, making the ratings even more positively biased than they already are.
PS: I would be interested if anyone can find out the statistics of how many stories have an overall rating of 3 stars or less (excluding stories that have no ratings at all) compared to 4 stars or more. I can find some with less than an overall 4 star rating, but they are very rare. Especially considering how there is no minimum level of story-writing skill before someone can upload their work on this site. 

I like the idea of Point 5 (Emoticon feedback on chapters). Mainly because I like the idea of authors being able to get feedback on specific chapters, rather than just the story as a whole. Arguably they can get that to a certain extent from comments, but am wondering if also having a chapter rating system would result in feedback from a greater number of readers than just comments on their own. Much like how there are many more ratings than there are written reviews.
Though I am wondering if a thumbs-up/thumbs-down option might possibly be better for this as I feel there is less need for more nuance on a chapter scoring than a story scoring. Also am not sure whether some authors will be able to cope with another avenue for receiving potentially negative feedback (e.g. imagine getting a vomit or just unhappy emoticon). A few people might just focus on the one or two who are negative rather than the overwhelming majority who liked it. Humans might be capable, but are not always logical. It’s one of our failings and hard to get around. Maybe a thumbs down would hurt less than a low score, but maybe that is just my opinion. There also might still be authors who want people to write a reason/story for why people give a thumbs-down. I can fully understand why, but if the thumbs-downer didn't leave a comment about it, not sure they would be bothered enough to write an essay for the chance to give feedback. Especially when it is easier to just skip the emoticon/thumbs-up-or-down option and just leave a comment. So yeah might be good to consider but possibly has some problems of its own for those already sensitive to ‘bad’ ratings.

Point 6 (Replace Technical Ratings on the front page of a fiction to Comments or Endorsements not tied to technical feedback) also sounds interesting. But note that this is another avenue for people to receive negative feedback. And if getting 'low' number of stars and 'bad' reviews is already too tough, can you imagine if your story has 'negative' comments or endorsements? Imagine if one was the author that had 'terrible' or 'stupid MC' on their front page, despite having a high overall star rating, and mostly positive reviews in their first page of reviews? And if it was always positive, would it be helpful to readers or just very prominent ego-stroking for the author and providing them a way to echo-chamber themselves into thinking everything is awesome and all the less than positive feedback are just haters who are simply jealous or stupid? 

I think this comes from a well-intentioned place, but might need some further thought on the potential issues or implications. I do think though this sort of discussion should occur, but we also shouldn't kid ourselves that there is some sort of silver bullet that will fix everything. Let's not be the people who introduce new animals in a new location to control a real or imagined pest problem only to cause a bigger one. Instead we should consider each possible change holistically and figure out potential abuses and implications, not just what it might 'fix'. Maybe even though there are new issues they might be better than existing issues. But hopefully site admins try and figure out as much of this before implementing the changes. Rather than trying something new and just hoping it works out for the better. Also introduce them in a controlled manner and preferably on a small scale before applying huge changes everywhere that we might not fully understand. Because even intelligent people can make mistakes in analysis and miss huge potential problems with the change. Even brilliant scientists can screw things up. If you are human, you are prone to error.
And hopefully this forum thread helps provide some of that more detailed analysis rather than just purely focusing on a single aspect of a suggestion. It’s why I appreciate the discussion about it. Many minds seeing things from different angles and seeing what is good and not so good about it. 

Will end this with lyrics from what I think is an awesome song which is even partially relevant to my last paragraph (or you can listen to it at the link here):

Everything's not awesome
But that doesn't mean that it's hopeless and bleak
Everything's not awesome
But in my heart, I believe

We can make things better if we stick together
Side by side, you and I, we will build it together

Everything's not awesome
Things can't be awesome all of the time
It's an unrealistic expectation
But that doesn't mean we shouldn't try
To make everything awesome
In a less idealistic kind of way
We should maybe aim for not bad
'Cause not bad, well that would be real great

Everything is better when we stick together
Side by side, you and I, we will sing it together
Party forever (Together)

This song's gonna get stuck inside your
This song's gonna get stuck inside your
Heart