I find that outlier accounts don't last. Many extremely low and triggered reviews I get are from relatively new accounts. People with a triggered mentality do not last in RR because of the community. As cliche as it might sound, people who read for enjoyment tend to more often than not, be relatively intelligent. So a triggered person leaving a triggered comment will either be met with backlash or be ignored.

My solution would be to limit the impact that new accounts have when reviewing.

*Please be informed that the above comment was created in the absence of coffee.*

Honestly, and I know I'm gonna get shit for this, but I would just do as Youtube did and remove the ratings in favor of a like-to-dislike ratio. I get wanting to keep the system, and change it in such a way that it works, by altering the math surrounding it. However, it doesn't change the fact that the majority of people vote either 0,5 stars or 5 stars anyway. 

I remember when Youtube switched over to the like-to-dislike system they have, they ended up releasing the statistics of how people voted (tried to find it, but it's buried under mountains irrelevant search results), and I believe 95% of people voted in the two extremes. It essentially just boils down to people voting "like" or "dislike" anyway, so Youtube decided that trying to maintain a broken system was a pointless exercise, since it didn't change the fundemental flaw in the way people voted.

RoyalRoad could just imitate the exact way Youtube did it, with a bar displayed underneath the book title and everything. It's a much better system to indicate the ratings of internet content.

edit: not sponsored by Youtube lol

I think people are missing the forest for the trees here. I could reiterate how broken the review system is, but there's something much more important going on. And it's something very very sad. It's not the fault of most readers, and it's not the fault of authors either, but it happens anyway.
Take a look at this. This is what a review is worth, depending on your rating.

A lot of writers here are hobbyists, doing it for the love of it. And the validation is nice too. It's always good to see a reader enjoying the chapter.
We've all been there. Press submit. First review comes in, five star. Next review, five star. Next review, five star. Next review, five star. Next review, four star. Instantly, that author is down to 4.8
Is a four star good? Usually, yes. Does it mean the reader likes the story? Usually, yes.
And what does the writer see? they've lost 0.2 in their rank. :feelsbadman:
If they're higher up, that four star will sink their rank as well.
These are people who like their stories. Usually.

But what does this mean?
The author develops a warped view of the fiction. Until they reach 4.5, anything except a five star will directly harm their average, and potentially their ranking as well.
These are people who like the book, these are fans. But the nature of the ranking is such that authors only believe a five star review is a good review.

This continues. The author gets more five stars, more four point fives, and more fours. By some miracle they reach 100 ratings with 70 five stars, 15 four point fives, and 15 fours. Nothing else. Not even that sneaky 0.5 ninja who patrols the listings.

The author's at 4.775, and things are really great, they've gotten so much validation for their fiction, they're enthusiastic about their writing, all is good. And they enjoy writing.

Then the big 101st review comes in.

0.5

The listing ninja has found the fiction at last. Not even a comment to say what they liked or disliked.

4.733

Down 0.042

Doesn't seem too bad, does it? We'd all kill for a 4.733

The author doesn't see it that way. Someone hates their fiction. Or, even worse, they just wanted to see it burn. That rating is worth 15.833 of their current five star fans. And they don't have the excel spreadsheet that I have, but they get the point anyway. It will take 7 five stars to get back up to 4.75. 19 five stars to get back to where they were at 4.775.

Every five star is a rush of dopamine, everything less than that is its withholding. 

You see, this author is new. They've been getting validation from this. Maybe their hobby is turning into a passion. And each five star, they get that dopamine hit. They check their ratings incessantly, like an addict waiting for their dealer.

And it teaches them something. That no matter what, their haters are right. Their votes matter more, after all. The author might not have the numbers, but they know that it takes multiple people loving the story to counteract even one person hating it. That even when their fiction is at rock bottom, the dead zone of Royal Road, even then, their haters still matter more.

The author begins resenting their audience, not enjoying it as much as they once did. They struggle, and they make it. Top 150 of royalroad. Then the floods of negative ratings come in, each anonymous and unexplained. The author wonders what happened. Everyone liked the fiction at the start? Where did all the reviews come from? 

It's like they crossed a magical line and they found themselves inundated with downvotes. Some might begin to suspect that they're getting targeted by other authors, some decide that maybe their fiction wasn't that great after all, some of the suspicious ones retaliate by downvoting everything around them - continuing the cycle - a rare few brush it off and keep on writing.

But now writing is a chore. It feels forced. They hate having to keep going when the people the rating system has taught them dislike their novel (and have a more important opinion) become more and more numerous. And what are they giving the fiction? Most of them are giving a 4, maybe a 3.5, even the occasional 3 with a "good job, keep it up" attached (and those ones hurt the most of all, because they liked the story, but they hurt it anyway, especially when that good job is equal to 3-5 five stars, drops their rank, and requires 7-10 five stars to fix). There are lower ones, yes, but the majority are those, so called "decent" votes, the "average", or "above average" votes.

And eventually, the dopamine drip stops. Fiction goes on hiatus. Writer stops writing. Never writes again.

The Royal Road readerbase hates abandoned novels, but because of the way the rating system works, they become weaponized against the writers who actually write. How many low rated fictions have you seen that continue on for a long time after? I can count them on maybe one hand, maybe two. They don't get seen, they don't get read, and thus they usually get dropped.

Why? Because a small percentage, maybe 5-10%, of the readerbase who give low reviews have the same power as the other 90-95%.

The readerbase is toxic to the writerbase. But it's not the readerbase's fault. No one understands the rating system, hell, the tagging system is f*cked - everyone expects different things from a tag, and if they don't get it:

0.5

There was a person crowing on discord the other day that they give 0.5 for any fiction that does not adhere to how they believe tagging should work. That's how you know they have never written a word on Royal Road in their lives.

The readerbase is toxic to the writerbase. But it's not the readerbase's fault. Okay, sometimes it's the fault of individual readers, but not the readerbase as a whole.

And if the ratings are malicious? How do you prove it? The author certainly can't. They're powerless. They just have to live with it. The only thing they can do, is stop writing.

And many do. Most do.

And they never write again.

Quote:There are, as best as I can tell: 22,356 dropped novels on RoyalRoad, and 1,897‬ ongoing ones. 24,253 in total. You can do that last bit of math yourself.

Quote:It's not the readerbase's fault, but you do have to wonder just how many fictions would still be around, if they had been better informed, more understanding, more encouraging, a little kinder. 0 it's not their fault. Period. At the end of the day, the blame lies squarely at the feet of a rating system that is frankly bad for an author's mental health. The fact that is has remained that way, really begs the question of whether anyone even cares.

And just for note, i agree with the above comment, upvote or downvote is patently a better system than what it currently is.

Itz how I try to find better stories to read, but I get annoyed at trawling through the list. Just make recommendations a thing. If you liked delve, you might like defiance of the fall etc...

Your idea seems like a good rating system but it would be better to give the 10% firsts chapters more weight in the rating so that disliker can also exprime themself ^^

Not a natif speaker sorry for the error

This thread is too long to read thoroughly in its entirety, so forgive me if this has already been suggested:

How about a trinary score not per chapter but as an average of a group of chapters, and capping it?

e.g., Every five chapters for up to and including chap 25, voting is possible AND once you break the 25 new voting only gets enabled every 25 chaps.

Something to that effect.

One immediate defect to such a modification might be that it incentivizes shorter chapters... In any event, maybe a cap and an aggregate for a bundle of chapters could be included to mitigate some of the negatives introduced by the original trinary suggestion.

EIDT: Just thought of it, but OP mentioned weight by word count rather than by chap. Availability of more voting options could follow suit.

This is simultaneously the best post and the worst shitpost I've ever seen

Or just include a place that says what the standard deviation is on those ratings. 

Perhaps a graph would be easier to show though... 

Dunno man, the problems you are explaining are the same in statistics and because statistics have to deal with super important stuff, they have figured out ways to deal with it. 

Should we apply their solutions to this problem? Sure!

Should we make it as complicated? Nope.

I'm not saying that what you are saying is complicated... well, my wording says it, I don't.

Perhaps a graph... *cough*! 

You are saying to start a new method all of a sudden and that won't be fair to old novels, trust me. 

For all the fairness you are talking about, this system you are describing creates an advantage for a group, which makes it inherently flawed.

Also something I've heard, but haven't confirmed, is that ratings without reviews count less towards the novel than those with reviews. 

Obviously if someone rates a novel 0.5 and leaves there is a higher possibility he is a troll than if he rates a novel 0.5 and leaves a review. For the reviews that are troll, you can report them. 

The same goes for 5 star reviews. 

So, has the problem been solved? Certainly not, like you said, it isn't something that can be fixed so easily.

Could it be improved? Definitely.

So, where do we start? If it indeed can be improved. Why haven't companies done something if it's so easy? 

Well... some have, some try other ways and others don't even bother. 

Like I said, RoyalRoad already has done something by valuing ratings without reviews less. 

Amazon, rates the sellers differently from the items (if I remember well I don't use amazon that much). There is some correlation to these ratings, but it's something more that a buyer can consult to get a better view of the overall picture. Plus you can comment on reviews! 

And where does the problem stem from? How much of that view the reader gets at first glance. 

Yeah, now that I think about it a graph would be amazing to have... publicly. Or at least the site tell us the deviation between the ratings. 

Anyways, it was a nice read. 

Something to consider. Is rating valuable? Consider what you want from it. A) To determine some ranking of sories? B) To find a story you want to read? C) To receive advice from a review. D) Other reasons that I can't think of right now.

And consider the (already mentioned previously in this thread) downsides of a rating system. 

Disproportionate power given in one way or another to one time readers or long time readers. Addressable by normalizing scores (see post #13), applying a method of sliding weighting to each vote based on chapter of vote (see post #16), or giving one vote (current system).

Negative effects on author morale. Best explained by post #23. No solutions provided by anyone as of this writing.

Ratings don't reflect successful fulfillment of tags. Suggested solutions are tag based ratings. I'm not going to be addressing this one.

Consider then what other systems fill the needs while addressing or sidestepping the negatives.

First to mind are positive only systems. You can see them in effect widely on Twitter, Facebook, and even here on royalroad with the rep system. 

Let's consider what a positive only system could look like. There are a few things to Shortconsider first. Short term and long term. Repeat and unique. They best two examples that I can think of are short term repeat and long term unique. The actual system could be called anything, like, favorite, bookmark, view (If it wasn't obvious, I'm saying that already existing systems could be used in this way.) For now let's say it's a new likes system.

The only important difference between my two propositions is overall ranking and trending (also trending is it's own rabbit hole of a complicated topic). Let's evaluate how it meets the set goals and avoids the negatives. Rank based on total likes. Search for new fictions no worse than current system. Reviews can still exist to provide feedback, even without a rating system. (You can often see reviewers give ratings even in systems that don't allow reviewers to assign ratings. Steam reviews come to mind.) Technically non-likers and likers have equivalent power, as in the current system. You have one vote per fic. Is it used? In a more philosophical sense, the power of a non-likers is nearly non-existent, as only likers have any noticable effect. Does it avoid negative author morale? I would think so. Now the only measuring stick is size of number, and rate of increase. Number can't go down (at least in long term system, short term system it can. I really guess I mean for them to be used together. Or even more preferrably for a more nuanced short term system (the current one is interesting, and not as straightforward as you'd initially think) edit: unverified information from others). There are certainly better ways to implement some system of this type, but in the end it's a simple way to address (almost all) concerns. (Tag fulfillment is a problem that isn't tackled by this. It's a serious problem and deserves some good ideas.)

A bit of unorganized rambling. Negative ratings are pointless. The purpose of  the  to find rankings here isn't to find what you don't want to read. It's to find what you do. A negative ranking symbolizes that someone didn't like something. If 10 people like something, and 10 people dislike something. It's a .5. if those numbers were instead 10000 each. It's still a .5. I would rather read the story that reached more people though. If all 20 of the people who read the first story were positive, I'd still probably want to read the more popular story rather than the higher rated.

Addendum: I've considered several counters to my points. Several very valid. A popular story can be popular through luck and not quality. A positive only system, with a low bar of entry can lead to high amounts of erroneous positive noise. Said otherwise, accidental massive score inflation (see retweet challenges) (addressable by timegating likes like rep). One common counter is invalid here. This system could be implemented with existing data like viewcount, and therefore automatically include existing finished stories. There would still be hiccups though. I'm sure there are others. Feel free to challenge this idea. If it fails to something simple then it's not good enough.


A bit of a non-analysis here at the end.

Apologies if anyone considers this parroting others. I read through the topic and found myself considering the viewpoints brought up. At this point I don't remember if anyone else suggested this, but I know that no one else did a point by point comparison against the pros and cons of the current system. 

I consider writer attrition to be the single largest problem with the current system. Post #23 sums up the problem with data very well and I thank them profusely for it. 
I've been reading on RR since it swapped from TL to fic hosting. Of all the stories and writers from back then, very few still write today, and many of them were promising and interesting. I selfishly want more people to willingly wordslave on the internet for me, and while the current system "works" (no shade or shame on site devs) it could absolutely be better.

What I want to rly know, is how you did that matrix-style Cthulhu-episode? 
At the very least the name, so I can look it up.

     I think a decent solution to high rating bias based on negative reviewers having higher attrition rates as stories get longer, would be to weigh ongoing ratings after the first chapter on a diminishing logarithmic scale. 

     Example: A stories' first ten chapters are bad, receiving more negative votes than positive. As the story progresses the writing improves; chapters 11-20 receive more favorable votes but each one is worth 8/10ths of a chapter 1-10 vote. For chapters 21-30 the story continues to improve and votes are now mostly positive, but votes are worth 8/10ths of a chapter 11-20 vote (64% of a chapter 1-10 vote). 
     This means that stories where the negative reviewers read fewer chapters have ratings less skewed towards the longer lasting positive reviewers. Positive reviewers can overcome negative ratings over time, but the diminishing returns will favor the number of positive reviewers over the number of reviews per person, while not making them worthless.
     The actual rate it diminishes is an example, but I believe the only way to fairly weigh positive reviewers versus negative reviewers against each-other in a chapter based voting system is to have the value of a vote be lower as the story length increases. It may even be necessary to slightly lower the starting point of positive votes on the curve if the intent is to have average stories tend towards a 0 on the 1/0/-1 scale.
     
     
     

this is a serious problem, but you lost me at the math. 0.5/5 stars.

I don't think the 5 stars are the problem. The problem is that that data is reduced to a single number and that number is used to order the fictions from best to worst.

As soon as you use the ratings to declare "best" or "top" you get the 5-star inflation problem.
It gets worse as due to avaraging the ratings giving a good rating that is not a top rating hurts the position of the fiction on the list.

You can try to fight that with more complicated formulas, but that will only make it more absurd and have more absurd effects.

Just don't combine the rating numbers but show the readers the numbers for each number of stars (and also the number of "not for me" voters)

Replace "'best' rated" with a list ordered by some vote system.

For example rename favorites recommendations and change them so that you can refresh them. Then have a list ordered by the number of recommendations that were added or refreshed in the last 3 months. And for the completed works order it by the number of recommendations (refreshed or unrefreshed) but only counting those from users that were logged in once in the last three months.

(I'd also like a way to write a review for a story without the need for a rating, but that is a different issue, though its lack is more annoying as long as 4-star ratings are hurting fictions).

I agree with a lot of the things pointed out in this post. Speaking from the perspective of someone who is new to Royal Road, I have found that the rating, as it is, to be oddly punitive. It is hard to get any traction on the trending threads unless you have a significant spike. I would love to know what the math behind some of their algorithms is. 

I feel like the new book I put out got a relatively warm response. I probably released too many chapters, too fast and at the wrong time, but I am rambling. My point is, that it was doing well pulling in ratings and it got a review into the trending review section and then I got my first .5 star review and everything fell off a cliff. The rating went from an exaggerated 4.8 (with too few ratings - around 20ish, but most in the 4-5 star range) to 4.1 with a single review. 

Before I got a chance to wrap my head around what had happened it felt like I had just done something personally wrong. It was a good learning experience as most of the negative feedback at the point was correctable or something that I could use to improve as a writer. However, the .5 outlier was so far off the scale that of course they didn't post a review so that I could figure out what in the writing had gotten that sort of reaction out of someone. 

I finally had some time to read other folks works on this website and start poking around the ones that I had flagged to read later when I first found this treasure trove. Once I talked to a few folks and got some context from commentary on other works I had a better understanding. I think that there are several proposed solutions here that would alleviate the kinks in the current system. 

I particularly like the solution of giving more weight to users with higher experience and reputation, i.e. those folks who have been around longer and been rated by their peers to have contributed. That just makes sense in this community. I mean, besides the personal accolades and the information that the experience and reputation represent as far as I can tell they do not have a tangible effect on the weight of a persons rating or review. That would help solve the issue of knee jerk reactions from relatively new readers to the site, myself included. Although I would like to think I fall into the "honest reviewer" category, but with a bit of fan hiding inside. 

I agree with many of the points raised here. The current 5-star system is flawed. What could be used to replace it.. I'm not sure.

Also with the recent addition of the "Not Interested" button/feature, 0.5 star ratings should have decreased (at least according to RR).

Which helps against people using the system as a way of marking the story as something they're not interested in. But doesn't avoid the 'trolls' and those who legitimately dislike the story.

I just want an option to disable the rating on my stories. 

Please, give an option to disable it to at least those who has less than 100 followers. Or until the story is completed. Or some sort of opt-out option.

Thank you.

I think the reality is relatively simple, a sizeable chunk of voter don't simply give a personal score and carry on, witch would have an arbitrarily personal meaning known to no one else.
Instead the review process goes more or less like this. 

ReaderA think your novel is (ex.) Appreciably trash. It worth 3.5 to readerA but can see it staying at around 4.0 rating.  Alas the overall reviews give it a 4.6 "this is outrageous"  readerA thinks, the situation could create stress in readerA mind for a variety of reasons. People don't like being wronged, in the example case readerA is not a generally agreeable person, so readerA slap a 0.5 on the novel. 
The overall rating drop to 4.4, "still way above what it deserve" readerA thinks, no recourse or appeal can sway the reader at this point. 

The same is true for readerB only he really like a trashy novel and give five star to bump the less then stellar average score. Only the five star are worth way less mathematically.

Rr could dictate the rating system meaning top down to the users, but surprise! It didn't have one for a reason, this system is supposed to work this way, more a weighted thumb up thumb down then a reviews score. 

If less people stressed over arbitrarily decided meanings they personally give to the numbers, it all would work has intended. 

13lack12ose Wrote: Also with the recent addition of the "Not Interested" button/feature, 0.5 star ratings should have decreased (at least according to RR).

I wonder it has any effect at all. It is totally broken.

• At least I would expect the story you mark as "not interested" to be hidden in the lists (Last Updated, ...). Instead they get highlighted there.
  

• There is no feedback about it,: There is no count of it in the statistics. Nothing visible outside of your account.
  

So that button is quite useless. You cannot share your hurt with it (as no one else sees it) and you even hurt yourself as the highlight forces you to remember that annoying story you'd rather forget whenever you look at a list.

So I can see why someone that wants to give a 1.5 star rating to use the "Not interested" button instead. But for the very disagreeable stories that need some outlet and thus get a 0.5 star rating the author might deem undeserved I cannot imagine that feature to be used (at least more than once, the first time needed to learn how broken it is).

Tldr?
Anyway, single rating is pretty stupid anyway, no doubt will be flawed, I want to place ratings on all genres and tags. (+ Grammar rating)

There is only one problem with that idea and that is that percentage of story read isn't valuable. More valuable would be time spent in the story. 

Both are cheat-able though.