So, this "more is better" idea is why I've been looking at the ratings for books on Goodreads much more than Amazon or B&N to vet potential book purchases. But the switch got me thinkin': Is there a difference between the three in terms of how each site's users rated novels? More fundamentally, is there a huge difference between average book ratings between sites? If so, what might account for such a difference?
So I thought I'd spend a post and take a look. Now, what follows is hugely unscientific. It's just a random sampling of five novels. But the interesting thing to me, and hopefully to you too, is the conclusions that can be (however tenuously rooted in logic) drawn about the ratings, both on a book-by-book and also on a sitewide basis.
(Liked = Four- and five-star ratings, Neutral = three-stars, and Didn't Like = one- and two-star ratings.)
1. Freedom, by Jonathan Franzen
Amazon: 895 total ratings. Average: 3 stars. Liked: 365 (41%). Neutral: 113 (13%). Didn't like: 417 (47%).
B&N: 2,143 total ratings. Average: 3 stars.
Goodreads: 19,079 ratings. Average: 3.66 stars. Liked: 11,627 (61%). Neutral: 4,494 (24%). Didn't like: 2,783 (15%).
My thoughts: I picked this one to look at because it of its huge hype — and the resulting push-back against Franzen for having the temerity to write a very good, popular literary novel. Looks like the push-back was most pronounced on Amazon. The percentage of "didn't like" ratings compared with Goodreads is astounding. My opinion is that this was a very good novel, and therefore Goodreads is far and away the most accurate here.
2. The Help, by Kathryn Stockett
Amazon: 3,435 total ratings. Average: 4.5 stars. Liked: 3,132 (91%). Neutral: 120 (3%). Didn't like: 183 (5%).
B&N: 6,911 total ratings. Average: 4.5 stars.
Goodreads: 123,656 ratings. Average: 4.46 stars. Liked: 111,937 (90.5%). Neutral: 8,760 (7%). Didn't like: 1,827 (1.5%).
My thoughts: Yep, everyone loved it — but, again, the same rating over more than 120,000 ratings is much more statistically relevant than only 3,400.
Amazon: 880 total ratings. Average 3 stars. Liked: 419 (47%). Neutral: 67 (7%). Didn't like: 394 (45%).
B&N: 1,407 total ratings. Average 3.5 stars.
Goodreads: 5,696 ratings. Average 4.0 stars. Liked: 4,271 (75%). Neutral: 1,128 (20%). Didn't like: 282 (5%).
My thoughts: As Goodreads' ratings show, when people actually rated this novel on its merit, it did well. This analysis shows how much the deplorable practice of rating a novel poorly to protest its eBook pricing can affect a novel's rating — when Fall of Giants came out last fall, it was ground-zero for this type of idiotic protest. Please, if you're one of the offenders, stop doing that.
4. Super Sad True Love Story, by Gary Shteyngart
Amazon: 166 ratings. Average: 3.5 stars. Liked: 93 (56%). Neutral: (17%). Didn't like: 44 (27%).
B&N: 229 ratings. Average: 3.5 stars.
Goodreads: 4,592 ratings. Average: 3.44 stars. Liked: 2,355 (51%). Neutral: 1,415 (31%). Didn't like: 787 (17%).
My thoughts: I was pretty lukewarm on this book, so I wanted to see if that lukewarmness carried over in all three sites. It did — and surprisingly consistently. Just about as many people liked it as didn't on both Amazon and Goodreads, and it got the lukewarm 3.5 on B&N.
Amazon: 443 ratings. Average: 4 stars. Liked: 312 (70%). Neutral: 31 (7%). Didn't like: 100 (23%).
B&N: 113 ratings. Average: 4.5 stars.
Goodreads: 9,455 ratings. Average: 4.29. Liked: 7,643 (81%). Neutral: 926 (10%). Didn't like: 805 (9%).
My thoughts: Of course, I wanted to find out how one of my favorite novels of all time did. And again, Amazon raters were the most critical (read as: Didn't get it), and Goodreads raters were closest to my own feeling about it. What sticks out here is the very low number of raters on B&N. I have no idea why that'd be the case.
Conclusions: This mini-analysis seems like good justification for continuing to use Goodreads. It has the most comprehensive, and in my view, most accurate cumulative ratings. I'll continue to be skeptical of (Read as: avoid) Amazon ratings. Raters there seem to rate books on lots of tangential issues like eBook pricing, slow shipping and cynicism, which drives the rating down. Boo. B&N to me is sort of a non-factor for ratings, as they don't give you an easy-to-read breakdown of the different ratings.
(Final Disclaimer: I'd never advocate that you pick books solely on these sites' ratings. Just making sure we're clear on that.)
What do you think? Any surprises here? Any of these three sites you tend to rely on more than the others?
What an interesting idea for a post. I love to see bloggers doing their own bit of research into the literary world, especially when it is out of their own personal curiosity. I've always trusted Goodreads reviews however I will often look at Amazon's 1-star reviews to see if I can find any legitimate reasons for potentially passing up a title. Hope to see many more posts by you in this fashion.
ReplyDeleteThis was a really interesting survey - and it only confirmed what I have long noticed over at Amazon - the number of smack downs (ie: rude and critical reviewers) is enormous. It is one reason I do not post reviews there...the commenters are just plain vicious sometimes. I much prefer sites like Goodreads and Library Thing to get my book recommendations (not to mention my favorite bloggers!). thanks for an interesting post!
ReplyDeleteI think it's a slippery slope saying that a site's reviews are "more accurate" than others. Yes, marking something down because of price point is not accurate, but you can't say that, because a site reviews generally match up with your own feelings on a book, they're more right. It would be better to say that you trust Goodreads more because the readers there tend to align more closely with your own taste in books. (I would suppose that is because Goodreads readers are probably more well-read in general, but again, that's just a guess). I mean, I hate Franzen, but that doesn't make Goodreads more or less accurate does it?
ReplyDeleteAgree
DeleteVery interesting to see it all laid out. I do sometimes check LibraryThing ratings. But I have an advantage in not reading reviews until after I've read a book, never before. Then I read some of each; high, low or medium-rated, concentrating on the whys of how much they liked it or didn't. I suppose that begs the question how do I know if a book is worth reading? A dozen little ways like others do, I just don't use reviews to help me decide.
ReplyDelete@PadfootandProngs - Thanks! I wish I'd had more time to put this info in bar chart form - so it's easier to read quickly. Ah well, maybe next time.
ReplyDelete@Wendy - Yeah, Amazon has become really bad regarding the increased incidence of ignorance. If you haven't seen The Ken's Amazon Wednesdays feature, it's worth checking out (http://trippingabout.blogspot.com/2011/05/amazon-wednesday-regrets-to-inform-you.html). I often get a chuckle out of some of the one-star reviews there.
@home - I suppose you're right that "accurate" isn't the most accurate word there. But that's really beside the point of the post. It's not just about Goodreads (or whatever site) matching up better with my own taste, though Goodreads does seem to fit that bill. My point is that Goodreads gives a better feel for what the majority of readers think about a book based on its merit, and therefore a more complete picture about how good readers in general think a novel is. You may hate Franzen, but most people (61%, in fact) like him and thought his book was really good - and that's what I want to know when I'm using those sites to vet books (among many other criteria - like, your blog, for instance). Goodreads seems to do a better job of separating the BS from the legitimate commentary. Would you agree with that?
@Sandra - You know, I've never looked into Library Thing at all - but you're the second commenter in four posts who checks it. And I agree that it's very worthwhile to read negative or lukewarm reviews of a book to find out why a reviewer wasn't a fan. A lot of times, on Amazon, though, you get "I was bored" or "I didn't connect with the characters" with not a whole lot of supporting evidence why.
I was just using the goodreads app while I was browsing around Barnes and Noble today to help narrow down my daunting pile of books I wanted to buy. I do like the sheer number of reviews that are posted on goodreads, it presents a bigger scale and since you can't but anything from them you don't get any of the biased price bullshit.
ReplyDeleteAlso I haven't experienced anyone getting irrationally angry on goodreads and trashing reviews. I'm sure it happens, but it's gotta be less than over at amazon.
I'm actually surprised that more people don't rate books on Amazon. It shouldn't surprise me - I buy from Amazon all the time, but only post reviews once in a while. It's kind of a pain in the ass process, with little reward.
ReplyDelete@Greg - You're absolutely right. Goodreads does give a better feel and is more legitimate because of the kinds of readers it attracts. I have my own biases as well, of course. I tend to think of Amazon readers as less serious and therefore less capable of really "getting" literature. I don't think I would really pay attention to those reviews because I know I can get better elsewhere. Like Goodreads. I completely agree with you on the conclusion you make and I really only go to GR for reviews anymore. You know I like to call you on your word choices though - I couldn't let it pass ;)
ReplyDeleteVery interesting! I really like the reviews on LibraryThing. First of all I like how easy they are to access. But mostly, I think, I like the fact that readers seem to feel more free to post negative reviews. And I also think I "trust" them as "authentic" more than I do the reviews at Amazon. I don't think GR is as helpful because people can't post half stars. And I think B&N readers seem to be more gushy than LT. It's sort of interesting too how the different review sites attract (or develop) different reviewing personalities!
ReplyDeleteThanks for doing this bit of research and posting it. I've long suspected ratings are skewed by publishers, friends and fans of authors and various people with political axes to grind so I've always taken them with a grain of salt. It's interesting to see how close they all are.
ReplyDeleteThat said, I've never based a reading decision or book buying decision on any ratings systems. I do pay attention to blogger reviews and mainstream reviews as well. But rating systems, never.
Besides, most of what I read has very few reviews anyway.
I don't normally go by the opinions of these sites; as sometimes they aren't exactly correct. Some people's thoughts on a book might not be mine... and what I see in a book might not be what other see in it. So, what I do is talk to people who have read the books I'm reading and see what they have like and disliked about them; and then see if I can find those things in those books.
ReplyDeleteHowever, I also read reviews and find out if the books are also of the same opinions of book reviewers; if not, well, then it gives me an even bigger scope to work by.
Good thoughts. First off, I'd equate "reviews" and "comments". Amazon's system for reviewing makes it very easy for trolls. In fact, it almost rolls out the red carpet for them. Systematically, a person can comment 100% anonymously. At Goodreads, reviewers must use their real identity (or systems are in place to make it difficult to not). As such, they are more likely to leave civil reviews knowing that reprisal may occur. Note the recent switch on TechCrunch to Facebook Comments for the purpose of eliminating the trolls: http://techcrunch.com/2011/03/01/pros-cons-facebook-comments/
ReplyDeleteMy best guess on this is that Amazon and B&N are book buying sites, while Goodreads is a book readers' site. I'd guess that most serious readers get at least a significant portion of their books from libraries, personal loans and physical bookstores. If you don't buy that book from one of the online retailers, you don't usually rate it there. To test this explanation, I would predict that books published before 1995 (which people read before Amazon existed) would have more accurate ratings than books published more recently.
ReplyDeleteAnyway, great post. Thanks.
I've never paid too much attention to reviews -- especially the ones on Amazon. It's a site that specializes in selling everything, and therefore attracts everyone, and not everyone is bookish. Which is fine, but it makes for some really unhelpful reviews ("I liked it. I was better than CATS.")
ReplyDeleteI find most of the additions to my TBR list from people I know, blogs I read, and my mom (of course). Generally the only time I'll look at the reviews on Barnes and Noble is when I'm doubtful about ordering a book, and want to see what other people are saying.
I've never hopped onto the Goodreads wagon. I was on LibraryThing for awhile, but I have other methods of cataloging my books (Listography and Bookpedia), so being there felt redundant. And it's the same with Goodreads. I had never really considered that the reviews sections would be good places to look. I'll definitely be doing so in the future. Great post!
I haven't trusted Amazon for book reviews in a long time,due to the likes of Harriet Klausner(their generic reviewer gal)and people putting out vindictive one star ratings based on agendas other than the book's merit.
ReplyDeleteThe only good thing about either B&N or Amazon in the review section is checking out their links to the book's reviews in publications like Publisher's Weekly(B&N does the best job in that department).
Good Reads does offer a nice diversity of opinion,a bit more so than Library Thing,in my opinion and it's become one of the first places I go for some real insight into a potential new read. By the way,I'm reading one of the books on your profile list here(The Help)and it's better than I had anticipated.
Couchpotatocritic - your parenthetical, "I liked it. It was better than CATS" made my morning.
ReplyDeleteIt's slightly off topic, but I do love The Complete Review, with many links to what the litarary critics have to say. It is limited, but I always like to check the site.
ReplyDeletehttp://www.complete-review.com/main/main.html
I stay away from Amazon ratings of books. There are too many people who complain about eBook pricing or slow delivery times.
ReplyDeleteGoodreads is great, though I've noticed that literary novels tend to rate lower than other genres like YA, SFF, and romance.
And I never read reviews at B&N, for whatever reason.
What about LibraryThing? The ratings there are usually closest to my own opinions.
@Letter4no1 - Yeah, you're right - the key is that they're not a seller. Still, though, a lot of the Amazon reviews are a lot more negative than the Goodreads ones, which isn't beneficial to the seller. Maybe it's because the jerk reviewers who think their negative reviews will negatively affect sales - so they choose to post there. At any rate, you're right - Goodreads is still more civil.
ReplyDelete@Sarah - Little reward, is right! I do post reviews there once in awhile (that I've already written for here), and the reward is often some jerk making a totally off-base comments, sometimes involving crazy politics and/or conspiracy theories. ;)
@Home - Okay, then. Common ground achieved. And thanks for being my pseudo-editor. ;)
@rhapsodyinbooks - Interesting - the worst thing about Amazon seems to be that many of its negative reviews (as couchpotatocritic points out below) are not rooted in any reasoning. So I'm intrigued by the idea that Library Thing seems to attract better negative reviews. No, GRs doesn't allow you to post half stars, but I do like that their average rating is a decimal (3.68, i.e.) instead of just in half-star increments.
@CB - Aw, my resident cynical reader. ;) Sure, every review has to be taken with a little bit of a grain of salt - there's usually some sort of agenda, even if it's subconscious.
@Mozette - Well, yeah, of course some people's thoughts may not be your own. The idea of glancing at those ratings is just to get some feeling for what the community thinks. I've never made a book-purchase decision solely based on Amazon ratings, either. But it sounds like you're saying that a book review is only valid if you agree with it. That seems like it could be problematic.
@Clinton - Ha - easy for trolls. But good point about how Goodreads reviewers aren't nearly as anonymous - at least in the sense that most are connected to others, and then are moderately accountable for what they say. Whenever I get an anonymous comment on the blog, my first instinct is to delete immediately - especially if it's negative and doesn't say anything of import. (And I agree with you about couchpotatocritic's comment - high comedy!).
ReplyDelete@Listener - I think you've put it best, as the way to account for the differences. I don't know that I'd agree that most serious readers get their books not from B&N or Amazon, though (that's where I get most of mine - the former, not the latter...and with some indie bookstores mixed in). But your explanation still makes sense. Interesting theory on pre-1995 books, too. Would that I had the time to test it. ;)
@couchpotatocritic - Well, as I think we've established, you win today's Commenter Award for Funniest Comment "I liked it. It was better than CATS." And I agree - blogs and other recs and a million other criteria are more important to me than those ratings - but they are still an interesting factor. It seems as though GoodReads has evolved quite a bit since its infancy - from a place primarily to catalog books, to an actual full-fledged social networking readers' site. I really like it there!
@lady t - Good point about B&N linking to other sources. Superheidi below points out another site that does that well - I think that's a huge plus for a review. I just finally bought The Help a few weeks ago - succumbed to the hype. I'll be interested to hear your thoughts!
@superheidi - Thanks for pointing that out. I hadn't heard of that before, but I love the idea behind it! I wish that site was a bit more comprehensive, though.
@Melody - Interesting point about literary novels being rated lower than other genres - though, I suppose that makes sense (broad over-generalization alert) given that readers of those other genres generally don't put forth the "intellectual rigor" into reviews as readers of literary novels put into theirs.
Greg, I hadn't thought of it that way. I read a variety of books; however, I find that going by what other reviewers say can sometimes be miss-leading. It's the same with movies; what one reviewer says about one movie could be totally different to what another says about the same movie. However, until I go and see it, I'm not going to go by their judgement.
ReplyDeleteI usually check reviews on both goodreads and amazon (don't usually go to B&N). I don't usually pay much attention to the distribution or the number of reviews (to see which has a bigger sample size), because I usually just read the negative reviews to see why people didn't like it, to see if we share similar views on books. A lot of times on Amazon, 1 star = "books didn't arrive", "you're charging too much for kindle!" so not always real reviews.
ReplyDeleteAs a former business analyst, I got a huge kick out of this post; however as a book reviewer, I only read other reviews on the same book if it leaves me feeling shocked, confused, or simply curious. Chelsea Handler and Sarah Silverman memoirs are perfect examples. While their books please and humor me, I know that the general public generally disagrees. I always visit Amazon if I want to seek out negative reviews and I visit Goodreads if I seek fair, constructive criticism or in-depth literary analysis. Mr. Peanut by Adam Ross was one of those novels I visited Goodreads for (since it took me a few days to "get it" after I read it); whereas I loved to read the trashy reviews readers left for Chelsea Handler on Amazon. So there you have it; I get sick pleasure reading negative reviews on Amazon and I feel love for other reviewers after perusing their opinions on Goodreads.
ReplyDeleteI've only rated books that I've bought on Amazon - & always favorably; I think I'd better take a look at Goodreads.
ReplyDeleteI agree. I prefer Goodreads when it comes to ratings. It gives a fairly accurate rating on books solely because it is more intrinsic. Library Thing is also a good place to look for fair ratings but because it has certain fees (if you wanted to keep ticking more than 200 books), it does not rave to the freebie-loving public.
ReplyDeleteGoodreads ratings are mainly fake and purchased by the publisher. Amazon ratings are more genuine because they are reviewed and you can see if the review is from a purchaser. Simples!
ReplyDeleteYou've got that exactly reversed.
DeleteI just did an analysis on my blog of how well Goodreads, Amazon and LibraryThing predicted my own ratings of books. While none of them did great, all three sets of ratings did in fact correlate positively with my own ratings. Goodreads and Amazon did about equally well, which I think is interesting given all the suggestions that Amazon ratings are skewed by various factors. LibraryThing seemed to match my tastes slightly better which surprised me since it usually had fewer ratings.
ReplyDeleteYou are of course correct that "data" is a plural, although I wonder if language experts accept its being used with a singular verb, as that seems to be a common preference. In any event, if you are going to be strict about honoring its plurality, your first sentence should read:
ReplyDeleteI'm no mathelete, but I do know this: The more data there ARE about something, the more complete the picture of whatever it is THOSE data are measuring.
It's a slow day.
One issue with this is how you've decided that the scores match on all 3 sites. When you hover over the stars to rate on Goodreads, it gives you strict guidelines, and most stick to it because it makes the site more understandable. 3 is 'I liked it' , 2 is 'it was OK', whereas on the others 'i like it' is 4 points and 'OK' is 3. Goodreads basically shifts everything one point lower, with only one negative score (1), one neutral one (2) and 3 positive ones (3,4 & 5). It makes sense seeing as people don't usually finish what they don't like and can't really give subtle gradations of 'don't like', and also to give subtler differentiation between those they do like. Though, of course, that makes the differences even more pronounced for most of your results, where the Goodreads ratings are actually numerically higher, despite those numbers meaning something even higher than they do on the other sites. On Amazon you have to actually write a review to get your rating counted on the page, which the far lower numbers is a result of. Someone who really likes it would feel that if they rate it they have to write something substantial about it, and then maybe put it off because it's time and work. Whereas to say something is crap is not time-consuming (it seems to me negative reviews are generally shorter). That would explain why you get more negative ratings on Amazon, because most who like it are not rating it, because they haven't got the time for the reviews. Probably it's a younger demographic on Goodreads too, so that will make a difference. Also this might exacerbate the first factor, because younger people are less likely to sit for a while indoors and write a review of what they like, but they'll happily just click the stars on Goodreads. So you end up with more ratings from the young on Goodreads, and more ratings from those who like the books on there too. My guess at least. Ratings are spread internationally on Goodreads too. The Amazon site you look at would be for your locality. The ratings on amazon.com are separate from the ones on amazon.co.uk, amazon.de, amazon,jp, etc., etc., but Goodreas ratings are international. Not that I know how that would change it, but surely in some way. So it's surely not just more data, but different data to an extent, via a system that encourages different interaction, and that collects data from a probably younger and more international demographic group.
ReplyDelete