Study Confirms Amazon Reviews Are as Reliable as Media Critics
“A new study from Harvardsuggests that Amazon reviews are, taken together, just as trustworthy as those from professional critics.
The study, carried out at Harvard Business School, analyzed the top 100 non-fiction reviews from 40 media outlets, including the New York Times, the Guardian and the Washington Post, between 2004 and 2007. The researchers then compared data from review aggregator metacritic.com to that from Amazon.”
On the whole, there was broad agreement—when reviews were taken in aggregate— between critics and Amazon reviewers.
The subtle differences, however, are interesting. Amazon reviewers tended to give debut writers more favorable write-ups—which the Harvard academics point out is in fact a weakness on the critics’ parts. They are, according to the researchers, a little closed-minded and “slower to learn about new and unknown books”. Conversely, paid critics are more likely to write positive reviews about prizewinning authors. That also, if anything, reflects badly on the critics.”
What Makes a Critic Tick? Connected Authors and the Determinants of Book Reviews
|Published:||April 26, 2012|
|Paper Released:||March 2012|
|Authors:||Loretti I. Dobrescu, Michael Luca, and Alberto Motta|
“The professional critic has long been heralded as the gold standard for evaluating products and services such as books, movies, and restaurants. Analyzing hundreds of book reviews from 40 different newspapers and magazines, Professor Michael Luca and coauthors Loretti Dobrescu and Alberto Motta investigate the determinants of professional reviews and then compare these to consumer reviews from Amazon.com. Key concepts include:
- The data suggest that media outlets do not simply seek to isolate high-quality books, but also to find books that are a good fit for their readers. This is a potential advantage for professional critics, one that cannot be easily replicated by consumer reviews.
- Expert ratings are correlated with Amazon ratings, suggesting that experts and consumers tend to agree in aggregate about the quality of a book. However, there are systematic differences between these sets of reviews.
- Relative to consumer reviews, professional critics are less favorable to first-time authors. This suggests that one potential advantage of consumer reviews is that they are quicker to identify new and unknown books.
- Relative to consumer reviews, professional critics are more favorable to authors who have garnered other attention in the press (as measured by number of media mentions outside of the review) and who have won book prizes.”