Amazon’s review-for-free-stuff policy wrecks its reviews
These days, you can access a massive library of independent product reviews from sites like Consumer Reports and The Sweethome, and you don’t even have to get off the couch. But who wants to bother with all that when you have Amazon (AMZN) product reviews right next to the “add to cart” button?
This is the power Amazon product reviews wield today. That power has swelled as more people now shop on Amazon using mobile devices than desktop computers, where it’s harder to jump from Amazon’s app to a browser and back if you want more than just the Amazon reviews.
Unfortunately, the whole review system is corked, thanks to an Amazon policy that permits its vendors to trade free or discounted products for reviews.
Amazon forbids fake and paid reviews, but the company has a massive loophole—a “sole exception” in its policies for free or discounted product. If vendors provide free products in exchange for reviews, they have to tell people getting the freebies they welcome positive and negative feedback. For their part, reviewers just have to add a disclaimer in the review text itself.
These disclaimers have become so common—one in five reviews, according to review aggregator and analyzer ReviewMeta—many consumers may accept them without question.
As if diluting more objective reviews isn’t bad enough, Amazon accommodates vendors even further by factoring these paid reviews in the actual one-to-five star rating system, although an Amazon spokesperson noted verified purchases—which now must be at a typical price—and helpful reviews were given more scoring weight last year.
Even at a lesser weight, counting these incentivized reviews can skews scores, making a big difference in a product’s overall rating. According to ReviewMeta’s data, incentivized reviews are 0.38 stars more positive than their more-objective counterparts. That might not seem like much, but when the average overall review is 4.4, it can boost products into the very highest percentiles on the curve. That makes them come up at the top of the Amazon search results.
This isn’t just a problem for vendors who give away products in exchange for reviews—it’s a problem for consumers. ReviewMeta pointed to a black umbrella that got 121 reviews with an average score of 4.7 stars. Only four reviews came from people who didn’t get a free umbrella. They averaged an abysmal 2.7 rating.
Amazon’s dominance in the marketplace has lit a fire under vendors to thread this loophole. Last year, survey data of 2,000 users from e-commerce company Bloomreach showed that 44% of people bypass the web and search for products directly on Amazon.
The space has gotten so hot that it’s not hard to find a single product with thousands of reviews anymore. With stakes like those, how could a company not buy five star reviews?
Quantifying the damage Amazon’s policies have done to its own rating system, ReviewMeta used an algorithm to crawl reviews looking for common disclaimer language and found that 20% were incentivized.
ReviewMeta’s data isn’t just an interesting stat—it’s a consumer tool you can use to separate paid reviews out, letting you see what ratings would be without the fawning, paid reviews that could skew the scores. Consumers can also look at other interesting metrics, such as the average reviews changed over a time period.
In action it’s a slightly annoying process, partly because it feels like an unnecessary stopgap considering Amazon allows this behavior. To use it, you copy and paste an Amazon link into the ReviewMeta website—real hoops you need to jump through if you’re shopping on the app. There is a Chrome plugin option if you’re on a desktop, though.
If that sounds like too much to do, it might help just to remember the stat that one in five reviews should raise an eyebrow. It also might be helpful to read the three-star and “most helpful” reviews. Three-star reviews are often worth reading, standing apart from the usual feedback binary of “awesome” and “awful.”
Of course, sometimes you’ll see a “most helpful” review with that disclaimer, showing just how numb we’ve become to this phenomenon—or how effective a reviewers may have become at playing down the fact that they got a free product in exchange for a review.
Update: Amazon has responded with clarification of its scoring algorithm, noting its weighting and verified purchase requirements. This has been reflected in the text.
Ethan Wolff-Mann is a writer at Yahoo Finance focusing on personal finance and tech. Follow him on Twitter @ewolffmann.
Read more:
New iPhone can’t support two pairs of wireless headphones
This four-letter word will get you better customer service on the phone
Apple spits on history by removing the headphone jack
Owners of the $10k gold Apple Watch react to news their watches are now obsolete
