Hotel review sites aim for more credibility By Dennis Schaal / September 17, 2009 Share 1 -- Though consumer hotel reviews are popular, they have also often been suspect. So, recently, several consumer websites and one professional service have introduced new methodologies designed to inject more credibility into hotel ratings. The latest to offer a solution is Shermans Travel, a deal publisher and media company. Shermans recently debuted its Meter Rating, which it positions as a hedge against problematical consumer reviews because its algorithm gives roughly equal weight to consumer reviews drawn from numerous websites, professional reviews and its own independent critiques. Shermans grades hotels by aggregating consumers' reviews from sites such as TripAdvisor, Yelp and Yahoo Travel; uses so-called professional critiques from Concierge, Frommer's and Fodor's; adds Shermans' own assessments of the properties; computes the percentage of positive reviews; and designates a corresponding color code. Shermans claims that if a hotel or perhaps the property's competitors successfully manipulate "consumer" reviews on TripAdvisor or another consumer-review website, the Shermans rating system would offset that manipulation. Shermans Travel is not alone in grappling with the user-review issue. Although a recent Forrester Research study found that 70% of travelers trust user reviews, mainstream media and the blogosphere have reported instances in which hoteliers or their public relations firms have abused the system. Oyster Hotel Reviews, which predominantly publishes hotel reviews by journalists under contract, recently felt the need to add consumer reviews, but it took measures it believes will limit ratings manipulation. Oyster now requires travelers to log in using Facebook Connect if they want to gripe or fawn about a property. Facebook Connect requires user names and passwords, and readers of the reviews can view the public portion of the reviewer's Facebook profiles. Oyster believes that forcing consumer reviewers to shed their anonymity will result in more honest reviews and will give readers the ability to determine if the reviewer and reader have parallel interests regarding family or business travel. Oyster added consumer reviews because more travel content encourages users to spend more time on the website doing trip-planning, which gives advertisers a better chance to entice them to book hotels or vacations. The downside for Oyster.com in enforcing the Facebook Connect prerequisite for review writing is that it won't get as many consumer reviews as sites such as TripAdvisor, which takes consumer reviews without restriction. The same drawback applies to Priceline.com. which requires consumers to complete a stay before they can review a hotel. Fewer reviews and less content could mean lower advertising revenue for Oyster. The context for all of this experimentation in hotel reviews "is that it's still a bit of the Wild West out there, and some people are trying to tame it," said Jay Karen, president and CEO of the Professional Association of Innkeepers International, based in Haddon Heights, N.J. Karen, a critic of TripAdvisor, characterized Shermans Travel's steps as "a move in the right direction. It gives consumers more well-rounded reviews in one place." He was less enthusiastic, however, about Oyster's use of Facebook Connect because he feels anonymous reviews have their place. For example, repeat guests might feel more comfortable penning anonymous reviews, Karen said. These new review models are designed to tap into the success of TripAdvisor as a global media company while avoiding the controversy that its reviews have generated as hotels and their representatives have been caught manipulating critiques over the years. TripAdvisor itself has not made any significant moves to change its review model because the system is very successful financially and is spearheading parent Expedia Inc.'s international expansion. Even so, it has made a few tweaks to review policies over the years. For example, a couple of years ago, the company began flagging hotels with red warning notices next to their displays on TripAdvisor if the properties were accused or suspected of foul play. As a critic of its policies and a hotel association executive, Karen meets periodically with TripAdvisor officials to provide them with a hotel industry perspective. He said that in these get-togethers, TripAdvisor officials always seem very open to new ideas, but Karen likened the prospect of TripAdvisor making wholesale modifications in its hotel-review policies to the difficulties in repositioning an ocean liner: "They are slow to change direction." Under Shermans' color-coding system, if the reading is green, it means the reviews generally are positive. Yellow means the reviews are mixed, and red points to a preponderance of negative reviews. "Ours is the only site that factors in user and expert reviews, displaying all results side by side," said Darren Frei, editorial director for ShermansTravel.com. "Most savvy travelers seek out a variety of opinions from a variety of sources before making a hotel decision. We make that evaluation process several degrees easier by taking the guesswork and confusion out of the equation, not to mention the pesky task of searching for the same hotel across multiple sites." When researching the Waldorf-Astoria in New York a couple of weeks ago, the Shermans Meter aggregated 1,310 user reviews from a variety of websites, professional critiques from Fodor's and Frommer's (Concierge doesn't rate the Waldorf) and the opinions of Shermans' own experts to arrive at a 76%, or green, rating. In comparison, TripAdvisor, aggregating 1,051 traveler reviews of the Waldorf, assigned the property four stars and a 72% TripAdvisor Traveler Rating. In this one comparison of a large well-known property, the TripAdvisor rating comes out looking fair, given that it was roughly on par with Shermans' review system. Waldorf's respective ratings on TripAdvisor and Shermans largely are consistent with most review sites. However, analysts note that it is much easier to tip the balance of a rating for smaller, nonchain properties. In the end, of course, no review model is perfect. For example, the professional reviews that Shermans assesses as a counterpoint to consumer reviews include Fodor's ratings, which aren't actually professional ratings. Instead, Fodor's provides a 3.4 "member rating" (out of five) of the Waldorf, meaning the Fodor's rating is another consumer-review compilation. And, Frommers.com, another professional service that Shermans throws into its algorithm, doesn't post negative reviews at all, which could skew results, depending on how much weight Frommer's gets in the Shermans Meter Rating. Frommer's hotel ratings are: zero stars (recommended), one star (highly recommended), two stars (very highly recommended) and three stars (exceptional). Frommer's rates the Waldorf three stars [its highest rating], or exceptional, and is lone among the review sites used by the Shermans Meter Rating in giving the Waldorf such high marks. Still, the Shermans Meter Rating provides a counterweight to instances of consumer-review hanky-panky, and thus makes a valuable contribution to the hotel-review conundrum. Meanwhile, Star Service Online, a sister company of Travel Weekly, has a different slant on the consumer-review trend, accepting "consumer" reviews of hotels and cruise ships only from travel professionals. Star Service Online, a subscription service offering professionally written reviews for travel agents, recently added an Agent 2 Agent feature, enabling travel professionals to rank things like whether a hotel is easy to book, how quickly it pays commissions and whether it satisfied a client's needs. Like TripAdvisor's consumer reviews, Star Service Online doesn't require an agent to prove that a client actually stayed at a property as a prerequisite for a review. But unlike TripAdvisor, Star Service sees little value for travel agents in critiques written by travelers. "Travel agents should not trust consumer reviews, and as professionals, they should not use the same sources that are available to consumers," said Ned Tobey, director of sales and marketing for Star Service Online. "They should utilize professional tools designed specifically for travel agents rather than putting their businesses at risk with the use of untrustworthy consumer comments." Forrester Research analyst Henry Harteveldt agrees that agents need an industry-oriented forum. "Agents need and value professional insight from one peer to another," Harteveldt said. "Agents will be looking at things through a certain lens." But he also cautioned that the issue of professional-vs.-consumer reviews should not be an either/or proposition. "It is good for the agent to have both," Harteveldt said. "The consumer reviews serve as a reality check. Getting the sense for how consumers feel allows the agent to say to a hotel sales rep or a hotel-management group, 'You may think you are great, but your customers don't.' " As for what tack hotels should take, there has been much debate among hoteliers about whether they should post TripAdvisor and other consumer reviews on their websites. Harteveldt argues that hotels "should control their own destiny" regarding consumer reviews and not cede control of guest reviews to third-party websites. Hotels should consider posting reviews from TripAdvisor, Yelp, Yahoo or other review sites on the hotels' own websites, but they should encourage their guests to post reviews directly on their websites, as well, he said. "Negative reviews are out there," Harteveldt said. "Take steps to fix the problems, not [public relations] campaigns."