How could they get it so wrong? A travel marketing expert critiques traveler sentiment surveys

Sandra Thomas-Comenole is an award-winning travel marketing professional and behavioral economist. She can be contacted at sandra.thomas
[email protected]

Several months into the Covid-19 pandemic, companies started turning to market research, such as travel sentiment surveys, to make seemingly informed decisions. That is unfortunate, because these surveys are not effective or accurate in predicting current and future travel trends, according to the findings in my newly published white paper, "Are Customer Sentiment Surveys a Good Indicator of Current and Future Travel?"

The white paper is a deep dive into the accuracy of travel sentiment surveys in predicting travel behavior through three data-driven studies. These studies compare air travel sentiment to U.S. weekly air travelers and air ticket purchases, respectively. The most striking finding is that while travel sentiment has remained relatively steady since April, travel behaviors, including air ticket purchases and air travel passengers, have increased dramatically.

These differences, in addition to a lack of correlation between travel sentiment and objective data, is astounding, pointing to travel sentiment surveys not being a good indicator of current and future travel behavior.

How could they get it so wrong? Surveys are known in market research for having the opportunity to be too subjective to be accurately representative. There are many reasons why subjective data can fall short. These errors usually fall into two main categories: survey taker errors and poor methodologies.

From the survey taker standpoint, we can use behavioral economics to explain much of these discrepancies.

Survey takers are experiencing what economists like myself describe as "availability heuristic" and "affect heuristic." Availability heuristic means that people make decisions based on the information that is currently and readily available. In this context, that means that the negative information that is circulating about safety in travel skews answers more negatively.

Meanwhile, affect heuristic describes the fact that subjects can't always predict their future actions, desires and motivations accurately. They are judging how their future self will feel about a certain topic with today's feelings. This can be biased due to whatever the current feelings and information they are experiencing while taking the survey or immediately prior to it. For example, someone may have just read a headline about a new bunch of Covid-19 cases. They then take a survey and their response will be skewed negatively.

Further, there are several reasons why subjects give false information during market research: subjects can boast, become defensive, want to be socially accepted and acceptable or want to be polite. They might be in a hurry, annoyed or downright mischievous; they can't always predict their future actions or they simply forget. These are well-known issues in all survey and interview research.

By the same token, many companies -- even big, well-respected ones -- are using poor methodologies in their research. While considering multiple data sets for this white paper, I observed several data-collection methodologies that skewed the results, including sample selection bias and question framing.

In a number of data sets, the researchers screened for "regular travelers." This is a form of Sample selection bias, a systematic error caused by choosing nonrandom data for statistical and qualitative analysis.

The bias exists due to a flaw in the sample selection process, where a subset of the data is excluded due to a particular attribute, sampling technique or even geographic location. This results in a biased sample, defined as a statistical sample of a population in which all participants are not equally balanced or objectively represented. The exclusion of the subset can influence the statistical significance and produce distorted results.

A classic example of a biased sample and the misleading results it produced is the 1936 U.S. presidential election. In the early days of opinion polling, the American Literary Digest magazine collected over 2 million postal surveys and predicted that the Republican candidate, Alf Landon, would beat the incumbent president, Franklin Roosevelt, by a large margin.

The result of the election, of course, was the exact opposite. The Literary Digest survey represented a sample collected from readers of the magazine, supplemented by records of registered automobile owners and telephone users. This sample included an over-representation of affluent individuals who were more likely to vote for the Republican candidate.

Moreover, I observed several travel sentiment surveys that framed questions. In behavioral economics, framing means the manner in which a question has been presented. Survey makers frame questions by:

• Asking leading questions (for example, "How dangerous is it to travel during the pandemic?")

• Asking loaded questions or multiple questions at once.

• Using absolutes (such as Yes/No questions or using terms such as "always").

Framing questions can skew results. I even saw one company that framed each of their questions, "given the global pandemic." This type of framing will lead to negatively skewed results and can completely invalidate the responses of the survey takers.

With the current state of the industry and several green shoots that seemingly signify the road to recovery, it is important to understand the recovery signals that are out there. While travel sentiment surveys are seemingly everywhere and can be a good resource, they shouldn't be the only tool that business leaders use to make multi-million- or billion-dollar decisions.

Instead, business leaders should look toward objective data, such as how many people are flying, how many tickets are purchased, etc., to make these decisions.

It is also important to look at data from destinations, hotels, attractions, restaurants and national parks that have opened to see the progression of travel behavior, while keeping in mind the restrictions, fears and risks that are in place while they are opened.


From Our Partners

2021 Quebec Webinar
Bonjour Québec: It’s Vibrant Cities and Culture
Watch Now
USTOA 2021 horizontal
USTOA 2021
Read More
2021 Outlook Webinar
Outlook 2021
Watch Now

JDS Travel News JDS Viewpoints JDS Africa/MI