Destination marketing organizations spend heavily to measure visitation numbers. But one of the most essential metrics for the travel industry -- total visitation -- is also one of the industry's most elusive.
Since there exists no single third-party organization that tracks total visitation for cities, counties or states, there is no common barometer. Convention and visitors bureaus each have their own methodologies and data collection models, often built by or with the help of independent research firms, for calculating how many people are traveling to their destinations.
"It's very difficult to measure this animal. There isn't a turnstile at the city or county limits," said Warren Hull, vice president at San Diego-based CIC Research, which is contracted by the U.S. Commerce Department's Office of Travel and Tourism Industries to conduct its survey of international air travelers. OTTI data provide one of the most widely used and accepted indicators for international arrival figures and international traveler behavior.
So, when New York Mayor Michael Bloomberg last month decreed that for the first time since 1990, New York was "the most popular tourist destination in the U.S., ahead of both Las Vegas and Orlando," the veracity of that statement was difficult to prove or dispute.
For one thing, comparing the number of total visitors to each city is not necessarily comparing apples to apples, because it generally involves comparing one city's self-derived estimate with another's.
"There is a challenge in finding resources where we can make comparisons" with competing markets, said Kevin Bagger, senior director of marketing at the Las Vegas Convention and Visitors Authority. "In terms of visitor volume, there is not one methodology for determining visitor counts."
Many CVBs start with the OTTI's international traveler data. The OTTI's base numbers come from the Department of Homeland Security forms filled out by nonresidents arriving from overseas, meaning the numbers exclude Canada and Mexico.
The Transportation Security Administration's numbers are supplemented by the OTTI's survey of international air travelers. In all, the OTTI surveys at least 70,000 non-U.S. resident travelers annually, excluding Canada. The survey consists of 29 questions that cover such topics as destinations visited while in the U.S., airlines booked, accommodations used, methods of booking and demographic information.
According to OTTI Director Helen Marano, the OTTI's numbers are objective.
"We're the honest broker. We are the unbiased, third-party resource putting the information out there," she said.
Even so, she conceded that some other research firms are also "decent and honest brokers."
The survey is handed out on flights of airlines that have agreed to cooperate with the OTTI, but increasingly, with airlines struggling with labor shortages, the surveys are being handed out at international departure gates, often with the support and cooperation of airport staff.
The OTTI tested an e-survey approach in 2008 in which travelers could fill out an online form when booking an international flight on the Web, but the OTTI deemed the e-surveys less cost-effective than the paper surveys.
When the survey was established in 1983, "the primary use was to provide a more robust and accurate estimate for the balance-of-trade data," Marano said. "Now, it's a multipurpose industry service."
And while the survey is an ambitious project, the OTTI acknowledges that improvements could be made in its data collection process.
"The survey produces very stable results where we have robust sample sizes -- i.e., at the national level and from the top 20 origin countries producing visitors," the OTTI states on its website.
"It is at the sublevel of country/destination, aka the origin-destination level, that the volume estimates get jumpier, when compared year over year, due to smaller sample sizes and normal sampling error."
As for how destinations use or manipulate the OTTI's data, such as its list of top international arrivals by city and state, Marano acknowledged that there's no controlling the marketing messages that destination marketing organizations put out there. "Look, if I'm No. 1, I'm going to tout it," she said.
Whether CVBs get their international visitor data from the OTTI or directly from airports in their region, they still need to complete the picture with data about domestic travelers. And the ways for doing that are "all over the map," Hull said.
"About the clearest picture that you can say is standard is Smith Travel Research, or STR reports," Hull said. "It's the performance of the lodging industry, and the nationwide occupancy, [revenue per available room]. You have something that's a standard ruler."
Still, STR data make up just "one piece of the puzzle," cautioned Duane Vinson, vice president of content management at Smith Travel Research.
"Our business is primarily performance benchmarking reports for the hotel industry," Vinson said. "[Hotels] give us room revenue figures and the room demand figures. They don't tell us how many people were in the room. The only thing we can do really, for example, is to tell the Atlanta CVB how many hotel rooms were occupied in a given month. We don't tell them how many people come to Atlanta."
Additionally, STR's reports rely heavily on data from chain hotels, which leaves a major gap in a market such as Las Vegas, where many of the hotels are independently owned. So, the LVCVA has over the past 30 years built its own database of hotel research to be able to compare itself with other cities for which STR has more reliable data.
Bagger said, "We have a survey that we conduct with our hotels to get actual occupancy and daily room numbers." The hotels are willing to give the authority that proprietary information, he said, because trust between the LVCVA and Las Vegas hotels and resorts has developed over the years.
In order to complete the math and get additional information about visitor behaviors or spending habits, many CVBs partner with travel and tourism research firms such as CIC Research, McLean, Va.-based D.K. Shifflet & Associates Ltd. or Portland, Ore.-based Dean Runyan Associates. The Orlando CVB, for example, works with D.K. Shifflet.
"They do a monthly survey, a sample of U.S. residents nationally," said Daryl Cronk, director of research at the Orlando CVB. "They do a couple hundred thousand surveys. That's how we get our domestic volume. Being a panel survey, it's weighted. That tells us how many U.S. residents came to Orlando."
To get the total number of visitors, each firm has its own survey and data collection methodology, which is a complex science unto itself.
"Most destinations use a nationally syndicated household survey to estimate total volume of visitors," said Adam Sacks, managing director of Tourism Economics, which works with NYC & Company to compile tourism stats for New York. "We then work with destinations to provide trend analysis based on industry indicators which are real counts as opposed to survey-based. This provides a more statistically sound method of tracking the change in visitor volumes over time than a national survey.
"We also use this approach, along with government data on international visitation and air arrivals by port of entry, to provide more rigorous trend results than a survey alone would provide," Sacks said.
Translating numbers to marketing
CVBs invest lot of money, effort and analysis into tracking these numbers for one key reason: marketing.
Visitor data "serves several purposes," Cronk said. "We make the information available to the public." Additionally, "we can look at which markets we should focus on. Summer travelers are different than people who come in the winter. The core of it is used for marketing."
To put it into perspective, the LVCVA's annual research budget, for example, is approximately $1 million, according to Bagger. But its marketing budget is about $120 million. One hand is clearly washing the other, much larger, hand.
The interplay between research and marketing is a delicate dance. Marketing teams are under pressure to publicize positive messages about their destinations, while research teams are charged with simply getting the most accurate data.
This is one of the challenges the Hawaii Tourism Authority's research department has been grappling with since being moved under the HTA from the state's Department of Business, Economic Development and Tourism.
Daniel Nahoopii, tourism research director for the HTA, said, "I am always struggling to keep [our research] independent." At the same time, he said, his department invests a great deal of effort in getting accurate visitor information. In 2008, Hawaii's tourism research team conducted 52,600 international traveler surveys and collected 2.9 million domestic traveler survey forms. "We spend big money to get accurate data," Nahoopii said.
Bagger added that fudging the numbers would simply be counterproductive to the objective of CVBs, whose ultimate goal is to find out as much as they can about their visitors so they can better target each traveler with more focused and informed marketing and advertising strategies.
As for any potential temptation to game the data, Bagger said, "There's no benefit to us to do anything to those numbers."
The problem most CVBs face is that their funding has been cut short just when they arguably need it most.
Most CVBs get their funding from a variety of sources. Many are financed through a share of hotel taxes. Some collect revenue from convention center fees. No matter where they get their money, most sources of revenue have suffered recently.
"The reality has been, in the last 18 months, those budgets have really been devastated," said Hull of CIC Research. "Cities are hard-pressed to come up with a budget that makes it."
Without additional income, many CVBs are struggling to make do with less. What that often means is cutting back on areas like special-interest research programs. Where many are still able to collect total visitation numbers and several other standard metrics, they might not be able to do a study this year of a particular market, such as Canadian or gay and lesbian travelers.
"Our budget has certainly been affected," said Orlando's Cronk. "We've been able to protect the core research project, but what we have not available is for custom research."
For instance, in the summer of 2008, the Orlando CVB did a custom research project in Toronto and southern Ontario to get a better understanding of Canadian travelers. "OTTI is primarily overseas, which leaves kind of a black hole for Canada and Mexico," Cronk said. But, he added, that is exactly the kind of research project Orlando will have to scale back given current budgetary restraints.
With CVBs losing research and marketing dollars, suppliers have begun to take notice and take action.
In some municipalities, Hull said, hotels have united to create a tourism marketing or tourism improvement district. "The hotels together decide to tax themselves, a special fund that goes to a promotion management group."
San Diego on Jan. 1, 2008, became one of the first cities to establish a tourism marketing district.
A year later, the San Francisco Tourism Improvement District was established. Governed by a board of San Francisco tourism industry representatives, such as hotels and tourism attractions, the San Francisco Tourism Improvement District established a 1% or 1.5% levy (depending on proximity to the main San Francisco tourism infrastructure) on gross hotel room revenue to secure funding for the CVB's selling, marketing and promotional efforts.
Cities such as Anaheim and Los Angeles are in the process of trying to establish tourism improvement districts, an evolution toward more private funding and less reliance on tax revenue for CVBs.
"Tourism improvement districts are the way of the future" for destination marketing organizations, the Anaheim Orange County CVB wrote in a recent update on its website. "When constructed correctly they provide for a seamless transition from public to private funding. ... The other obvious advantage of shifting to a [tourism improvement district] funding program is removing the bureau's dependency on local government and the uncertainty that accompanies it during these unsure economic times."
Indeed, said the OTTI's Marano, who relies on government funding to keep her $2 million survey program running, "the research side of life is what everyone says is the most boring side of life. But it is the most demanded part of life. It's the one area that's cut first. Yet, it's the one area where everyone says we need better data."