By CARL BIALIKEach day, 5,000 Americans fill out an online survey rating corporate brands on such criteria as quality, value and reputation. Their responses are compiled into a daily tracking index that aims to show the ebbs and flows of brand value.
The tracking poll, conducted by U.K. survey company YouGov PLC, has shown how much an automobile recall cost Toyota in reputation, and how long it took to bounce back. Its German arm has made headlines for measuring the deep dive in the reputation of a German fitness-studio chain after its founder's music festival last month ended in a deadly stampede. And BP PLC has used the poll to follow public response to the massive oil spill this year from its well in the Gulf of Mexico, according to a person familiar with the matter. (A BP spokeswoman declined to disclose which measures the company uses.)
Daily tracking polls are becoming a must-have for companies who have suffered blows to their public standing. Online polls are fast, and often less expensive than telephone surveys. But getting measurements quickly on so many companies involves taking steps that could dent the reliability of the numbers.
Those pitfalls, including questions about maintaining a random sample of participants, worry data experts who study survey methodologies. Still, the growing acceptance of Web-based surveys marks a turning point in the polling industry. Until recently, many established pollsters had shunned their online counterparts or said their methods were dubious. Now, some traditional telephone pollsters have begun incorporating online polling themselves.
The American Customer Satisfaction Index, a poll founded at the University of Michigan, until this year conducted surveys exclusively by phone. "We used to think telephone sampling was the way to go," says David VanAmburg, managing director of the index, which now is produced by a for-profit company in Ann Arbor, Mich. But then people started ditching landline phones—one in five households, at last count, were cellphone-only. This spring, ACSI began blending online and telephone polling.
YouGov began tracking American opinion of corporate brands three years ago for its poll, which it calls BrandIndex. (YouGov has conducted polls for U.K. papers the Sun and the Times, which like The Wall Street Journal are owned by News Corp.) Each day, the company sends out enough surveys to members of its one-million-person panel of U.S. adults in order to receive back at least 5,000 completed surveys on 1,100 brands. Each brand is rated by between 50 and 125 people per day, and the results are combined into a single score.
"If you have a crisis or a potential crisis, you can have a sense of, how much of an impact is this having with consumers?" says Ted Marzilli, the New York-based global managing director for BrandIndex.
Other research companies take a different approach to using the Web to monitor corporate reputations. New York firm NM Incite, for example, monitors chatter on blogs and social networks, using automated tools to detect whether comments such as Twitter posts are positive or negative. While some respondents to an online poll might not be customers of the brand they rate, or might not have much influence over others, people who comment on blogs and social-networking sites are likely to have an outsize impact on brands.
A problem, though, is ensuring that the software correctly categorizes online buzz. "That's the hard part," says Pete Blackshaw, executive vice president of digital strategic services for NM Incite, a joint venture of the media-measurement firm Nielsen Co. and McKinsey & Co. "Anyone who claims perfection is misleading you." The company continually updates its algorithm and occasionally has analysts review material, he adds.
Some data experts wouldn't consider YouGov's survey panel, recruited exclusively online, to be randomly chosen. For one, not all Americans have Internet access. Also, polls that recruit participants online favor the heaviest Web users, who are more likely to spot the ads seeking brand raters.
Mr. Marzilli says the company ensures respondents are representative of the overall population by such factors as age and gender. He also questions whether those who aren't online "would view BP or Toyota inherently differently than people with Internet usage." And he pointed to the company's success using similar panels for political polling—for instance, predicting Barack Obama would beat John McCain by six percentage points in the national popular vote for president in 2008. President Obama won by seven percentage points.
Even survey experts who have doubts about the reliability of online polling say it is a useful way of tracking changes in public opinion over time. "There is probably no other mechanism available" for the intensive daily results like those the BrandIndex seeks to generate, says Michael Brick, vice president of Westat, a company that conducts polls for the U.S. government. "If you use caution with the results, you can get something valuable out of them."