We’ve all heard that telephone surveys are falling out of favour as the trend to use mobile devices instead of household landlines continues to grow. Social media monitoring and online straw polling are increasingly touted as the latest and best ways to gather public opinion. But are they?
Although the debate is usually framed as telephone versus online data collection, the real issue is respondent “self-selection” versus respondent “random selection.” Put simply, whenever a person participates in a survey through a website or social media platform (a choice that is limited to a select segment of the population), he or she will be a specifically motivated type of respondent who is acting for personal reasons. Those reasons often involve a rewards program, or other incentive provided by the poll’s owner, or a specific interest/expertise in the topic area. Therefore, as soon as a respondent “self selects” to do such a survey, any claim to population representation is lost.
Data gathered in this way can never be used to extrapolate or explain any other population than the exact one from which the data was collected. So, when we see a statement such as, “gentlemen prefer blonds” and the data source is a media industry online poll, what does that mean? All we really know, because we can’t verify that the participants are gentlemen or even unique respondents rather than repeaters, is that an uncertain-but-larger proportion of those who used this Web platform during an unknown-but-limited window in time clicked “blond” for unknown reasons in preference to the other options offered. Not accurate, not useful—and so, not even interesting. Just who really prefers “blonds” and why is it still open to question? Here’s my answer: I certainly do – stouts and lagers are just not to my taste!