Lies, Damn Lies, and Surveys

Sometimes they make me want to laugh; sometimes they make me want to launch into a profanity-laced tirade—the bottom line is that surveys and poll results are a necessary evil, but are often of limited value and shouldn’t be taken too seriously. This is only exacerbated during an election year. Every time President Barack Obama delivers a speech, the talking heads gleefully serve up the latest poll numbers. If Mitt Romney makes a gaffe, the news networks examine how much ground he's lost.

If you believe the news networks, Americans change their minds on a weekly basis about which candidate they're going to vote for come November. Yet in my experience, only a handful of people hadn’t already made up their minds before election season even began.

In financial technology, surveys and polls abound. Indeed, we at WatersTechnology produce them on a regular basis. They provide an interesting snapshot of what people are thinking, but always take the results with a grain of salt—and not just because the results aren’t scientific.

Often, survey and polltakers bear out the principle that “you don’t know what you know,” and don’t answer the questions accurately. A good example of this came in May when Waters conducted a webcast looking at cloud adoption on the buy side.

During the online event, 35 percent of respondents said they don’t use cloud at all. Really? Not for email? Not for data storage? No deployments of software-as-a-services (SaaS)? Nada?

I just don’t buy it—and I believe most respondents don’t understand exactly what cloud encompasses. If they don’t employ Amazon or Google, they assume their firms are cloud-free.

Survey and poll inaccuracies are not only the fault of the respondents. I receive surveys that are either conducted by, or sponsored by, vendors and trade associations. In these cases, people might answer questions based on what they think the pollster wants to hear, or because they have an agenda to advance.

So if surveys and polls are flawed in some way or another, they must be useless, right? Wrong.

What they do is allow firms and vendors to look out for knowledge gaps, misunderstandings—as in the cloud question—or general areas that need attention. They are not the be-all, end-all—they’re guides.

I never allow someone to win an argument by citing a survey. But if the numbers are out of whack, there's a reason to take a deeper look. The numbers are useless; the topics can be fruitful.

Let's take an informal poll: Who thinks surveys are useless? Email "yay" or "nay" to me at anthony.malakian@incisivemedia.com or give me a call at 646-490-3973.