Surveys are not Enough (You Need to Talk to the Customer)


As tempting as it is to take shortcuts, a survey is not sufficient market research – particularly when discussing strategic decision making and product design. Though cost-effective and well suited for tasks such as determining how much one option is favored over another, surveys also have significant shortcomings that limit their effectiveness, especially when it comes to innovation.

One reason that some executives favor surveys over qualitative approaches is that they see safety in numbers – in other words, by receiving more data (because more people are able to be engaged in a survey), the results are assumed to be more useful.  This however, is a major fallacy, in part because surveys cannot truly produce new categories of information. They can only help quantify and refine already existing categories of knowledge.  

Another major reason surveys find favor is because they are easy to bound. One knows what they will get, and for the most part, what they can expect to learn from a survey (e.g. “do they prefer A or B”). In-depth qualitative research, on the other hand, is by it’s nature more difficult to draw simple boundaries around. But despite the fact that it may seem to those inexperienced in such research that qualitative experience and findings could be potentially infinite, this is simply not the case. Rather, what always occurs in any qualitative project is a point of saturation, when one hears nothing new from respondents and material is simply repeated. And on most projects, it does not take a significant amount of time to reach this point. Regardless of size and scope, the point at which saturation is achieved is typically found between 12-24 respondents. Although it is not quite as simple as it may be for surveys, an intuitive and experienced researcher can bound qualitative programs quite effectively. 

To make effective strategic and design decisions, businesses need to get out there and properly engage the customer – which means going beyond the survey.  Such impersonal methods simply cannot provide the level of insight necessary to provide meaningful answers to the innovation challenges of today.

Greg VanderPol