The story is told of a focus group for a new $100 electronic gadget. The response in the focus group was fabulous. All the people talked about the features of the new device with excitement.
At the end of the session, the moderator said “Thanks for coming. As our gift to you, you can have your choice of the device or $25.”
Everyone took the cash.
Surveys that ask your customers about their preferences, their net promoter intent, their media habits — they’re essentially useless compared to watching what people actually do when they have a chance.
(1) Seth Godin is right that surveys which ask people what they think or would do are ineffective. But that doesn’t mean your only option is to watch directly what people do. Sometimes you don’t have the opportunity to observe actual customer behavior. For example, if you’re trying to assess potential demand for a product you haven’t yet developed, you might want to know what similar products your customers buy. In cases like that, surveys can be very useful.
(2) How then should you run a survey to avoid the pitfall Seth describes? Ask questions that get your customers to share facts and experiences rather than opinions. In the words of Benson Garner: Don’t ask “Would you..?”. Ask “When is the last time you..?” or “Tell me about a time when you..?”. See How to interview customers to get great product insights.
(3) Cf. The survey question you should never ask.