How Many Interviews

One of the most common questions we get is “how many customers should we interview?”.

• 20-30 interviews from a well defined segment is a good start. And usually you only need 20-30 interviews from a well defined segment to start seeing some patterns emerge from the dataset. When hearing those numbers your first thought is probably that 20-30 is far below what’s required for statistical significance, and you’re right. But when you’re doing Win/Loss interviews that’s the wrong mental framework to be using.
  
  
  • Why so few? Customer interviews are expensive, so we have to keep the numbers relatively small. One big limitation of Win/Loss data-sets is that a single interview is ‘expensive’ in time and energy expenditure, whereas other data-sets such as surveys and product analytics are cheap. But that’s okay, because Win/Loss (qualitative) datasets are a totally different animal than quantitative datasets. Instead of thinking about these data-sets as needing statistical significance to draw conclusions, think of them on a different axis altogether. Think of them as tools that unlock all of the secrets of particular pain points. Your product analytics data can tell you there’s a problem with a certain feature, but your Win/Loss interviews can unlock the exact details of what that problem actually is - and reveal the solution. Furthermore, customer interviews are potent tools for getting decision makers to act. Once you show them a real flesh and blood human being, an actual customer struggling with an important issue and expressing their frustration, they can’t unsee it. It’ll stick with them and now be part of their mental model when making decisions.
    
    
• 50-60 interviews is better. At 50-60 interviews from a well defined segment you’ll start to see a lot more patterns. At 20-30 interviews you’ll start to see the first big patterns emerge (an issue with pricing, an issue with a particular feature). At 50-60 interviews you’ll see a whole new layer of insights starting to emerge. There might be issues with the sales team, or more likely, a single sales person. The pricing issue you thought you had might contain some hidden complexity. All kinds of interesting new patterns will start popping out of a bigger data-set.
  
  

10 Interviews per month, ongoing, is the best. If you can, aim for doing regular interviews every month, and piping the insights into your organization. Not only will you get a much bigger data-set over time and much richer insights, but you’ll start building workflows to accommodate the insights, and your team will get used to seeing these insights and taking action. Bigger data-sets lead to better insights and more actionability. So don’t think of it as a “project”, think of it as a new ongoing workflow to help drive revenue.