Product feedback is one of the central themes of a Win/Loss interview, and can be one of the more complicated areas of the interview to get right. If you have a very simple product, like a widget that performs a single task, then product feedback is pretty straightforward. But if your product is software, or in the digital space, product feedback gets complicated quickly in a Win/Loss interview. This article is going to focus on products that are big and complicated (i.e. nearly all software and digital products).

• Products are big and complex, that’s a problem for getting good feedback. If you think about all the different types of experiences users can have in your product, you quickly realize that the overall surface area of your product is vast, and that while one user is experiencing one set of features, another user may be experiencing a totally different set of feature, and so asking about people’s experience with your product comes with a big “what part of the product are we talking about” problem. By contrast, think about an interview question about the sales experience. The total number of responses the participant can provide is pretty small. How many different way sare there that a sales experience can go poorly? Not that many (the salesperson was unresponsive, their questions didn’t get fully answered, there was a personality issue, etc.). The upper limit on what could have gone wrong is probably around 10 possible things, and that’s pushing it. Whereas the total number of possible answers you can receive from a product question is:
  
  The total number of features you have X the number of ways an a users experience with that feature can go wrong (it was buggy, not designed well, didn’t behave the way I expected, etc).   
  
  Let’s just say you have 100 features in your product, and there are 5 ways the experience with a single feature can go wrong. That’s 500 different possible answers you could receive to a question like “how was your experience with our product?”. Whereas with a question about their experience with the sales team there are around 10 possible answers they could give you. See the problem?
  
  
• Gather product feedback in the extremities. Instead of generically asking about their experience with your product, try focusing on two main themes. First, ask them what their favorite parts of your product are. What is it that they find most valuable or useful in your product? And then ask the inverse of that question. What are their least favorite parts of your product? By structuring your product questions this way you’ll start seeing trends and patterns in your product feedback, and most importantly you’ll start seeing patterns around the two most important areas of your product - what’s working really well, and what’s broken.
  
  By contrast, if you ask a non-open ended question about your product, a question like “what do you think about X feature”, you’re usually going to get bad feedback. It’s not immediately obvious that just because you’ve built and deployed a new feature that everyone on the team is excited about, your customers are equally excited. In reality, most of your customers probably don’t know about it yet. Think about your own experience with products you use. Uber might be releasing a ground-breaking feature that’s going to totally change their revenue model. But are you really paying close attention to their feature changes and updates? Probably not. You’ll eventually stumble onto that new feature at some point in the future, but if someone called you today to ask what you think about it, you’d probably wouldn’t know what they’re about.
  
  Additionally, when you ask about specific product features people feel pressured to give you answer back. Some people will be very honest and say they haven’t used that feature, but a lot of other people will give you some kind of half-hearted “yeah, I believe I’ve seen that, yeah that does seem quite valuable” when in reality they haven’t, and this moment is the first they’ve heard of it. That type of feedback isn’t valuable, and if you get enough of it polluting your data-set it can become dangerous misinformation.
  
  
  • What’s missing from your product (feature requests). When people talk about what they don’t like about your product, a nice distinction you can draw in the data is to listen for the difference between things that are broken in your product (poor design, poor user flow, poor utility, buggy, etc) vs. things that don’t exist yet, also known as feature requests! When you ask about what pain points people have in your product have a hard time naturally distinguishing between these two things so you’ll need to catch the differences as you listen. If they don’t share any pain-points that are feature requests, we’ll usually then ask them “is there anything missing in our product that you wish existed”. That question will usually provoke a new set of thoughts from them.
    
    

• Connect interview feedback with other product feedback tools. There are a lot of great tools out there to help you figure out what features you should be building next, and what features you should consider sunsetting. Product feedback in a Win/Loss interview is not a replacement for those tools, not even a little bit. But it is very (very) complimentary to them! Let me give you a few examples.
  
  
  • Tallying votes on which product features to build. You may have a process for showing your customers what features you’re considering for the product roadmap, and for gathering feedback from customers in the form of either comments or up-votes. That tooling and process generally sits outside of the Win/Loss data and reporting cycle. That process is managed by the product team and the product team has all kinds of workflows they’ve built to gather that data and make smart decisions. Where Win/Loss interviews can be helpful is they can start contributing ideas to what features ought to be added to the list. Specifically, when you ask for critical feedback about your product, if you’re listening carefully for feedback about poorly built feature vs features that don’t exist (feature requests), you’ll have a nice list of feature request ideas that comes from a totally unique and high-value source of data. This list of features can and should be added to your list of features for consideration. The product team will have pretty good instincts on how to weigh the different items appropriately. And your list of feature requests from Win/Los interviews will start to become a vital input for product roadmap decision-making in the future. some text
    ‍
    • Getting votes from Win/Loss interviews. Another way that Win/Loss interviews can be complementary to product decisions is requesting that the participant go vote on what features they’d like to see built in coming months. If you do have a voting or feedback process in place, there’s a great mechanism in a Win/Loss interview for requesting that they cast a vote for their favorite features, and that mechanism is the exit survey [link]. Since you’re already sending them a survey after the call, you can add a little link to the bottom of the survey that tells them about the product roadmap and lets them know they can vote for different features.
      ‍‍
• Use Win/Loss product feedback for depth (not breadth). It’s important to understand that product feedback in Win/Loss interviews is not broad, it’s deep. You’re not going to get a broad overview of all of the different aspects of your product. Instead you’ll gain a very deep understanding of certain parts of your product (like the extremities of what people love and what they hate). The extremities also happen to be the parts of your product that drive the most revenue, and lose the most revenue, so that’s good. But product teams tend to be focused on a more holistic view of the product, taking as much as possible into account. If you think about product feedback via Win/Loss interviews through that lens, it’s a very small percentage of your total features you’ll see feedback on. However, if you set your expectations correctly for Win/Loss product feedback, it can be absolutely invaluable for going deeper on what’s working (what’s actually driving your revenue), and fixing the biggest pain points (what’s actually losing you the most revenue). Once you start thinking about it in this way, then the product team can start watching the interviews to gain a very deep and nuanced understanding of how these high-value parts of your product are functioning. These types of Win/Loss product insights can drive a lot of revenue, and they nearly always open the eyes of the product team to previously hidden aspects of how these areas drive (or destroy) value.  The details of that happens simply aren’t available in your product analytics. They’re only available in the storytelling and experience your customers can share via conversation. So as long as you have your expectations set correctly, and the product team does as well, these insights can play a crucial function in how the product team makes decisions in the future. 

‍

The punchline on how to gather product feedback in a Win/Loss interview is to structure the question appropriately up front (ask about what’s best and worst about your product). Listen for the difference between negative feedback about existing features vs feature requests. And think about product feedback is a deep dive on the most value parts of your product, not a comprehensive view of everything across your product. Product Analytics can answer the “what is happening” across all of your features. Win/Loss interview feedback can answer “why it’s happening” across your most valuable features.  

‍