When you’re designing your Win/Loss program, one of the biggest and most pressing questions is “what questions am I going to ask during the interview”. There are a handful of questions that you already know you’ll ask, questions like “why did you decide not to move forward with us” or “how was your experience with the sales person”. Those questions are pretty easy. But in order to conduct good interviews, it’s a very good idea to sit down and create a proper interview guide. Brainstorm all of the questions you’d like to ask, and then refine which ones you’ll actually ask, and at what point of the interview you’ll ask them. Here are some things to keep in mind while you’re building out your interview guide:

1. Ask the same set of questions to every person. All win interviews should be asked all the same questions, and all loss interviews should be asked all the same questions. By making sure everyone gets asked everything you’ll get well structured data and enough feedback for each question to derive insights in each area. Additionally, you’ll want to ask each question the same way each time, if possible. For perfectly structured data it helps actually read the exact question, word for word, each time. That said, you want to balance the advantage of a structured “word-for-word” approach with the disadvantage that it can make your voice and tone sound robotic, and can make it sound like your disengaged from the conversation, which will usually elicit an unengaged response from your participant. Alternatively, as long as you feel like you have your head around the essence of the question, you can ask it in your own words each time, just make sure and carefully listen to their response. If they’re not answering the question fully, it might be because you didn’t quite ask it in the right way,
   
   
2. Keep questions focused on the participant’s actual experience. If you don’t remember anything else about asking interview questions, this is the thing to remember. Qualitative and conversational feedback can be unruly and hard to tame. You’re in what feels like a free-flowing conversation with another human being, and the conversation can very easily veer off the rails into territory that just isn’t useful. And if you’re not careful, the conversation can yield data that’s actually misleading. In order to keep the conversation focused on feedback that will be useful, the key is to develop a keen ear for the difference between someone’s opinion and someone’s experience. The person you’re speaking with will naturally drift (if you let them) back and forth between “what happened” when they were using their product and “what they think” about well, all kinds of things.  It’s your job to keep the conversation laser focused on “what happened”, and listen carefully for things like “hey, you know what would be cool, what if you built an XYZ and included that in your product”. It’s not that that data doesn’t have any value. It depends on who it’s coming from, and the context in which it’s delivered. But it’s a very different type of data from “I tried introducing this feature to my colleague and they hated it because they said it was too complicated”. That’s an example of a real thing that actually occurred, and a piece of information you should definitely pay attention to. The other information is mostly speculation, and people tend to be very bad at predicting their own behavior when it comes to future feature use.
   
   
3. Keep questions open ended. It’s easy to fall into the trap of asking yes/no questions, or questions that simply don’t yield much of a response. As you’re constructing your questions, think carefully about whether the question is restricted to 1 or 2 possible answers (like ‘yes’ or ‘no’), or whether the question is open ended, like “How do you think the sales person could improve their pitch and demo?”. Usually the structure of the question will dictate the richness of the answer you get back. If you ask “who’s got a better user experience, Competitor-X or us?”, a lot of people will give you a one word answer back. They’ll say “I think you guys do”. So a much better way to approach a topic like this is to either ask an open ended question up front like “what are the big differences between the user experience between us and Competitor-X”. Or, if you do end up asking a simple question like “who’s got a better user experience, Competitor-X or us?” be prepared to quickly follow it up with detailed questions about what makes the user experience better, ask for as many examples as possible and go deep with them. 
   
   
4. Ask a lot of followup questions. There’s a tremendous amount of value that will surface in from followup questions. When you ask a participant a question, they’ll usually respond with an incomplete or partial answer, and then turn the conversation back over to you. As the interviewer, you can either move onto the next question, or you can evaluate whether that was a rich enough answer and if not (which it usually isn’t) go back for more. It’s almost always a good idea to ask followup questions to almost every question.
   
   One really good technique to try is to simply repeat back to the participant what they’ve just said. This technique will cause them to go deeper into the thoughts they were just sharing and round out a number of ideas they touched on. It may not seem like it to you, but it gives the participant significant time to think more deeply about what they were just saying. We usually try to repeat what the person said to us almost every question (without sounding awkward) and most of the time it’s astounding how much more data it yields. As a rule of thumb always try to ask followup questions, and if you don’t know what to ask, just repeat their answer back to them and they’ll share more.
   
   
5. Warm up the participant with ice-breaker questions. When you start the interview, don’t dive straight into the most important questions like why you lost. A great ice-breaker question is “how did you originally hear about us”. Get them talking about the Google search they did, or the referral they received from someone using your product. This ice-breaker intro is important to get them comfortable, and to get them used to doing the talking on the call. In contrast, if the first question you ask is “why did you decide not to move forward with us”, it will feel strained, and it's too big of a question too quickly. Before they come forward with the big reasons behind their decision, they need to be comfortable in the conversation, and comfortable with you (it’s just like any conversation with another human being). You have to build up to the big stuff. We usually ask the big question about why we lost about 5-10 minutes into the interview. Basically whenever you feel like they’re comfortable, and the conversation feels like it’s flowing. 

Best of luck building your interview guide and sequencing your questions. After you’ve completed your first 10 or so interviews you’ll start noticing a lot of structural elements in the interview questions. And once you get to 20-30 interviews under your belt you can start iterating and experimenting with your own structural ideas. It takes some time to develop these skills, but they will come!

‍