Win/Loss interviews require some forethought. Here's a guide that will help you get the most out of your interviews.
Interview Setup
Use Calendly to schedule: Scheduling calls can be surprisingly time consuming, and often requires multiple back and forth exchanges. Simplifying the process by simply handing someone your Calendly link will take a lot of the friction out of the process.
You don’t need more than 30 minutes. Most participants don’t have more than 30 minutes to spare, and as long as you’re asking questions efficiently, you usually don’t need any longer than 30 minutes to conduct a good Win/Loss interview.
Record the interview: Use a video platform (Zoom, Google Hangouts, etc) And record the interview. We’ve done thousands of interviews at this point, and I think I can only recall maybe 2 people who had concerns about the call being recorded, and after I told them that the call would only be seen be a handful of people internally, they were okay with the it being recorded.
Be clear about the incentive: Make sure you know what kind of incentive the participant is expecting (Amazon Gift Card, donation to the charity of their choice, etc), and that you can explain how exactly they’ll receive that incentive. You don’t need to explain these details until the end of the call, but the participant will be expecting this information once the call is finished.
Conduct the interview alone. There’s a temptation to add teammates to a call because they’ll often come up to you and ask if they can join. When this request comes in occurs it’s very easy to drift into a response like “sure, you’re welcome to join the call”. Allowing additional people into the call is almost always a mistake. some text
Group dynamic: If you have more than one interviewer it automatically creates more of a ‘group-dynamic’ on the call, making it feel a bit more formal, and causing the participant to be a little less free with their answers. It also causes the interviewers to be overly conscientious about their questions, since their colleague (or worse, boss) is on the call with them. Introducing these types of variables into a Win/Loss interview always degrade the quality of the data you’ll receive.
Rabbit holes: A good Win/Loss interview has a very clear beginning, middle, and end. There’s a natural arc to the conversation, and the interviewer is leading the participant through the conversation. When you have more than one person the flow of the conversation starts getting interrupted, and you’ll find yourself going down rabbit holes that don’t yield any valuable data, and which are hard to come back from in a way that feels natural.
When your colleague asks to join the conversation, cite these reasons as to why it’s probably not a good idea and instead, ask them for a list of questions they’d like to see asked. Spend some time integrating those questions into your interview guide.
Keep the interview questions focused on their actual experience. There is a temptation to ask "what if" questions in a customer interview like "If we changed the price to x, how would that affect your decision" or "if we built X feature, would you use it?". These questions tend to yield little to no value, because people are not good at predicting their own future behavior in these ways. You'll get much better data if you keep all of the questions tightly focused on "what happened" in their experience, and you steer clear of their opinions or ideas.
Interview Questions
Create an interview guide. An interview guide is simply the list of questions you want to ask, with a few reminders for yourself (like the participant incentive information at the end of the call).
The most important question of any Win/Loss interview is “Why did you decide not to move forward with us” (or for a Win interview, “why DID you decide to move forward with us). Before you ask this question, you want to warm up the participant first. Ideally you ask this question somewhere between the 5-10 minute mark of the conversation. Good warmup and ice-breaker questions include: “how did you originally hear about us”, oro “what was it that initially attracted you to us… why did you decide to go ahead and take a demo”. Once the conversation is flowing a bit, then you can lean ask this question.some text
Ask for 3 reasons: It can be surprisingly easy to miss one of the key reasons for why they decided not to go with you because they spent too much time on some other reason. When you ask this question, a helpful tip is to ask for 3 reasons. “What would you say are the top 3 reasons you decided not to move forward with us”. This ensures that they’ll move through all of the reasons, and if they only have 1 or 2 reasons they’ll let you know.
Ask followup questions, but don’t get stuck. Followup questions are always very important when doing qualitative interviews, but you need to balance your followup questions with how well you understand the issue. If the participant hasn’t sufficiently answered the question, or you simply don’t understand the reasoning, keep asking followup questions until you do. That’s very important. BUT, once you do sufficiently understand what occurred, avoid lingering too long on the issue. The time you have in the interview is valuable, and usually once the issue is clear and out in the open, there’s not much more to be gained.
Ask about the sales experience. If they went through a demo, you really want to find out 2 different things. You want to find out what they think the sales person did well, and then you want to find out what they think the sales person could improve on. Sometimes people have a hard time delivering criticism about the sales person, so a good followup question is often “if you were on the sales team at [myCompany], how would you sell this product differently than it was sold to you”. some text
This data is sensitive. Be careful with this data because sometimes it will reveal that a single sales person lost a deal for reasons that are avoidable (poor response times, personality conflicts, etc.). Naturally the sales team is often apprehensive about Win/Loss interviews precisely because of this question, so handle this data with care.
Share this data with the head of sales. This data is ideal for the head of sales to share during 1-on-1s. While there is a lot of criticism that comes through in this data, it’s also true that there’s a lot of praise for individual sales people. And what the sales person is doing well is often just as valuable to hear as critiques on how they can improve. Outside of the head of sales, be selective on who you share this data with because if it’s decontextualized and passed around freely, the sales team will feel like they’re being unfairly singled out.
Ask about Pricing. The way we typically frame the pricing question is "When you first encountered the price of our product, did you think it was a) higher than you expected, b) lower than you expected, or c) about what you expected. By framing the question this way it will allow you to bucket answers into 3 nicely differentiated categories. If you don't structure the pricing question up front, the feedback will start to get unruly, and generating insights will eventually become too difficult.
Ask this question again in an "exit survey" (we send participants their gift card after they've completed a survey, which we send as soon as the call is over). It's a good idea to ask this question in the interview, but to also ask it again in a followup survey, so you have a nice clean survey answer. In the interview people will give you an answer like "I thought it was higher than I expected". But they'll also expand on different areas of pricing, like your pricing model, and whether there are any pricing clarity issues.
In the exit survey, ask a followup question for precision. After they've answered that they felt the price was 'higher' than they expected, follow that up with "how much lower did you expect it would be". And offer scale of percentages that starts at 10% and then goes to 20%, 30%, 40%, etc. This question will allow you to see whether you're off on price by 10%, or 50%. This question is especially rich as you start comparing different customer segments. You''ll very likely see interesting differences here between customer types.
Ask about product, but structure it carefully. If your product is a software, asking their opinion about the product can be challenging because most software products can have hundreds and hundreds of features. We've found that for large complicated products (which most software products are), interviews are most helpful for figuring out what's working best in your product, and what's most broken in you product.
Asking about a single feature doesn't work. Most teams tend to think that if they could just get a handful of user interviews lined up, they can ask everyone about a new feature that was just deployed, or a new feature they're thinking about building. Both of those approaches tend to fail in customer interviews. The reason you can't usually ask about a single feature (again, if your product is software) is because you can't assume that most people have meaningful experience with the feature. When we'e tried to do this we've found that the team thinks almost everyone will have an opinion on the feature, but in reality only about 10% of people do. And you can't ask people's opinion about a potential feature build, is because people don't actually know what they'd use until you put it in front of them. Most people's opinions about future features are not dependable.
Ask loss interviews whether they're open to using you in the future. This is an important one. Around 1/4 of your loss interviews will have some kind of win-back potential. We recommend asking this question near the end of the interview. If you ask it too early, it can make the call feel too salesy. Once they give you some feedback, make sure and uncover as much detail as possible about what would need to happen for them to come back. And ask them specifically when they'll be in the market, or open to chatting again. With those details you can cycle them back into the sales funnel at the appropriate time, and provide the sales team with the key details they need to close the deal.
Ask "is there anything else you want to share" at the end of the interview. You can ask this question in a few different ways, like "do you have any additional advice", or "is there anything else that we haven't covered". After a long interview, surprisingly, people tend to hold back certain key details about their experience and this question will typically open up that feedback for you.
