Analyzing Win/Loss interviews can feel a bit overwhelming because there is so much data to sift through. A single Win/Loss interview generates 30 minutes of video, and 5-7 pages of transcript. That’s a lot. And when you scale that to 10s and hundreds of interviews, the mountain of data gets pretty intimidating. Here are a number of key steps to be thinking about when you start your analysis:
This seems like an obvious one, but because there’s so much data to analyze, it can be easy to forget that a stack of 50 interviews can be reduced to 15 with appropriate segmentation, and 15 interviews is a lot easier to think about than 50. Additionally, segmentation is absolutely crucial for deriving actionable insights out of the interviews. You’ll want to segment based on whatever the top segments at your company are. Many of our clients have a “customer size” distinction that functions as their primary segmentation, something like SMB, mid-market, and enterprise. Whatever your company’s primary segmentation is, remember that your Win/Loss data should simply follow whatever your company has deemed most important. Your own internal company segments are the first and probably most important segments to be thinking about.
After that, there really are 3 other “types” of segments you’ll want to be thinking about. First, you’ll want to separate wins from losses, since they’re different interview types with different data and feedback. Next, you might have multiple products. If so, you’ll want to make sure that you’ve got all feedback headed into the right product. It’s surprisingly easy to mix Win/Loss feedback across different products. And the third, you’ll want to separate interviews across time. Interviews from 2 years ago are going to be very different from interviews conducted last month, so make sure that you’re comparing data from the same slice of time when you’re doing your analysis.
You’ll need a clean transcript to work from when you start analyzing your interviews. There are plenty of tools out there that will auto generate a transcript for you, but if you’re serious about quality and doing deep analysis, we strongly recommend using a human transcriber, yes even today with all of the AI innovations around us. At some point in the near future this will change, but today there are still enough mistakes in an AI generated transcript that we rely on humans for our transcription for two big variables. The first is tiny differences in language can totally change the meaning of a sentence. For example the difference between “can” and “can’t” have opposite meanings, but linguistically sound very similar, and it’s very easy for an AI to transcribe this incorrectly. And secondly, many of the interviews you do will probably have accents. The stronger the accent, the worse the auto-AI transcription. We conduct interviews all the time where the feedback being given is incredibly valuable while the accent of the participant is incredibly strong. An auto-generated transcript for that type of interview will be full of mistakes, and make your analysis more time consuming and frustrating.
A great place to find human transcribers is Upwork. There is a very big audience of human transcribers on Upwork all with references and ratings available so you can make an informed choice on which person is the best fit for you.
Once you’ve gotten a full transcript in hand, now’s the time to actually go through the interview and highlight all the valuable parts. But before you do that, you’ll want to actually create “buckets” for each question you asked in the interview. For example, one of the questions will be “Why did we lose”, another will be “How was the sales experience”. Create separate Google Docs for each question, and as you analyze each individual interview you’ll want to lift the text for each answer and drop it into the appropriate section. Eventually you can go to a single source and see all of the answers to a single question like “Why we lose”. And from there, you can start generating insights on that question.
As you can see, the analysis process is a heavy lift, and takes a lot of time. The challenging thing about analyzing customer interviews is that they produce so much data it requires a lot of analysis to make the information usable. And in order to make it usable you basically have to take it through 3 different stages. 1) you have to analyze the full interview and distribute all the valuable moments into their own information buckets. 2) you have to go through each individual information bucket like the “why we lose” bucket and identify all the trends occurring in that bucket. And then 3) you need to organize those insights into some kind of presentation format that you can share with others, probably executives or team leads within your company. It’s a heavy lift and takes time, but the insights you’ll reveal will be exceptionally high value to your company.
A final thought on tooling for this process. If you’re not using a 3rd party Win/Loss vendor, you probably don’t have access to proper Win/Loss software, which does most of the above work for you. So a simple way to do this is with either Google Docs or Google Sheets. Some of us prefer using a new Google Docs for every new information “bucket” because of all of the styling and visual advantages a word processor like Google Docs offers. Remember, this data is language, so you can highlight, bold, underline, change font sizes and colors, etc. A word processor organizes this data nicely and makes it visually appealing. Others like to use a tool like Google Sheets to organize this data. So instead of beautiful visualizations of the language on the page, you get the advantage of columns and rows and the ability to build more sophisticated data structures and organizations. If you’re unsure which one makes more sense, trying both at first and see which one feels easier. The analysis process takes a lot of energy, so it’s a good idea to always break in the direction of whatever feels easiest to you, and causes the least amount of mental strain.
