When you think about what constitutes “Win/Loss Data”, the default for almost everyone is the same. It’s the notes your sales person left in the CRM about why the deal was lost. Pretty much everyone we’ve ever worked with has a “reason for loss” note from a sales person in the CRM. So almost everyone has at least some kind of data when it comes to understanding the reasons behind why you win and why you lose. The down-side to that note is that for most companies, that’s the ONLY Win/Loss data the have. And if there’s one thing we’ve learned from conducting so many Win/Loss interviews it’s that the accuracy of those notes from sales people tend to vary somewhere between being incomplete all the way to being actually wrong. For example if the note from the sales person said “price was too high” that’s probably true, but it’s only part of the truth. The more important question is “how much” of the whole story is it? For example, in a good interview you’d likely uncover that while they did think the price was high, the reasons they didn’t move forward are far more nuanced, and the price obstacle may have been overcome if something else had occurred, like the length of the free trial had been increased, or if they were able to get another stakeholder into the demo call (which they weren’t). That’s how sales note tend to be true, but totally incomplete. On average I’d say that the sales note tells about 20-40% of the actual story. So if you inverse those numbers, the sales note is missing 60-80% of the actual story. That’s a lot. Not to mention that a decent percentage of sales notes are actually wrong altogether. It’s not that the salesperson is knowingly putting incorrect information into the CRM, it’s that for whatever reason (a comment that was made on a call, a feeling they have, etc), they think they know why the deal was lost when they actually don’t.  

There are four main types of Win/Loss data you can aggregate. Notes from the sales team, survey data, interviews with customers, and interviews with sales people. Each of these channels has its own strengths and weaknesses, and they’re all pretty different from each other. Let’s go through each of them.

‍

Data-source 1: Notes in your CRM from the salesperson (partial truths with heavy bias). 

–Easy to get. Low value– 

As I just mentioned above, most companies have notes on why each deal was lost from the sales person who lost the deal. But those notes are usually missing 60-80% of the actual story. For teams who haven’t built out a Win/Loss program, the aggregate of these notes are your Win/Loss data-set. And while they’re better than nothing, they’re surprisingly weak in their ability to answer big questions like why you’re losing deals. There are a number of interesting reasons why the sales notes tend to be so misleading: 

• No proper feedback structure in place. The first reason is because there’s usually no meaningful feedback structure in place to gather the information correctly. To actually unpack why a deal was lost, it requires an in-depth conversation with someone who isn’t the sales person. Most teams simply haven’t built that type of workflow. And that missing workflow is largely the reason that 3rd party Win/Loss vendors exist, to plug that hole.
  
  Instead, the sales person tends to develop their own view on why the deal was lost based either on passing comments or questions the person asked, or from the feedback they received when they asked the person directly.
  
  
• People tend to hide the truth from the salesperson. You’d think that if they asked the person directly that it’d be a pretty reliable indicator of why the deal was lost, but when a salesperson asks the question directly people tend to be guarded about their answer. Oftentimes they have issues with the sales person themselves, and they’re uncomfortable giving directly critical feedback to them. They could have other reasons for not choosing your company - maybe they have a personal relationship with another vendor and they feel awkward about that being such a motivating factor. Realistically there could be all kinds of reasons. Human beings are funny and weird about these types of things. Ultimately if a salesperson asks you why you decided not to move forward with them it’s an uncomfortable question, no matter what the reasons are. While some people are comfortable giving very direct feedback, the majority of people aren’t, and they’re much more comfortable either ghosting the salesperson, or giving a reason they think the salesperson might not want to hear (it’s just not the right time for us, we had some budget changes last week, etc.).  
  
  People are also uncomfortable about sharing too much with the salesperson because they’re hyper aware of the fact that the salesperson is trying to sell them something. That sales dynamic by nature creates a bit of an offense/defense game where people tend to be a bit guarded about what information they share. When a salesperson asks “why did you decide not to move forward with us” most people don’t hear that question as an opportunity to provide open and honest feedback, they see it as a possible extension of the sales strategy at some level. So instead of simply opening up, they tend to still have their guard up when answering the question.
  
  
• Salespeople usually have strong biases. Let’s be honest, salespeople aren’t known for their humility and soft-spoken / thoughtful demeanors, right? They’re usually known for their confidence, and their ability to connect with other people - as they should be. Their job is to sell. But when it comes to gaining an accurate understanding of the reasons a deal was lost, these traits tend to make it more difficult for salespeople to self-reflect and accurately identify the reasons why a deal was lost. I’m not trying to pick on sales people here, this skill is difficult for everyone. It’s one of the biggest reasons Win/Loss interviews should be conducted by an uninterested 3rd party, it strips almost all of the personal bias out of the conversation.
  
  
• Salespeople aren’t incentivized to look backwards. Salespeople are highly incentivized to look forward to the next opportunity, to charge towards it, and to close it as quickly as possible. Capturing good Win/Loss data is by definition a backwards looking activity. And a salesperson’s job just isn’t structured to stop and think deeply about the reasons a deal was lost. They’re usually adding notes to the CRM simply because it’s the company policy to do-so. But when a given deal is officially considered lost, most people are ready to close it and immediately move to the next thing.  

‍

Data-source 2: Conversations with your sales people

–Pretty easy to get. Pretty low in value–

We’ve encountered a number of companies who conduct Win/Loss interviews with their internal salespeople to understand why deals get lost. This approach is well intentioned, and certainly goes further than relying on the sales-notes in the CRM, but it’s deeply flawed for all the same reasons I’ve just outlined above. Even though a conversation with a salesperson will generate a lot more data than what you’d get from the notes they leave in the CRM, the salesperson will only be able to provide partial information 60-80% of the time, and will be operating with some really heavy biases that will pollute your data.

We’ve noticed that the impulse to interview salespeople tends to come from a combination of two things. 1) a dissatisfaction with how short the sales-notes are in the CRM and 2) a recognition that getting participants into Win/Loss calls is really hard, whereas getting salespeople into Win/Loss calls is really easy. 

If you have a really high-stakes deal, or if you’re the type of business that only closes a handful of deals per quarter, then interviewing your salespeople can add very helpful color to the interviews with the customers themselves, and might be worth building into your interview workflows. But don’t rely on the conversation with the salesperson instead of the customer. That’s a big mistake, and will really mess up the quality of your Win/Loss data-set.

‍

Win/Loss Survey Data

–Somewhat difficult to get. Somewhat valuable–

Sending out Win/Loss surveys to everyone who goes through your sales cycle is a well-worn strategy, and is considered by many companies to be a best practice. I’ve talked a bit about this idea here [link]. But surveys come with important limitations when it comes to gathering Win/Loss data.

• Conversion rates are low.. Lower your expectations about the total number of people who will respond to your survey. We’re all bombarded with feedback requests all the time, so if you want someone to reply to your survey, do your best to make sure the survey is dead simple, they can fill it out from their email inbox, and includes some kind of meaningful incentive (try a $1,000 prize for 5 different participants).
• It’s too easy to ask the wrong questions. Asking good survey questions is hard. There’s a lot of garbage-in / garbage-out. The best survey questions actually come from other sources of data like long-form Win/Loss interviews. If you conduct 40 Win/Loss interviews and then analyze them carefully, you’ll come up with a dozen excellent and high-value survey questions that you can ask. If you’re just thinking about survey questions void of important context like other data-sets, you’ll end up with a lot of useless survey data. The key thing about getting surveys right, in our opinion, isn’t getting enough responses. It’s asking the right questions.
• Survey questions aren’t open-ended. Most survey questions are multiple choice questions. And if they are open ended,what people tend to type-out with their fingers is far less expansive than what they would say if you could ask them. So you have this 

‍

Interviews with Customers

–Difficult to get. High in value–

Customer interviews require the most work, but are also the highest value. A customer interview allows you to go really deep with customers, and to fully understand the reasons they decided not to move forward. There’s really no substitute for customer interviews because, well, the individual is the source of truth, and there’s no medium like long-form conversation to surface that truth. Everything else is too limiting.

• Interviews should be the foundation of your program. Because Win/Loss interviews are able to capture the entire story, they should be seen as the most vital data source and the foundation of your Win/Loss program. All of the other data-sources (surveys, sales team interviews, CRM notes) should not be replacements for this source of data, but compliments to it. Use interviews to figure out what the actual picture looks like, and then use those other data-sources to help fill in some of the details, instead of the other way around.
• Interviews cost time and money. Be prepared to make a meaningful investment of both time and money for customer interviews. They take a lot of work to procure, but isn’t that how value works in general? Valuable things are hard to capture. So it’s not surprising that interviews are harder to get than other sources of data. The reason they’re harder is because they’re a lot more valuable. Don’t fall into the trap most teams fall into of believing that you can rely on other data-sources. This trap is laid by telling yourself first that interviews are going to be too hard to get, and then slowly convincing yourself that these other sources of data are good enough. They’re not. Settle in, accept that interviews are the most potent and important source of information, and build a strategy to get them. If you do you’ll eventually start driving meaningful new revenue for your company and if you don’t, you won’t.
  ‍

Ultimately the best Win/Loss programs are the ones that figure out how to prioritize these data-sources correctly, and keep that prioritization in place over time. That prioritization should be 1) long-form interviews 2) surveys that go to participants after the interview, and separately, surveys that go to a larger audience and are built with questions derived from the interview feedback. 3) Salespeople interviews and 4) Notes from your CRM.

‍