There's popular Win/Loss ROI quote out there from Todd Berkowitz of Gartner Research that says “ a good Win/Loss program can improve win rates by up to 50%, and revenue increase 15-30%”. Those are some very big numbers. Could they be true? Can you actually grow revenue and win-rates by that much with a Win/Loss?
At Kaptify we run a lot of Win/Loss programs for our clients, so we have a front-row seat to the impact a good program can have on revenue and growth. Our opinion is that yes, he is technically right. It definitely CAN grow revenue at that rate. BUT, notice that what he's saying is very different than "it WILL grow revenue" at that rate. The key word in his quote is "good". A GOOD Win/Loss program can dramatically improve win rates and drive significant new revenue. It’s important to be realistic about what it takes to build a good program. Unsurprisingly, building a good Win/Loss program takes a lot of work.
So if driving Win/Loss ROI is all about how much work you put into your program, then you should definitely read our article about how much work a good Win/Loss program actually takes. Nearly every company we've ever talked to about Win/Loss has severely under-estimated how much work it takes. And it's not that they're not up for doing the work. It's that Win/Loss programs are gnarly and complicated beasts. The amount of time and energy it takes to run a good Win/Loss program dramatically exceeds the expectations of almost everyone who endeavors to set one up (btw, this is why considering a 3rd party can be a really good idea).
For this article let's assume you've accepted that building a Win/Loss program takes a lot lot lot of work. So then the question becomes how do we actually drive ROI with Win/Loss, and can we actually measure the ROI it produces?
At first glance you probably have thoughts arising like "well, if we can figure out what we're doing wrong, we can simply fix those things and our win-rate will go up." And yes, that is theoretically true. But the devil is very much in the details here.
Here are the categories your "reasons for loss" data will fit into:
Seems easy enough. You learn that there's an issue with how the sales team is selling the product, you fix the issue, and win rates go up. Right? Not really. Sales issues tend to be the easiest to understand, and often times the easiest to fix. But they still take a lot of work to figure out and fix because sales issues usually occur at the individual sales-person level. The sales team is not a monolithic creature behaving in lock-step. It's a bunch of unique individuals with their own unique personalities that are impacting prospects in their own way. To drive ROI with insights on the sales time three key steps need to happen:
As you look at this list of categories, think for a minute how diverse the issues are that will arise within each category, and how different the categories are from one another. They're totally different. And the patterns you'll start observing within each category will also be numerous and diverse from one another. In order to grow your win-rate, you need to identify lots of little things happening, and then fix them.
If you’re building and running your own program you don’t need to get any budget approvals or go through a vendor selection process. All you need to do is get approval from your boss and team to start building the program. You can get off and running quickly, and that’s valuable. Additionally, the cost of a Win/Loss program can vary widely from vendor to vendor. Some of the more well-known vendors out there are, in our opinion, very overpriced. The Win/Loss service provider market is new, and it’s not an efficient market, so you have to proceed cautiously when it comes to spend. You’ll find some providers that charge double the price for half the quality of service. As the market matures these inefficiencies will begin to disappear, but today that’s where things stand.
As you’re trying to evaluate cost, a helpful lens to look through is the “cost of a single interview”. If you’re talking to a provider and they’re providing you with pricing information, break down the price to a “per interview” price, and do the math on the following inputs: time spent recruiting, time spent interviewing, time spent analyzing, access to their software (if they have their own software), interviewing skill-set (which the provider will have, and your team probably won’t). They might pitch you on additional services like surveys and such, but the nuts and bolts of cost really boils down to the interview itself. As you think about these inputs, keep in mind that doing these interviews is a serious time input, and the skill-set required to do them well is a real thing. Though if you do the interviews yourself, you’ll eventually get there once you’ve completed 40-50 interviews. And that the total time input for a single interview is 30 minutes for the interview, 30-60 minutes for the analysis, and about 15 minutes for the recruiting (see our article on recruiting if you haven't already). With that information you should be able to get to some helpful conclusions on cost.
Hiring a good Win/Loss provider on the low-end will cost you around $60k a year (that’s where we typically land), but other providers charge $120k per year and up, so it can get expensive. That said, running a robust Win/Loss program can become a full-time job, depending on how many interviews you want to conduct. And the outcome of an internal program can be limited since you're not inheriting the lessons and best practices from thousands of interviews and many clients. So if you’re planning on building a robust program, the cost of a 3rd party might equate to what you’d pay a full-time salaried employee. But again, the best way to think about cost is to break it down to the cost per interview, and do the math from there.
The amount of time it takes to build and run a Win/Loss program is substantial. It’s way more time than most people estimate. In addition to the obvious stuff (scheduling and conducting the interview, recruiting participants, analyzing the interviews), there are also plenty of hidden costs. On average, 25-30% of interviews don’t show up for their interview, so you’re sitting there for 20 minutes waiting for them to show up. You can be working on other things, but it tends to feel like broken time, and it’s difficult to be productive in those windows. You’ll have an executive who comes to you and says “what do people think about X new feature”. And suddenly you’re off on an unplanned 3 hour adventure scrubbing each interview for any morsels of information about that feature. You have rescheduling time, you need to get each participant their gift card as a thank you for participating. If you’re sending an exit survey after the call, you’ll need to get that sent, and collect and organize everyone’s answer to the survey, as well as combine the survey data with the Win/Loss data in a complimentary way. It’s a lot of time, and can easily turn into what feels like a full time job.
With a 3rd party, 90% of the time investment gets outsourced to the service provider, so it’s a major win if you’re trying to save time. A big part of the cost for a 3rd party provider is this time component. It’s just a lot of time and energy, there’s no way around it.
Interviewing participants well takes real skill. On average, it’ll take you at least 40-50 interviews to start really building out your skillset. When first started Kaptify we had a lot of clients who wanted to join our interviews because they were keen on seeing and hearing what people had to say. Seems innocent enough, right? We don’t let clients join interviews these days because on average, unskilled interviewers (which is basically everyone) have a hard time distinguishing what information is important from what information isn’t. It seems like an easy enough thing. But once you’re actually in a real conversation, you’re dealing with a brand new personality whose bringing all kinds of new dimensions to the conversation. If you don’t guide them correctly to the right questions, go deep enough in the right areas, and avoid the pitfalls, you can easily fill up 30 minutes of time with very little valuable data. You can also unintentionally pollute the data by simply asking for their opinion about things instead of keeping the conversation focused on their experience with your product.
The big difference between a 3rd party and doing it yourself is that as an internal analyst at you probably don’t have these skills, and it’ll take some time to get them, but you will eventually get them. Just plan on a learning curve of at least 3-6 months. Once you’re past the 40-50 interview mark you’ll start getting the hang of things.
When it comes to the big differences between a 3rd party provider and doing it yourself, ultimately this is the big difference. There are a small handful of Win/Loss companies that have built their own software to handle Win/Loss feedback interviews. The reason we’ve done that is because Win/Loss data-sets are really valuable, really big, and require a lot of massaging. If you think about a single interview, you’re talking about 5-10 pages of text on the page, and a 30 minute video. A single interview is a lot of data to review. And a Win/Loss program is trying to scale to hundreds and hundreds of interviews. You can start to see how keeping all of your interviews in Google Drive and Google Docs eventually breaks, and you stop being able to find the insights you need because the data is simply too scattered and unstructured. There are tools out there like Dovetail that can be helpful, and tools like that are usually better than just Google Docs, but they’re also severely limited given all of the nuanced needs a Win/Loss program has. When considering whether to build in house or use a 3rd party, if the cost of the 3rd party seems reasonable, this is really the thing that should swing your decision one way or the other.
Building a high-quality and robust Win/Loss program requires adequately covering all of these bases. If you want to drive high ROI, you need to figure out whether to outsource your program or do it internally, but either way, you need to kick the tires on all of the above issues to make sure you’re building a program that’s going to deliver the high-quality actionable insights you need to drive meaningful ROI.
