Appfire-BEST-WM-Show-E6-Christopher-Penn-KPIs

“How do I choose the right key performance indicators for my business team?” Find out on this episode of Appfire Presents: The BEST Work Management Show by Appfire.

Data scientist Christopher Penn joins Kerry O’Shea Gorgone to explain how to select key performance indicators (the kind of metrics that get you promoted or fired). Chris explains how to use data to figure out which metrics matter, and how to test whether or not they truly contribute to your team’s success.

About the guest

Christopher Penn is the chief data scientist at Trust Insights, Inc. He’s an authority on analytics, digital marketing, and marketing technology, a recognized thought leader, best-selling author, and keynote speaker. Chris is the author of more than two dozen books, including AI for Marketers: A Primer and Introduction.

About the show

The BEST Work Management Show by Appfire features smart leaders sharing their secrets for optimizing business processes and increasing productivity. Get the goods on how they handle everything from setting up workflows to automating processes. Every episode is 10 minutes or less, packed with insights you can use right away to supercharge your team’s productivity.


For your convenience, here is the transcript of this episode:

How do I choose the right key performance indicators for my business team?

Kerry:  Today we’re going to address the question how to choose the right key performance indicators for my business team. To help us answer that question is Christopher Penn, one of the smartest people I know, hands down. He is the chief data scientist at Trust Insights, he’s an authority on analytics, digital marketing, marketing technology, and AI. Stick around for 10 minutes of really valuable advice.

Thanks for joining us today, Christopher Penn, @CSPenn.

Christopher:  Thank you for having me. 

Kerry:  We’ve talked a lot over the years about business and what success looks like and how you measure it. Where do we start, how do we choose the right key performance indicators for a business team?

Christopher:  There’s the simple answer and then there’s a complex answer. The simple answer is what do you get your bonus for, that’s your key performance indicator. If you don’t know what that number is, there’s a problem. Right? What does your boss get their bonus for? Again, if you can’t measure that, it might be time to update your LinkedIn profile.

Kerry:  Okay. That’s the short answer, but we have 10 minutes. Let’s say an HR team, a marketing team, a legal team… Software developers obviously understand what you’re supposed to be doing, what kind of things to look at, but for a business team it can get a little squishy. That’s a technical term.

Christopher:  It can, but at the end of the day, a KPI is one of a set of metrics. Other metrics, other quantitative measures exist, and then the hard part for a lot of people is saying out of this buffet of numbers we have, which one should we be paying the most attention to. There’s a couple of different ways to get at that. 

The most mathematically sound way to do that is to have some sort of business outcome that has a quantifiable outcome. Revenue, it could be, maybe it’s employee retention rate, maybe it’s closed one deal. It really depends on your role within the organization. You essentially put a column of that number by as granular as you can make it in a spreadsheet, and then you put in all of your other metrics. Everything else that is under your purview goes in that spreadsheet. 

Using statistical and mathematical techniques like regression analysis and decision trees, you mathematically look at every other number alone and together compared to this outcome that you care about, and you say which has the strongest mathematical relationship to the outcome we care about. That helps you figure out your KPIs because these are the numbers that have that relationship.

I work a lot in marketing. In marketing, particularly in B-to-B, marketing qualified leads in the number. Then you have all these other things, like email open rates, click rates, YouTube followers, Twitter mentions, you name it, and you have all of those variables. You put them in that big spreadsheet. You say, machine learning software of your choice, which of these have the highest statistical correlation to marketing qualified leads. It may come out with emails sent, and it may come out with number of tweets that have a poop emoji in them, and it may come up with a couple of different numbers. 

That then becomes your testing plan to say I think these are my KPIs. Do I have control over them? Because a number that is important that you have no control over is not a KPI because your performance can’t change it. You have to have control over it. Then is it something that essentially you can manipulate in some way?

Kerry:  You say manipulate, but not a bad thing. We’re in the business of moving needles one way, the good way. 

Christopher:  Yes. We’re in the business of trying to get things done. Let’s say it is number of emails sent on Tuesdays. That could very well be one of the contributing factors. If that’s the case, then you say if I know that’s the number, if I double the number of emails sent on Tuesdays, do I see a corresponding doubling of my KPI? 

So, the first part is to do the mathematical analysis to build a hypothesis. Then the second part is to test the hypothesis by changing the inputs that you presumably have control over and see if the KPI comes out. 

Kerry:  So, you don’t just choose and then leave it. You choose and then test it. 

Christopher:  Yes. Because as every stats 101 course has ever taught us, correlation is not causation. Just because two numbers march in lockstep does not mean that they are actually related. Tyler Vigan has a fantastic website called Spurious Correlations. The number of drowning deaths and the number of movies Nicholas Cage has been in, and there’s a very strong correlation but they have nothing to do with each other.

Kerry:  I did see once, a long time ago, that Demi Moore’s movies that she did with short hair outperformed the ones she did with long hair. I feel like there was definitely some causation there. But I see what you’re saying.

So, select your KPIs, set your hypothesis, and then you test it.

Christopher:  Then you test them. Granted, there are going to be some instances where that might not be possible. If you are working in HR and employee retention is your key outcome, you might not be able to test certain things to see does this reduce retention. 

Kerry:  Do people quit or not? 

Christopher:  Exactly. In those cases, you would do a different type of statistical testing as your second half. The technical name is propensity score modeling, but really it’s retroactive A/B testing. You look at a period of time where you’ve made a change and then you look to see compared to other similar periods of time, did the KPI change one way or the other. 

Kerry:  Okay. That way, instead of testing in advance, you’re looking backward to test it. 

Christopher:  Exactly. Because there are some things you couldn’t test. For example, an employee benefit to just a certain percentage of the employees if it was a mission critical benefit, like we’re not going to offer healthcare to half of our employees. That won’t fly.

Kerry:  I’m pretty sure it won’t happen. You probably don’t have to test that one.

Christopher:  Right. That’s the third part, I guess the unspoken part. All of the mathematics are great, but if something doesn’t pass the common sense test, then obviously you know that something has probably gone wrong in your data itself. Again, it’s the silly things, like only tweets with poop emojis do well. You might have done something wrong in your data, or you just have a really weird correlation, and you probably don’t need to test that. 

Kerry:  It could be something else, like you were saying, too. If those are the ones that you’ve sent at normal business hours, and the ones that you sent without poop emojis went out during off hours or something. I don’t know. Maybe there’s another reason is what I’m saying. 

Christopher:  Exactly.

Kerry:  You’re talking about a lot of math here, and it’s a little scary for some people. Probably not for developers, but it’s a little scary for me. For business teams who can’t custom code their own thing to run math, what do you recommend?

Christopher:  Beer. I mean this in the sense of if your organization has these capabilities in another department, which is the case particularly in mid-market and enterprise companies, bring some beer down to that department on a Friday, get together with those teams, and do some cross-team collaboration. Say, “Hey, this is a thing that we’re interested in. Is it something that you folks could build or help us build, or do you know of a vendor, an agency, or a contractor that could build this kind of model?” 

These things certainly do exist, all of the consulting firms do stuff very similar to this, but you might have that talent internally within your organization. It doesn’t have to be beer. It could be the beverage or food of your choice, whatever your internal teams prefer.

Kerry:  Muffin basket. 

Christopher:  Muffin basket, flowers, other allergens. Whatever the thing is, that would be the approach to take is to say who in our company can do this. In every larger client that we’ve worked with, there’s been a data science team or an AI team or a dev team that for them it’s like that’s kind of cool, that’s interesting, maybe we could build this, and it becomes part of our secret sauce. 

Obviously, for that team, it reinforces their value to the company as well, to say we can build an internal model of what’s working, and the maintenance of that model then becomes a mission critical business imperative for that team.

Kerry:  The squeaky wheel gets the grease is what you’re saying.

Christopher:  I suppose. Or the beer.

Kerry:  When you’re going to a new company and you need to help them figure out what is strategically important before the math, what’s the step before the math to make sure? 

Christopher:  The step before the math is something that my business partner and CEO Katie Robbert talks about, which is the five Ps. Purpose – what is the purpose that you’re trying to fulfill, what is the overall thing that you’re trying to accomplish? 

With KPIs, it’s what performance are you trying to measure, what outcome are you trying to measure, why are you measuring it, is there a logical reason? We had one client a long time ago whose KPI was number of Twitter followers. He was really hellbent on having more Twitter followers than a competing company’s CEO. Cool. That was his purpose.

The next step is you look at the people. Who is involved, what kind of people do you have, what skills do they have, and can you bring them to bear? If they don’t, what kind of agencies and partners do you have? 

Your third part is what is the process for which you currently gather and analyze data, and are there things that you can reuse, are there gaps that you need to fill somehow? 

Then you get to the platform. That’s when you get to all of the math and stuff. That purpose, people, and process comes before the math. If you don’t do that part, then you can run into issues.

Kerry:  For more information or to hire Chris and his team, go to TrustInsights.ai. For more episodes of The Best Work Management Show, check out Hub.Appfire.com

Thanks for joining, Chris.

Christopher:  Thank you for having me.

Last updated: 2022-11-22

Recent resources

Back to Top