Second-Order Metrics

Avoid metrics that you can directly affect, it’s too easy to game them.

Metrics are amazingly powerful tools. If you get your team to focus on a specific metric, it brings clarity and alignment to the entire team. A team that knows it needs to increase revenue is going to only pursue things that increase revenue.

You can only focus on so many things, however, so choosing your metrics is a critical and difficult decision. One of the most common mistakes leaders make when choosing metrics is that they choose first-order metrics. 

Before we talk about first-order metrics, let’s talk about first order effects. When you take an action, it has some direct effects (aka first-order effects) but it also (likely) has some secondary effects as well. Here’s an example of an enterprise software company where you decide to start sending out more marketing emails:

The biggest difference between first and second order effects is that you directly impact the first order, while the second order is a result of customers, users or the market reacting to your actions. You can send emails and set up lots of sales meetings, and if that was your only goal you might set up a lot of useless and pointless sales meetings. What you really want is to close more customers, and while sending more emails and having more meetings help they do not get you there the entire way. 

First-order metrics track first-order effects while second-order metrics track second-order effects. The problem with first-order metrics is that they are too easy to manipulate. Even if you don’t intend to, the desire to achieve metrics can lead to cutting corners. Consider the following examples of first order metrics and how we might cheat to make them look good:

---

First-Order Metric: Size of Sales Pipeline. 

Cheats: You can add plenty of sales prospects to your pipeline by lowering your criteria or changing the definition of “prospect”. You can also provide overly-optimistic estimates for the potential contract value, all of which inflate the presumed value of your sales pipeline.

---

First-Order Metric: App Installs. 

Cheats: There are plenty of services where you can pay to have people install your mobile app, even though they will likely never use it more than once after they get paid. You can also have free giveaways and other gimmicks that trick people into installing without realizing what it means. 

---

First-Order Metric: Website Visitors

Cheats: You can run some misleading ads that trick people into visiting your website, or advertise to users who will likely never use your product but are cheap to advertise to! There are plenty of services that promise website visitors but really only send bots that look like visitors. 

---

These cheats manipulate the metrics to improve their values but not the business! You don’t need to be malicious to manipulate metrics, some of the most well-intentioned and honest people fall into this trap when under extreme stress. (Note that none of these were recommendations on how to juice your numbers for your next investor update.) There are other, more obvious, examples like total funding and valuation that I won’t cover here. 

A much better approach is to choose second-order metrics. These metrics measure things that you cannot affect directly yourself, and as a result force you to focus on the fundamentals of the business. For example, you can always spend money to generate more ad impressions but you cannot force someone to buy your product. As a result, measuring purchases might be a great way to measure advertising effectiveness, even though it’s not a metric about advertising at all!

Here are some more great second-order metrics and why we can’t cheat to make them look good:

---

Second-Order Metric: Net Retention

Why: While there are plenty of tactics to juice your user acquisition, those users will only stick around if you offer them real value. There is no single direction action you can take to convince them to stay if they aren’t happy. As a result, the number of users you retain is a great measure of how much people actually value and use your service. Whether you measure retention on a daily, weekly, monthly or yearly basis it’s one of the best metrics you have. 

---

Second-Order Metric: Margins

Why: Knowing how much money you are earning compared to your costs is critical to understanding your business. There are many ways to measure margins (gross, net, contribution) and each has different value to different businesses. All of them give you a blended view of how much you are spending to earn $1 which is hard to cheat without elaborate (and fraudulent) accounting tricks.

---

Second-Order Metric: Average Revenue Per User (ARPU) or Average Contract Value (ACV)

Why: Any measure of the average value of a user or customer is valuable to understanding how much someone is worth to your business. While it’s possible to skew the average with one really big (or small) contract, the larger your business the harder this becomes to do. And, if it really is a problem you can use the Median instead of the Average. Either way, you have a very robust measure of how much value you are creating. 

---

There will always be a gray area where you need to be careful. For example, Total Purchases might sound like a great second-order metric since you cannot force people to buy your product. However, you can reduce the price and drive a lot more purchases for a lot less money! You see this happen with struggling sales teams that offer higher and higher discounts to close deals and at retailers looking to liquidate merchandise. 

Second-order metrics are always a better choice for metrics, even if they make the job seem harder. The marketing team might complain that they have no control over the product experience, and hence a purchase, so you cannot measure them by total purchases. However, everyone is on the same team and as a result, these kinds of metrics require collaboration to achieve them. If the team can’t work together, the metrics will show exactly that.

Choosing your metrics is hard, and I know that this kind of approach makes it even harder. It’s all worth it when you find the right metrics, since your team will execute faster, and the business will grow faster than you ever thought possible.