You Have Too Many Metrics

The more metrics you track, the less you know.

Businesses are run by people, but the business itself is really just a set of numbers including revenue, profit, customers and capital. It’s a mistake to ignore the people that make those businesses work, but you need to understand the numbers to know how the business is doing. The language of business is metrics, and there are plenty of them. 

Most companies have far too many metrics. Companies might have a dozen dashboards, each with 4-5 metrics, leading to 50-100 metrics being tracked at any given time. With that many metrics, at any given time it’s likely that half of them are up and half of them are down. How do you know if you’re doing well?

Metrics are meant to simplify the complex parts of the business and help you understand, at a high level, how you are doing. When the number of metrics explodes this benefit is lost as the metrics themselves become noise! 

A Better Way

A much better approach is to assign a high cost to each metric you track. Imagine that every metric you choose costs you a million dollars a year. At that cost you’ll choose as few as possible! 

It may seem impossible to choose only a few metrics that describe how your business is doing. Your business is complex! In reality, that’s usually not true. Even complicated businesses can be described with just a few metrics, you just start small and only add one at a time as needed. 

For example, let’s consider a marketplace that connects buyers and sellers. Marketplaces are the most complex businesses because they involve so many moving parts. We want to track how our marketplace is doing with as few metrics as possible. 

To understand how the business is doing we need at least two metrics: One to measure supply (number of active sellers) and one to measure demand (number of active buyers). If supply is up and demand is down, that’s bad because sellers can’t sell. If demand is up and supply is down, that’s bad because buyers have nothing to buy. If both supply and demand are up, we’re doing well! If both are down, not so much.

That gives us a pretty simple dashboard:

Dashboard

Supply: # of Sellers

Demand: # of Buyers

However, those two metrics are not enough since we need to understand how well the supply is connecting with demand. Our sellers could be listing products our buyers don’t want, which would mean we have a lot of supply and demand but no sales! So, we need a third metric which covers the interaction of supply & demand. There are a number we can choose including total transactions or Gross Market Value (aka GMV, the total dollar value of all sales on the market). 

Let’s choose GMV, giving us an updated dashboard:

Dashboard

Supply: # of Sellers

Demand: # of Buyers

Marketplace health: GMV

You might have also noticed that we have two kinds of metrics. GMV tells us how much we’re selling right now, while supply & demand tell us how much we might sell in the future. That means we have both Success (GMV) and Velocity (supply & demand) metrics in our small set of 3 (see Success & Velocity for more details)! We covered a lot of ground with so few metrics. 

Are those three metrics enough? Well, the problem with GMV is that we might have sellers who sell really expensive items that throw off the total. So we might not sell a lot, but the GMV might be high because of a few, extremely high priced sales. This happens a lot in marketplaces, so we need a way to understand if that is happening or if the sales are well distributed. In this case we might choose the median sale price, a good measure of the average price of a sale. 

That gives us an updated dashboard:

Dashboard

Supply: # of Sellers

Demand: # of Buyers

Marketplace health: GMV

Transaction health: Median Sale Price

Okay, are THOSE metrics enough? That depends on the marketplace! For some, yes, in other cases you need to keep going. Whatever the case, based on our artificial cost of metrics we want to beware of adding anymore because it would get expensive. Instead of adding new metrics, we might replace one of the ones we’ve already chosen. In general, this kind of process forces you to be thoughtful and intentional about every metric you choose and avoids the explosion of metrics that is such an easy trap to fall into. 

How did we get here?

It’s worth a quick history lesson to talk about how we arrived in a place where our metrics exploded and became a source of frustration. There was a time when metrics were a competitive advantage! What happened? 

Much of this is the result of the collision of the plummeting cost of tracking metrics and a focus on data driven decision making. Decades ago, it was extremely expensive to collect data and track metrics so very few were used. At the same time, business had not yet adopted a belief in data driven decision making and many decisions were made through personal experience and judgment. 

Today, modern business intelligence tools make it very easy to add and track new metrics, and so there is little actual cost to doing so. At the same time, the rise of quantitative management means executives want to be seen as data driven so they want to use more and more data.

Eventually, the cost of metrics went so low and the demand so high that we saw an explosion in metrics. The end result is a perversion of the original goal of data, where the data itself is the goal!

What does this history lesson teach us? If we rely on the market to provide the constraints on our decisions, we can easily make mistakes when the market conditions change. Having discipline as a leader, and instilling your own constraints on the business, is a critical leadership skill. The framework we discussed above is a simple example of making better decisions by enforcing your own constraints in a system that imposes none on you. 

What About the Details?

What about a product manager focused on your registration flow, surely they need more detailed metrics on conversions to do their jobs? Absolutely! In fact, the same process we’ve discussed here works recursively throughout your organization. The key is that any given person shouldn’t be using any more metrics than absolutely necessary to do their job well. Those metrics might vary based on their goals and their task, but there shouldn’t be a huge pile of them. 

Using data to make decisions is great, but metrics theater is not. There are huge numbers of people that make it look like they are working hard by constantly juggling data, when in reality they are just running in place. The goal of this framework is to help everyone tell the difference between motion and progress.

The Final Word

You’re running a business, and your goal is to make money, not track metrics. Metrics are a tool, but not the destination! You want to use the fewest metrics possible to cover all the fundamentals of your business. If you do, you’ll find that the data becomes an asset instead of a burden. And who needs more burdens?