Data is a People Problem
Your data is only as good as the people that use it.
If you liked reading this, please click the ❤️ button on this post so more people can discover it on Substack. Thanks!
It’s easy to think about data in terms of math. You collect data on your business, do some calculations and view charts that show you the results. It feels a lot like doing math homework!
As a result, it’s tempting to think that the better your data collection, the better your calculations and the better your charts, the better you are using data. If it was math homework, it would mean that you get a better grade! Unfortunately, that’s not the way it works.
Businesses are made of people, and the data is only as good as the organization, communication and motivation of the people who use it. Perfect data, even if it was achievable, is still only as good as the actions taken by the people who see it. If those people are fighting, or not honest with each other or, worse, don’t understand the business the perfect data means nothing.
For example, let’s say we work at a large e-commerce company where the most important metric is “Active Buyers”. The VP of Growth sees Active Buyers as the number of new customers he brings to the site, showing how great his growth strategies work. The VP of Product sees Active Buyers as the number of Buyers who come back and buy more than once, since it shows how great her product experience can be. These people see the same metric in different ways because it’s in their best interest to do so!
Well, why not just standardize on the definition of “Active Buyers” you say? Won’t that solve the problem? No, because a better definition won’t address the underlying people problem: They don’t want to use the same definition. Instead of having different definitions of Active Buyers, these executives will just focus on different segments of Active Buyers. The VP of Growth will look at Active Buyers calculated from first time customers, while the VP of Product will look at Active Buyers calculated from repeat customers. They will just sample the data differently, while using the same definition, and get two different results. Until their incentives are aligned, no set of rules will help.
This might seem like an extreme example, but most companies suffer from this to some degree. If you have ambitious, motivated people on your team they will want to find ways to get ahead. That leads to internal competition, which in turn breaks down transparency and communications leading to some kind of dysfunction. There is a reason that corporate politics exist in so many companies!
You can spot the worst kinds of dysfunction because the your teams use data as weapons to throw at each other:
“Clearly, our revenue problems are the fault of the low marketing user volume!”
“No, lead quality is fine but we’re not converting enough of them into buyers due to product problems.”
“The real problem is price, we’re too expensive for the kind of users marketing can attract.”
You might not be able to completely eliminate these problems, but the more you do the better you will be able to use data. Data requires you to be open minded, objective and honest if you want it to drive sound decisions. It shouldn’t be a weapon used to harm your rivals and get a promotion.
That doesn’t mean everyone at your company is friendly and gets along! Some organizations are absolutely cutthroat and make excellent use of data. They are just honest about being cutthroat and hire people who thrive in that kind of environment. Own who you are, and make that the foundation of your culture, organization and communication.
So, data can help you make better decisions IF…
… You’ve built strong trust across your teams, with a foundation of honest and transparent communication.
… You’ve ensured that everyone’s incentives are aligned, and the roles and responsibilities of your org chart are crystal clear.
… Your team understands the business and the levers that affect performance.
If you don’t have those things, data can still help you optimize small things like conversion rates but it won’t really have a major impact on your business. Companies that use data for strategy have actually figured out these people problems first, even if you can’t see it from the outside.
Solve your people problem first, then worry about the data.
For more on Metrics and People, see:
Earlier in my career, I couldn't figure out ... Why don't we just publish the data? Why do the metrics we're showing keep changing? In part, it's because the business circumstances changed. But also, it's because there's a certain narrative to tell. But to your point, business success is much easier when there's transparent alignment across teams.