Every metrics-driven manager needs to watch out for this risk

From There’s a Name for the Big Flaw in Our Obsession With Assessment and Metrics:

We live in an age obsessed with assessment. Tough-minded CEOs and numbers-driven wonks are forever looking for the perfect metric: the most objective way to define, measure, and reward success in a given arena. But there’s a tragic flaw in this hardheaded approach, and it’s known as Campbell’s Law.

Here’s the gist: The more a given metric—say, a national college ranking—is used to evaluate performance in some domain, the less reliable it becomes as a measure of overall success. Why? The people whose performance is being measured will neglect other parts of their job just to focus on boosting the relevant numbers, sometimes to the point of cheating. The chosen metric will inevitably “distort and corrupt the social processes it is intended to monitor,” suggested the social psychologist Donald Campbell back in 1974.

Notes:
(1) Cf. the Wikipedia definition of Campbell’s Law.
(2) An example: Social networks discovered that engagement was determined by the number of people you followed / friended / connected to. So they encouraged users to follow / friend / connect to as many people as possible (= the metric). But because the added relationships were weaker, they gradually destroyed the value of their social networks.
(3) Team leaders in Seeking Alpha are given a goal and a metric. The goal is a verbal formulation of what they’re trying to achieve. The metric is the best available way to measure the goal. Can a properly formulated goal eliminate the risk of Campbell’s Law?
(4) Thank you Rachael Granby for the tip.

One thought on “Every metrics-driven manager needs to watch out for this risk

  1. Our approach to this has been to assign two metrics to each group/person. A metric and a counter metric, typically one business health metric and financial metric.

    The most basic version of this is measuring increase in sign-ups coupled with 7d & 30d customer retention.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s