From There’s a Name for the Big Flaw in Our Obsession With Assessment and Metrics:
We live in an age obsessed with assessment. Tough-minded CEOs and numbers-driven wonks are forever looking for the perfect metric: the most objective way to define, measure, and reward success in a given arena. But there’s a tragic flaw in this hardheaded approach, and it’s known as Campbell’s Law.
Here’s the gist: The more a given metric—say, a national college ranking—is used to evaluate performance in some domain, the less reliable it becomes as a measure of overall success. Why? The people whose performance is being measured will neglect other parts of their job just to focus on boosting the relevant numbers, sometimes to the point of cheating. The chosen metric will inevitably “distort and corrupt the social processes it is intended to monitor,” suggested the social psychologist Donald Campbell back in 1974.
Notes:
(1) Cf. the Wikipedia definition of Campbell’s Law.
(2) An example: Social networks discovered that engagement was determined by the number of people you followed / friended / connected to. So they encouraged users to follow / friend / connect to as many people as possible (= the metric). But because the added relationships were weaker, they gradually destroyed the value of their social networks.
(3) Team leaders in Seeking Alpha are given a goal and a metric. The goal is a verbal formulation of what they’re trying to achieve. The metric is the best available way to measure the goal. Can a properly formulated goal eliminate the risk of Campbell’s Law?
(4) Thank you Rachael Granby for the tip.
Our approach to this has been to assign two metrics to each group/person. A metric and a counter metric, typically one business health metric and financial metric.
The most basic version of this is measuring increase in sign-ups coupled with 7d & 30d customer retention.