Goodhart's Law

"When a measure becomes a target, it ceases to be a good measure."

Charles Goodhart
Goodhart's Law
Image: Virpi Oinonen, Business Illustrator

Goodhart's Law means that when a particular metric is used as a goal or target for performance, individuals or teams may manipulate their behavior to achieve that metric, often leading to unintended consequences and undermining the original purpose of the measurement.

Impact of Agile Teams

Agile teams frequently use metrics to track progress, productivity, and quality, such as velocity, sprint burndown, code coverage, or number of defects. While these metrics are valuable for monitoring and improving processes, Goodhart's Law warns that turning these measurements into targets can lead to:

  1. Gaming the System:
    • Team members might focus on achieving the metric rather than delivering actual value.
    • Examples:
      • Inflating story point estimates to show higher velocity.
      • Spliting stories at end of Sprint to show predictability.
  2. Misaligned Prioirities:
    • Important but non-measured activities (like refactoring or knowledge sharing) may be neglected.
    • Quality may suffer if the focus is solely on quantifiable outputs.
  3. Short-Term Focus:
    • Teams may prioritize quick wins that improve metrics over strategic, long-term goals.
    • Neglect of technical debt because it doesn't immediately impact key metrics.
  4. Reduced Collaboration:
    • Individuals may prioritize personal metrics over team success.
    • Examples:
      • Developers might avoid helping others to complete their own tasks and meet personal targets.
      • Developers cherry pick stories & tasks to meet personal targets.

Scenario

An Agile team is measured primarily on velocity, calculated by the total number of story points completed in each Sprint. Management has set an expectation that the team's velocity should increase by 15% every year to demonstrate continuous improvement.

How Goodhart's Law Manifests:

  • Inflating Story Points:
    • To meet the target velocity increase, the team begins assigning higher story point values to user stories without a corresponding increase in complexity.
    • A task previously estimated at 5 points is now estimated at 8 points.
  • Compromising Quality:
    • The team rushes through development to complete more story points, leading to inadequate testing and more defects.
    • Bugs are deferred to future Sprints to keep the current sprint's velocity high.
  • Neglecting Unmeasured Work:
    • Activities like code refactoring, documentation, and team learning sessions are postponed because they don't contribute to velocity.
    • High-priority but complex features are delayed, impacting product value.
  • Avoiding Complex Stories:
    • The team cherry-picks easier, less valuable user stories to inflate velocity figures.
    • Bugs are deferred to future Sprints to keep the current sprint's velocity high.
  • Reduced Innovation:
    • Fear of not meeting velocity targets discourages experimentation and innovative solutions that might take longer to implement.

Outcome:

  • Misleading Metrics:
    • Reported velocity increases, but actual productivity and value delivered to the customer decrease.
  • Customer Dissatisfaction:
    • The product quality declines due to an increase in defects and lack of critical features.
  • Team Morale Issues:
    • Team members become disillusioned as they feel pressured to meet arbitrary targets rather than focusing on meaningful work.
  • Erosion of Trust:
    • Stakeholders lose trust in the team's reported metrics and ability to deliver genuine value.

Mitigate Goodhart's Law in Agile Teams:

To prevent the negative effects of Goodhart's Law, Agile teams and organizations can:

  1. Use Metrics as Indicators, Not Targets:
    • Treat metrics as tools to gain insights rather than goals to achieve.
    • Avoid setting rigid targets based solely on quantitative metrics.
  2. Focus on Outcomes Over Outputs:
    • Prioritize delivering customer value and quality over hitting specific metric numbers.
    • Example:
      • Emphasize user satisfaction and product usability.
  3. Implement Balanced Scorecards:
    • Use a combination of metrics to provide a holistic view of performance.
    • Include qualitative measures like customer feedback and team morale.
  4. Align Metrics with Agile Principles:
    • Ensure that metrics support Agile values like collaboration, working software, and responding to change.
    • Example:
      • Measure the number of successful deployments rather than just code committed.
  5. Regularly Review and Adjust Metrics:
    • Continuously assess whether the metrics used are driving the desired behaviors.
    • Be willing to change or discard metrics that lead to unintended consequences.

Conclusion:

Goodhart's Law highlights the risks of over-relying on specific metrics as targets within Agile teams. When team members focus on meeting metric-based goals, they may:

  • Manipulate Metrics:
    • Alter their behavior to improve numbers rather than deliver value.
  • Neglect Important Work:
    • Overlook critical but non-measured activities.
  • Sacrifice Quality:
    • Compromise on best practices to achieve short-term metric gains.

Key Takeaways:

  1. Metrics Should Inform, Not Drive Behavior:
    • Use them to guide decisions, not dictate actions.
  2. Emphasize Value Delivery:
    • Keep the customer's needs and product quality at the forefront.
  3. Promote Healthy Team Dynamics:
    • Encourage collaboration and collective ownership over individual performance metrics.

Summary

By being mindful of Goodhart's Law, Agile teams can better leverage metrics to enhance performance without falling into the trap of optimizing for the metric at the expense of genuine progress and value.