arxivst stuff from arxiv that you should probably bookmark

Counterfactual Fairness: Combat the Inherent Social Biases of Your Dataset

Post · Mar 21, 2017 19:03 ·

dataset sociology

When developing ML models with real-world impacts (such as loan lending or predictive policing), it is important to take into account the different social biases that may arise towards individuals of a particular race, gender, or sexuality and compensate for these biases effectively. The Counterfactual Fairness model attempts to do just that.

Arxiv Abstract

  • Matt J. Kusner
  • Joshua R. Loftus
  • Chris Russell
  • Ricardo Silva

Machine learning has matured to the point to where it is now being considered to automate decisions in loan lending, employee hiring, and predictive policing. In many of these scenarios however, previous decisions have been made that are unfairly biased against certain subpopulations (e.g., those of a particular race, gender, or sexual orientation). Because this past data is often biased, machine learning predictors must account for this to avoid perpetuating discriminatory practices (or incidentally making new ones). In this paper, we develop a framework for modeling fairness in any dataset using tools from counterfactual inference. We propose a definition called counterfactual fairness that captures the intuition that a decision is fair towards an individual if it gives the same predictions in (a) the observed world and (b) a world where the individual had always belonged to a different demographic group, other background causes of the outcome being equal. We demonstrate our framework on two real-world problems: fair prediction of law school success, and fair modeling of an individual's criminality in policing data.

Read the paper (pdf) »