AI and the World's Hard Problems · Economics & social sciences · Anthropic · 2026-10-20
Evaluating discrimination in language model decisions
As language models are considered for high-stakes societal decisions such as financing or housing eligibility, their potential for discrimination raises ethical concerns. The study presents a method that uses a language model to generate a wide array of potential prompts across 70 diverse decision scenarios while systematically varying the demographic information in each. Without interventions, patterns of both positive and negative discrimination appeared in select settings in Claude 2.0, and careful prompt engineering significantly reduced both; the dataset and prompts are released.