AI Reports · Stanford HAI · 2026-10-30 · Report / white paper
What Makes a Good AI Benchmark? | Stanford HAI
This brief presents a new framework for assessing the quality of AI benchmarks and scores 24 benchmarks against it. It finds large quality differences between them, including among benchmarks widely relied on by developers and policymakers. Most benchmarks are highest in quality at the design stage and lowest at the implementation stage.