AI Reports · GovAI (Oxford) · 2026-10-17 · Research paper
AI Lab Policy Model Evaluation for Extreme Risks May 2023 Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, et al. Current approaches to building general-pu
The paper explains why model evaluation is critical for addressing extreme risks from general-purpose AI, which can have both beneficial and harmful capabilities. Developers must identify dangerous capabilities through 'dangerous capability evaluations' and the propensity of models to apply those capabilities for harm through 'alignment evaluations'. These evaluations will be critical for informing policymakers and for decisions on model training, deployment, and security.