AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-23
TrustedCode: Evaluating Large Language Models for Developing Trusted Code through Automated Program Verification
Because AI-generated code is unreliable, formal verification is a promising way to establish trust, but existing tools mostly target Java and C rather than Python. The authors present TrustedCode, which uses LLMs to type-check Python, translate it into WhyML via Why3, and verify it against formal specifications with Why3's automated provers. In experiments on ten Python programs, 50% of the observed errors came from gaps in domain knowledge, 30% from syntax, and 20% from semantic and type issues, and the tool is open source.