AI and the World's Hard Problems · Physics · arXiv · 2026-10-22
Fine-Tuning Small Reasoning Models for Quantum Field Theory
The authors perform the first academic comparison of reinforcement learning (RL) and supervised fine-tuning (SFT) for LLMs on verifiable problem-solving tasks in theoretical physics, choosing quantum field theory (QFT) as the primary domain. Because verifiable training data was scarce, they built a data generation pipeline that both creates over 2,500 synthetic problems and adapts existing human-authored problems from arXiv and standard pedagogical resources for training. Both methods improve over the base model on every dataset, but SFT gains concentrate on in-distribution synthetic tasks while RL generalizes better and beats SFT on the human-adapted datasets and TPBench, and the authors release their data pipeline, verifiable QFT training data, and about 200M tokens of QFT reasoning traces alongside an analysis of how chain-of-thought errors evolve during training.