AI and the World's Hard Problems · Astronomy & space · arXiv · 2026-10-26
FacilityAudit-Agent: Beyond Mention-as-Usage for Fine-Grained Facility Contribution Attribution in Astrophysics Publications
FacilityAudit-Agent is a framework that rates, on a six-level scale, how much a facility's data supports a paper's argument beyond whether it is merely mentioned. It uses a tool-calling LLM to fetch evidence from paper sections, combines it through a decision tree and a confidence score, and routes uncertain cases to human review. On 39,358 annotated paper-facility pairs it reached 89.1% accuracy and 79.8% macro-F1, outperforming three baselines, and it is deployed at the National Astronomical Data Center for LAMOST, FAST, and EP publications.