AI and Cancer Research ── Prostate ── 2026-11-07
Prostate cancer: treatment, trials and AI
- Abstract
Aim: Large language models (LLMs) may help interpret clinical guideline content. However, their performance in guideline-based prostate cancer (PC) scenarios remains insufficiently validated. This study evaluated the agreement between a multidisciplinary expert panel and responses generated by three leading LLMs (Claude, Gemini and ChatGPT) when addressing structured, guideline-derived clinical questions on PC. Materials & methods: A multidisciplinary tumor board created 25 structured questions based on the 2025 European Association of Urology guidelines. The guideline figures and tables were transformed into plain-language summaries for use by the LLMs. Each model was prompted with a simple, unmodified query and a panel of experts in PC management independently rated their level of agreement with the responses. The evaluation used a two-round modified Delphi method with predefined consensus criteria.
Results: Eleven experts evaluated 25 guideline-based questions. After two Delphi rounds, consensus in agreement was reached for 23/25 responses (92%) with Gemini, 20/25 (80%) with ChatGPT and 14/25 (56%) with Claude (p = 0.0033). Explicit disagreement was noted in three questions (12%) with Claude after the second round. Median scores of at least 7 out of 9, indicating strong agreement, were achieved in 24/25 responses (96%) for Gemini, 22/25 (88%) for ChatGPT, and 14/25 (56%) for Claude (p = 0.0002). Most consensus difficulties involved complex cases, advanced imaging, genetic testing or questions with limited data or controversial topics.
Conclusion: Advanced LLMs, when provided with PC guidelines with plain-language adaptations of figures and tables, can achieve strong alignment with expert consensus in routine situations. However, performance varies among models, with Claude showing lower agreement rates, especially in complex cases. These results emphasize the importance of expert oversight, continuous validation and standardized guideline formats interpretable by both clinicians and AI systems. What is this article about? Clinical guidelines help doctors make decisions about prostate cancer (PC), but they can be long and complex. Artificial intelligence tools such as Claude, Gemini and ChatGPT may help interpret guideline information. However, the answers generated by these tools need to be checked by medical experts before they can be used in practice. What methodology was used in this study? PC specialists created 25 questions based on European PC guidelines. The original guideline text was provided to the three artificial intelligence tools, with figures and tables converted into clear, structured text. For each question, experts reviewed the answers and rated whether they agreed with them. What were the results? Gemini and ChatGPT agreed with expert opinion more often than Claude. After two rounds of expert review, Gemini reached agreement in 23 of 25 answers, ChatGPT in 20 of 25 answers, and Claude in 14 of 25 answers. The tools performed better for routine PC questions than for more complex situations, such as advanced imaging, genetic testing, incomplete clinical information or controversial topics. Why are these results important? These results suggest that artificial intelligence tools may help interpret information from PC guidelines, especially in straightforward situations. However, their responses are not always reliable and should be checked by healthcare professionals with expertise in PC.
Journal IF-equivalent: 2.7 (OpenAlex 2-year mean citedness, value as of 2026-11-05, retrieved 2026-11-06; not the official Clarivate IF)Reference: Borque-Fernando Á, Ledo Cepero MJ, Rodríguez-Fernández A, Espinós EL, Breijo SM, Gordoa TA, et al. Evaluation of large language models for clinical decision support in prostate cancer: a multidisciplinary Delphi study. J Comp Eff Res. 2026 Nov 4 [Epub ahead of print]. doi:10.57264/cer-2026-0081.Checked: Abstract only