AI and Cancer Research ── Kidney ── 2026-11-07
Kidney cancer: treatment, trials and AI
- Abstract
Aim: Large language models (LLMs) may facilitate the interpretation of clinical guideline information, but their performance in renal cell carcinoma (RCC) remains insufficiently characterized. We evaluated agreement between a multidisciplinary expert panel and the responses generated by Claude, Gemini and ChatGPT to RCC guideline-based questions. Materials & methods: A multidisciplinary tumor board developed 25 clinical questions based on the 2025 European Association of Urology RCC guidelines. Guideline figures and tables were converted into structured, plain-language textual summaries and supplied to the models. Each LLM generated one response per question using direct prompting. An 11-member multidisciplinary expert panel evaluated the responses in a two-round modified Delphi process using a 9-point scale and predefined consensus criteria.
Results: Consensus in agreement was reached for 24/25 Gemini responses (96%) and 23/25 responses (92%) for both Claude and ChatGPT. No response reached consensus in disagreement. The proportion of responses with a median score ≥7 was 25/25 (100%) for Gemini, 24/25 (96%) for Claude and 23/25 (92%) for ChatGPT. No statistically significant differences were detected among the models overall or in pairwise comparisons. Responses requiring further discussion or failing to reach consensus mainly involved clinically uncertain or individualized scenarios. Panelists identified occasional factual inaccuracies, omitted options and limited clinical nuance.
Conclusion: When provided with structured, guideline-derived material, selected LLMs generated responses that frequently aligned with multidisciplinary expert consensus. These findings support their potential for guideline interpretation but do not demonstrate independent clinical reasoning or real-world decision-making performance. Expert oversight remains essential, particularly in complex scenarios. What is this article about? Clinical guidelines help doctors make decisions about renal cell carcinoma, the most common type of kidney cancer, but they can be long and complex. Artificial intelligence tools such as Claude, Gemini and ChatGPT may help interpret this information. However, their answers need to be checked by medical experts before they are used in practice. What methodology was used in this study? Specialists created 25 questions based on the European Association of Urology guidelines for kidney cancer. The original guideline text was provided to the three artificial intelligence tools, with figures and tables converted into clear, structured text. An expert panel of 11 specialists then reviewed the answers and rated whether they agreed with them in a two-round process. What were the results? The expert panel reached agreement with 96% of Gemini responses and 92% of responses from both Claude and ChatGPT. There were no clear overall differences between the three tools. Difficulties were more common in complex situations, such as careful monitoring without immediate treatment in kidney cancer that has spread, expected outcomes in a less common type of kidney cancer, choosing treatments that destroy the tumor without removing it surgically, and considering imaging tests as possible alternatives to kidney biopsy. Experts also identified some incorrect information, missing options and limited detail. Why are these results important? These tools may help interpret kidney cancer guideline information. However, they were tested using information supplied from the guidelines and were not shown to make independent clinical decisions. Their responses should therefore be checked by healthcare professionals with expertise in kidney cancer.
Journal IF-equivalent: 2.7 (OpenAlex 2-year mean citedness, value as of 2026-11-05, retrieved 2026-11-06; not the official Clarivate IF)Reference: Montes SFP, García IO, Martín RB, Agustín Ferrández MJ, Arrea FA, Esteban EC, et al. Evaluation of large language models for clinical decision support in renal cell carcinoma: a multidisciplinary Delphi study. J Comp Eff Res. 2026 Nov 4 [Epub ahead of print]. doi:10.57264/cer-2026-0082.Checked: Abstract only