Does the care an LLM recommends cost more than the treating physician's? ── Pricing AI-written plans and physicians' plans from the same visit records against one fee schedule

1. What problem does the paper address?

Most studies of large language models (LLMs) in medicine have asked about accuracy. Can the model name the diagnosis, pass the exam, beat the physician? Yet when an AI suggestion is acted on in a clinic, the first things that move are test orders, referrals and prescriptions. Each of those carries a cost.

This original article, published in NEJM AI after peer review, moves the question from accuracy to cost. Given the same visit record, does the care plan written by an LLM cost a different amount from the plan of the physician who actually saw the patient? If so, what drives the difference, and does the way the model is prompted change it^[1]?

An AI with a high accuracy score can still raise spending and patient burden if it tends to stack up tests "just in case." Cut costs too hard and things get missed. Accuracy and cost have to be measured on separate axes; that is the paper's starting point.

2. What does it propose?

The core of the method is to price the AI plan and the physician plan against the same fee schedule. According to the authors' public description of their data and analysis code, the presenting information was taken from outpatient and emergency visit records, and a set of AI systems, both general-purpose LLMs and specialized medical systems, was asked to write an assessment and plan. The comparator is the treating physician's documented plan^[3][4].

Tests, referrals and medications in each plan were extracted by several independent LLMs and mapped to US procedure codes (CPT). Orders were then priced with the Medicare Physician Fee Schedule for calendar year 2026, and drugs with a US national drug price survey. Besides a standard prompt, two prompting arms were run: one asking for a parsimonious plan and one asking the model to be cost-aware^[4].

So the study does not give AI a new capability. It re-measures existing AI output with a health-economics yardstick.

3. What did it show, within what could be checked?

A caveat first. In the bibliographic record collected for this site, the abstract field contained only the line "NEJM AI, Ahead of Print." At the time of writing the NEJM AI article page could not be accessed, and the full text has not been read. What follows rests on the summary the authors' group posted on its own publications page^[2].

According to that summary, the care recommended by the AI systems cost more per visit than the care recommended by the treating physicians. Most of the gap came from tests the AI added at visits where the physician had ordered none. Cost-aware prompting narrowed the gap but never closed it, the authors write.

The summary also gives a ratio for the size of the gap. This article does not print numbers that are absent from the primary text it collected, so the ratio is left out here. Readers who want the magnitude should go to the NEJM AI paper and the authors' summary directly.

The public repository also lists analyses of physician appropriateness ratings, diagnostic agreement between AI and physician, and test-retest reliability^[4]. How those results are reported in the paper could not be confirmed within the scope of this article.

4. A worked example

Consider a medical affairs team at a drug maker weighing an AI support tool for healthcare professionals.

The paper brings in the second speaker's view. In an accuracy comparison, a plan that over-orders can score well as "complete." Line the plans up by cost and the skew becomes visible.

5. What is not new, and where are the limits?

Studies comparing LLMs with physicians are plentiful: diagnostic accuracy, exam questions, image reading. The comparison frame itself is not new. What is new here is the yardstick. The building blocks, extraction from records, code mapping and pricing against a fee schedule, are standard in health-economics research. Several limits can be read off the available material.

• The physician plan is not ground truth. The reference is what the treating physician did, and a physician ordering no test was not necessarily right. Some AI-added tests may have covered something the physician missed. Cost differences alone do not say which plan was better for the patient.
• Pricing depends on the US system. Because the Medicare fee schedule and US drug price data are used, the results do not transfer directly to Japanese reimbursement or drug prices. The direction of the gap may be informative; its size depends on the system.
• The visit record is text. The AI read a note and wrote a plan; the physician met the patient and weighed preferences and insurance. It is hard to say both sides had the same information. The repository includes a snapshot of a public medical-transcription corpus, but exactly where the cases came from could not be confirmed here.
• Reproduction is limited. The repository itself warns that model versions and API availability change, so re-running the queries would not give the same results. The saved outputs reflect model behavior observed in spring 2026^[4].
• Extraction is also done by LLMs. Pulling tests and drugs out of the plans relies on LLMs, so extraction errors can feed into the cost estimates. Using several models is meant to contain this, but the error level could not be confirmed here.

6. How was the paper covered?

This site's selection step chose the paper as one that passed peer review and was covered by several independent outlets. The outlets listed were Nature, Science, Frontiers, Neurology Today and medRxiv, five in all. Line up the headlines, though, and none of them is about this paper.

• Nature: "Multimodal large language models challenge NEJM image challenge" (accuracy on image questions)
• Science: "Performance of a large language model on the reasoning tasks of a physician" (physician reasoning tasks)
• Neurology Today: "Large Language Model Beats Doctors in Accuracy of Diagnostic Coding" (coding accuracy)
• medRxiv: "Multimodal Large Language Model Passes Specialty Board Examination…" (board exam scores)
• Frontiers: "Blinded two-phase evaluation of large language models in complex cardiac surgery…" (task performance in cardiac surgery)

All five are separate studies whose titles happen to pair LLMs with physicians. The selection step matched them on shared words; they are not evidence that this paper was reported. For this paper, the stated selection reason of independent coverage does not hold.

The mismatch is still informative. Every captured headline sits on the question "is AI more accurate than doctors?" This paper sits outside that question: equal accuracy does not mean equal cost. That a cost yardstick reached a peer-reviewed journal while press attention is fixed on accuracy is worth recording. Neither publication in a peer-reviewed journal nor selection by this site guarantees that the results are correct, clinically valuable or novel.

7. Where does this connect to pharma and regulatory work?

First, evaluation criteria for AI support tools. If a system presents care options to healthcare professionals, the evaluation should cover not only accuracy but how much the suggestions increase tests, referrals and prescriptions. The study's design offers one template for doing that.

Second, prompts move the outcome. In the authors' summary, cost-aware instructions narrowed the gap. The system prompt is therefore part of the product specification and belongs under change control and record keeping. If one instruction can shift the pattern of recommendations, an evaluation cannot be reproduced unless the prompt used is recorded.

Third, health-economic assessment. As AI takes part in more recommendations, the way drugs and tests are used may itself shift. Cost-effectiveness work and post-marketing utilization studies may need a way to capture how far AI recommendations were involved.

None of this means the results can be applied to the Japanese system as they stand. When citing the study, it is worth stating that the pricing rests on US rules and that the reference standard is the treating physician's judgment.