Utah's Health AI Gets Six Outside Evaluators, but No Rule on What AI Must Not Decide── Outside evaluators test the developers' claims; what the AI may not decide is written by the state, agreement by agreement

On October 5, 2026, Utah's Office of Artificial Intelligence Policy widened its regulatory sandbox for medical AI and named six outside evaluators to check what AI developers say about their products. Once outside evaluators are in place, does anyone decide which judgments the AI must never make? The evaluators test whether the AI does what its developer claims. The judgments it must not make are written by the state, one agreement at a time, and no general rule exists yet.

01Utah's evaluators test developer claims, while the limits on AI sit in each agreement

The six evaluators Utah named this month have one job in common: to find out whether an AI product behaves the way its maker says it does. The list includes the Coalition for Health AI, a group that works on standards for health AI, and Stanford's Clinical Excellence Research Center.

Deciding what the AI may not do is not part of that job. In the pilot that handles prescription refills, the AI is barred from writing new prescriptions and from changing doses. That bar is not in any evaluator's report. It is in the agreement the state signed with the company.

So the evaluation and the list of forbidden decisions live in two different documents. Anyone bringing AI into their own work has to check both: whether the system has been evaluated, and where the decisions it must not make are written down. I read the Utah announcement as practice in keeping those two questions apart.

02The sandbox is a state agreement that eases penalties for up to 12 months

The agreements Utah signs with companies rest on a 2024 state law, S.B. 149, usually called the Utah AI Policy Act. It created the Office of Artificial Intelligence Policy and a "learning laboratory," whose purpose is to draw lessons from AI trials that can feed into the state's regulatory framework.

The statute defines "regulatory mitigation." One item in that definition is reduced civil fines during the participation term. An initial mitigation agreement may last no longer than 12 months.

Some agreements go further than lighter fines. In January, Doctronic, a medical AI company, signed an agreement with the state to use AI in renewing prescriptions. The law firm Polsinelli, analysing that agreement, reported that the state will forgo enforcement actions for unlawful conduct as long as the company keeps to the agreed conditions.

The current pilots sit between the third and fourth steps of that chain. The last step, where pilot reports return to the state as rules, has not been reached.

03Prescription refills, acne telehealth and postpartum therapy are the three new pilots

The state is now using these penalty-easing agreements in three areas of care. Set side by side, the three pilots differ in how much of their limits has been made public.

Only the August AI pilot came with a clear public statement of what the AI will not do. According to the Salt Lake City station FOX 13, the AI will not write new prescriptions, change doses or substitute medications.

For Nolla Health, AI Weekly reported that in the first stages a licensed clinician will review the AI's decisions. There is a rule that a person looks at the output. There is no published list of decisions the AI is forbidden to make.

There is one fact here that touches the pharmaceutical side directly: AI has entered the renewal of existing prescriptions under a state agreement. Nothing more about medicines can be drawn from this coverage, and I will not try.

04A hospital forecast that up to 40% of providers will be gone by 2035 drives the push

The pilots were narrowed to refills and mild acne for reasons the people running them stated openly. Here they are, one speaker at a time.

The 40% figure is a forecast from Intermountain Health, one of the large hospital systems in the state. It is not a state statistic. Still, the expectation that doctors and therapists will be in short supply is shared by state officials.

Margaret Woolley Busse, who heads the state's Department of Commerce, said that if the state gets this right, people will see better access and lower costs. Keeping the door to care open while the workforce shrinks is the stated aim.

Boyd's reason for choosing acne was that it is common and carries comparatively low risk. My own reading is that starting with low-risk care keeps the harm small when the AI gets something wrong. The coverage does not say who judged acne low risk, or by which standard.

05The six evaluators examine technical attestations and the AI's decision logs

Having started with low-risk care, the state hands the checking to six outside bodies. According to AI Weekly, their work falls into three parts.

First, they verify the technical attestations that developers submit. Second, they audit the records of decisions the AI has made. Third, they provide an independent clinical evaluation against medical standards.

Boyd said outside review would ensure that "every claim made by AI developers is independently verified." Mark Sendak, chief executive of Vega Health, one of the six, said the evaluators would independently test whether products meet the state's standards of evidence.

The power to decide whether a company enters the sandbox stays with the state. The medical profession has also changed its role. Members of the Utah Medical Licensing Board had criticised the earlier prescription-renewal pilot for going ahead without their input.

The board has since set up a working group that supports the AI office's reviews. Dr. Bill Hamilton, a board member, said the working group does not amount to blanket approval of AI pilots.

The upper row in the figure checks whether a product does what it claims. Writing down what the AI must not do happens only in the lower row.

06Claim checks, forbidden decisions and paused enforcement are settled in three separate places

Even after the evaluators have checked attestations and logs, a reader is left with gaps. I count three, and each one sits in a different document.

Passing evaluation does not tell you how much to trust

Outside evaluation stops at verifying the developer's claims. The tasks listed for the evaluators were checking attestations, auditing decision records and judging against medical standards. Deciding how far the AI may be trusted is not on that list.

A report that says a product has been evaluated therefore says nothing about the scope of decisions it may take. An adopter has to find that scope somewhere else.

Forbidden decisions are written agreement by agreement

Asked where the line should fall, Commissioner Busse said: "I don't think that we know that answer yet, because the capabilities of AI are changing rapidly." She added that the state always wants a human involved somewhere.

The bar on new prescriptions, too, sits inside the August AI agreement. Each time a new use enters the sandbox, its forbidden decisions will have to be written again from the start.

Enforcement can pause while a pilot runs

The state law allows fines to be reduced during the agreement term. The January agreement with Doctronic was analysed as pausing enforcement so long as its conditions are met.

If the AI errs during a pilot, state penalties may not reach the company. How errors are handled is set by a clause separate from anything in the evaluation report.

None of the three shows up if you read only the evaluation report. They appear when you open the agreement and the statute.

07Whether a general rule follows depends on the pilot reports after 12 months

Evaluation, forbidden decisions and the handling of errors are now scattered across company-by-company agreements. Does the state intend to gather them into rules?

The reporting chain gives a partial answer. Participating companies file quantitative metrics with the state every month. The AI office reports to the Legislature's Business and Labor Interim Committee before November 30 each year.

Lawmakers said the results will inform future legislation. The topics they named included liability, patient privacy, transparency for consumers and the role of licensed professionals. Senator Cullimore's interest in a federal version is just that, an interest, not a plan.

What I could not find in the coverage is anyone promising to write the forbidden decisions into a general rule. The commissioner herself said the answer is not known yet. Whether a general rule will be written after the 12-month agreements end cannot be confirmed today.

Decisions to bring AI into work will not wait for that rule. The Utah case does suggest an order in which to read the documents before adopting a system.

Having an evaluation is only the first step. If the decisions the AI must not make are not written down, I stop there and have them written before going further.

Outside evaluators can check medical AI, but they are not the ones who decide what the AI must never decide. The state decides that, and Utah writes it down one agreement at a time.

Anyone adopting AI has to check the evaluation and the written limits as two separate things. When I see the word "evaluated," the next thing I look for is the page that says what the AI will not be allowed to do.