Special ── 2026-10-24

AI Fluency Index: A Baseline From 9,830 Claude Conversations

Anthropic analyzed 9,830 chats: users who iterate show more evaluation behaviors.

What changes

What organizations measure about AI skill can shift from how much people use AI to how they behave when they use it. Anthropic's report counts behaviors that can be observed in human-AI collaboration.

According to the report, people who keep refining their work with Claude also question its reasoning more often. In conversations that produce artifacts, however, evaluation behaviors were less common. Anthropic says that as AI produces more polished-looking outputs, the ability to critically evaluate them becomes more valuable.

Why it matters

Anthropic writes that people are building AI tools into daily routines at a pace that would have been hard to predict a year ago. The company says adoption alone does not show the impact of these tools.

The question it poses is whether people are developing the skills to use AI well. This study is a first baseline, meaning a reference measurement, for tracking that over time. Because behavior may change as models change, an early measurement matters.

What the report says

Anthropic's report presents the AI Fluency Index, a measure of AI fluency, meaning how well people show behaviors for safe and effective AI collaboration. It covers 9,830 conversations.

In 85.7% of the conversations, users showed iteration and refinement, meaning they built on earlier exchanges rather than accepting the first answer. Those conversations showed 2.67 additional fluency behaviors on average, against 1.33 for non-iterative ones. They were 5.6 times more likely to include questioning Claude's reasoning and 4 times more likely to include identifying missing context.

Artifacts such as code, documents or interactive tools appeared in 12.3% of conversations. In these, users were more likely to clarify their goal (+14.7 percentage points), specify a format (+14.5pp) and provide examples (+13.4pp). They were less likely to identify missing context (-5.2pp), check facts (-3.7pp) or ask Claude to explain its rationale (-3.1pp).

Only 30% of conversations included users telling Claude how they wanted it to interact with them.

Who and what scope

The report comes from Anthropic. The authors listed include Kristen Swanson, and the 4D framework was developed by Rick Dakan and Joseph Feller in collaboration with Anthropic.

It examines 9,830 anonymized multi-turn conversations on Claude.ai over seven days, January 20 to 26, 2026. The sample includes six languages: English, French, Spanish, Chinese, Japanese and German.

Anthropic cautions that these users likely skew toward early adopters. It says the results are a baseline for this population, not a universal benchmark.

Why this report

Earlier Anthropic Education Reports studied how university students and educators use Claude. This report asks how people using AI develop fluency with it over time.

Anthropic's view is that adoption does not by itself show whether skills are growing. It therefore counted behaviors that indicate good AI use across a large set of conversations. The aim is a starting point for measuring change.

How they studied it

Anthropic used the 4D AI Fluency Framework, developed by Rick Dakan and Joseph Feller with Anthropic. The framework defines 24 behaviors that exemplify safe and effective human-AI collaboration.

Eleven of the 24 can be observed directly in conversations on Claude.ai or Claude Code. The other 13 happen outside the chat and were not assessed; they include being honest about AI's role in work.

Anthropic used its privacy-preserving analysis tool. It ran 11 separate binary classifiers, one per behavior, marking each as present or absent in each conversation. Claude Sonnet 4 did the behavior classification and Claude Haiku 3.5 detected language. A screener removed greetings and chitchat.

Anthropic reports that results were broadly stable across days of the week and across languages.

Key points

1. In Anthropic's analysis of 9,830 conversations, 85.7% showed iteration and refinement, and those conversations had roughly double the other fluency behaviors.
2. In conversations that produced artifacts, directing behaviors rose, but identifying missing context fell 5.2 percentage points and fact-checking fell 3.7.
3. The findings are correlational and come from one week of Claude.ai users, so Anthropic cautions against generalizing them.

Conclusion

Anthropic's report sets a first baseline for AI fluency using 9,830 conversations. Users who iterate showed more evaluation behaviors, while artifact conversations showed fewer checks. Anthropic plans cohort comparisons of new and experienced users and tests of causal questions.

Quotes and sources

“85.7% of the conversations in our sample exhibited iteration and refinement: building on previous exchanges to refine the user's work, rather than accepting the first response and moving to a new task.”

── Anthropic, 2026-10-23

“users are less likely to question its reasoning (-3.1 percentage points) or identify missing context (-5.2pp)”

── Anthropic, 2026-10-23

“In only 30% of conversations do users tell Claude how they'd like it to interact with them.”

── Anthropic, 2026-10-23

“Our sample should be understood as providing a baseline for this population, not as a universal benchmark.”

── Anthropic, 2026-10-23

Source articles

• Anthropic 2026-10-23

Written by AI (Claude) from the report and its coverage, then checked by Jev (TypeSafe's judgement model), a Gemini fact-check and a verbatim check of every quote.