An agent's capability is set by its harness, not its model
When an agent is assembled, the language model at the center is usually frozen. What actually moves performance is everything around it — prompts, control flow, tooling, memory, context management. The authors call that surrounding apparatus the harness, and state that an agent's capability is largely magnified by it.
Harnesses have been built by hand. Recent work automates the job: a machine proposes component-wise edits to the harness, a machine selects among them, and the loop repeats. The authors read this as a practical form of recursive self-improvement operating at the agent-system level.
This is an unreviewed preprint. Every figure below is a claim made by the authors in the abstract, not a result reproduced by anyone else.
The proposal: put constraints on the evolution
The problem the authors identify is singular: evolve a harness recursively and it memorizes the tasks it trained against. Gains are large in distribution and shrink, or vanish, on out-of-distribution benchmarks.
Their method, RRSI (Regularized Recursive Self-Improvement of Agent Harnesses), applies the principles of regularization to that loop, constraining both the side that proposes and the side that selects.
- The proposer operates under a temporally annealed budget. It is limited in how many edits a single candidate can bundle, and it is pushed toward trajectories not yet explored, judged against the evolution history.
- The selector gains a critic and a pruner. The critic screens out benchmark-specific proposals. The pruner removes changes that are too small, too expensive, or no longer useful.
The intent is that what survives is a reusable mechanism rather than a benchmark-specific trick or plain noise.
What was shown, within the abstract
Evaluation covers eight benchmarks spanning coding, agentic workspace, and engineering design tasks. The authors report:
- Up to 14.1 points gained on the split the harness evolves against.
- Up to 4.7 points gained on the five out-of-distribution benchmarks.
- A resulting harness that runs on 30% fewer policy tokens than unregularized evolution produces.
The shape of those numbers is itself the argument. The in-distribution and out-of-distribution gains differ by a wide margin — the inside figure is more than three times the outside one. The phenomenon the authors criticize, large in-distribution gains that shrink outside, has not disappeared under their constraints. The claim is that the shrinkage was reduced. Code is available[2].
A concrete look: what survives and what gets cut
What kinds of edits can a self-rewriting harness end up keeping? Set the two regimes side by side.
Unregularized evolution
Prompts get rewritten to match phrasings common in the evaluation set; branches get added that assume a particular input format; many edits land in a single revision. Scores rise. Step outside that task family and the footing is gone, and there is no way to trace which edit did the work.
Constrained evolution
Edits arrive a few at a time, and proposals tuned to one benchmark are screened out by the critic. Changes with little effect, or with effect not worth their cost, get pruned. What remains is machinery that still functions when the task changes — what the authors call reusable mechanisms.
Translated into pharmaceutical practice: the difference between a document-handling agent that scores well only on the formats prepared for its evaluation, and one that behaves the same way when the format changes. The first stops holding the moment it leaves the evaluation setting.
What is not new, and where the limits are
Frameworks that improve a system automatically are not new, and constraining search to suppress overfitting is a long-standing idea in machine learning. What is new is applying that idea to harness self-improvement specifically, and placing concrete constraints on both the proposing and the selecting side.
The limits sit in the reported numbers. Out-of-distribution gains are smaller than in-distribution ones, meaning overfitting was suppressed rather than removed. The word "up to" also matters: the per-benchmark distribution is not in the abstract. Whether the average held, and whether any benchmark failed to improve or regressed, cannot be read at this range.
Nor can the abstract say how far "out of distribution" the five outside benchmarks really are. If they were constructed in similar ways, they may look outside without being outside. The compute cost of the improvement loop, the variance of the results, and reproducibility across random seeds are likewise absent. A self-improvement loop spends cost on every iteration, so the size of the gain has to be read next to that cost before any practical judgment is possible.
Why this subject is clustering now
The same cluster holds work on recursively self-improving AI research agents themselves[4]. Improving the harness and improving the entity that conducts research share the same recursive structure. This site has covered the theme repeatedly; harness self-improvement is not a single paper but a cluster that has been forming.
The reaction figures are worth stating plainly. At collection time the paper carried 176 upvotes, its repository 190 GitHub stars, in a cluster of size 2. Three independent signals — readers, implementers, and other researchers — rose together, which is a different shape from a one-off talking point. Even so, upvotes and stars record that something was read and that something was run; they record nothing about whether the claim is correct. Before peer review, whether the harness really generalized is a question only independent reproduction can answer.
What this connects to in pharma and regulatory work
First, a system deforming to fit its evaluation tasks lands squarely on internal validation design. Fix a validation set, tune against it repeatedly, and the score rises; whether the risen score reflects operational performance is a separate question. This paper measures that divergence head on. How the validation distribution is separated from the operating distribution is a decision to make at design time, not afterwards.
Second, change control for a system that rewrites itself. If a machine revises the harness, there has to be a record of what changed, when, and how. Systems used in regulated work require approval and documentation of changes. A propose-critique-prune structure does make it easier to define the unit of record, but having records and being able to justify a change are not the same thing.
Third, making cost visible. Reporting a reduction in policy tokens evaluates the harness on an axis other than accuracy. For anything run continuously in production, per-item cost and latency enter the decision with the same weight as accuracy. This is not an argument for adopting a preprint-stage method as is; it is an argument that whoever defines what counts as a good harness should not define it on accuracy alone.