Building Agent Harnesses Separately and Composing Them ── Raven constructs domain-specific harnesses automatically and reworks them through experience

1. One working setup cannot carry work across domains

Turning a large language model into a working AI agent requires scaffolding around it: the tools it can call, its instructions, the control loop that drives the work, and how it stores intermediate state. That scaffolding is called a harness. The same model can do very different quality work depending on how good its harness is.

The paper starts from the observation that agent work is moving away from short, single-domain tasks toward long-horizon workflows that cross domains. Two problems follow. As harnesses grow more complex, designing them by hand stops scaling. And the more tightly a harness is built for one domain, the less useful it is anywhere else.^[1]

The authors argue that this changes the question. Instead of engineering a stronger harness for one domain, the problem becomes how to build specialized harnesses automatically, improve them through experience, and orchestrate them across domains. The primary source is a preprint posted on arXiv; it has not been peer reviewed.

2. The core idea: treat each model-harness pair as one component

Raven, described as "The Harness of Harnesses," is an open-source multi-agent ecosystem. The core of the method is to treat each executable pairing of a model and a harness as a composable unit. Raven automatically constructs harnesses fitted to particular models and domains and keeps reworking them as they are used.

A Host Agent ties the units together. It breaks a goal into subtasks, matches each subtask to a specialized agent, coordinates the order in which they run, and integrates the results. The authors call the overall arrangement an All-Domain Collaboration Network.

Experience has its own machinery. A host archive and a component called EverOS keep experience across tasks, and Skill Forge turns that experience into reusable procedures. The design intent is that methods learned on one job are stored as procedures that later jobs can call.

3. What the abstract reports

On the theory side, the authors say they establish sufficient conditions under which this kind of composition expands reliable task coverage beyond what the available individual agents can achieve, under a shared resource budget. These are sufficient conditions; they do not say that composition always expands coverage. The conditions themselves are not stated in the abstract.

On the empirical side, the authors report that Raven significantly outperforms state-of-the-art agent systems on complex, long-horizon tasks. The abstract does not say which benchmarks were used, which systems were compared, or by how much. How large "significantly" is cannot be checked from the abstract alone.

4. A worked example: from literature search to a briefing table

Suppose an agent is given one request: collect recent literature on a disease area, summarize the key points, and arrange them in a table for an internal briefing.

The point of the comparison is that in the second case the convenience comes with a system that changes every time it is used. From the standpoint of whoever checks the results, the thing being checked is not fixed.

5. What is not new, and what the abstract leaves open

Frameworks in which multiple agents take roles and converse are already widely used as open-source tools.^[5] Storing experience as executable skills and calling them on later tasks has been shown before, for example in agents operating in game environments.^[4] A coordinator that decomposes goals and routes subtasks to specialists is not new either. The paper's contribution is to bring these together in one system that makes the harness itself the object of automatic construction and revision, and composes model-harness pairs across domains, and to attempt a theoretical account of when that composition helps.

The limits are substantial. First, as noted above, the abstract gives no experimental conditions and no numbers. Second, it does not show whether people can trace when and how an automatically built, experience-driven harness has changed. Third, it does not address what happens when an error enters the experience that Skill Forge stores as a procedure and is carried into later tasks. Fourth, there is no cost comparison for cases where composition is more expensive than a single agent. Since the theory is stated under a shared resource budget, actual cost is an important axis of comparison.

6. Why this paper is getting attention: a single paper, not a cluster

This paper was selected on its own. Its cluster size is 1; no paper from another group addressing the same question turned up in this collection. By the selection design's own definition, this is not a topic rising as a cluster but a single paper that drew strong attention.

The attention is large. The paper has 511 upvotes on the paper-sharing page, and the public repository has 5,067 stars.^[2][3] However, the only author listed is a company, and the repository sits under that company's organization. The stars show interest in a usable open-source tool. They do not show that the paper's claims, in particular the claim of significant outperformance, are correct.

As a topic, how to build and improve harnesses has been taken up repeatedly in separate papers recently, and this site has reviewed earlier preprints on harness self-improvement. Within that line of work, this paper tries to move the question from improving one harness to composing many.

7. Where this connects to pharmaceutical and regulatory practice

When a computerized system enters regulated pharmaceutical work, the basic expectation is that it is shown to work as intended and that the verified state is then held under change control. A system like Raven, whose harnesses are built automatically and rewritten through experience, makes it hard to pin down a single verified state. Unless there is an outer layer that records which version produced each output and requires human approval for changes, it is difficult to bring such a system into regulated work as it stands.

A second connection is the step where Skill Forge turns experience into procedures. In practice, a written procedure has an author and a separate approver, and the reason for each revision is recorded. Procedures generated automatically from experience are convenient, but the question of who verified them tends to be left blank.

At the same time, the Host Agent's role of splitting work, assigning it to specialists, ordering the steps and assembling the result resembles how people divide work. If it is settled in advance who reviews the assembled result and takes responsibility for it, systems of this kind have room to be used at the research and drafting stage. The performance claims come from a preprint and are given without numbers. Anyone considering adoption would need to test it on tasks close to their own work first.