Building training data from model debates without answer labels ── A preprint introducing AbCD, which turns multi-agent debate transcripts into preference data without gold labels

1. What problem does the paper address?

The problem is how to build reliable training material from model answers when there is neither a gold answer nor a checker. Post-training for reasoning typically relies on gold answers, task-specific verifiers or learned reward models. The authors argue that this reliance limits scalability where such signals are scarce^[1].

Multi-agent debate is one alternative. Several model instances propose answers and reasoning, then exchange them over several rounds to reach a common answer, and this has been reported to improve mathematical reasoning and factual accuracy^[4]. But better answers through debate and usable training data from debate are separate questions.

The authors name three difficulties in converting debate into supervision. Majority voting weights agreeing responses equally, has no principled tie-breaking rule, and is ill-defined for open-ended tasks without a canonical answer. Likelihood-based scoring can confuse fluency with correctness. Homogeneous committees may reinforce correlated errors^[1]. This article discusses a preprint posted to arXiv that has not been peer reviewed, and what follows should be read as the authors' claims.

2. What does it propose?

The core idea is to group the final answers of a debate by semantic equivalence and build preference pairs from confidence-weighted disagreement between those groups. The authors call the method Alignment by Confident Debate (AbCD)^[1].

The procedure runs in three steps. A committee made diverse through different sampling debates each problem. The final answers are then clustered by task-level semantic equivalence, so that answers written differently but meaning the same land in one group. Comparisons between groups yield pairs of preferred and dispreferred answers, which are used for post-training.

Each part of the design maps to one of the three difficulties. Grouping by meaning reduces vote splitting caused by surface differences. Weighting by confidence addresses the way majority voting counts every agreeing vote the same. A sampling-diverse committee is meant to limit correlated errors. The abstract concludes that debate traces can yield useful preference supervision when answer equivalence is combined with confidence-weighted disagreement^[1].

3. What does it show? (Only what the abstract states)

The evaluation uses verifiable tasks: mathematical reasoning, multiple-choice quantitative QA, and executable code generation. These serve as a controlled test bed in which the quality of debate-derived supervision can be checked against ground truth. The authors stress that the method itself never uses gold labels^[1].

• Comparison: post-training on the resulting preferences outperformed inference-only ensembling and debate, as well as supervised fine-tuning (SFT) on the same traces.
• Scope: five model families spanning 1B to 8B parameters and four datasets. Consensus accuracy improved, with a maximum gain of 21.5 percentage points.
• Further analysis: ablations showed transfer to a larger model and to code generation, and also revealed limitations of heterogeneous committees and agreement-based weighting.

The 21.5-point figure is a maximum. The abstract gives no average gain and does not describe the combinations where gains were small. It also does not say how large the "larger model" was.

4. A concrete example

Consider three models that have debated the same arithmetic problem, and compare how majority voting and AbCD treat their answers. This is a hypothetical illustration, not an experiment from the paper.

This works because it is a maths problem, where semantic equivalence is easy to check mechanically. For tasks such as summaries or explanations, where the same meaning can be phrased in countless ways, how well the same grouping would hold up lies outside this paper's evaluation.

5. What is not new, and where are the limits?

Improving answers by letting several models debate has been reported before^[4]. Tuning a model directly from preference pairs is also an existing technique^[5]. The addition here is a procedure for turning debate transcripts into preference data without gold labels, combining semantic grouping with confidence weighting, and testing it on tasks with known answers.

There are at least four limits. First, all evaluation is on verifiable tasks. Whether the method works on the open-ended tasks the paper cites as motivation is not shown in the abstract. Verifiable tasks were chosen to measure supervision quality; they are not evidence of effectiveness on open-ended work. Second, the models range from 1B to 8B parameters. Transfer to a larger model is mentioned, but its size is not given.

Third, the authors themselves report limitations of heterogeneous committees and agreement-based weighting. Mixing model types is the natural response to correlated errors, yet this suggests it does not always work, so committee design is not a solved question. Fourth, gains are measured as consensus accuracy. If a committee agrees on the same wrong answer, a path for that error into the training data remains. Within this paper's scope, the claim is that on verifiable tasks, post-training on debate-derived preferences beat the comparison methods.

6. Why is this topic clustering now?

The selection system placed this paper in a cluster with two papers on multi-agent reinforcement learning (cluster size 3). One shows that, in cooperative multi-agent reinforcement learning, a population problem can be separated into learning a single-agent policy and coordinating agents through optimal transport^[2]. The other presents SEER, a runtime that executes reinforcement learning environments with many interacting entities in parallel, event by event^[3].

The shared keywords include multi-agent, reinforcement and learning. On closer reading, though, AbCD builds training data from debates among language models, while the other two address the theory and computing infrastructure of agents acting in an environment. The vocabulary overlaps; the questions do not. The cluster is better read as a broad shared interest in how to coordinate and train multiple agents than as several independent studies converging on one question.

The paper has 0 upvotes on Hugging Face and 0 GitHub stars. A cluster size of 3 counts overlapping vocabulary; it says nothing about whether the findings are correct or important.

7. What connects to pharmaceutical and regulatory practice?

The connection is the question of whether agreement among several AI systems can serve as evidence of quality in document work where model answers are hard to prepare. When language models assist with drafting or reviewing pharmaceutical documents, building large sets of reference answers is difficult. Having several models review a document and adopting what they agree on is an attractive way to save effort.

This paper adds two cautions to that practice. One is the motivation the authors themselves give: a homogeneous committee may reinforce the same errors. Several models built on the same foundation may not amount to an independent check. The other is that the authors tested the method only on verifiable tasks; they did not show that it works on open-ended tasks such as summarisation or judging whether wording is appropriate.

For regulated documents, responsibility for the final decision rests with people. Agreement among AI systems can help narrow down what a person should check, but recording agreement itself as evidence of correctness is not supported by this paper. Anyone considering such a setup would first need to measure, on their own tasks and against a small set of expert-confirmed answers, how closely AI agreement tracks correctness.

8. Summary

AbCD turns transcripts from debates among language models into preference data without gold labels, by grouping answers by semantic equivalence and using confidence-weighted disagreement. The authors claim that on verifiable tasks it outperformed inference-time ensembling, debate, and SFT on the same transcripts.

Agreement reached in debate is not the same as being right. What this paper tested were tasks with checkable answers, and its effectiveness on tasks without a correct answer has not been shown. As a preprint, its average gains and the limits of its committee design should be checked in the full text.