Source-linked AI summary
UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Model
Xing Zhang, Guanghui Wang, Yanwei Cui, Mengdie Flora Wang, Peiyang He
TL;DR
Compound LLM systems usually delegate assembly, later-call allocation, and stopping to a higher-level generative model, raising concerns about opaque and order-sensitive control. UnitBoost replaces that manager with a deterministic unit-level operator and residual-directed loop. Across held-out evaluations, it improves candidate selection, generative-manager comparisons, and fixed compound-system configurations, while its gains depend on identifiable units and can be reduced by feasibility repair.
Problem
Compound LLM managers commonly use a generative model for assembly, call allocation, and stopping, which can introduce unsupported content, depend on proposal order, and obscure worker provenance.
Method
UnitBoost maps worker outputs to slot-value proposals, uses a constrained argmax and feasibility predicate to assemble outputs, and directs later calls to a literal residual.
Results
UnitBoost improves candidate selection and fixed compound-system configurations while replacing only the management step; without cross-unit constraints, unit-wise maximization dominates complete-candidate selection under the same score.
Takeaways & Limitations
The manager trades semantic freedom for order invariance, unit provenance, and explicit, testable control over admission, residual allocation, and stopping.
Takeaways & Limitations
Benefits require multiple identifiable units with different worker coverage, and feasibility repair can consume the unit-wise gain.
Abstract
from arXiv · showhide
Compound LLM systems often solve a coordination problem by adding a higher-level LLM. The resulting meta-agent reads workers' outputs, writes the final answer, allocates later calls, and decides when to stop. It is expressive, but it also concentrates three control decisions in an opaque, order-sensitive model call. We ask whether the manager needs to be generative at all. UnitBoost replaces that model with a defined meta-level operator: a task-given unit map turns worker outputs into slot-value proposals, a constrained argmax assembles the output, and the slots left unfilled or unsupported become an explicit residual for the next round. The operator is order-free, records unit provenance, and gives a simple guarantee: without coupling constraints, unit-wise maximization under the same admission score dominates selection of any complete candidate. On three held-out benchmarks, it exceeds the best single candidate chosen with gold labels by 0.060-0.195 absolute task-score points and input-matched generative managers by 0.048-0.076. Replacing only the management step improves six compound-system configurations by 0.013-0.182. Residual-directed rounds raise FanOutQA cell F1 from 0.4778 to 0.5524; matched controls show that the true residual outperforms random targets and ordinary rereading, while a label-free supply signal flags exhaustion after one unproductive round. The same analysis measures three conditions in which no such gain is available (one indivisible unit, unavailable unit identity, and an endpoint that charges for every emitted unit) and quantifies cross-unit coupling as a repair cost. The manager gives up semantic freedom and gains order invariance, unit provenance, and testable failure conditions.
1 Introduction
UnitBoost replaces a generative manager with a deterministic operator that assembles structured units, allocates residual work, and controls stopping. This makes management order-free and auditable, but requires multiple identifiable units that workers cover differently.
- Generative managers combine assembly, allocation, and stopping in an opaque call that can introduce unsupported content, depend on proposal order, and hide worker provenance.
- UnitBoost maps worker outputs to task-defined slot-value proposals, selects values across workers under feasibility constraints, and sends unfilled or unsupported slots to the next round.
- The merge is order-free, records each accepted value’s source, score, and feasibility trace, and retains values from different rounds in one persistent unit table.
- The experiments hold workers, prompts, rounds, evidence, and call counts fixed while replacing only the candidate-to-output management step.
- UnitBoost requires multiple identifiable units with different worker coverage and cannot help with one unit, unavailable unit identity, or endpoints that charge for every emitted unit.
2 Related work
Related work places UnitBoost among generative managers, defined combiners, and classical fusion methods. Its central distinction is to keep model freedom in proposing units while code controls admission, allocation, feasibility, and stopping.
- Generative management: Prior multi-agent systems use models for assembly, allocation, stopping, or program generation, whereas UnitBoost assigns those control decisions to code once a worker topology exists.
- Combination and its ceiling: Classical metasearch is UnitBoost’s verifier-free special case, but lacks its feasibility predicate, provenance requirement, and residual round.
- Management contract: The management contract requires exposing accepted values, admission reasons, future call allocation, and stopping rationale rather than evaluating only the returned sentence.
- The UnitBoost operator: UnitBoost maps candidate outputs into task-defined units, scores values including an empty option, and assembles feasible assignments through a constrained argmax.
- The UnitBoost operator: Without cross-unit constraints, unit-wise maxima score at least as highly as the best complete candidate under the same score; feasibility repair can consume that gain.
- Scoring tiers: Deployable admission scores use worker agreement, within-worker rank, retrieved-passage support, and a retrieval-backed check rather than reference labels.
- Residual allocation: The inference loop queries workers on the residual, inserts proposals into a persistent table, applies an admission margin, and stops when residual supply falls below a threshold.
4 Experimental setup
The experiments evaluate held-out manager replacement across multiple task endpoints while keeping workers, prompts, evidence, and calls fixed. Scores are macro-averaged task measures, with bootstrap-based significance testing.
- Testbeds: The testbeds include QAMPARI, ASQA, FanOutQA, and ELI5, with held-out question splits used for evaluation.FanOutQA uses table-valued answers and reports cell F1; the other listed tasks use distinct answer or claim units.
- Endpoints and statistics: Task scores are external measures macro-averaged over questions on [0, 1].QAMPARI uses capped recall, ASQA reading coverage, ELI5 atomic-claim coverage, and FanOutQA answer-table cell F1.
- Endpoints and statistics: Held-out differences use 4,000 one-sided paired bootstrap resamples, while residual-size correlations use 4,000 permutations.Development data determines all choices, and p-values are uncorrected.
- Experimental design: Table 1 compares manager replacements while fixing workers, prompts, evidence, and calls across generation-round and protocol-step settings.Panel A replaces one generation-round manager, whereas Panel B replaces one protocol step.
A. Passing the candidate-selection ceiling
UnitBoost surpasses oracle candidate selection on QAMPARI and ASQA under held-out evaluation, showing gains over the best complete candidate selected with gold labels.
- Held-out comparison: QAMPARI: UnitBoost reaches 0.4685 versus 0.3967 for oracle candidate selection, a +0.0718 gain at p < 0.001.The comparison isolates the management step.
- Held-out comparison: ASQA: UnitBoost reaches 0.4191 versus 0.3595 for oracle candidate selection, a +0.0596 gain at p < 0.001.This is the corresponding held-out candidate-selection comparison.
B. Drop-in management inside compound protocols
Replacing existing generative managers with UnitBoost improves every reported compound-protocol configuration, while the comparison controls for the surrounding workers and execution setup.
- Drop-in replacement: UnitBoost improves six compound-system configurations over their original managers by 0.0129 to 0.1820.The configurations span five protocol families, with the largest gain on the sequential chain.
- Experimental control: The replacement experiments keep workers, prompts, evidence, rounds, and calls unchanged while changing only candidate-to-output management.This isolates the management step rather than introducing a new end-to-end agent.
5 Passing the candidate-selection ceiling
UnitBoost’s advantage comes primarily from unit-space assembly and persists across several controls, though the evidence also identifies boundaries and trade-offs for the method.
- Candidate-selection ceiling: FanOutQA rises from 0.2829 with one article-level worker to 0.4778 with unit-wise management, a +0.1949 margin.A generative manager reaches 0.6019 only after reading every retrieved article, so that comparison is not input-matched.
- Sources of gain: Unit-wise assembly contributes most of the QAMPARI gain: scoring adds 0.0063 over the deployable admission rule, while selection in unit space adds the rest.Reciprocal-rank fusion already exceeds oracle candidate selection by 0.0655 without labels.
- Generative proposal control: Unrestricted Claude Sonnet 5 reaches 0.4918 by emitting units absent from worker proposals, but restricting managers to the candidate union places every configuration below UnitBoost.Adding the unrestricted output as an eleventh proposal raises the controlled merge to 0.5132.
- Robustness: UnitBoost preserves its QAMPARI advantage with Qwen3-32B, reaching 0.4180 and exceeding oracle candidate selection by 0.0615.It also exceeds the matched Qwen3-32B evidence-reading manager by 0.0398.
- Drop-in management: UnitBoost improves every tested compound-protocol configuration by 0.0129 to 0.1820 when only the management step changes.Pooling units from every debate round adds a further 0.0467.
- Order and exchange: Sequential-chain outputs change on 57.5% of questions when worker order is permuted, whereas UnitBoost’s fixed-candidate merge is order-free.UnitBoost also gains 0.0150 from critique-only exchange, which preserves complementary units.
6 Residual-directed allocation and stopping
UnitBoost directs later workers to residual units while preserving accepted values in a persistent table, and uses supply signals to identify when additional rounds stop paying. Matched FanOutQA controls show targeted residual allocation improves over random targeting and rereading, while coverage expansion matters more than admission-margin shrinkage.
- Residual-directed allocation: Residual units are unfilled, infeasible, or unsupported slots, and accepted values persist in one table across rounds.Doubtful slots have no accepted positive-score value; new proposals compete with incumbents under the admission rule.
- Residual-directed allocation: The true residual beats matched random targeting and ordinary rereading, with gains increasing from −0.0092 for one named slot to +0.0516 for three or more.The true residual’s advantage over an untargeted prompt is not independently significant; the reported evidence is the dose response and matched-control comparison.
- Stopping: ρt reaches 0.055 in QAMPARI round four, which loses 0.0073, while new answers fall from 5.32 to 2.34 between rounds two and three.The label-free supply signal is one round late and supports stopping after the unproductive round.
- Stopping: Sweeping the admission margin from 0 to 0.40 changes held-out QAMPARI by at most 0.0023, whereas persistent coverage expansion drives the larger round gains.On ASQA, a margin of 0.10 adds 0.0036.
- Residual-directed allocation: Residual allocation expands the candidate table: ASQA round-two performance exceeds round-one oracle ordering by 0.0145, and FanOutQA round two exceeds it by 0.0237.Rescoring earlier values cannot create a slot value that no worker proposed.
7 Preconditions and boundaries
UnitBoost helps when tasks expose identifiable, complementary units whose locally selected values can be admitted. Unit identity, cross-unit coupling, and endpoint pricing define the boundaries of that benefit.
- Preconditions: UnitBoost requires multiple mechanically identifiable units, workers that fail on different units, and manageable repair or emission costs.These conditions form the paper’s pre-deployment checklist.
- Preconditions: 0.4878 versus 0.4550: on a resampled pool, UnitBoost still outperforms the candidate preferred by the deployable score, while partitioned evidence creates complementarity.UnitBoost never passes oracle selection on the resampled pool at any output budget, but its margin grows with worker count under partitioned evidence.
- Unit identity: A lexical relation catches only 5.9% of same-fact pairs at a 1.3% false-merge rate, and embeddings add just 0.0025 to five-statement atomic-claim coverage.Perfect reference identity would add 0.1413, showing that unavailable semantic unit identity limits the gain.
- Coupling: 0.038 is the ClassEval coupling gap when sibling calls are present: standalone methods transfer cleanly 0.985 of the time versus 0.947 when calling siblings.Exhaustive product search and score-order admission agree on all 95 held-out classes, while coupling remains a repair cost.
- Endpoint pricing: Under set F1, UnitBoost falls 0.0650 below oracle selection because every emitted answer is priced through precision.A price rule gets 36 of 40 combine-versus-select decisions right, compared with 24 for a constant decision.
8 Discussion
UnitBoost frames management as an auditable control layer rather than an end-to-end generative task. It extracts more value from fixed workers and allocates later rounds, but does not reduce the cost of workers or complementary calls.
- Discussion: UnitBoost licenses explicit, auditable control of admission, allocation, and stopping while models retain semantic proposal and repair.Table 7 pairs these design rules with held-out measurements.
- Discussion: The merge makes no model call, but deployable scoring adds one retrieval-backed check per value and complementary workers cost calls.The system extracts more value from fixed workers without making workers cheaper.
9 Limitations
UnitBoost's benefits depend on identifiable units and compatible evaluation settings, while its current evidence and controls leave several scope and deployment boundaries.
- UnitBoost cannot help when outputs have one indivisible unit, equivalent units cannot be identified, or every emitted unit is charged.Feasibility repair taxes can also reduce the gain that remains.
- Its labeled development score transfers across three retrieval regimes and two worker pools, but not yet across corpora.Across corpora, agreement and rank fusion remain fallback methods.
- The score lacks per-worker trust, detects residual exhaustion one round late, and positive results come mostly from knowledge-intensive language tasks.Code testbeds measure coupling rather than the primary language-task setting.
- UnitBoost may scale harmful information gathering or code generation, so deployments should restrict tools and data, cap calls, validate predicates, and retain a human halt.The reported benchmarks involve no personal data and no human subjects.
A Additional experimental detail
Additional experimental detail specifies the management contract, testbeds, model roles, and fixed-worker-call comparisons used to evaluate UnitBoost.
- Management contract: Table 3 defines equivalent generative and operator implementations for the same management responsibilities.The contract covers how proposals are managed rather than changing the worker topology.
- Testbeds: Table 4 records each testbed's unit interface and its development/test split.The testbeds include multi-answer, ambiguous-question, table-valued, and long-form factual tasks.
- Model roles: The retrieval-backed per-value check and judge-model baseline run on the same model as the candidate pool, adding no model beyond the listed roles.This keeps the scoring comparison aligned with the candidate pool.
- Experimental findings: 0.4848 agreement ranking versus 0.4778 calibrated ranking is reported on FanOutQA, while signal selection on 85 development questions does not transfer.All seven tested signal subsets nevertheless remain more than 0.19 above oracle candidate selection.
- Token accounting: 5,108 input tokens per question are used by one worker reading one ten-passage window, compared with 18,879 for one worker reading all passages.The ten-worker partition uses 51,128 tokens and ten full-list resamples use 188,788; merging itself makes no model call.
- Compound comparisons: Table 6 compares compound protocols while keeping workers, prompts, evidence, rounds, and call count fixed; only the replaced management step changes.Table 7 pairs each design rule with a held-out measurement in the body.