PaperlayerView exact revision on arXiv

Source-linked AI summary

Asymmetric Capacity Allocation in Self-Refinement Pipelines

Zhuoyi Yang, Ian G. Harris, Salar Hashemitaheri, Cassie Huang, Yuangang Li, Hyunwoo Oh, Paul Dourish, Tony Givargis, Mohsen Imani, Li Zhang

arXiv:2608.21345v1cs.LG

TL;DR

Existing self-refinement systems rarely study how model capacity should be allocated across generation, critique, and revision, despite the stages’ different demands. This paper conducts a controlled stage-wise analysis across five benchmarks and two model families. Generators and refiners generally benefit from scaling, critics are comparatively insensitive to size, and capacity should therefore be allocated non-uniformly.

  • Problem

    Prior work has not systematically examined how model size affects the generator, critic, and refiner stages or whether they require equally capable models.

  • Method

    The paper independently varies generator, critic, and refiner sizes while holding the other stages fixed across 5 benchmarks and 2 model families.

  • Results

    Generators and refiners are highly performance-sensitive to capacity, whereas critic size has limited effect and even the smallest critic outperforms a no-critique pipeline.

  • Takeaways & Limitations

    Model capacity should be allocated non-uniformly across self-refinement pipelines to support more compute-efficient multi-stage language-model systems.

  • Takeaways & Limitations

    The study uses a single refinement iteration and may not directly generalize to more complex agentic systems with retrieval, planning, tools, memory, or multiple rounds.

Abstract

from arXiv · show

Self-refinement, typically structured as generation, critique, and revision, is a widely adopted paradigm for improving LLM generation and serves as a core mechanism in many LLM agents. While the three stages involve different cognitive demands, most existing approaches conveniently treat the model size as an implementation detail rather than a subject of study, which may lead to a waste of resources. Little work has systematically examined how model size affects each stage or whether effective self-refinement requires equally capable models for generation, critique, and revision. We present the first stage-wise model size study of the self-refinement pipeline on 5 benchmarks from different domains using 6 model sizes of Qwen3 and 4 model sizes of Gemma 3. We conclude that larger generators and refiners generally improve the pipeline, whereas an undersized refiner can even harm performance. Second, performance is highly insensitive to the size of the critic, although including even a small critic consistently outperforms omitting critique altogether. Our findings demonstrate that model capacity should not be allocated uniformly across self-refinement pipelines. Instead, different stages exhibit distinct size scaling characteristics, providing practical guidance for designing more computationally efficient multi-stage language model systems.

1 Introduction

Self-refinement is widely used in LLM systems, but prior work has not systematically examined how model capacity should be allocated across generation, critique, and revision. A controlled study finds that generators and refiners benefit more from scaling than critics, supporting non-uniform allocation.

  • Motivation: Self-refinement uses generation, natural-language critique, and revision to improve solutions in LLM systems and agentic workflows.The generator produces an initial solution, the critic gives feedback, and the refiner revises using the original input, solution, and feedback.
  • Research gap: Prior approaches typically select model sizes heuristically or scale all pipeline stages uniformly without testing whether capacity benefits them equally.The paper frames this as a resource-allocation question because the stages perform different functions.
  • Study design: A controlled analysis independently varies generator, critic, and refiner sizes across 5 benchmarks and 2 model families.Holding the other two stages fixed isolates each stage’s contribution and size sensitivity.
  • Main findings: Scaling generators and refiners generally improves pipeline performance, while an undersized refiner can reduce performance below the initial generation.The study identifies generator and refiner capacity as the stages to which performance is highly sensitive.
  • Main findings: Pipeline performance is comparatively insensitive to critic size, although even the smallest critic consistently outperforms omitting explicit critique.Larger critics provide only marginal gains, whereas including a lightweight critic remains beneficial.

2 Related Works

Prior research studies self-refinement and model scaling largely separately, leaving capacity allocation across interacting pipeline stages underexplored. This paper addresses that intersection with a systematic stage-wise analysis distinct from query-level computation allocation methods.

  • Self-refinement research: Self-refinement research has expanded from single-model feedback to stronger critics, verification, execution feedback, retrieval, and tool use.These extensions target reasoning, planning, coding, and text-generation tasks.
  • Model scaling research: Model-scaling research examines how model size, training data, compute, and inference-time computation affect language-model performance.Related work includes scaling laws and additional reasoning computation without increasing model parameters.
  • Inference computation allocation: Routing, cascading, adaptive selection, and mixture-of-experts methods allocate computation dynamically according to query difficulty or budget.These approaches decide how much computation to spend on each query, rather than how capacity is distributed across refinement stages.
  • Paper focus: The paper instead studies how model capacity is distributed across the interacting generator, critic, and refiner stages of a multi-stage pipeline.This distinguishes stage-wise capacity allocation from query-level computation allocation.
  • Research gap: The intersection of self-refinement and model-size scaling remains largely unexplored, motivating the first systematic stage-wise analysis of capacity allocation across these three stages.The claimed gap concerns systematic investigation of generator, critic, and refiner capacities together.

3 Method

The method models self-refinement as sequential generation, critique, and revision, then independently varies one stage’s model size while holding the other stages fixed. It also compares explicit critique with a matched no-critique pipeline.

  • Self-Refinement Pipeline: Self-refinement uses a generator, critic, and refiner in three sequential stages.The generator produces an initial solution, the critic provides natural-language feedback, and the refiner applies that feedback to the original solution.
  • Self-Refinement Pipeline: The generator produces an initial solution from input x, after which the critic analyzes it and generates actionable natural-language critique.The critique includes potential errors and actionable feedback.
  • Self-Refinement Pipeline: The refiner generates the final solution by applying the critique to the original solution.
  • Stage-wise Model Size Analysis Protocol: Generator, critic, and refiner sweeps vary their respective stage sizes while the remaining stages are held fixed.The highlighted panel identifies the stage under evaluation, and the other two stages remain fixed.
  • Stage-wise Model Size Analysis Protocol: Stage-wise experiments vary exactly one stage while fixing the other two, isolating that stage’s contribution to performance.The protocol repeats generator, critic, and refiner sweeps across benchmarks and model families.
  • No-Critique Baseline: The method compares standard self-refinement with a matched no-critique pipeline in which the refiner revises the generator output directly.The critic is fixed to Qwen3-0.6B for the critique comparison, which is averaged across generator and refiner sweeps.

4 Experimental Setup

The study evaluates stage-wise model-size sensitivity across five heterogeneous benchmarks using Qwen3 and Gemma 3, independently varying pipeline stages while fixing the others. It measures performance variability with range and standard deviation and uses representative examples to illustrate the pipeline.

  • Experimental Design: Each experiment independently varies the model assigned to one stage while holding the other two stages fixed.The study includes multiple Qwen3 sizes and Gemma 3 sizes, with Qwen3 configurations capped at 32B, 14B, or 8B.
  • Model Families: The analysis uses two open-weight model families, Qwen3 and Gemma 3, to study capacity effects across pipeline stages.The families provide diverse parameter scales while maintaining consistent architectures and training paradigms.
  • Benchmarks: The experiments span planning, summarization, logical reasoning, code optimization, and story generation across five diverse benchmarks.The benchmarks use task-specific evaluation metrics.
  • Pipeline Illustration: The pipeline is illustrated by showing each benchmark’s objective and input, initial generation, critique, and final refined solution.The examples provide intuition before the quantitative stage-wise scaling analysis.
  • Metrics: Stage-wise sensitivity is quantified by performance range (∆) and standard deviation (σ), where larger values indicate greater sensitivity to model-size increases.Table 1 reports variability while the other two stages are fixed at the largest model.

5 Results

Generator and refiner capacity strongly influence self-refinement performance, while critic size has limited effect. Explicit critique helps even when produced by a small critic, but weak refiners can degrade the initial output.

  • Generator size: Generator scaling consistently improves end-to-end pipeline performance across all five benchmarks.The trend is reproduced across multiple model configurations.
  • Generator size: Qwen3 generator variability ranges from 0.96 standard deviation on CollaboSentGen to 10.43 percentage points on PIE.These statistics quantify the effect of generator size across benchmarks.
  • Critic size: Critic size has limited impact: its standard deviation stays below 0.21 on CollaboSentGen and below 3.1 percentage points on other benchmarks.Critic variability is lower than generator and refiner variability across both model families.
  • Critic size: The 32B and 0.6B critics usually produce similarly non-misleading feedback, although the larger critic detects additional errors.On ZebraLogic, scores 3–5 cover 70% of 32B critiques and 64% of 0.6B critiques; the refiner may not exploit extra critique information.
  • Critique contribution: A Qwen3-0.6B critic improves performance over a matched no-critique pipeline across all five benchmarks.On Meeting Planning, the average gains are 9.67 and 10.34 percentage points for the generator and refiner sweeps.
  • Refiner size: Weak refiners can harm initial performance: 12 of 30 Qwen3-32B pipelines underperform their corresponding initial generations.Among 50 analyzed degradation events, 41 followed non-misleading critiques, with the weak refiner unnecessarily modifying correct content.

6 Conclusion

The study finds distinct scaling behavior across self-refinement stages, supporting non-uniform capacity allocation. Larger generators and refiners help, critics are comparatively insensitive to size, and lightweight critique still beats omitting critique.

  • Conclusion: Across five benchmarks and two model families, generator and refiner capacity strongly affect performance, whereas critic size has comparatively little effect.The study evaluates stage-wise model size allocation in self-refinement pipelines.
  • Conclusion: A lightweight critic consistently outperforms a no-critique pipeline, while a weak refiner can reduce initial performance.The findings provide guidance for more compute-efficient multi-stage language model and agentic systems.

Limitations

The analysis is limited to single-iteration generate–critique–refine pipelines evaluated with Qwen3 and Gemma 3 on five benchmarks. Its findings may not generalize to richer systems, additional architectures, tasks, modalities, or other inference-time scaling methods.

  • Scope: The study analyzes a canonical generate–critique–refine pipeline with only one refinement iteration.Generalization to systems with retrieval, planning, tools, memory, or multiple refinement rounds is not established.
  • Scope: The evaluation covers two model families and five benchmarks, leaving additional architectures, tasks, and modalities for future validation.The authors identify broader evaluation as needed to establish generality.
  • Scope: The study focuses on model capacity allocation rather than adaptive routing, larger decoding budgets, or refinement beyond one critique–refine cycle.These forms of inference-time scaling are outside the reported analysis.

A Implementation details

Experiments use Qwen3 and Gemma 3 on NVIDIA H100 GPUs with task-specific few-shot prompts and fixed greedy decoding settings across stages and model sizes.

  • Experimental setup: All experiments use Qwen3 and Gemma 3 models on NVIDIA H100 GPUs.The implementation uses task-specific few-shot prompts for generator, critic, and refiner stages.
  • Decoding: Greedy decoding uses temperature 0.0 and a maximum output length of 700 new tokens across model sizes and pipeline stages.The common decoding configuration is intended to attribute observed differences to model capacity.

B Dataset Specifics

The evaluation uses subsets of five benchmarks, with sampling or difficulty restrictions chosen to keep experiments computationally tractable. Each benchmark contributes a specified number of examples or controlled problem instances.

  • Evaluation subsets were used to keep the experiments computationally tractable.
  • Benchmark-specific settings: Meeting Planning uses 500 examples restricted to problems involving at most five people.The restriction reflects the difficulty of the benchmark for the largest model used.
  • Benchmark-specific settings: CNN/DailyMail uses the first 500 examples from version 1.0.0 of the official test split.
  • Benchmark-specific settings: ZebraLogic uses 320 puzzles restricted to instances with at most 10 cells.The restriction reflects the difficulty of the benchmark for the largest model used.
  • Benchmark-specific settings: PIE uses a random sample of 500 examples because evaluating each generated program is computationally expensive.
  • Benchmark-specific settings: CollaboSentGen uses 500 randomly selected examples.

C Additional Results on the Gemma Family

The Gemma 3 replication shows that the stage-wise scaling pattern generalizes beyond Qwen3. Generator and refiner size matter most, while critic size matters least, and weak refiners can underperform initial generation.

  • Gemma 3 results closely mirror the Qwen3 observations across all five benchmarks.The evaluation uses Gemma 3 1B-IT, 4B-IT, 12B-IT, and 27B-IT models.
  • Pipeline performance is largely sensitive to generator and refiner size across all five benchmarks.
  • A weak refiner can perform worse than the initial generation.
  • Performance is least sensitive to critic size.
  • The Gemma results indicate that the conclusions are not specific to a particular model family.

D Qwen3-14B Configuration

The Qwen3-14B configuration compares independent generator, critic, and refiner size sweeps while holding the other stages fixed. The accompanying prompts instantiate the three-stage pipeline across five benchmarks, and the figures report the sweep encodings and baseline.

  • Stage-wise scaling: Increasing generator and refiner size substantially affects pipeline performance in the Qwen3-14B configuration.The comparison independently varies each stage while fixing the other two stages at Qwen3-14B.
  • Stage-wise scaling: Critique scaling has a smaller effect on pipeline performance than generator and refiner scaling.The same qualitative observation is reported for the Qwen3-8B configuration.
  • Pipeline prompts: The generator produces an initial solution from the task description with examples.
  • Pipeline prompts: The critic analyzes the initial solution for errors and provides actionable critique.
  • Pipeline prompts: The refiner uses the task, initial solution, and critique to produce a revised final solution.
  • Benchmark prompts: Prompts are provided for Meeting Plan, CNN/DailyMail, ZebraLogic, PIE, and CollaboSentGen.The prompt groups are organized in Figures 10–24 by benchmark and pipeline stage.
  • Figure encodings: The stage-wise sweeps plot model sizes on the x-axis and use blue, green, and red lines for generator, critic, and refiner sweeps.A grey dashed line reports initial-generation performance without refinement; steeper curves indicate greater stage-size sensitivity.
  • Refiner degradation: Table 4 identifies refiner sizes whose refined outputs perform below corresponding initial generations under different model configurations.A checkmark marks a refiner size associated with lower performance than initial generation.
Loading 2608.21345v1…