PaperlayerView exact revision on arXiv

Source-linked AI summary

Jev-LiteMAS: Separating Decisions from Generation in Multi-Agent Workflows

Kaixiang Wang, Zhaijiacheng Zhou, Jiong Lou, Jie Li

arXiv:2610.21411v1cs.AI

TL;DR

Multi-agent workflows spend substantial generative-LLM resources on routing and review, although these operations often require bounded decisions. Jev-LiteMAS combines decision-based routing and screening with generative execution and conditional feedback; across four benchmarks and two backbones, it reduces token use relative to LAS while maintaining similar accuracy and usually lowering latency.

  • Problem

    Routing and review account for 31.7–43.5% of generative-LLM tokens, but replacing generative planners and reviewers with bounded decisions leaves gaps in handoff instructions and actionable repair feedback.

    Source: §1 · p. 1

  • Method

    Jev-LiteMAS uses Jev to select experts from task context and prior outputs and screen artifacts, escalating flagged artifacts to generative review and possible expert repair.

    Source: §1 · p. 2

  • Results

    Jev-LiteMAS reduces generative-LLM token usage by 12.7–54.6% relative to LAS, with accuracy differences within 1.34 percentage points and lower latency in seven of eight configurations.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Grounding routing in substantive expert feedback matters: removing that feedback reduces accuracy by 18.00 points on IDRBench I3 and 9.02 points on API-Bank.

    Source: §1 · p. 2

    Routing uses a predefined pool of expert roles and legal actions, and savings from gated review may decrease when more artifacts require generative review or revision.

    Source: B Limitations and Future Work · p. 11

Abstract

from arXiv · show

Multi-agent LLM systems rely on generative models for expert execution, workflow planning, and artifact review. Yet many coordination operations require only bounded decisions, such as selecting the next expert or judging artifact quality. Profiling three representative MAS frameworks, we find that routing and review account for 31.7-43.5% of generative-LLM tokens. However, replacing these operations with low-cost decision models introduces two interface gaps: generative planners produce execution plans and handoff instructions (planning gap), while reviewers provide actionable feedback for repairing flawed artifacts (review gap), which bounded decisions alone cannot provide. We introduce Jev-LiteMAS, a multi-agent framework that bridges both gaps by combining a lightweight decision model (Jev) with generative execution and feedback. To address the planning gap, Jev selects experts dynamically from the task and prior expert outputs, using accumulated context to ground subsequent execution without a pre-generated trajectory. To address the review gap, Jev screens intermediate artifacts against task criteria: passing artifacts proceed directly, whereas flagged ones undergo LLM re-review for acceptance or corrective feedback. Across four benchmarks and two backbones, Jev-LiteMAS reduces generative-LLM tokens by 12.7-54.6% over LAS, maintaining accuracy within 1.34 percentage points and lowering latency in seven of eight settings. Grounding routing in substantive expert outputs prevents up to 18.0 points in accuracy loss.

1 Introduction

Jev-LiteMAS assigns bounded coordination decisions to a lightweight model while retaining generative models for execution and corrective feedback. Across four benchmarks and two backbones, it reduces generative-LLM tokens relative to LAS while keeping accuracy differences within 1.34 percentage points.

1 Introduction

  • 31.7–43.5% of total generative-LLM tokens go to routing and review across six measured configurations, with latency shares of 29.5%–46.1%.The profiling covers MetaGPT, AgentNet, and LAS on MATH and MBPP+ using DeepSeek-V4-Flash.Source: §1 · p. 1
  • Bounded decision models can select experts or judge whether artifacts proceed, but do not provide planners’ handoff instructions or reviewers’ actionable repair feedback.These are the planning and review interface gaps Jev-LiteMAS addresses.Source: §1 · p. 1
  • Jev selects each next expert from the task and prior outputs, while flagged artifacts receive LLM re-review and, when needed, corrective feedback for expert repair.Qualified artifacts proceed without a generative reviewer.Source: §1 · p. 2
  • Compared with LAS, Jev-LiteMAS uses 12.7–54.6% fewer generative-LLM tokens, keeps accuracy differences within 1.34 percentage points, and achieves lower end-to-end latency in seven of eight configurations.Source: §1 · p. 2
  • The paper contributes an information-preserving workflow combining feedback-driven handoffs, bounded artifact screening, conditional generative review, and expert-guided repair.Its evaluation also examines workflow organization, semantic feedback, and review reliability through ablations and error injection.Source: §1 · p. 2

2 Related Work

Prior work studies adaptive workflow construction, coordination, and bounded decision models in agent systems. Jev-LiteMAS applies choice-based decisions to multi-agent workflows while retaining generative reasoning and corrective feedback.

LLM Multi-Agent Systems

  • Workflow research includes searched or learned architectures, task-aware communication, adaptive routing, and recovery through evolving memory or earlier execution states.Examples include AFlow, MetaAgent, MaAS, G-Designer, MasRouter, AnyMAC, AgentDropout, LAS, AgentFlow, and ReAgent.Source: §2 · p. 2

Jev and System-One Decision Models

  • Jev offers typed choices, rubric scores, and binary judgments, while related studies examine its decision quality, cost, calibration, and stability.Other work applies bounded decisions to memory operations, routing, agent control, and multi-agent coordination.Source: §2 · pp. 2–3

Jev-LiteMAS

  • Jev-LiteMAS integrates choice-based decisions through feedback-driven expert handoffs and gated artifact review, retaining generative reasoning and corrective feedback.The stated aim is to balance task quality, review reliability, and execution efficiency.Source: §2 · p. 3

3 Problem Setting and Design Rationale

The section distinguishes open-ended expert execution from bounded routing and review, then motivates a workflow that allocates these capabilities by operation. Its design uses current execution context for handoffs and conditional generative review for artifacts needing further attention.

3.1 Operation-Level Multi-Agent Workflows

  • The framework separates expert execution, expert routing, and local artifact review, whose outputs respectively require generation or can use bounded decisions.Routing selects an eligible expert or termination action; review determines whether an artifact can proceed.Source: §3.1 · p. 3

3.2 Capability Mismatch in Workflow Control

  • Choice-based routing selects from supplied actions, unlike generative planning that can produce task-specific plans and handoff instructions.Handoffs must remain informed by the latest artifacts, and the next expert must retain the information needed for execution.Source: §3.2 · p. 3
  • A bounded review decision can screen an artifact, but artifacts needing attention still require generative reassessment and actionable repair guidance.The section frames capability allocation as bounded decisions guiding execution while generative operations supply content for completing and revising tasks.Source: §3.2 · p. 3

3.3 Capability-Matched Workflow Design

  • The design assigns bounded routing and initial review to a choice-based model while generative experts retain task execution and artifact revision.Subsequent experts are selected after observing the latest execution results, and screening determines whether an artifact proceeds or receives LLM re-review.Source: §3.3 · p. 3
  • Jev-LiteMAS combines feedback-driven expert handoffs with Jev-gated LLM re-review.The section presents this as the capability-matched workflow design.Source: §3.3 · p. 3

4 Jev-LiteMAS

Jev-LiteMAS coordinates generative experts with Jev-based routing and artifact screening, while retaining generative review for escalated artifacts. Its workflow uses accumulated task context and expert outputs to guide successive execution.

4.1 System Overview

  • Jev-LiteMAS combines generative experts, Jev-based routing and screening, and an LLM reviewer for escalated artifacts.Experts produce task outputs and revisions; Jev chooses the next expert and screens intermediate artifacts.Source: §4.1 · p. 3, §4 · p. 3

4.2 Feedback-Driven Expert Handoffs

  • Routing selects an eligible expert from the current task state and candidate set, adapting each handoff to the latest expert outputs without a pre-generated trajectory.The selected expert receives the original task and available intermediate results; STOP finalizes the workflow, while a local review pass only releases an artifact to continue.Source: §4.2 · pp. 4–5

4.3 Jev-Gated LLM Re-review

  • At review checkpoints, Jev passes acceptable artifacts onward and escalates others to an LLM reviewer for a verdict and, when needed, corrective feedback.The reviewer may accept an escalated artifact or request revision; its verdict controls the workflow branch, while feedback informs repair.Source: §4.3 · p. 5
  • When revision is requested, an assigned generative expert receives the artifact and review information, produces an updated artifact, and re-enters the configured review procedure.The review–revision loop may continue within configured limits, with each revision treated as a new artifact.Source: §4.3 · p. 5

4.4 Design Benefits

  • When most artifacts satisfy local criteria and false escalations are low, screening lets passing outputs skip generative re-review and avoids a substantial fraction of review calls.Generative experts retain responsibility for producing and refining artifacts, and the LLM remains available for escalated outputs.Source: §4.4 · p. 5

5 Experiment

Across four benchmarks and two backbones, Jev-LiteMAS reduces generative-LLM use relative to LAS while keeping accuracy differences small; ablations examine workflow, routing, and review choices. The overhead analysis finds Jev adds a small share of latency and cost.

5.1 Experimental Setup

  • The evaluation spans MATH, MBPP+, API-Bank, and IDRBench I3, compares five baselines, and measures accuracy, generative-LLM tokens, latency, and API calls.Main experiments use DeepSeek-V4-Flash and GPT-5 backbones, with Jev as the lightweight decision model; analyses also test injected errors, review judges, call budgets, and heterogeneous experts.Source: §5 · p. 5, §5.1 · p. 6

5.2 End-to-End Quality and Efficiency

  • 17.5–54.6% fewer total LLM tokens with DeepSeek-V4-Flash and 12.7–38.6% fewer with GPT-5 than LAS, with accuracy differences of -0.67 to +0.66 and -1.34 to +0.67 percentage points, respectively.Latency falls on all four DeepSeek benchmarks and three GPT-5 benchmarks; GPT-5 MATH is the exception (30.04 s versus LAS’s 25.91 s).Source: §5.2 · pp. 6–7, Table 1 · p. 6

5.3 Workflow and Routing Ablations

  • Jev-LiteMAS reaches 75.33% accuracy on IDRBench I3 and 66.29% on API-Bank; removing semantic feedback lowers those scores to 57.33% and 57.30%, with nearly unchanged LLM call counts.Jev Router also exceeds LLM Router and SLM Router in accuracy on both tasks while using fewer generative-LLM tokens; Rule Router uses fewer tokens but substantially degrades accuracy.Source: §5.3 · p. 7, Table 2 · p. 7

5.4 Jev-Review and Screening Performance

  • Under injected errors, Jev-LiteMAS reaches 94.00% and 84.67% accuracy on MATH and MBPP+, with attack success rates of 3.33% and 4.00%.Jev screening TPR is 95.33% on MATH and 91.33% on MBPP+, close to LLM judging; under corruption, the cascaded configuration uses 14.6–21.7% fewer generative-LLM tokens.Source: §5.4 · pp. 7–8, Table 3 · p. 7, Table 4 · p. 8

5.5 Decision-Layer Overhead

  • Jev adds 1.13–2.21 seconds per evaluation unit, accounting for 2.92–6.85% of end-to-end latency and 1.56–2.14% of model cost across four GPT-5 benchmarks.Overall latency ranges from 30.04 to 75.57 seconds, and model cost ranges from $0.00664 to $0.01766 per evaluation unit.Source: §5.5 · p. 8, Figure 3 · p. 8

5.6 Additional Analyses.

  • Additional analyses examine quality–resource trends under different generative-LLM call limits and heterogeneous expert backbones, with detailed results reported in Appendix C.Source: §5.6 · p. 8

6 Conclusion

Jev-LiteMAS combines dynamic, feedback-driven expert handoffs with Jev-gated LLM re-review. Across four benchmarks and two backbones, it reduces generative-LLM token use while keeping accuracy close to LAS and lowering latency in most configurations.

  • 12.7–54.6% fewer generative-LLM tokens than LAS, with accuracy differences within 1.34 percentage points and lower latency in seven of eight configurations.Results span four benchmarks and two generative backbones.Source: §6 · p. 8
  • Feedback-driven handoffs route experts using prior outputs, while Jev-gated re-review reserves generative feedback for artifacts needing further examination or revision.Source: §6 · p. 8
  • Ablations highlight the importance of preserving substantive execution feedback alongside lightweight decisions.Source: §6 · p. 8

A Reproducibility Statement

The paper reports experimental configurations, dataset sampling procedures, and evaluation protocols in the main text and appendices. Full implementation details and prompt templates are available through an anonymous repository.

B Limitations and Future Work

Jev-LiteMAS routes only among predefined expert roles and legal actions, and its review savings depend on the quality distribution of intermediate artifacts. The authors identify more adaptive collaboration structures and review policies as future directions.

  • Feedback-driven routing uses a predefined pool of expert roles and legal actions, so it does not create experts or reconfigure roles for unseen task requirements.The authors identify more adaptive collaboration structures as future work.Source: B Limitations and Future Work · p. 11
  • Token and latency savings from Jev-gated re-review may decrease when more intermediate artifacts require generative review or revision.The authors suggest adaptive review policies that account for task difficulty and artifact quality.Source: B Limitations and Future Work · p. 11

C Additional Experimental Analyses

The additional analyses examine call limits and heterogeneous expert assignment; the call-budget study finds quality gains beyond the tightest limits, alongside additional resource costs as budgets grow.

C Additional Experimental Analyses

C.1 Sensitivity to Generative-LLM Call Budgets

  • Accuracy on I3 rises from 46.67% at a budget of two calls to 70.00% at four and 73.33% at six.Average calls rise from 2.00 at budget two to 2.64 at budget six, so the larger allowance is not exhausted on every task.Source: §C.1 · p. 11
  • On API-Bank, accuracy rises from 56.18% at two calls to 64.72% at four and 65.84% at six.Average calls increase from 2.00 to 4.46 between budgets two and six.Source: §C.1 · p. 11
  • Larger call budgets create opportunities for further expert contributions but also add token and latency costs.API-Bank token usage and latency fluctuate across intermediate budgets as executed trajectories change; the strongest quality gains occur beyond the tightest limits.Source: §C.1 · p. 11

C.2 Heterogeneous Expert Backbones

Heterogeneous expert backbones produce task-dependent accuracy and resource trade-offs, while cascaded review conditionally escalates artifacts for generative re-review and repair. Its savings and reliability depend on screening behavior and the prevalence of artifacts requiring correction.

C.2 Heterogeneous Expert Backbones

  • On IDRBench I3, accuracy rises from 75.33% with AAA to 78.00% with ABA as token use increases from 10.233K to 14.536K.The intermediate ABB configuration reaches 76.67% accuracy and 11.387K tokens.Source: Table 5 · p. 11, §C.2 · p. 11
  • On API-Bank, accuracy is 66.29%, 66.07%, and 64.49% for AAA, ABB, and ABA, while token use rises from 9.869K to 13.518K.Average calls also rise from 4.00 to 4.39; the results show task-dependent trade-offs when switching backbones.Source: Table 5 · p. 11, §C.2 · p. 11

D.1 Escalation and Re-entry

  • At review checkpoints, Jev can pass an artifact to ordinary routing or escalate it for LLM review; requested revisions return to an assigned expert and the next screening checkpoint.LLM acceptance resumes routing, while STOP—not local acceptance—initiates task finalization.Source: §D.1 · p. 11

D.2 Escalation, Savings, and Reliability

  • When acceptable artifacts are common and false escalations are low, most can bypass generative review; review-stage savings require Jev screening cost below the avoided LLM-review cost.Full-workflow accounting also includes routing, expert generation, revisions, and repeated checks.Source: §D.2 · p. 12
  • Erroneous releases bypass LLM re-review, and controlled error-injection class ratios do not estimate the natural workflow prevalence of artifacts requiring correction.Detection and false-escalation rates therefore need assessment alongside final task quality and resource use.Source: §D.2 · p. 12

D.3 Experimental Protocol

The experimental protocol uses fixed benchmark subsets, task-native evaluation criteria, and separate accounting for task quality and workflow resources. The workflow pseudocode combines expert generation and feedback-driven handoffs with Jev-gated review, repair, and budget handling.

D.3 Experimental Protocol

  • Four benchmarks use identical fixed subsets of 150 instances each, except API-Bank, which uses 150 episodes totaling 445 API-call prediction points.Sampling uses seed 42, with stratified sampling for MATH, MBPP+, and IDRBench I3.Source: §D.3 · p. 12
  • Final outputs are evaluated with benchmark-native criteria, including mathematical equivalence, MBPP+ pass@1, strict API-Bank accuracy, and IDRBench I3 label accuracy.Invalid IDRBench I3 outputs count as incorrect.Source: §D.3 · p. 12
  • Resource accounting averages generative-LLM input, output, and total tokens, model costs, end-to-end latency, and API calls across instances, with API-Bank resources aggregated per episode.This separates task accuracy from resources consumed by the complete workflow.Source: §D.3 · p. 12

E Workflow Pseudocode

  • The workflow generates expert contributions, screens review-point artifacts with Jev, conditionally invokes LLM review and expert repair, then routes to another expert or finalizes on STOP.Execution is bounded by the workflow budget, with a separate handler for budget exhaustion.Source: E Workflow Pseudocode · p. 12

Related papers

Loading 2610.21411v1…