Loop-Back Authority in LLM Agent Teams: A Paired Experiment on Flat and Hierarchical Coordination

1 Introduction

• The introduction contrasts organizational theory’s prediction that authority improves quality with evidence that authoritative prompts and iterative critique can induce sycophancy and reduced divergent reasoning.The paper also distinguishes its question from research on AI managers coordinating human workers.Source: §1 · p. 1
• Prior LLM-team comparisons give conflicting results and often change whole frameworks or use verifiable short-answer benchmarks rather than open-ended synthesis.The study addresses these limits with a controlled test of business-intelligence reporting, where strategic content lacks an answer key.Source: §1 · p. 1
• The study isolates one authority link across otherwise identical teams and pairs 43 products, using a five-model judge panel and a deterministic specification check.Roles, prompts, tools, models, and data are held fixed; the link lets the Manager require revision in the hierarchical form but only comment in the flat form.Source: §1 · p. 1
• Flat teams scored higher on Utility (d = 0.42, p = 0.009) and Writing Clarity (d = 0.34, p = 0.030), while hierarchical reports used 53% more hedging words.The reports were the same length; each Manager-issued loop was associated with a 0.14-point drop in Writing Clarity, and first drafts were indistinguishable from flat reports on text measures.Source: §1 · pp. 1–2
• Loop-back authority costs 51.5% more tokens without improving a quality metric, and the quality loss tracks revision loops rather than authority alone.The authors present a paired experiment that isolates the link while holding roles, prompts, tools, models, and data fixed.Source: §1 · p. 2

2.1 Organizational Design in Multi-Agent Systems

• Classical multi-agent research treats organizational design as consequential, while this study adopts its distinction between structural links and role obligations to isolate one authority link.The paper’s two forms share roles and communication links but differ in whether Manager input is merely permitted commentary or an obligation on the worker.Source: §2.1 · p. 2

2.2 Coordination and Influence Structures

• Kocak, Levinthal, and Puranam’s framework predicts that hierarchy can improve similarity and resolution but risks entrapment; the paper treats influence asymmetry as authority to force revision.The paper notes that its sequential shared-state teams differ from the framework’s simultaneous, fully connected flat teams.Source: §2.2 · p. 2

2.3 LLM Agent Teams

• LLM-team frameworks and surveys often use hierarchical orchestration, but prior work varies whole systems rather than isolating one authority link.Research on critique also reports authoritative compliance and reduced divergent thought under iterative feedback; both forms here retain tool-grounded critique, making authority the manipulation.Source: §2.3 · p. 2

2.4 Competing Predictions

• Classical theory predicts loop-back authority raises Writing Clarity, whereas LLM-specific accounts predict lower clarity through compliance; both predict flat teams will score higher on Utility.The organizational forms hold roles, prompts, tools, models, data, and the blackboard constant while differing in the Manager-worker authority link.Source: §2.4 · p. 2, Figure 1 · p. 3

3.1 Agents, Roles, and Communication Topology

• Both forms use the same five roles, shared blackboard, and worker tools; roles are randomly assigned from a five-model pool, with each product’s assignment held constant across forms.The Critic’s tool-grounded review is present in both conditions, so the treatment is the Manager’s coordination policy.Source: §3.1 · p. 3

3.2 The Treatment: Loop-Back Authority

• In the hierarchical form, the Manager can issue up to two directives requiring a worker to revise; in the flat form, it routes work and may comment but cannot require revision.The directive is an obligation, but the link carries no sanction; workers face no penalty for ignoring it.Source: §3.2 · p. 3

3.3 Task and Data

• The task is business-intelligence reporting on gaming laptops with RTX 4060 GPUs; 43 products with at least five reviews were run under both forms, yielding 86 runs.Reviews were stratified by star rating to avoid retrieval skew toward positive feedback.Source: §3.3 · pp. 3–4

3.4 Evaluation and Metrics

• A five-model judge panel scores Writing Clarity and Utility, while a deterministic script checks asserted product specifications against listing metadata.Writing Clarity and Utility each average three dimensions; the authors describe their exploitation/exploration mapping as an interpretive assumption, not a validated correspondence.Source: §3.4 · p. 4

3.5 Analysis

• Paired-sample t-tests compare the 43 product pairs, with a mixed-effects model checking form effects across 86 runs; loop-count analyses are exploratory and correlational.The model controls for writer model, number of reviews, and mean rating, with product as a random intercept; Table 2 reports the paired and mixed-effects comparisons.Source: §3.5 · p. 4

4.1 Overall Comparison

• Writing Clarity favored flat teams (d=0.34, p=0.030), as did Utility (d=0.42, p=0.009); specification accuracy did not differ.The flat form won on Final Score in 28 of 43 products; both primary tests survived Holm correction, while Writing Clarity was borderline under the Wilcoxon alternative (p=0.051).Source: §4.1 · p. 4

4.2 Where the Gap Comes From

• Flat reports scored 4.18 versus 3.98 on Conciseness (d=0.44, p=0.007); hierarchical reports used 53% more hedges per 1,000 words (5.03 versus 3.30, p<0.001).Structure and Coherence did not differ; reports had similar word counts, and lexical diversity was identical between forms.Source: §4.2 · pp. 4–5, Figure 2 · p. 5

4.3 Specification Accuracy: No Difference, and a Ceiling

• On validated specification checks, mean recall and precision were 1.000 for flat reports and 0.997 for hierarchical reports; 85 of 86 reports restated every specification correctly.The authors attribute variance in the original accuracy measure to parser noise; both forms were at ceiling on the repaired metric.Source: §4.3 · p. 5

4.4 Cost

• Hierarchical reports used 51.5% more tokens (74,781 versus 49,370), cost 20.2% more overall, and ran 34.3% slower, without a gain on any quality metric.Estimated generation cost was 40.5% higher.Source: §4.4 · p. 5

4.5 Is the Harm from Hierarchy Itself, or from its Revision Loops?

• Within hierarchical runs, each Manager-issued loop was associated with a 0.142-point drop in Writing Clarity (p<0.001), but loop assignment makes this relationship correlational.In the 11 zero-loop pairs, hierarchical reports were numerically higher on all three metrics, with neither primary comparison significant.Source: Figure 3 · p. 5, §4.5 · p. 6
• Before intervention, hierarchical first drafts were indistinguishable from flat reports; revised drafts added hedges and words while largely retaining their predecessors.Among 31 revised runs, second drafts added 1.95 hedges per 1,000 words and 38 words on average; the authors interpret revisions as softening and padding rather than rewriting.Source: §4.5 · p. 6

4.6 Judge Reliability and Robustness

• Utility was robust across judge omissions, but Writing Clarity was judge-sensitive; Conciseness remained significant under every omission.Absolute agreement among judges was low, though each judge agreed directionally with the panel mean on at least 68% of pairs for Writing Clarity and 77% for Utility.Source: §4.6 · p. 6
• In a blinded human check of 20 products, flat reports won 54% of nontie overall choices, but the small sample and rater disagreement could not confirm the direction.The two raters had modest agreement with each other (Cohen’s κ=0.30 for overall preference).Source: §4.6 · p. 6

5 Discussion and Conclusion

• The paired experiment found higher Utility and Writing Clarity for flat teams, with specification accuracy at ceiling in both forms.The hierarchical form cost half again as many tokens; its first draft was indistinguishable from the flat report, and runs without an exercised loop showed no measurable deficit.Source: §5 · p. 7
• The authors recommend loop-back authority when a Manager can verify work, and verifier-triggered rather than Manager-judged revision when it cannot.For short-context synthesis, they recommend a flat sequential pipeline by default and advise that a Manager used for routing should not hold loop-back authority.Source: §5 · p. 7
• The findings contrast with evidence that AI managers improve human-team performance, suggesting the observed failure may be specific to revision between RLHF-trained models.The authors frame the coordination-theory divergence around a difference in how human writers and language models adapt to evaluative audiences.Source: §5 · p. 7
• The design rule is bounded to one short-context product-reporting task, one generation of five models, one Manager prompt, and 43 pairs.The authors do not extend it to long-context generation, code, or tasks with richer verifiers; factuality of analytical claims was not measured.Source: §5 · p. 7
• Future tests include manipulating verifier-triggered versus Manager-judged revision, adding worker sanctions or persistent stakes, and measuring deference in model activations.The authors also note that the hedge lexicon cannot distinguish calibrated uncertainty from evasion, so interpreting hedging as harmful relies on quality scores.Source: §5 · p. 7

• The listed authors are Burak Agachan, Max van Duijn, and Amirhossein Zohrehvand, affiliated with Leiden University’s LIACS.Source: Supplementary Material for: Loop-Back… · p. 9
• The supplement collects robustness checks, judge concordance figures, regression output, construct-validity evidence, and the verbatim Manager prompt.Source: Supplementary Material for: Loop-Back… · p. 9
• The listed keywords cover multi-agent systems, organizational coordination and authority, LLM-based agents, and LLM-as-a-judge evaluation.Source: Supplementary Material for: Loop-Back… · p. 9

1.1 Amazon review and product-specification data

• Product specifications and reviews were collected from publicly visible pages via third-party services in January 2026, without account data or non-public metadata.Review author display names were not analyzed or retained; raw scraped material is not redistributed, partly because collection may conflict with Amazon’s terms for commercial purposes.Source: §1.1 · p. 9, §1.3 · p. 10

1.2 AI-use disclosure

• AI assistants helped debug code, edit prose, and search literature; the authors reviewed outputs and take responsibility for the content.Source: §1.2 · p. 9

1.3 Data and code availability

• The code and analysis repository is public, while the analytic dataset and per-run evaluation outputs are available from the authors on request.Raw scraped material is not redistributed.Source: §1.3 · pp. 9–10

A Robustness Checks: Wilcoxon and Holm-Bonferroni Correction

• Both primary paired-t tests survive Holm correction; under Wilcoxon, Utility survives while Writing Clarity is borderline (p=0.051).The primary-test table reports these correction and alternative-test checks.Source: Table 1 · p. 10, A Robustness Checks: Wilcoxon and… · p. 10
• Strategic Depth survives Holm correction under both base tests; Conciseness survives only under the t-test (Wilcoxon Holm p=0.084), and four dimensions remain nonsignificant.The per-dimension table covers six follow-up dimensions.Source: Table 2 · p. 10, A Robustness Checks: Wilcoxon and… · p. 10

B Per-Judge Directional Concordance

• Every judge matches the panel-mean direction on at least 68% of Writing Clarity pairs and 77% of Utility pairs; three of five exceed 80% on Utility.The paired design evaluates within-product directional differences rather than absolute score calibration.Source: B Per-Judge Directional Concordance · p. 10

C Architectural Main-Effect Mixed-Effects Regression

• Controlling for writer model, review count, and mean rating, Hierarchical is associated with lower Writing Clarity and Utility but no architectural effect on Accuracy.The mixed-effects model uses all 86 runs and a product random intercept.Source: C Architectural Main-Effect… · p. 11

D Paired-Difference Inter-Judge Correlations

• Paired-difference correlations are uniformly positive across judges, with mean r=0.29 for both Writing Clarity and Utility.The correlations assess whether judges align on treatment differences across products.Source: D Paired-Difference Inter-Judge… · pp. 11–12

E Construct Validity: Inter-Dimension Correlations and Rubric Anchors

• The exploitation–exploration mapping is interpretive, and weak inter-dimension correlations mean a result on one dimension cannot stand in for the broader construct.The dimensions are weakly inter-correlated, so the composites average distinct signals.Source: E Construct Validity: Inter-Dimension… · pp. 12–13

F Hierarchical Manager Prompt (Verbatim)

• The hierarchical Manager reviews the full blackboard and can require revision, whereas the flat Manager is a peer that gives non-binding commentary in a forward-only workflow.The flat prompt removes loop-back authority, fixes the workflow to one pass, and limits review to the latest worker output.Source: F Hierarchical Manager Prompt (Verbatim) · p. 13

• The Researcher retrieves product metadata and reviews, while the Critic uses verification to detect hallucinations via similarity scoring.Source: G Implementation Details · p. 15
• Verification labels use thresholds of ≥0.7 for VERIFIED, 0.5–0.7 for PARTIAL, and <0.5 for UNVERIFIED.The cut-points were calibrated heuristically using representative claim–evidence pairs outside the analytic sample; labels are not numeric inputs to downstream metrics.Source: G Implementation Details · p. 15
• The overall score is the arithmetic mean of Writing Clarity, Utility, and Accuracy.Source: G Implementation Details · p. 15

• The composite Writing Clarity difference is driven entirely by Conciseness; Structure and Coherence do not differ significantly between architectures.Source: H Additional Results Tables · p. 16
• At the conventional small-effect bound, Structure equivalence is not established (TOST p=0.238); the lenient bound supports equivalence only up to a small-to-medium effect.A ceiling effect is present: 83.7% of Flat and 88.4% of Hierarchical reports score at least 4.5 on Structure.Source: H Additional Results Tables · pp. 15–16
• Hierarchical operation has 40% higher estimated generation cost, 20% higher estimated total cost, runs 34% slower, and uses 51% more tokens per report, without corresponding quality gains.Source: Figure 1 · p. 17
• The loop-count slope survives ORIGINAL, DROP, and RECODE treatments of three leverage-prone censored observations, but its magnitude approximately halves under DROP.The observations represent third revisions blocked by the MAX_LOOPS guard.Source: H Additional Results Tables · pp. 16–17
• In the loop_count=0 subset (N=11), Hierarchical is numerically higher than Flat on all three metrics, so all reported d values are negative under the paper’s sign convention.Source: H Additional Results Tables · p. 17, Table 15 · p. 17
• With 43 paired observations, the design has approximately 80% power for d=0.44 and 61% for d=0.35; per-dimension follow-ups are individually underpowered for small effects.Source: H Additional Results Tables · p. 18

I Report Length Control

• The 43 paired reports are not distinguishable in length: Flat averages 1036 ± 293 words, Hierarchical 1046 ± 278, with a −9-word difference (d=−0.047, p=0.760).Source: I Report Length Control · p. 18
• Report length correlates negatively with Writing Clarity (r=−0.241, p=0.026) and more strongly with Conciseness (r=−0.459, p<0.001); eight other metrics show no significant correlation.Source: I Report Length Control · p. 18
• Within the hierarchical arm, each additional Manager loop is associated with 105 more words (p=0.021).Source: I Report Length Control · p. 18
• The Conciseness rubric partly incorporates report length, so its length-control results should be read alongside length-independent dimensions.The length-control inference uses classical OLS standard errors on 43 paired differences; borderline p-values warrant caution.Source: I Report Length Control · p. 18

J Judge Sensitivity: Per-Judge and Leave-One-Judge-Out

• Per-judge tests at N=43 are individually underpowered, so signs and effect sizes are more informative than their p-values alone.Dimension-level judge entries are missing in 12 of 2,580 cells, leaving some per-judge rows with N=41 or 42.Source: J Judge Sensitivity: Per-Judge and… · pp. 19–20
• Dropping any one judge leaves the headline result significant or close to significant in the leave-one-judge-out checks.The checks cover Writing Clarity and Utility.Source: Table 22 · p. 20, Table 23 · p. 20, J Judge Sensitivity: Per-Judge and… · p. 21

• The repaired metric scores recall and precision over each product’s 5 to 7 non-null ground-truth specifications, rather than precision alone over asserted specifications.Recall is correct assertions divided by non-null ground-truth specifications; precision is correct assertions divided by asserted specifications.Source: K Repaired Specification-Accuracy Metric · p. 21
• Matching requires exact hardware variants and numeric specification matching; family-only assertions count as partial and are correct only when the report does not contradict ground truth.Price is matched within 1 percent, while storage uses marketing size classes.Source: K Repaired Specification-Accuracy Metric · p. 21
• On the repaired metric, 85 of 86 reports score recall 1.0, and the original score’s 3.0–5.0 variation occurs despite identical checkable specification content.The authors interpret that original-score variation as judge noise rather than a real difference in specification accuracy.Source: K Repaired Specification-Accuracy Metric · p. 21
• The repaired recall and precision metrics correlate weakly with the original accuracy score (r=0.151, p=0.1664, n=86), but their near-constant values make that correlation uninformative.The metrics coincide almost everywhere in this sample.Source: K Repaired Specification-Accuracy Metric · p. 21

Draft-Level Text Measures: Hedging, Lexical Diversity, and Revision Overlap

• Hedging density counts lexicon matches per 1,000 words; MTLD and length-insensitive MATTR measure lexical diversity, while overlap uses content-word Jaccard and longest-common-subsequence similarity.The hedge lexicon contains 51 single-word items and 18 multiword phrases; multiword matches are masked to prevent double-counting.Source: L Draft-Level Text Measures: Hedging,… · p. 23
• Loop count is not the number of Writer rewrites: it also counts Manager turns that may redelegate work to the Critic or Analyst.There are 77 hierarchical Writer drafts in total, with 31 runs having at least two drafts.Source: L Draft-Level Text Measures: Hedging,… · pp. 23–24
• Hierarchical reports hedge about 53% more per 1,000 words, while lexical-diversity measures show no consistent overall decline.MTLD is unchanged, while MATTR falls slightly but reliably; MTLD is the primary specified measure.Source: L Draft-Level Text Measures: Hedging,… · pp. 25, 27
• None of the 34 transitions is a substantial rewrite; revised drafts retain a median 88% of predecessor text and 85% of content-word vocabulary while adding 30 words.The pattern is marginal editing and appending, changing stance and length rather than substance.Source: L Draft-Level Text Measures: Hedging,… · p. 26
• The full −1.734 hedging gap opens between the hierarchical Writer’s first and last drafts; first drafts are indistinguishable from flat reports on measured text features.The first-draft comparison uses all 43 pairs.Source: L Draft-Level Text Measures: Hedging,… · p. 27, Table 42 · p. 27
• Across 31 revised runs, later drafts have roughly 56% more hedging per 1,000 words, run about 38 words longer, and retain roughly 88% of prior text.The between-run loop-count comparisons are vulnerable to selection into revision; within-run draft comparisons are the estimates to lead with.Source: L Draft-Level Text Measures: Hedging,… · pp. 27–28
• Hedge counts cannot distinguish calibrated uncertainty from evasion, so interpreting increased hedging as harm depends on the judge quality scores.Loop-count dose slopes are descriptive, not causal, because the Manager assigns loops; their small-sample estimates are fragile when adjusted for word count.Source: L Draft-Level Text Measures: Hedging,… · pp. 26–27