PaperlayerView exact revision on arXiv

Source-linked AI summary

LabourCrew: A Multi-Agent RAG Framework for Trustworthy Adversarial Deliberation and Statutory Reasoning over Labour Law

Fatema Tuj Johora Faria, Mukaffi Bin Moin, Jubayer Al Mahmud, M. F. Mridha, Md. Alam Hossain

arXiv:2609.27814v1cs.CLcs.AIcs.IR

TL;DR

Existing systems do not unify statute-aware retrieval, protocol-enforced grounding, fault-tolerant reasoning, and calibrated release, a gap in a domain where unverifiable labour-rights answers carry real costs. LabourCrew combines a structure-preserving statutory graph, ledger-restricted adversarial deliberation, fault-tolerant supervision, and conformally calibrated release decisions. The evaluation examines whether the calibrated Trust Gate outperforms an uncalibrated categorical decision and whether its guarantee holds empirically.

  • Problem

    Existing systems do not unify statute-aware retrieval, protocol-enforced grounding, fault-tolerant reasoning, and calibrated release, a gap in a domain where unverifiable labour-rights answers carry real costs.

    Source: §2.4 · p. 3, §7 · p. 8

  • Method

    LabourCrew combines a structure-preserving statutory graph, ledger-restricted adversarial deliberation, fault-tolerant supervision, and conformally calibrated release decisions.

    Source: §3.1 · p. 4, §3.6 · p. 6, §7 · p. 8

  • Results

    The evaluation examines whether the calibrated Trust Gate outperforms an uncalibrated categorical decision and whether its guarantee holds empirically.

    Source: §1 · p. 2, §6 · p. 6

  • Takeaways & Limitations

    The authors argue that statutory faithfulness should be an architectural constraint rather than left to prompting conventions.

    Source: §7 · p. 8

    The work is scoped to one labour statute in one low-resource jurisdiction, and its retrieval graph does not link cross-chapter references.

    Source: Limitations · pp. 8–9

Abstract

from arXiv · show

In statutory question answering, every claim must be traceable to evidence, not merely relevant, since unverifiable labour-rights answers carry serious legal consequences. Current systems fall short: single-pass RAG cannot detect insufficient evidence, while multi-agent legal-debate systems treat grounding as a prompting convention, letting agents cite unretrieved evidence. To address this gap, we introduce LabourCrew, a multi-agent RAG framework built around three grounding mechanisms: StatuteGraph, a graph index that explicitly links chapter, section, proviso, and cross-reference structure rather than fixed-length spans; an Evidence Exchange Protocol that confines advocates and an interpreter to an evidence ledger, making citation to unretrieved text impossible, while a fault-tolerant supervisor board runs advocates in parallel so individual failures degrade rather than crash the system; and a Calibrated Trust Gate that replaces categorical accept/reject decisions with a trust score, thresholded via conformal risk control for a distribution-free bound on the false-accept rate. We evaluate on LabourActQA, a 500-item Bangla question set from the Bangladesh Labour Act, 2006, spanning seven reasoning categories and three difficulty tiers. The framework drives the empirical false-accept rate to 0.081, within the target level ($α= 0.10$), achieves the highest Answer Relevancy among HyDE RAG, Graph-RAG, and Hierarchical RAG (0.862 $>$ 0.839, 0.815, 0.828), and degrades gradually rather than catastrophically as question difficulty increases. These results show that calibrated abstention, not retrieval quality alone, is what makes legal question answering auditable in low-resource statutory domains.

1 Introduction

LabourCrew addresses statutory QA’s grounding and reliability gaps with structure-aware retrieval, evidence-constrained deliberation, fault isolation, and calibrated release. It is evaluated on LabourActQA, a 500-item Bangladesh Labour Act question set spanning seven categories and three difficulty tiers.

1 Introduction

  • Legal RAG audits report hallucination rates of 17–33%, while fixed-size chunking can separate statutory sections from qualifying provisos (Magesh et al., 2024).Surface correctness also does not establish that a claim was derived from its citation.Source: §1 · p. 1
  • Prior multi-agent legal systems improve reasoning, but leave evidence access to instructions and do not recover from agent failure.Related work also leaves structure-aware retrieval, protocol-enforced grounding, fault isolation, and calibrated release uncombined; statistical calibration has not been applied to audited deliberation in law.Source: §1 · p. 2
  • StatuteGraph links chapters, sections, provisos, and cross-references, while an interleaved retrieval loop plans queries and expands along statutory links.The Issue Spotter, Retrieval Planner, Link Hopper, and Statute Retriever implement that loop (Yao et al., 2023).Source: §1 · p. 2
  • The Evidence Exchange Protocol limits advocates and the interpreter to ledger evidence, while parallel supervision isolates role failures and citation checks precede release.A Trust Auditor monitors missing evidence and contradictions; the conformally calibrated Trust Gate releases a grounded opinion or marks it degraded or undecided with a distribution-free error guarantee.Source: §1 · p. 2
  • LabourActQA contains 500 Bangladesh Labour Act questions across seven reasoning categories and three difficulty tiers.The benchmark addresses a domain under-served relative to prior legal NLP’s high-resource corpora.Source: §1 · p. 2
  • The five research questions test structure-aware retrieval, evidence restrictions, fault-tolerant orchestration, calibrated trust decisions, and graceful performance under difficulty.They compare retrieval grounding and efficiency, unsupported claims, failed-run recovery, empirical calibration guarantees, and reliability on the hardest categories.Source: §1 · p. 2

2 Related Work

Prior work identifies unresolved limits in legal RAG faithfulness, multi-agent grounding, and structured retrieval and evaluation. LabourCrew addresses their shared gap by combining statute-aware retrieval, enforced evidence grounding, fault-isolated deliberation, and calibrated release decisions for Bangladesh Labour Act questions.

2.1 Faithfulness and Trust Calibration in Retrieval-Augmented Legal Generation

2.2 Multi-Agent Legal Reasoning and Debate

  • Role-differentiated legal agents improve reasoning quality, but grounding remains a prompting convention and prior systems report no failure-recovery mechanism.Examples include judge, prosecutor, and lawyer court simulation (He et al., 2024), evolvable adversarial lawyer roles (Chen et al., 2025), and multiagent legal-theory collaboration for charge prediction (Yuan et al., 2024); these systems target case-law prediction in high-resource jurisdictions rather than statutory interpretation.Source: §2.2 · p. 3

2.3 Structured Retrieval and Benchmarking for Legal Multi-Agent Systems

  • Structured legal-agent retrieval and benchmarks address specialization and verification, but existing approaches leave open-form statutory QA and calibrated, fault-isolated release decisions unresolved.MASLegalBench uses GDPR compliance questions and multiple-choice evaluation (Jing et al., 2025), while LegalGraphRAG uses sequential graph routing (Chen et al., 2026) and L-MARS verifies open-web citations without a calibrated release gate (Yuan and Wang, 2026); a survey identifies unresolved hallucination, verifiability, and evaluation challenges (Liu et al., 2026).Source: §2.3 · p. 3

2.4 Research Gap and Positioning of the Proposed Work

  • No existing system unifies statute-aware retrieval, protocol-enforced grounding, fault-isolated multi-agent reasoning, and statistically calibrated release decisions for low-resource statutory law.LabourCrew is positioned at this intersection for the Bangladesh Labour Act.Source: §2.4 · p. 3

3 LabourCrew Framework

LabourCrew aims to produce statutory opinions whose claims are traceable to existing text, using statute-aware indexing, evidence-confined deliberation, trust auditing, and gated release. Its agents coordinate through a shared ledger, with explicit release states for trustworthy, degraded, or undecided answers.

3.1 Problem Definition

  • The system targets a grounded legal opinion with explicit release status, coordinating eleven specialized agents through a shared Formal Case Ledger rather than a linear pipeline.It assumes the statute graph is complete and correctly versioned at query time, and that agents cannot introduce evidence absent from the graph.Source: §3.1 · pp. 3–4

3.2 Phase I: Statute-Aware Indexing (StatuteGraph)

  • StatuteGraph builds a version-tagged index with typed links among statutory units, preserving provisos and cross-references that fixed-length chunking can separate from governing rules.Its deterministic parser follows statutory numbering, gives provisos their own chunks, and resolves cross-references to every named section.Source: §3.2 · p. 4

3.3 Phase II: Agentic Retrieval

  • Agentic retrieval combines semantic search with bounded graph traversal, recovering structurally linked evidence and recording each expanded node’s path in the ledger.The Trust Auditor can request further retrieval by specifying seed nodes, edge types, and a hop budget.Source: §3.3 · pp. 4–5

3.4 Phase III: Evidence-Gated Deliberation

  • Worker and Employer Counsel argue opposing readings of retrieved evidence while a Legal Interpreter states the governing rule; advocates cannot retrieve evidence outside the pack.The interpreter can run on one side’s claims if the other advocate fails, and agent failures are recorded without aborting the round.Source: §3.4 · p. 5

3.5 Phase IV: Trust Auditing and Fault-Tolerant Orchestration

  • The Trust Auditor scores each claim’s grounding, while a supervisor retries failed roles, requests retrieval when warranted, or proceeds to composition under a fixed round procedure.A claim with no evidence receives trust(c) = 0; failures are isolated per agent so a transient failure does not abort the round.Source: §3.5 · p. 5

3.6 Phase V: Composition and Calibrated Release Gate

  • Under calibrated mode, claims are admitted at a conformally selected threshold; citation checks then determine whether the opinion is released as TGLO, DEGRADED, or UNDECIDED.The gate fails closed if no threshold meets the target risk, and accepted citations are mechanically reverified against retrieved text.Source: §3.6 · p. 6

4 Dataset Construction

LabourActQA fills the gap in Bangla labour-law statutory reasoning benchmarks with 500 question-answering items built directly over the Bangladesh Labour Act, 2006, for evaluation.

4 Dataset Construction

  • No existing benchmark evaluates statutory reasoning over Bangla labour law.Prior legal benchmarks cover contract review, English-language legal understanding, general legal reasoning, or civil-code statute tasks in another jurisdiction, while Bangla NLP resources target non-legal tasks.Source: §4 · p. 6
  • LabourActQA is a 500-item question-answering resource built directly over the Bangladesh Labour Act, 2006.The resource is used exclusively for evaluation.Source: §4 · p. 6
  • The full construction protocol, annotation schema, and quality-control details are provided in Appendix A.Source: §4 · p. 6

5 Experimental Setup

LabourCrew is evaluated against end-to-end baselines and architectural ablations using human, RAGAS, and deterministic agent-internal metrics.

5 Experimental Setup

  • The evaluation compares LabourCrew with two end-to-end baselines and four ablation families using human, RAGAS, and deterministic agent-internal metrics.Full model, retrieval, calibration, and evaluation configuration is provided in Appendix B.Source: §5 · p. 6

6 Results and Discussion

Results across the five research questions show gains from structure-aware retrieval, evidence-ledger restriction, supervised fault tolerance, and calibrated release, while grounding quality remains nearly flat as question difficulty rises.

6.1 RQ1: Structure-Preserving Retrieval and Indexing

  • Adding structural signals and graph linkhopping raises retrieval precision monotonically; structure-aware chunking improves faithfulness, while query replanning raises precision and reduces retrieval rounds.The chunker leads tested indexing alternatives on internal faithfulness.Source: §6.1 · pp. 6–7
  • LabourCrew leads every RAGAS column against three external retrieval baselines on the 500-question evaluation.The comparison uses the same LabourActQA questions for the full framework and the retrieval baselines.Source: Table 1 · p. 7, §6.1 · p. 7

6.5 RQ5: Robustness Under Increasing Difficulty

  • Across seven reasoning categories, gate-verified citation precision and decisive-opinion rate stay nearly flat as difficulty rises, while Citation-Based Precision declines on the hardest categories.Latency and retrieval rounds also rise with difficulty, indicating greater effort to find sufficient evidence rather than a relaxed citation standard.Source: Table 2 · p. 7, §6.5 · p. 8

6.2 RQ2: Evidence-Access Restriction

  • Restricting advocates to the Retrieval Planner’s shared evidence ledger sharply reduces unsupported claims while improving faithfulness, citation precision, and decisive-opinion rate.The compared configurations differ only in whether advocates can retrieve directly or must cite ledger evidence.Source: §6.2 · p. 7

6.3 RQ3: Fault-Tolerant Orchestration

  • Each added orchestration layer recovers more failed runs and raises decisive-opinion rate, while latency falls relative to the unsupervised sequential pipeline.The progression adds failure isolation, retry, and then the full fault-isolated supervisor.Source: §6.3 · p. 7

6.4 RQ4: Calibrated Release Gate

  • At the target risk level, calibrated trust-score thresholding keeps empirical false accepts within bound, unlike categorical LLM decisions, and improves citation precision and decisive-opinion rate.The threshold is selected using conformal risk control. Citation error falls as the Citation Checker and Trust Auditor are added and combined; the Trust Auditor and Link Hopper lead under different citation-precision metrics.Source: §6.4 · p. 8

7 Conclusion

LabourCrew makes faithfulness an architectural constraint for statutory question answering through structure-aware retrieval, evidence-bounded deliberation, fault isolation, and conformal calibration. The conclusion reports improved retrieval and faithfulness, calibrated false-accept control, and weaker reliability on multi-hop questions.

7 Conclusion

  • 0.081 ≤ α for the calibrated threshold, versus 0.147 > α uncalibrated; aggregate citation precision remains 0.991 as performance degrades gradually from easy to hard categories.Multi-hop citation precision is 0.412 versus 0.763 for direct lookup, reflecting the difficulty of verifying evidence across cross-referenced provisions.Source: §7 · p. 8
  • LabourCrew combines StatuteGraph, an Evidence Exchange Protocol ledger, fault-isolated supervision, and a Trust Gate whose release threshold uses conformal risk control.The shared ledger makes citation to unretrieved text structurally impossible, while conformal risk control provides a distribution-free bound on the false-accept rate.Source: §7 · p. 8
  • Structure-aware retrieval raises initial retrieval precision from 0.52 to 0.78; EEP reduces unsupported claims from 6.4% to 1.2% and raises faithfulness from 0.77 to 0.82.Fault-isolated supervision reduces failed runs from 14.2% to 1.0%.Source: §7 · p. 8
  • LabourCrew releases LabourActQA, a 500-item Bangla evaluation resource, and identifies multi-hop reasoning and generalization beyond one labour code as future work.The conclusion notes that chained evidence across cross-referenced provisions is harder to verify than a single-rule lookup.Source: §7 · p. 8

Limitations

LabourCrew is evaluated in a narrow setting: evidence-gated statutory question answering over one Bangla labour statute in a low-resource jurisdiction. Dataset, methodological, and evaluation constraints limit what the results establish, while future extensions would test broader applicability.

Generalizability and future extensions

  • The current work defines a specific scope: evidence-gated statutory question answering over one labour statute in one low-resource jurisdiction.The framework’s structural assumptions suit codified texts with explicit hierarchy; applying it to case law or common-law reasoning would likely require a redesigned indexing scheme.Source: Limitations · pp. 8–9

Dataset scope

  • LabourActQA covers only the Bangladesh Labour Act, 2006, in Bangla, so performance on other statutes, domains, or languages remains unestablished.Applying the framework elsewhere would require re-indexing and re-annotation. Two researchers constructed the dataset and two reviewers verified it; the released schema lacks machine-readable gold citation spans, limiting fine-grained citation-grounding evaluation.Source: Limitations · pp. 8–9

Methodological constraints

  • LabourCrew assumes explicit statutory hierarchy and same-chapter cross-reference markers, leaving cross-chapter provisions unlinked and retrieval potentially under-connected.Retrieval breadth uses fixed per-question-type ceilings that can narrow, but not widen, a conservative plan; calibrated release depends on a representative held-out, faithfulness-labeled set, and eleven specialized agents add latency and cost through multiple calls and retries.Source: Limitations · p. 9

Evaluation scope

  • The human evaluation used two legal-domain evaluators on sampled outputs and reported mean and standard deviation without a formal inter-rater agreement statistic.The authors identify larger evaluator panels, full coverage, and an agreement statistic as a next step. Judge metrics can penalize multi-provision retrieval and legitimate narrative restatements; evaluation also excludes longitudinal, deployment-setting, and adversarial-stress-testing studies.Source: Limitations · p. 9

Generalizability and future extensions

  • Testing additional statutes, jurisdictions, and languages, alongside newer foundation models and learned retrieval budgets, would clarify the framework’s boundaries.A larger practitioner panel is also identified as future work.Source: Limitations · p. 9

Ethics Statement · Appendix · A LabourActQA Dataset

LabourCrew is a research prototype scoped to the Bangladesh Labour Act, 2006, intended to support rather than replace qualified legal judgment. Its safeguards reduce risks from unsupported claims, but its reliability, fairness, and appropriate use remain bounded by the evaluated scope and human-evaluation limits.

Ethics Statement

  • The system is a research prototype for Bangla-speaking workers, legal-aid practitioners, and NLP researchers, limited to evaluated chapters of one statute and not validated as legal advice.It is intended to support, not replace, qualified legal professionals, particularly in disputes with material employment or income consequences.Source: Ethics Statement · p. 9
  • The Act is the sole textual corpus, and LabourActQA items are drafted from and traceable to it; no case records, third-party commentary, or user-submitted content are used.The source statute is public government text; the corpus, index, and evaluation set contain no personally identifiable information.Source: Ethics Statement · pp. 9–10
  • Structural coverage is limited to one jurisdiction, and Bangla legal-register mismatch, OCR errors, and small annotator and evaluator pools may constrain fairness and reliability.The dataset was constructed by two researchers and independently verified by a labour-law expert and a graduate researcher; human evaluation used two evaluators rating a sampled subset.Source: Ethics Statement · p. 10
  • Evidence checks reduce but cannot eliminate inaccurate claims, while automation bias and overreliance outside validated scope remain foreseeable risks.Use in a real labour dispute without qualified professional review is outside the work’s validated scope.Source: Ethics Statement · p. 10
  • Potential benefits include more accessible labour-rights information and reusable low-resource legal evaluation methods, while uneven reliability and reduced human oversight are risks that scale with deployment.Neither the potential benefits nor risks are unique to this system.Source: Ethics Statement · p. 10
  • The Evidence Exchange Protocol, Citation Checker, calibrated Trust Gate, and three-way release status support evidence verification and abstention, while supervision, version-pinning, and human review add safeguards.Documentation also calls for spot-checking OCR against source PDFs before treating it as verified.Source: Ethics Statement · p. 10
  • Future work will expand evaluator and user panels, assess how workers and legal-aid practitioners interpret abstention, and test broader statutory coverage with renewed fairness and bias validation.The authors also identify inter-rater agreement reporting and clear disclosure that the system supports rather than substitutes for qualified counsel as open considerations.Source: Ethics Statement · pp. 10–11

A.1 Data Sources … B.3 Baselines

The appendix describes how LabourActQA is built and checked against the Bangladesh Labour Act, 2006, and specifies LabourCrew’s implementation, retrieval limits, and comparison baselines. It emphasizes traceable statutory units while noting that per-item gold citation spans are not yet released.

A.1 Data Sources

  • The source is the Bangladesh Labour Act, 2006, as amended; digitized statutory text is divided into citable units rather than arbitrary spans.No third-party commentary or case law is used.Source: §A.1 · p. 12

A.2 Dataset Construction Pipeline

  • LabourActQA is constructed from OCR’d statutory units, with joint item drafting and independent review before disagreements are resolved by reviewer consensus.The pipeline targets 500 items across seven reasoning categories; reviewers check each item directly against the statute and note genuine textual ambiguities rather than forcing resolution.Source: §A.2 · pp. 12–13

A.3 Annotation Schema

  • The dataset defines seven reasoning categories and three category-assigned difficulty tiers, from easy factual or definitional questions to hard multi-hop or hypothetical reasoning.Questions and reference answers are provided in Bangla and English, with Bangla primary for evaluation; per-item machine-readable gold citation spans are planned but not yet implemented.Source: §A.3 · pp. 13–14

A.4 Quality Control

  • Independent reviewers’ pre-resolution judgments yield Krippendorff’s α = 0.86.Construction also requires both researchers to agree on every item field before verification.Source: §A.4 · p. 14

A.5 Dataset Statistics

  • All LabourActQA items are Bangla text with parallel English translations, and the dataset has no non-textual modality.Full per-category counts are reported in Table 4 (Appendix A).Source: §A.5 · p. 14

B.1 Implementation Details

  • LabourCrew uses LangGraph orchestration, specified OCR and embedding components, and type-specific retrieval ceilings that scale with retrieval demands.The implementation uses Gemini vision OCR, local BAAI/bge-m3 embeddings, and Milvus Lite for StatuteGraph; ceilings range from one query and five dense hits for direct factual questions to four queries, three hops, and ten dense hits for multi-hop questions.Source: §B.1 · p. 14, §B.2 · p. 14

B.2 Retrieval Configuration

  • Retrieval plans are clamped to per-question-type ceilings that narrow over-wide plans but never widen narrow ones, identically across compared configurations unless targeted by an ablation.Direct factual and definitional questions receive one query/hop and five dense hits; multi-hop reasoning receives four queries, three hops, and ten dense hits.Source: §B.2 · p. 14

B.3 Baselines

  • The end-to-end baselines are single-pass RAG without deliberation, auditing, or a calibrated gate, and unconstrained multi-agent debate with direct retrieval access outside the shared evidence ledger.Both hold the chat model and corpus fixed; the debate baseline isolates removal of the Evidence Exchange Protocol rather than deliberation itself.Source: §B.3 · p. 14

B.4 Ablation Configurations · B.5 Trust Gate Calibration Protocol

B.4 defines controlled ablations across LabourCrew’s retrieval, evidence access, trust-gating, and orchestration components. B.5 describes how the trust gate is calibrated using held-out claims and conformal risk control.

B.4 Ablation Configurations

  • Retrieval ablations vary strategy, chunking, query planning, and Link Hopper depth while holding the rest of the pipeline fixed.The strategies include the full StatuteGraph retriever; chunking ranges from fixed windows to structure-preserving parsing, and Link Hopper depth ranges from zero to three.Source: §B.4 · p. 14, Table 5 · p. 15, Table 7 · p. 16
  • Advocates’ unrestricted evidence access is compared with the Evidence Exchange Protocol, which limits citations to node_ids already on the ledger.Under the protocol, advocates have no retrieval tool and cannot cite evidence outside the ledger.Source: §B.4 · p. 15
  • Trust-gate ablations remove the Citation Checker and Trust Auditor individually or jointly, test six reasoning/gating agents one at a time, and compare categorical decisions with calibrated scores at α = 0.10.The calibrated mode thresholds the trust score at τ.Source: §B.4 · pp. 15–16
  • Three worked runs illustrate correct conditional, comparative, and multi-hop answers grounded in the cited statute nodes.The examples concern Section 15 exemptions, juveniles’ working-hour limits under Sections 41(1)/(2), and hazardous-work restrictions under Sections 39(2) and 40.Source: Figure 5 · p. 16
  • Orchestration ablations compare no supervisor, a supervisor without retry, a supervisor with retries, and the full fault-tolerant supervisor.The four configurations are evaluated on all 500 questions; Table 9 defines Recovery Rate relative to the immediately preceding configuration’s failures.Source: §B.4 · p. 16, Table 9 · p. 17

B.5 Trust Gate Calibration Protocol

  • Calibration uses a held-out LabourActQA split: the board logs each candidate claim’s Trust Auditor score, and a legal-domain reviewer labels faithfulness.The reviewer follows the same protocol as the dataset’s reference answers.Source: §B.5 · p. 16
  • Conformal Risk Control calibrates the gate at α = 0.10; if no threshold satisfies the bound, the gate defaults to reject-all with τ = ∞.The categorical mode requires no calibration set because an LLM call makes its decision directly.Source: §B.5 · p. 16

B.6 Evaluation Protocol

The evaluation combines human ratings, RAGAS judge-model metrics, and deterministic agent-level metrics. It specifies protocols and metric scopes across all 500 LabourActQA questions, with category-level analysis from easiest to hardest.

B.6 Evaluation Protocol

  • The evaluation combines human evaluation, RAGAS judge-model metrics, and deterministic agent-level metrics, reported in Tables 3, 1, and 2, respectively.Table 10 provides the aggregate deterministic evaluation across all 500 questions, and Table 2 breaks it down by category.Source: §B.6 · p. 17, Table 10 · p. 17
  • Two independent evaluators rate each sampled opinion on six five-point Likert dimensions, with Citation Quality highest at 4.79±0.42 and Completeness lowest at 4.52 ± 0.61.Evaluators have labour-law expertise or graduate-researcher qualifications; all six dimension means exceed 4.5. Ratings follow current human-evaluation best practices (van der Lee et al., 2019).Source: §B.6 · p. 17, Figure 8 · p. 20
  • Five RAGAS metrics cover all 500 questions, the full system, six leave-one-out agent ablations, and Hierarchical RAG, Graph-RAG, and HyDE RAG baselines.GPT-4.1 judges all configurations using BAAI/bge-m3 embeddings; context recall, answer correctness, and noise sensitivity are primary signals, while context entity recall is exploratory.Source: §B.6 · pp. 17–18
  • Execution-time ceilings are enforced per question type for Retrieval Planner dense-hit count, Link Hopper hop depth, and query count.The supplied table caption specifies these deterministic ceilings but does not provide their values.Source: Table 11 · p. 18
  • The evaluation includes six-agent leave-one-out ablations, four Trust Gate configurations, unrestricted-versus-role-isolated Evidence Exchange, and release-mode comparisons at α = 0.10.The supplied captions identify the conditions and metrics for these evaluations but do not provide their result values.Source: Table 12 · p. 18, Table 13 · p. 19, Table 14 · p. 19
  • Deterministic trace metrics include latency, retrieval/deliberation rounds, evidence coverage, path completeness, Gate-Verified Citation Precision, TGLO Rate, and Citation-Based Precision.Figure 7 reports nearly flat Gate-Verified Citation Precision and TGLO Rate across difficulty, while Citation-Based Precision drops sharply on the hardest categories; category N-weighted averages reproduce the aggregate row.Source: §B.6 · p. 18, Figure 7 · p. 19

C Detailed Analysis of Research Questions … D Prompt Specifications for the LabourCrew Agents

Across RQ1–RQ5, LabourCrew combines structured retrieval, evidence-restricted advocacy, fault-tolerant orchestration, and calibrated release decisions to support verifiable answers under increasing difficulty. The agent prompts operationalize this design by routing searches from case intake and limiting advocates to supplied evidence.

RQ1: Structure-Preserving Retrieval and Indexing

  • StatuteGraph’s structural links and adaptive planning close retrieval coverage gaps, while whole-chapter chunks undermine citation faithfulness.The full configuration reaches 24.3s latency versus 25.8s for dense-plus-link-hops alone; graph-aware retrieval and structure-preserving indexing jointly support grounding gains.Source: RQ1: Structure-Preserving Retrieval… · p. 21

RQ2: Evidence-Access Restriction

  • The Evidence Exchange Protocol reduces unsupported claims from 6.4% to 1.2%, while decisive opinions rise from 97.8% to 99.0%.Its tool-level access restriction confines advocates to the shared evidence ledger, addressing unverifiable claims even when advocates otherwise reach grounded opinions.Source: RQ2: Evidence-Access Restriction · p. 21

RQ3: Fault-Tolerant Orchestration

  • Supervision and retries recover distinct classes of failures, and the full supervisor adds a smaller further gain; latency decreases as failure rates fall.A residual failure rate of 1.0% remains, with unrecoverable cases surfaced through a degraded or abort status rather than hidden.Source: RQ3: Fault-Tolerant Orchestration · p. 22

RQ4: Calibrated Release Gate

  • 0.081 is within the target bound, while the categorical baseline’s 0.147 exceeds it; conformal calibration gives the release gate an empirically confirmed error bound.The Citation Checker verifies citation existence and verbatim quotation, while the Trust Auditor checks evidentiary completeness and contradiction; together they outperform either check alone.Source: RQ4: Calibrated Release Gate · p. 22

RQ5: Robustness Under Increasing Difficulty

  • As question difficulty rises, retrieval efficiency declines while accepted-answer grounding and release integrity remain nearly stable.Harder questions require wider evidence packs, more retrieval rounds, and more discarded evidence; the system raises the cost of reaching a verifiable answer rather than lowering its verification standard.Source: RQ5: Robustness Under Increasing… · p. 23

D Prompt Specifications for the LabourCrew Agents

  • The Issue Spotter structures parties, facts, issues, legal concepts, missing facts, and question type without answering the legal question.Those case seeds route subsequent retrieval and advocacy; the planner chooses semantic queries, link types, hop depth, and hit counts according to question type and difficulty.Source: D Prompt Specifications for the… · pp. 24–25
  • Worker and Employer Counsel argue independently from the supplied evidence pack, grounding each claim in a cited node and verbatim span and returning no claim when support is absent.The planner expands provisos for asserted duties or rights and cross-references when concepts suggest another operative section; advocates do not retrieve independently.Source: D Prompt Specifications for the… · pp. 26–27

Related papers

Loading 2609.27814v1…