PaperlayerView exact revision on arXiv

Source-linked AI summary

LawCompass: Navigating from Legal QA to Multi-Agent Deep Research with Grounded Evidence

Xiaoxia Cheng, Linnan Wang, Jiahao Ma, Zhichuan Ye, Xuemei Zhou, Chuanyu Tong, Bo Jiang, Qing Zhu

arXiv:2610.01027v1cs.CL

TL;DR

Legal assistants often remain limited to conversational QA despite legal tasks that require systematic evidence retrieval, multi-step reasoning, and report-level synthesis. LawCompass combines evidence-grounded QA, structured retrieval, and multi-agent Deep Research; its evaluations and case studies support its practical use for legal research.

  • Problem

    Many legal assistants remain confined to conversational QA rather than supporting complex tasks requiring systematic retrieval, multi-step reasoning, and report-level synthesis.

    Source: §1 · pp. 1–2

  • Method

    LawCompass combines evidence-grounded Legal QA and Professional Retrieval with a multi-agent workflow that decomposes research objectives, gathers cross-source evidence, and synthesizes reports.

    Source: §1 · p. 2, §3.4 · p. 3

  • Results

    Retrieval evaluation, a user study, and representative case studies validate LawCompass’s effectiveness in supporting practical legal research tasks.

    Source: §5.1 · p. 6, §5.2 · p. 6, §6 · p. 7

  • Takeaways & Limitations

    LawCompass offers a practical interface for diverse legal information needs with traceable evidence.

    Source: §1 · p. 2

    Retrieval coverage depends on local-database completeness and update frequency, while multi-stage Deep Research adds latency and computational cost.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Recent advances in Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) have significantly democratized access to legal information. Nevertheless, most existing legal assistants remain confined to multi-turn conversational QA, failing to support complex legal tasks that require systematic evidence retrieval, multi-step reasoning, and report-level synthesis. In this paper, we present LawCompass, an evidence-grounded legal assistant that navigates the transition from standard Legal QA to multi-agent deep research. LawCompass provides three task-oriented functions: Legal QA, which delivers precise, evidence-backed answers to legal questions; Professional Retrieval, which enables structured exploration of statutes and judicial cases via query rewriting; and Deep Research, which employs a multi-agent workflow to decompose complex legal tasks and synthesize comprehensive research reports. Crucially, LawCompass maintains explicit citation links across all modules, empowering users to directly verify system outputs against original legal sources. Evaluation results demonstrate that LawCompass provides a practical and scalable paradigm for transforming conversational AI into trustworthy and evidence-grounded legal research assistance.

1 Introduction

Existing legal assistants often stop at conversational QA, while legal work can require systematic retrieval, multi-step reasoning, and report-level synthesis. LawCompass combines evidence-grounded QA, structured retrieval, and multi-agent deep research in one system.

1 Introduction

  • Figure 1 contrasts existing systems focused on multi-turn QA with LawCompass’s extension to multi-agent deep research.Source: Figure 1 · p. 1
  • Legal practice demands verifiable and trustworthy evidence in addition to fluent answers.Source: §1 · p. 1
  • Existing legal assistants are often confined to conversational QA and rarely support tasks requiring systematic evidence retrieval, multi-step reasoning, and report-level synthesis.Legal information seeking progresses from a direct query to examining legal authorities and conducting cross-source analysis.Source: §1 · pp. 1–2
  • LawCompass combines Legal QA, Professional Retrieval, and multi-agent Deep Research, with explicit legal evidence supporting outputs across the modules.Deep Research decomposes complex objectives, gathers and verifies evidence, and synthesizes comprehensive reports.Source: §1 · p. 2

2 Related Work

Prior legal assistants combine retrieval or multi-agent designs with conversational assistance, but remain oriented toward short-form QA or dialogue simulation. LawCompass instead orchestrates research workflows for complex legal tasks.

2 Related Work

  • Legal applications require accurate, interpretable outputs grounded in authoritative sources, and RAG has been adopted to mitigate hallucinations.Source: §2 · p. 2
  • Existing legal assistants remain predominantly confined to short-form conversational QA, while LawCompass uses an orchestrated multi-agent workflow for complex legal research.Source: §2 · p. 2
  • RAG-based legal assistants such as ELLA fetch relevant articles, while MASER uses multi-agent simulation of lawyer–client dialogues for evaluation and data generation.Source: §2 · p. 2

3 Framework and Usage Example

LawCompass provides three task modes through a unified interface, moving from concise legal answers to structured authority exploration and multi-agent research. Each mode supports a distinct level of task complexity and output.

3 Framework and Usage Example

  • The Unified Task Entry routes requests to Legal QA, Professional Retrieval, or Deep Research; the modes share document input, citation rendering, session management, and access control.Source: §3 · p. 2

3.1 Unified Task Entry

  • Users can submit natural-language queries, template questions, local documents, or query-document combinations as grounded context.Source: §3.1 · p. 2

3.2 Legal Question Answering

  • Legal QA retrieves statutes, cases, or document fragments and returns concise answers with clickable citations and conversational memory for routine consultations.Users can inspect source metadata and original text through the citation-linked source panel.Source: §3.2 · pp. 2–3

3.3 Professional Retrieval

  • Professional Retrieval rewrites incomplete or non-legal queries, searches heterogeneous legal sources, and presents retrieved authorities in a structured report for comparison and evidence collection.The report covers disputes, retrieved materials, detailed legal analysis, and retrieval conclusions with recommendations and risk alerts.Source: §3.3 · p. 3

3.4 Deep Research

  • Deep Research decomposes objectives into subtasks, dispatches specialized executors in parallel across heterogeneous sources, and synthesizes findings into a six-section legal research report.Intermediate subtask reports let users assess coverage of analytical dimensions.Source: §3.4 · p. 3

4 System Architecture and Implementation

The architecture connects three legal-assistance modules to heterogeneous evidence sources and a shared evidence-grounded generation component. Its implementation combines retrieval, multi-agent research, and citation mechanisms.

4.1 LLM and Data Services

  • LawCompass uses DeepSeek-v4 as its primary reasoning engine to orchestrate core workflows; its data services collect 220,000 statutory provisions and 100,000 court cases, and integrate Tavily-powered real-time web search.Source: §4.1 · p. 4

4.2 Legal Question Answering

  • Legal QA uses conversational memory and a two-tier intent router: heuristics handle common inputs, while an LLM classifier routes only legal_query inputs to retrieval and generation.Short- and long-term memory support session context and persistent user preferences.Source: §4.2 · p. 4

4.3 Professional Retrieval

  • Professional Retrieval rewrites queries, searches original and rewritten forms through keyword and vector paths, then merges and reranks results into a unified evidence set.Results are organized by legal authority, and the evidence set can feed Deep Research.Source: §4.3 · pp. 4–5

4.4 Multi-Agent Deep Research

  • Deep Research plans 3–5 scoped subtasks, repairs invalid plans or uses a fallback template, retrieves and summarizes evidence through executor agents, then deduplicates sources and synthesizes a structured report.The report covers legal issues, analysis, statutes, cases, risk alerts, and a liability disclaimer.Source: §4.4 · pp. 5–6

4.5 Evidence-Grounded Generation

  • Shared evidence-grounded generation orders sources by legal authority and instructs the LLM to cite only numbered tags from a source catalogue rendered as clickable links.The catalogue records metadata such as statutory hierarchy, article and case numbers, courts, case reference levels, and source locations.Source: §4.5 · p. 6

5 Evaluation

LawCompass is evaluated through retrieval quality, a task-based user study, and a representative case study. Retrieval and user-study results are reported for legal queries and participants with and without legal backgrounds.

5 Evaluation

  • Evaluation covers retrieval quality, a user study, and a representative case study.The retrieval evaluation uses 20 legal queries; the user study asks participants to complete three representative tasks.Source: §5 · p. 6, §5.1 · p. 6, §5.2 · p. 6

5.1 Retrieval Quality

  • Two legally trained annotators rated the top-five passages for 20 legal queries; Table 1 reports Precision@5, nDCG@5, and average relevance, with results supporting dual-path retrieval.Passages were labeled relevant, partially relevant, or irrelevant.Source: §5.1 · p. 6, Table 1 · p. 6

5.2 User Study

  • Participants with and without legal backgrounds rated LawCompass on five-point Likert scales, with particularly strong ratings for usefulness, Deep Research value, and overall satisfaction.They completed three representative tasks and rated usability, usefulness, clarity, evidence-grounded trustworthiness, and Deep Research utility.Source: §5.2 · p. 6, Table 2 · p. 6

6 Conclusion

LawCompass combines Legal QA, Professional Retrieval, and multi-agent Deep Research in one evidence-grounded legal system. The authors report that its evaluations support practical legal research tasks.

6 Conclusion

  • LawCompass supports Legal QA, Professional Retrieval, and multi-agent Deep Research within a unified interface.It combines heterogeneous legal sources with task-oriented retrieval and a coordinated multi-agent workflow to support movement from direct legal information seeking to report-level research.Source: §6 · p. 7
  • The coordinated workflow and heterogeneous legal sources support movement from direct legal information seeking to complex, report-level research.The system's sources include statutes, judicial cases, and user-uploaded documents.Source: §6 · p. 7
  • Retrieval evaluation, the user study, and representative case studies validate LawCompass's effectiveness in supporting practical legal research tasks.Source: §6 · p. 7

Limitations

LawCompass's retrieval coverage depends on the completeness and update frequency of its local database, while Deep Research adds latency and computational cost.

Limitations

  • Retrieval coverage depends on the local database's completeness and update frequency, so new legal documents, recent cases, and local regulations may not be included promptly.Source: Limitations · p. 7
  • Deep Research decomposes complex questions into subtasks to improve structure and interpretability, but this multi-stage process adds latency and computational cost.Source: Limitations · p. 7
  • Because retrieval coverage depends on a local database, recently issued legal materials may not be available promptly.Source: Limitations · p. 7

A Case Study of Deep Research

The dismissal-compensation example distinguishes outcomes by the legality and type of termination, with additional conditions for notice and high-income employees.

A Case Study of Deep Research

  • Lawful termination requiring compensation generally yields N, while certain non-fault terminations without 30 days' prior written notice may yield N + 1.Source: A Case Study of Deep Research · p. 9
  • For unlawful termination, employees may seek continued performance or, when it is impossible or not requested, compensation generally equal to twice the statutory amount, or 2N.Source: A Case Study of Deep Research · p. 9
  • For high-income employees, the compensation wage base may be capped at three times the local prior-year average monthly wage and the compensable service period at 12 years.The applicable statistical measure depends on the employing unit's locality [Web 1][Web 2].Source: A Case Study of Deep Research · p. 9

2. Detailed Legal Analysis

The amount recoverable depends first on the legality and type of termination. The conclusions distinguish ordinary compensation, specified non-fault terminations, and unlawful dismissal remedies.

2. Detailed Legal Analysis

  • The amount recoverable first depends on whether termination was lawful and what type of termination occurred.Source: §2 · p. 9
  • “N” financial compensation is generally calculated as years of service multiplied by monthly wage.Source: §2 · p. 9
  • “N + 1” is limited to certain terminations on statutory non-fault grounds, rather than applying to every dismissal.Source: §2 · p. 9
  • For unlawful dismissal, employees may generally claim continued performance or 2N compensation; “N + 1” does not apply to every dismissal and is limited to certain statutory non-fault terminations.Source: §2 · p. 9
  • High-income employees may be subject to a cap of three times the local average wage and a maximum of 12 years.Source: §2 · p. 9

Conclusion 5: High-income employees may be subject to the “three-times-local-average-wage plus 12-year maximum” cap

Article 87 addresses unlawful revocation or termination of a labor contract, generally requiring compensation at twice the statutory financial-compensation rate.

Conclusion 5: High-income employees may be subject to the “three-times-local-average-wage plus 12-year maximum” cap

3. Statutory Provisions

The provisions list legal authorities by legal-effect hierarchy, and the examples distinguish a representative compensation dispute from the need to classify the termination before assessing payment outcomes.

3. Statutory Provisions

  • Article 87 generally requires twice the statutory financial-compensation rate for unlawful revocation or termination of a labor contract.Source: §3 · p. 10

4. Case References

  • The retained ordinary case concerns an employer’s disputed obligation to pay RMB 26,001.87 in financial compensation after termination.The case is presented as one representative example of a common dispute.Source: §4 · p. 10

5. Risk Alerts

  • Payment outcomes may differ substantially across voluntary resignation, mutual agreement, lawful fault-based termination, non-fault termination, redundancy, and unlawful termination.The risk alert advises identifying the termination category rather than relying only on the label “dismissed.”Source: §5 · p. 10, §4 · p. 10

6. Liability Disclaimer

  • The abridged bilingual Deep Research report omits detailed analyses, additional legal materials, and case references.Source: Table 3 · p. 10
Loading 2610.01027v1…