PaperlayerView exact revision on arXiv

Source-linked AI summary

JudgmentLens: Human-AI Sensemaking of Complex Legal Judgments

Xinyi Chen, Ruijie Li, Yuelu Li, Chen Liang

arXiv:2609.07607v1cs.HC

TL;DR

Non-experts can access judgments yet struggle to connect their facts, evidence, and reasoning to a court's decision, while AI-generated interpretations can be difficult to verify. JudgmentLens combines persistent case representations with source-grounded AI assistance. In a controlled comparison with PDF reading, it reduced completion time and subjective workload and increased self-reported decision understanding, while rubric-scored comprehension did not differ reliably.

  • Problem

    Non-experts have limited support and empirical understanding for reconstructing how evidence and reasoning within a judgment support a decision, and for inspecting the basis of AI-generated interpretations.

    Source: §1 · p. 2

  • Method

    The authors developed JudgmentLens, combining structured case representations, adaptive explanations, and links from generated interpretations to original judgment passages, informed by a formative study.

    Source: §1 · p. 3

  • Results

    JudgmentLens reduced completion time and subjective workload and increased self-reported decision understanding relative to PDF reading, while rubric-scored comprehension showed no statistically reliable difference.

    Source: §5.2.3 · p. 11, §6 · p. 14

  • Takeaways & Limitations

    Persistent structure and direct paths back to source passages shaped how users distributed sensemaking work between themselves and the system.

    Source: §5.2.4 · p. 12, §7 · p. 16

    Rubric-scored comprehension showed no reliable difference, limiting claims that the system improves factual knowledge or legal reasoning accuracy.

    Source: §6.4 · p. 16

Abstract

from arXiv · show

Judicial judgments are increasingly available, yet dense language and distributed relationships among facts, evidence, reasoning, and rulings remain difficult for non-experts to interpret. Through a mixed-methods formative study with Chinese non-expert readers (survey N=34; interviews N=6), we identified structural, interpretive, verification, and action breakdowns. We developed JudgmentLens, an AI-augmented reading system combining persistent case representations, adaptive explanations, and traceable links from generated interpretations to judgment passages. In a counterbalanced within-subject evaluation (N=16), participants completed tasks faster with JudgmentLens than with conventional PDF reading and reported lower workload and greater self-reported decision understanding, while rubric-scored comprehension did not differ reliably. An exploratory PDF+DeepSeek probe suggested that conversational AI supported formulated questions while leaving question formulation, answer integration, and source checking largely to users. We contribute an empirical account of non-expert judgment sensemaking and design strategies for inspectable, source-grounded AI mediation.

1 Introduction

Judgments are more accessible in China, but non-experts still struggle to reconstruct how evidence and reasoning support a decision and to verify AI-generated interpretations. The paper investigates these sensemaking gaps and presents JudgmentLens as source-grounded support for interpreting judgments.

1 Introduction

  • Expanded access to judgments does not ensure understanding: non-experts must reconstruct relationships among facts, evidence, legal rules, and decisions from documents designed for institutional practice.The challenge is understanding how and why a court reached a particular decision, not simply accessing legal information.Source: §1 · p. 2
  • Prior systems support document exploration, summarization, or collections of legal information, but offer less support for understanding evidence evaluation, judicial reasoning, and decisions within a single judgment.The paper identifies limited empirical understanding of judgment-specific sensemaking breakdowns among non-experts and a need for explanations that support inquiry and source inspection.Source: §1 · p. 2
  • Fluent AI explanations can be difficult to verify, so readers still need to assess whether interpretations are supported by and faithful to the authoritative judgment.Users may need to distinguish explicit support from synthesis across passages; hallucination and unsupported generation add to this burden.Source: §1 · p. 2
  • A formative study with Chinese non-expert readers (survey N=34; interviews N=6) identified four breakdowns and informed JudgmentLens, which combines structured case representations, visual scaffolding, adaptive explanations, and links to judgment passages.A counterbalanced within-subject evaluation (N=16) compared the system with conventional PDF reading; a separate PDF+DeepSeek probe was exploratory and descriptive, not a causal baseline comparison.Source: §1 · p. 3
  • The paper frames judgment interpretation as human-AI sensemaking and contributes design strategies for keeping AI-generated interpretations inspectable against their documentary basis.It also reports quantitative evidence on efficiency, workload, and self-reported understanding, alongside qualitative and descriptive insights into case organization, source return, and conversational support.Source: §1 · p. 3

2 Related Work

Prior research offers ways to navigate complex documents and legal information, but leaves open how non-experts can reconstruct the reasoning behind a single judgment. This paper positions source-grounded AI support and user control as central to that task.

2.1 Judicial Judgment Interpretation as a Sensemaking Challenge

  • Judgment interpretation requires non-experts to connect facts, evidence, rules, reasoning, and decisions; existing legal systems support broader information exploration but leave case-specific reconstruction within one judgment open.Judgment complexity can also make it harder for readers without professional support to understand rights, obligations, and available actions.Source: §2.1 · p. 3

2.2 Interactive Support for Complex Document Sensemaking

  • Complex-document support includes text reduction and interactive visual structures, but compression can lose context and visualizations make relationships implicit in linear text more navigable.Neither text reduction nor navigation alone ensures that underlying reasoning is easier to inspect.Source: §2.2 · pp. 3–4
  • AI-augmented legal reading has linked assistance to authoritative text, but contract reading differs from reconstructing a judicial decision from interconnected facts, evidence, rules, and reasoning.Prior systems leave open how structural and contextual techniques should be organized around judicial decision reasoning.Source: §2.2 · p. 4

2.3 AI-Augmented Sensemaking and Epistemic Agency

  • Because fluent AI outputs may be unsupported, high-stakes interpretation requires source inspection and user control over how much to rely on generated explanations.Epistemic agency is framed as the ability to inspect information, evaluate an interpretation's basis, question AI claims, and retain control over conclusions.Source: §2.3 · p. 4
  • The paper investigates this gap through JudgmentLens, integrating structured case representations, visual scaffolding, adaptive explanations, and source-grounded AI assistance for human scrutiny.The goal is to extend document augmentation toward judgment sensemaking in which generated interpretations remain connected to the authoritative document.Source: §2.3 · p. 4

3 Formative Study

A two-phase formative study combined a questionnaire and interviews to characterize how non-experts read judgments and where interpretation breaks down. Its findings informed design implications for structural support, attribution, traceability, and practical action.

3.2 Phase 2: Semi-structured Interviews

  • The questionnaire mapped reading practices and expectations; six purposively selected respondents from that cohort then completed 20–30-minute interviews about breakdowns, coping, prior AI use, and desired support.Interviews were transcribed and analyzed thematically by two researchers, with questionnaire patterns considered alongside interview themes.Source: §3 · pp. 4–5, §3.2 · p. 5

3.1 Phase 1: Online Questionnaire Survey

  • 50.00% (n=17) rated judgment-understanding difficulty at least 4/5; evidence admissibility (70.59%, n=24), statutory references (67.65%, n=23), and judicial reasoning (64.71%, n=22) were the most difficult sections.The questionnaire had 34 valid responses and was descriptive, intended to identify patterns for follow-up interviews rather than test hypotheses.Source: §3.1.2 · p. 5
  • 76.47% (n=26) sought help from lawyers or knowledgeable peers, 85.29% reported anxiety, helplessness, or dissatisfaction, and 91.17% (n=31) were willing to use an AI-assisted visual reading tool.Participants also reported online searches and looking up legal provisions as coping strategies.Source: §3.1.2 · p. 5

3.3 Formative Study Results

  • The formative study synthesized four recurring breakdowns: structural reconstruction, understanding why decisions were reached, verification of AI interpretations, and translating understanding into subsequent steps.These patterns were used to derive the design implications guiding JudgmentLens.Source: §3.3 · p. 5

3.3.1 Structural Breakdown: Reconstructing Hidden Reasoning

  • Readers struggled to connect information distributed across factual narratives, evidence descriptions, and court reasoning; the burden was reconstructing implicit relationships, not simply handling volume.Evidence admissibility, statutory references, and judicial reasoning were among the sections most often reported as difficult.Source: §3.3.1 · p. 5

3.3.2 Interpretive Breakdown: Understanding Why Decisions Were Made

  • Participants with unfavorable outcomes especially wanted to know how the court evaluated their evidence and claims, not merely what legal terms meant.One participant contrasted dictionary explanations with wanting to understand why transfer screenshots were judged insufficient.Source: §3.3.2 · p. 6

3.3.3 Verification Breakdown: Trusting and Tracing AI Assistance

  • Participants interested in AI remained concerned about fidelity to the judgment and safe handling of sensitive documents; they wanted interpretations traceable to exact source lines.The concern was whether users could inspect the basis of a particular interpretation, not simply whether they trusted AI in general.Source: §3.3.3 · p. 6

3.3.4 Action Breakdown: Translating Understanding into Next Steps

  • Participants connected judgment interpretation to procedural decisions and wanted help with concrete, reviewable preparation tasks rather than automated outcome predictions.The study translated the four breakdowns into implications: externalize reasoning structures, attribute interpretations, maintain traceability and data control, and bridge understanding to situated action without replacing human judgment.Source: §3.3.4 · p. 6, §3.4 · p. 6

4 The JudgmentLens System

JudgmentLens turns formative-study design implications into a workspace of persistent, coordinated case representations and source-linked AI assistance. Readers can move between orientation, detailed analysis, source inspection, and preparation without replacing the judgment or surrendering decisions about subsequent action.

4 The JudgmentLens System

  • JudgmentLens uses persistent representations to help readers examine case relationships, ask follow-up questions, and return to the court’s text rather than relying on a single summary or chatbot sequence.The system is guided by design implications tied to structural, interpretive, verification, and action breakdowns.Source: §4 · pp. 6–7, Figure 2 · p. 7

4.1 Entering and Configuring the Reading Workspace

  • Readers can configure explanation depth, reading perspective, and original or masked text, then change those settings without losing the active case or source links.Preparation runs asynchronously, exposes completed views progressively, and leaves the Judgment Text available if a generated module is delayed or unavailable.Source: §4.1 · pp. 7–8

4.2 Constructing a Case-level Mental Model

  • The Case Summary and Panorama orient readers, while coordinated Timeline, Logic Graph, and Evidence views foreground chronology, inferential relationships, and evidential status.The views are coordinated projections of one case representation, letting readers switch among questions while retaining shared source-linked entities and case context.Source: §4.2 · p. 8, Figure 3 · p. 9
  • The Timeline orders factual and procedural events; the Logic Graph links evidence and considerations to rulings; the Evidence view distinguishes submission from judicial acceptance and reliance.The Evidence view records each item’s proof target, the court’s stated reason for accepting or rejecting it, and its effect on the ruling.Source: §4.2 · p. 8

4.3 Moving between Representations, Questions, and Sources

  • Source links open the relevant judgment paragraph and highlight related text while preserving the active representation; the AI Guide links case-specific answers to source passages for inspection.Readers can formulate questions from visible case relationships, inspect answers against the judgment, and return to persistent representations to integrate them.Source: §4.3 · p. 8

4.4 Extending Understanding toward Preparation

  • Next Actions organizes source-linked deadlines, obligations, and issues for review, while Similar Cases supports bounded comparison without predicting success or substituting for professional analysis.The workspace organizes and compares source-backed information, but leaves its significance and decisions about subsequent action to readers and, where appropriate, legal professionals.Source: §4.4 · p. 9

4.5 Implementation of the Source-linked Workspace

  • The React/TypeScript application uses a FastAPI backend to build a versioned Case Fact Core and generate coordinated views with paragraph-level source anchors.Study materials were manually checked against the judgments to stabilize them, but this does not establish general model correctness; the implementation and supporting files are slated for public release upon publication.Source: §4.5 · pp. 9–10

5 User Evaluation

A counterbalanced comparison found faster task completion and better self-reported experience with JudgmentLens, but no statistically reliable difference in rubric-scored comprehension. An exploratory PDF+DeepSeek probe described additional work users retained, without supporting a causal comparison.

5.1 Evaluation Overview and Participants

  • Participants were 16 non-experts who completed one judgment task per interface; cases and interface order were crossed, and four primary outcomes were analyzed with paired tests and Holm correction.Tasks used comparable first-instance theft judgments; participants answered five questions per task.Source: §5.1 · p. 10, §5.2.1 · p. 11, §5.2.2 · p. 11

5.2.3 Quantitative Results

  • 4.25 minutes faster (28.9% reduction relative to PDF), participants completed tasks with JudgmentLens and reported higher decision understanding and lower workload.The primary comparison involved 16 participants; Figure 6 summarizes the four outcome distributions.Source: §5.2.3 · p. 11, Table 3 · p. 12, Figure 6 · p. 13
  • Rubric-scored comprehension was numerically higher with JudgmentLens (M=8.81) than with PDF (M=7.75), but the difference was not statistically reliable.The study therefore does not claim improved or statistically equivalent rubric-scored comprehension.Source: §5.2.3 · p. 11

5.2.4 How Structure and Source Links Shaped Use

  • Participants attributed reduced effort to persistent organization, while source links supported checking interpretations but could open several passages and add navigation costs.Other reported frictions included slow or missing AI responses, dense highlighting, small text, and insufficient onboarding.Source: §5.2.4 · pp. 11–12

5.3.1 Motivation, Procedure, and Evidentiary Scope

  • The PDF+DeepSeek probe followed the controlled comparison using Case C and a different procedure; its outcomes were treated descriptively rather than as causal interface effects.The probe always came third, so interface, case, order, learning, and fatigue could not be separated.Source: §5.3.1 · p. 13
  • In retrospective reports, JudgmentLens had the least cognitive/visual burden and greatest perceived control, while direct PDF access received the highest verifiability rating.These ratings reflect participants’ perceptions after experiencing the modes in a fixed sequence.Source: §5.3.1 · p. 13, Table 4 · p. 14

5.3.2 What Conversational AI Left to the User

  • With conversational AI, users still had to decide what to ask, assess answer coverage, integrate responses, and locate source support in the PDF.Participants described it as useful for local questions they had already formulated.Source: §5.3.2 · p. 13
  • JudgmentLens was designed to externalize more upstream sensemaking through persistent, source-linked case representations available before, during, and after individual questions.The probe is a contrast between distributions of human–AI work, not evidence that one interface universally outperforms the other.Source: §5.3.2 · pp. 13–14

6 Discussion

JudgmentLens externalizes parts of judgment sensemaking while preserving access to authoritative passages and user control. The discussion presents evidence of reduced task effort and improved reported experience, while delimiting what the evaluation establishes.

6 Discussion

  • JudgmentLens reduced completion time and subjective workload and increased self-reported decision understanding versus PDF reading, but rubric-scored comprehension did not differ reliably.The controlled evidence most clearly concerns task effort and self-reported experience, not equivalent comprehension.Source: §6 · p. 14, §6.1 · p. 14

6.1 Redistributing Sensemaking Work between Users and AI

  • Its case views externalize orientation, organization, relationship construction, and source navigation while leaving evaluation and conclusion formation to the reader.The Case Fact Core, Timeline, Logic Graph, Evidence view, and coordinated case views make this division of work explicit.Source: §6.1 · p. 14
  • The exploratory PDF+DeepSeek probe offers interaction insight, not evidence that JudgmentLens outperformed DeepSeek; conversational AI answered formulated questions while users handled question choice, integration, and source checking.The different case and fixed order confounded interface effects with case difficulty, learning, order, and fatigue.Source: §6.1 · pp. 14–15

6.2 Making AI Mediation Inspectable through Source Grounding

  • Provenance links make source origin and transformation navigable, connecting summaries, reasoning elements, and AI responses directly to judgment passages.Participants described the original judgment as necessary for checking explanations, particularly about consequential decisions; retrieved evidence alone does not ensure users can inspect its relationship to an output.Source: §6.2 · p. 15
  • Inspectability means users can locate claim support, distinguish direct statements from synthesis, and consider context that may change interpretation.This user-facing property complements measures of generation quality or hallucination rates.Source: §6.2 · p. 15

6.3 Supporting User Agency without Substituting Interpretive Authority

  • AI can reduce interpretive work while making users more likely to accept interpretations without engaging their basis, so assistance and meaningful scrutiny must remain compatible.The discussion treats source grounding as an inspection path and agency as the ability to use that path to accept, reject, or reinterpret outputs.Source: §6.3 · p. 15
  • Practical agency requires users to bypass explanations, inspect alternative evidence, question interpretations, and form their own conclusions, not merely remain nominally in the loop.The study did not behaviorally measure epistemic agency or show that source grounding eliminates overreliance; it supports a design direction that lowers the practical cost of exercising agency.Source: §6.3 · p. 15

6.4 Limitations and Future Directions

  • The evaluation involved 16 non-experts, short-term tasks, and selected Chinese judicial cases, so it addresses immediate interaction and comprehension rather than long-term legal behavior.The integrated-system comparison cannot isolate component effects; the DeepSeek probe was confounded, and source-checking, error-detection, and reliance behaviors remain untested behaviorally.Source: §6.4 · pp. 15–16

7 Conclusion

The conclusion frames judgment interpretation as more than access to text: readers must reconstruct relationships among facts, evidence, reasoning, and decisions. JudgmentLens combines persistent case representations with source-grounded AI assistance, while the appendix documents formative-study instruments and analysis.

7 Conclusion

  • Non-experts struggle to reconstruct relationships among facts, evidence, judicial reasoning, and decisions; JudgmentLens combines persistent case representations with source-grounded AI assistance.In the controlled comparison, it reduced completion time and workload and raised self-reported decision understanding, while rubric-scored comprehension did not differ reliably.Source: §7 · p. 16
  • The work argues for high-stakes AI that supports human sensemaking, epistemic agency, and accountable interpretation rather than substituting for human judgment.The conclusion presents this as a design direction, not a demonstrated guarantee of trustworthiness.Source: §7 · p. 16

A Formative Study Materials

  • The formative-study appendix provides English translations of instruments administered in Chinese and describes semi-structured interviews with prompts adapted to each conversation.The interview guide covered judgment difficulties, coping and AI use, source trust, and desired next steps or tool features.Source: A Formative Study Materials · p. 18, §A.2 · pp. 18–19

A.1 Phase 1 Questionnaire

  • The 15-item questionnaire covered participant background and judgment-reading behavior, including difficulty, responses to unfamiliar terms, and willingness to use an AI-assisted tool.Items also asked about emotional experience and expectations for charts, timelines, or flow diagrams with question answering.Source: §A.1 · p. 18

A.2 Phase 2 Interview Core Prompts

  • Interview prompts asked about case experience, reading and comprehension, responses to uncertainty, AI-answer trustworthiness, and the effects of visual explanations or source links.Interviewers could adapt prompt order and use condition-specific follow-ups.Source: §A.2 · p. 19

A.3 Condensed Analytical Framework

  • The condensed analytical framework traces how questionnaire and interview evidence was iteratively consolidated from initial codes into the four reported breakdown themes.It preserves the progression from original coding records to reported findings.Source: §A.3 · p. 19

B Evaluation Materials

The evaluation materials define a common rubric for comprehension tests and show how Case A questions operationalized judgment understanding across outcomes, reasoning, evidence, procedure, and action.

B Evaluation Materials

  • Each five-item comprehension test had a maximum of 10 points, with item responses scored 0 for incorrect or unsupported, 1 for incomplete, and 2 for all required components.The materials state that instruments were administered in Chinese and provide English translations.Source: B Evaluation Materials · p. 19

B.1 Case A Comprehension Test

  • Case A tested sentencing outcomes, a disputed voluntary-surrender argument, evidence relied on, the appeal deadline, and what a defendant should verify and prepare.The answer key specifies an appeal within 10 days from the day after receipt and, for a written appeal, one original and eight copies.Source: §B.1 · p. 19

B.2 Case B Comprehension Test

The Case B and Case C materials use case-specific comprehension questions to assess recall and reconstruction of outcomes, reasoning, evidence, procedure, source location, and subsequent action.

B.2 Case B Comprehension Test

  • Case B tested sentences and fines for three defendants and whether the court accepted L1’s voluntary-surrender claim, including its stated reason for rejection.The questions also addressed exclusion of the Price Appraisal Conclusion, the theft period and appeal deadline, and L2’s fine and recovery payment.Source: §B.2 · pp. 19–20

B.3 Case C Comprehension Test

  • Case C tested R’s and L’s sentences and fines, the court’s treatment of disputed cash and mutton amounts, and L’s lookout role in two joint thefts.The answer key identifies unsupported cash evidence, an uncorroborated mutton weight, and RMB 400 in resale proceeds as the basis used.Source: §B.3 · p. 20
  • Case C also tested whether readers could locate how R obtained a bank-card password in the PDF and state the appeal deadline and required copies.The source-location answer points to PDF page 2’s final complete paragraph; the appeal period is 10 days from the day after receipt, with one original and two copies.Source: §B.3 · p. 20

B.4 Post-task Scales

The appendices document how post-task understanding, workload, and feature perceptions were measured, followed by preference questionnaires and interview prompts. Exploratory PDF+DeepSeek comparisons were treated descriptively because they occurred last and used a different case.

B.4 Post-task Scales

  • Five seven-point items formed the self-reported decision-understanding composite, while adapted NASA-TLX items measured workload and analyzed perceived performance separately.The workload composite averaged mental demand, temporal demand, effort, frustration, and visual/interaction strain; perceived-performance anchors ran in the opposite direction.Source: §B.4 · p. 21
  • The decision-understanding items covered case facts and outcome, defendant roles, accepted or rejected evidence and reasons, appeal rights, obligations, and confidence in test answers.The feature ratings separately assessed explanation depth, perspective switching, multimodal presentation, and AI assistance with follow-up questions and source tracing.Source: §B.4 · p. 21

B.5 Preference Questionnaires

  • The preference questionnaire compared JudgmentLens with PDF reading on preference, speed, effort, fatigue, case-logic organization, answer confidence, and recommendation, then ranked nine modules.The modules ranged from Case Summary and Timeline to AI Guide and Judgment Text.Source: §B.5 · p. 21
  • The retrospective PDF-only, PDF+DeepSeek, and JudgmentLens probe ranked four dimensions and rated cognitive load, retrieval friction, fatigue, verification, and perceived control.Because the probe always came last and used a different case, its comparisons were descriptive rather than controlled outcome measures.Source: §B.5 · p. 21

B.6 Post-task Interview Core Prompts

  • Post-task interviews asked about structured views, source links, module use, interaction problems, AI question formulation, trust, verification, global understanding, and control.The evaluation interviews were semi-structured, and their core prompts were reconstructed from recordings and transcripts.Source: §B.6 · p. 21

C Additional Implementation Details

The implementation describes a web-based processing pipeline that prepares judgments for privacy-aware generation. Its modules are coordinated projections of a shared, versioned case representation, with routing and fallback mechanisms supporting generation.

C.1 Deployment and Processing Pipeline

  • JudgmentLens ran as a web application with a React/TypeScript frontend, FastAPI backend, Redis and Celery jobs, SQLite and local files, and hosted model endpoints.The evaluated prototype ran on a study-controlled cloud instance.Source: §C.1 · p. 22
  • Uploads accepted PDF, DOCX, and legacy DOC files; parsers extracted text, structure, case metadata, statutes, monetary amounts, and deadlines.The system maintained original and rule-based redacted representations, with privacy mode determining which went to downstream generation; redaction was not complete de-identification.Source: §C.1 · p. 22

C.2 Case Representation and Generation

  • A versioned Case Fact Core stored case parties, claims or charges, facts, evidence, reasoning, rulings, deadlines, and obligations with source-anchor identifiers.Summary, Timeline, Case Panorama, Logic Graph, Evidence, Next Actions, Similar Cases, and question starters were coordinated projections of this shared representation.Source: §C.2 · p. 22
  • Perspective and explanation-depth settings changed ordering, emphasis, vocabulary, and visible detail while retaining the same underlying case structure.Outputs were schema-validated and checked for source alignment; deterministic fallbacks produced more limited representations when the preferred generation path failed.Source: §C.2 · p. 22
  • Task-aware routing used a more capable DeepSeek endpoint for Fact Core construction and Professional-depth generation, and a lower-latency endpoint for routine projections and interactive responses.Judgment retrieval and curated classic-case retrieval used separate locally executed text encoders; versioned prompts, schemas, and hashes supported caching and invalidation of stale outputs.Source: §C.2 · p. 22

C.3 Source Support and AI Guide

Source anchors connect generated items to relocatable judgment passages, and the AI Guide retrieves paragraph-level context before responding. The implementation provides support cues and inspection mechanisms, but does not guarantee legal-interpretation correctness.

C.3 Source Support and AI Guide

  • Each source anchor retained a paragraph or chunk identifier, supporting excerpt, and relocation information; generated items received qualitative source-support states.The states included Direct Source, Source-supported Synthesis, Weak Source, and Requires Review, surfaced as cues rather than calibrated probabilities or legal-correctness estimates.Source: §C.3 · p. 22
  • The AI Guide retrieved active-judgment passages before generation, and selecting a linked item opened and highlighted its corresponding passage in the Judgment Text panel.The interface preserved the reader’s current view and settings while updating shared source focus.Source: §C.3 · p. 22
  • Validation, source-alignment checks, caching, and deterministic fallbacks supported graceful degradation and source inspection, but did not guarantee correct generated legal interpretations.These mechanisms isolated malformed or unavailable modules.Source: §C.3 · p. 22

C.4 Frozen Evaluation Materials

The evaluation used frozen, pre-generated Reading Packs for two cases, with participant-facing content manually checked against the judgments. The study-specific review supports confidence in those materials, not general model accuracy.

C.4 Frozen Evaluation Materials

  • Before evaluation, prompts, schemas, and routing were frozen, and 18 Reading Packs per case covered three perspectives, three explanation depths, and two privacy modes.Researchers checked facts, party roles, disputed issues, rulings, dates, obligations, and linked source passages against the judgments.Source: §C.4 · p. 22
  • All frozen packs passed the study-specific review without material factual or source-alignment discrepancies, but this does not establish general model accuracy.Prompt and schema versions, deployment instructions, and the review checklist were included in the anonymized artifact.Source: §C.4 · p. 22
Loading 2609.07607v1…