PaperlayerView exact revision on arXiv

Source-linked AI summary

ContraVis: Evidence-Grounded Visual Analytics for Contradiction Review in Legal Contracts

Luis Sante, Paula Lima, Mariana Rocha, Jorge Poco

arXiv:2609.27014v1cs.HCcs.CL

TL;DR

Legal contradiction review must account for distant, interconnected provisions, yet existing review is cognitively demanding and prior predictors can hide the context behind flagged clauses. ContraVis uses a typed paragraph graph both to condition LLM reasoning and to guide analyst exploration. Graph-conditioned reasoning recovered more injected contradictions than standalone analysis as contracts grew, and lawyers indicated that in-context evidence comparison supported validation.

  • Problem

    Contract review is cognitively demanding, and existing contradiction predictors can hide the contractual structure needed to inspect why provisions are relevant.

    Source: §I · p. 1

  • Method

    ContraVis uses a typed paragraph graph of explicit references and semantic relationships as both LLM reasoning context and an interactive representation for human review.

    Source: §I · p. 1, C. Structure-Aware Analytical Reasoning · p. 3

  • Results

    As contract length grew, graph-conditioned reasoning recovered more injected contradictions than standalone LLM analysis, while lawyers indicated that in-context evidence comparison supported validation.

    Source: A. Controlled Comparison with… · p. 4, D. Qualitative Findings · p. 5, §IX · p. 6

  • Takeaways & Limitations

    The paper frames contradiction analysis as a human-in-the-loop workflow in which evidence-grounded visual exploration supports expert validation and interpretation.

    Source: §I · p. 1, §IX · p. 6

    The evaluation relies on synthetic contradictions that may not fully capture the ambiguity of real disputes.

    Source: §VIII · p. 6

Abstract

from arXiv · show

Legal contracts are structurally complex documents in which contradictions may emerge across distant and interconnected provisions. Although large language models (LLMs) improve legal language understanding, contradiction analysis remains a human-centered and evidence-grounded review task. We present ContraVis, a visual analytics system for human-in-the-loop contradiction analysis in legal contracts. The system models contracts as typed paragraph graphs that combine explicit contractual references with semantic relationships between paragraphs. This graph plays a dual role: it conditions LLM reasoning and serves as the interactive representation the analyst explores, keeping model context and human inspection aligned across coordinated views. In a controlled comparison, graph-conditioned reasoning recovered more injected contradictions than standalone LLM analysis as contract length grew, while surfacing additional candidates for analyst validation. A formative study with contract-domain lawyers indicated that in-context evidence comparison supported contradiction validation, and we distill design implications for evidence-grounded, LLM-assisted document review.

I. INTRODUCTION

Contract contradiction review is cognitively demanding, and prior predictors often hide the contractual context needed to judge flagged provisions. ContraVis frames review as a structure-aware, evidence-grounded task and couples graph-conditioned reasoning with analyst inspection.

I. INTRODUCTION

  • Contract review is manual and cognitively demanding, while prior clause-pair or chunk predictors hide the context that makes flagged provisions relevant.This leaves analysts unable to trace a prediction to the contractual structure behind it.Source: §I · p. 1
  • ContraVis models contradiction review as comparing related provisions across distant regions and distinguishing conflicts from exceptions or refinements.Its typed paragraph graph combines explicit references with semantic relationships and serves as both reasoning context and an interactive representation.Source: §I · p. 1
  • The paper positions its contribution as an early-stage design study focused on problem characterization and task abstraction, rather than new visual encodings.It presents a human-in-the-loop workflow, an interactive system, a formative expert evaluation, and a controlled comparison.Source: §I · p. 1

A. Legal Contract Understanding with NLP and LLMs

  • Prior contract-understanding work spans extraction, question answering, metadata identification, and document-level inference, but end-to-end predictors offer no structural context for validating contradictions.ContractNLI and related work also report challenges for language models in long-document legal reasoning.Source: A. Legal Contract Understanding with… · pp. 1–2

B. Contradiction Detection and Retrieval-Augmented Analysis

  • Earlier contradiction methods predict over isolated clause pairs or retrieved chunks, whereas ContraVis keeps evidence spans and graph neighborhoods inspectable for expert validation.Related work also explores graph-aware retrieval and reranking for evidence discovery and contextual reasoning.Source: B. Contradiction Detection and… · p. 2

C. Visual and Interactive Analysis of Legal Documents

Prior legal visualization supports structural exploration, while ContraVis couples the structure shown to analysts with the structure conditioning model reasoning. Lawyer discussions inform its review tasks and workflow.

C. Visual and Interactive Analysis of Legal Documents

  • ContraVis couples the graph shown to analysts with the structure that conditions LLM inference, aligning model context with human inspection.Earlier legal visualization systems supported structural exploration, but did not couple that structure to contradiction reasoning in this way.Source: C. Visual and Interactive Analysis of… · p. 2

A. System Requirements and Analytical Tasks

  • Discussions with two contract-domain lawyers elicited review requirements, which the authors abstracted into five analytical tasks spanning triage, context inspection, verification, interpretation, and corrective reasoning.The tasks are mapped to four lawyer-centered requirements in Table I.Source: A. System Requirements and Analytical… · p. 2

B. Workflow

  • The workflow constructs a typed paragraph graph, retrieves local neighborhoods as evidence for graph-conditioned reasoning, and presents candidates for inspection through four coordinated views.The workflow begins with a legal contract and ends with analyst inspection of candidates and supporting evidence.Source: B. Workflow · p. 2, Figure 1 · p. 2

IV. GRAPH-CONDITIONED CONTRADICTION ANALYSIS

  • The analysis section describes the graph representation, controlled contradiction-generation process, and graph-conditioned reasoning strategy used in ContraVis.Source: §IV · p. 2

A. Graph Representation

ContraVis represents contracts as typed paragraph graphs with explicit-reference and semantic links. The graph structures controlled contradiction analysis and provides neighborhoods for contextual inspection.

A. Graph Representation

  • The typed paragraph graph represents each contract paragraph as a node and links paragraphs through explicit cross-references and semantic similarity.Referential edges preserve explicit legal structure; semantic edges connect paragraphs whose embedding similarity exceeds τ = 0.80.Source: A. Graph Representation · p. 2

V. CONTRAVIS SYSTEM

  • The interface coordinates document reading, candidate analysis, related-paragraph exploration, and grounded legal chat.The explorer distinguishes referential and semantic links, while chat supports follow-up reasoning grounded in selected context.Source: Figure 2 · p. 3

B. Controlled Evaluation Dataset

  • The controlled dataset inserts five validated, taxonomy-conditioned contradictions into each source contract while preserving its remaining structure.Candidates are filtered by NLI contradiction score and normalized perplexity change, then manually reviewed for plausibility and category alignment.Source: B. Controlled Evaluation Dataset · pp. 2–3

C. Structure-Aware Analytical Reasoning

  • For analysis, ContraVis reconstructs the graph and retrieves direct referential links plus the top five semantic neighbors as contextual evidence for candidate review.Candidates are presented for expert inspection and interpretation, not ranked as final decisions.Source: C. Structure-Aware Analytical Reasoning · p. 3

A. Document View

The interface keeps contract text central while synchronizing candidate evidence and graph context across views. Analysts can compare distant related paragraphs and classify candidates in context.

A. Document View

  • The Document View presents the ordered contract text as the primary representation and links paragraph selection to graph neighborhood, contradiction evidence, and assistant context.Candidates and related paragraphs are highlighted in the contract text.Source: A. Document View · p. 3

B. Contradiction Analysis

  • The Contradiction Analysis panel presents candidate category, assessment, scope, confidence, and evidence spans for contextual validation rather than as final predictions.Analysts can mark candidates as true contradictions, false contradictions, or contextual exceptions or refinements.Source: B. Contradiction Analysis · p. 3

C. Related Paragraph Explorer

  • The Related Paragraph Explorer distinguishes referential from semantic links and brings distant, structurally related paragraphs into a shared comparison space.Its proximity-based comparison supports reading potentially conflicting provisions together without navigating across contract sections.Source: C. Related Paragraph Explorer · pp. 3–4

D. Grounded Legal Chat

Graph-conditioned reasoning better retained injected contradictions as contracts grew, while surfacing additional candidates for analyst review. ContraVis links model context to the analyst’s inspection through graph neighborhoods and coordinated views.

D. Grounded Legal Chat

  • Grounded Legal Chat conditions answers on the selected paragraph, its graph neighborhood, and current contradiction evidence, keeping follow-up reasoning anchored to inspectable evidence.Analysts can ask global or targeted questions about clauses and candidates for explanation, risk inspection, and revision-oriented interactions.Source: D. Grounded Legal Chat · p. 4

A. Controlled Comparison with Standalone LLM Analysis

  • On shorter contracts, both approaches recovered 3–4 of 5 injected contradictions; as contract length grew, standalone recovery fell to 1 of 5 while graph-conditioned reasoning retained 3–4 of 5.Graph conditioning surfaced 2–4 versus 1–2 additional candidates per contract; analysts could validate, contextualize, or discard these inspectable candidates.Source: A. Controlled Comparison with… · p. 4

B. Chunking Limitations and Structure-Aware Neighborhoods

VII. FORMATIVE EXPERT EVALUATION

  • The formative expert evaluation sought domain-informed feedback on workflow usefulness, interpretability, and limitations rather than statistically generalizable usability measures.It was framed as an early-stage design study, with qualitative feedback valued for refining the design.Source: §VII · pp. 4–5

A. Participants

Three contract-domain lawyers found the workflow and explanations useful for contradiction inspection and validation, but adoption intent varied. They evaluated ContraVis in guided inspection and free exploration sessions before answering questionnaire items and giving feedback.

A. Participants

  • The evaluation involved three contract-domain lawyers from a large Brazilian law firm, all experienced in contract drafting and review, who had not participated in requirements elicitation.P1 had 6–10 years of contract-related experience; P2 and P3 had 3–5 years.Source: A. Participants · p. 5

B. Procedure

  • In approximately 30-minute live sessions, participants inspected a guided late-fees and bounced-check penalties contradiction, then freely explored candidates in contracts of 15–55 pages.They thought aloud, answered five-point Likert questions, and provided open-ended feedback; sessions were not recorded, and experimenters took notes.Source: B. Procedure · p. 5

C. Questionnaire Results

  • Experts rated usability and usefulness positively, but adoption intent diverged: P1 agreed, P2 was neutral, and P3 disagreed, preferring an AI legal-assistance tool already integrated into the firm’s workflow.The experts also said additional refinement was needed before deployment in professional legal settings.Source: C. Questionnaire Results · p. 5

D. Qualitative Findings

Lawyers valued in-context evidence comparison and structure-aware exploration for validating contradiction candidates, while identifying relationship classification and workflow integration as areas for improvement. The discussion situates these findings within a small formative evaluation and the conclusion frames CONTRAVIS as a human-in-the-loop approach.

D. Qualitative Findings

  • In-context comparison of conflicting spans helped participants inspect and validate candidates; one participant also noted that rule exceptions were sometimes classified as contradictions.Participants valued explanations and corrective capabilities, but these should remain grounded in evidence and legal context to avoid over-reliance on LLM reasoning.Source: D. Qualitative Findings · p. 5

E. Design Implications

  • Design implications include distinguishing contradictions from exceptions and scope refinements, grounding explanations and revisions, and addressing workflow integration as a separate adoption concern.Divergent adoption responses suggested that usefulness for inspection alone does not ensure fit with existing review practices.Source: E. Design Implications · p. 5

VIII. DISCUSSION, LIMITATIONS, AND FUTURE WORK

  • The formative study involved three experts from one Brazilian law firm, and its results do not support generalizable usability or effectiveness conclusions.The evaluation also uses synthetic contradictions, covers four contracts and one baseline, and has not tested CONTRAVIS in professional practice.Source: §VIII · p. 6
  • Future work includes enriching the graph with contract-level entities and obligations, testing broader contract-quality tasks, and evaluating robustness, larger collections, and longitudinal professional use.Planned studies include trust calibration, usability, analyst efficiency, cognitive workload, and naturally occurring inconsistencies.Source: §VIII · p. 6

IX. CONCLUSION

  • CONTRAVIS aligns LLM reasoning and analyst inspection through a shared typed paragraph graph, while expert in-context comparison supported validation in the formative study.The authors expect this human-in-the-loop design to transfer to other long, densely interlinked documents such as regulations and policy corpora.Source: §IX · p. 6

APPENDIX

The appendix defines six contradiction categories, describes a hybrid paragraph graph, and details a staged process for generating and validating synthetic contradictions.

A. Contradiction Taxonomy

  • The taxonomy covers Temporal, Numerical, Authority, Process, Policy Reversal, and Specificity contradictions, spanning dates, quantities, authority, procedures, rule reversals, and scope or granularity.Source: A. Contradiction Taxonomy · p. 7

B. Graph Construction Details

  • The graph combines paragraphs as nodes with directed edges for explicit legal references and semantic edges for paragraph pairs whose embedding cosine similarity reaches τ = 0.80.Reference links are extracted by identifying legal numbering, resolving identifiers to paragraphs, and filtering self-references and heading matches.Source: B. Graph Construction Details · p. 7

C. Synthetic Contradiction Generation and Validation

  • Synthetic contradictions are generated from high-degree seeds and reranked neighbors, then retained only when γ ≥ 0.70 and ∆PPLnorm < 0.01 and manually reviewed.The process prompts GPT-4.1 with a seed, its neighborhood, and a target category; validation checks semantic contradiction and linguistic plausibility.Source: C. Synthetic Contradiction Generation… · p. 7

D. Graph-Conditioned Detection Details

At detection time, CONTRAVIS reconstructs a typed graph from the contradicted contract and sends its connected paragraph context to an LLM, then presents structured predictions in coordinated views.

D. Graph-Conditioned Detection Details

  • The detector receives the contradicted contract with unknown contradiction locations and a serialized graph payload, enabling document-level reasoning over interconnected neighborhoods rather than isolated paragraph pairs.Source: D. Graph-Conditioned Detection Details · p. 7
  • Predicted evidence spans, involved paragraphs, categories, and justifications populate coordinated views, while the detector remains blind to injection positions so evaluation measures discovery rather than position-aware matching.Graph construction took 2.3–4.7 seconds for contracts of 208–517 paragraphs, and GPT-4.1 detection cost $0.21–$0.82 per contract.Source: D. Graph-Conditioned Detection Details · p. 7

E. Prompt Templates and Output Schema

The appendix describes released LLM prompts and a structured detection output, alongside the prototype interface and the scope of the questionnaire table.

E. Prompt Templates and Output Schema

  • Figure 4 shows a Document View highlighting contradiction evidence beside a panel listing candidate categories and confidence scores, with expanded assessments and conflicting snippets.Table III is identified as reporting usability and usefulness questionnaire results on a five-point Likert scale.Source: Figure 4 · p. 8, Table III · p. 8
  • The detection prompt gives the model a high-recall bias, requires verbatim evidence snippets, and supplies the target paragraph with typed reference and semantic neighbors for intra- and inter-paragraph passes.Source: E. Prompt Templates and Output Schema · p. 8
  • The detection schema includes a paragraph identifier, contradiction type, confidence from 0 to 100, brief reason, and two evidence snippets with source labels.The prompt descriptions are summarized in the paper, while full verbatim prompts are released with the source code.Source: E. Prompt Templates and Output Schema · pp. 7–8
  • The generation prompt uses a taxonomy and instructs the model to select a clause, create a contradictory sentence for insertion, and label its type.Source: E. Prompt Templates and Output Schema · p. 8

F. User Evaluation Questionnaire

After exploration, participants rated the questionnaire’s usability and usefulness items on a five-point Likert scale; Table III reports individual responses.

F. User Evaluation Questionnaire

Loading 2609.27014v1…