Source-linked AI summary
UCVE: Graph Structured Retrieval for Compliance Validation
Emily Chen, Michelle Menachery, Himanshu Kumar, Rahul Nenavath, Amirfarrokh Iranitalab
TL;DR
Compliance validation must assess coverage across interdependent obligations, but existing retrieval and classification systems often treat requirements independently. UCVE represents requirements in a Contextual Requirement Graph, learns edge weights from violation-rewarded traversal, and applies spectral reranking. Across three legal domains, it outperforms strong retrieval and graph-based baselines under document-level recall evaluation.
Problem
Existing retrieval and classification systems treat compliance requirements independently, although validation requires coverage across a structured space of obligations.
Source: §1 · p. 1Method
UCVE uses a Contextual Requirement Graph with reinforcement-learning-weighted edges and spectral reranking to propagate relevance across requirement communities.
Source: §1 · p. 1, §3.3.2 · p. 6Results
Takeaways & Limitations
Spectral reranking generalizes across all three domains without architectural changes, with RL edge learning and spectral reranking jointly shaping the community signal.
Source: §5 · p. 9On dense requirement graphs, spectral reranking has poor path efficiency and is sensitive to hyperparameter configuration.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
Structured compliance validation requires determining whether every obligation in a predefined requirement space is satisfied, violated, or unaddressed by a given document. This can be described as a coverage problem that existing retrieval and classification systems, by treating requirements as independent, are not designed to solve. We introduce \textbf{\ac{UCVE}}, a domain-agnostic framework that represents compliance requirement spaces as a \ac{CRG} and performs retrieval via spectral reranking over its reinforcement learning-weighted Laplacian. Edges between requirement nodes are learned via reinforcement learning with a traversal reward grounded in violation detection, reshaping the graph's community structure such that low-pass spectral filtering distinguishes relevant from irrelevant obligation clusters. Evaluating across three compliance domains, \ac{UCVE} averages 4.3\% matched-$K$ recall lift over flat cosine retrieval and consistently outperforms strong retrieval and graph-based baselines.
1 Introduction
The paper frames compliance validation as coverage over interdependent obligations, where missed violations carry greater costs than false alarms. UCVE uses a Contextual Requirement Graph and spectral reranking to recover relevant obligations, outperforming retrieval and graph-based baselines across three legal compliance benchmarks.
1 Introduction
- High recall across the obligation space is central because overlooked violations can trigger sanctions, litigation, or reputational harm, while false alarms cost additional human review.The benchmarks cover privacy policies, consumer contract fairness, and sustainability disclosures.Source: §1 · pp. 1–2
- Existing systems often treat obligations independently, missing violations with little lexical overlap and offering limited adaptability across jurisdictions or annotation schemas.Legal requirements can be hierarchically organized, cross-referenced, and expressed through varied drafting and paraphrase (Edge et al., 2024); adapting systems may require exhaustive reannotation (Zhong et al., 2020).Source: §1 · p. 1
- UCVE represents obligations as nodes in a Contextual Requirement Graph and learns edge weights with a violation-rewarded traversal, then propagates relevance through spectral reranking.Each node carries a natural-language validation question, supporting traceability; the requirement-defined graph is designed to transfer across legal domains.Source: §1 · pp. 1–2
- Across three legal compliance benchmarks, UCVE consistently outperforms strong retrieval and graph-based baselines, particularly when obligation evidence spans multiple clauses.The paper also reports generalization across structurally distinct domains with lightweight per-domain adaptation.Source: §1 · p. 2
2 Related Work
Prior compliance systems and graph retrieval approaches leave gaps in modeling dependencies across a closed set of obligations. UCVE instead learns requirement-graph structure from validation outcomes and applies spectral filtering over that structure.
2.1 Legal NLP and Compliance Checking
- Earlier legal compliance systems include fixed-hypothesis classification and curated GDPR requirement checking, but model obligations independently or omit dependencies across the requirement space.ContractNLI (Koreeda and Manning, 2021) classifies each hypothesis independently; Cejas et al. (2023) and Abualhaija et al. (2022) use a closed-set framing without requirement dependencies.Source: §2.1 · p. 2
2.2 Graph-based RAG and Knowledge Graph Approaches
- UCVE differs from graph-based RAG approaches that rely on static connectivity, schemas, or heuristic traversal by optimizing structural metadata edges with reinforcement learning.This lets the system learn domain-specific co-relevance patterns for high-recall obligation recovery.Source: §2.2 · pp. 2–3
2.3 Reinforcement Learned Edges
- Unlike RL methods that learn navigation policies or regenerate graph topologies, UCVE uses RL to weight discrete domain-specific metadata edges according to recovered obligations.The traversal reward is grounded in recovery of missed obligations during ϵ-greedy traversal (Xiong et al., 2017; Das et al., 2018; Darvariu et al., 2021).Source: §2.3 · p. 3
2.4 Spectral Graph Theory
- UCVE projects scores onto bottom eigenvectors of an RL-weighted, sparsified Laplacian, smoothing relevance across learned requirement communities and suppressing high-frequency noise.This is contrasted with flat similarities and sequential random walks; related spectral approaches include Fast Spectral Ranking and Query-Aware Flow Diffusion (Zhou et al., 2026).Source: §2.4 · p. 3
3 Methods
UCVE builds a document-independent graph of domain requirements, enriches semantic links with metadata relationships, and learns metadata-edge weights from retrieval outcomes. At inference, it combines graph traversal and spectral reranking to smooth requirement scores across learned community structure.
3.1 Context Requirement Graph
- The Contextual Requirement Graph is a per-domain weighted, undirected graph whose nodes are requirements and whose edges combine semantic similarity with structural metadata.Nodes also store natural-language validation questions and positive/negative class definitions, allowing a uniform grounded binary judgment across domains.Source: §3.1 · p. 4, §3.1.1 · p. 4
3.1.2 Embedding
- The main results use all-mpnet-base-v2, which achieves higher or comparable recall across all three domains than the alternative encoder.The comparison is motivated by using a domain-general encoder to capture requirement semantics and graph structure rather than domain-specific wording.Source: §3.1.2 · p. 4
3.1.3 Semantic Edge Construction
- Semantic edges connect requirement nodes when cosine similarity between their L2-normalized embeddings meets the 0.5 threshold.Their cosine similarity supplies the initial edge weight.Source: §3.1.3 · p. 4
3.1.4 Structural Edge Enrichment
- Six metadata-derived edge types augment semantic similarity, producing seven edge types whose raw signals are combined with learned per-type weights.Metadata types are drawn from fields available in each dataset; the paper notes that semantic similarity alone cannot capture all expert-encoded relational structure.Source: §3.1.4 · pp. 4–5
3.2 RL Edge Weight Learning
- Reinforcement learning optimizes six metadata-edge weights for retrieval recall, using an ϵ-greedy breadth-first traversal with violation-based credit and bounded weight updates.Training begins at the most query-similar node and traverses for 15 steps; credit decays for later violation discoveries, nearby nodes receive small positive credit, and other traversed edges receive small negative credit. The method weights fixed metadata edges rather than learning a navigational policy; its breadth-first strategy suits graphs of 24–82 nodes.Source: §3.2 · p. 5
3.3 Requirement Retrieval Engine
- At inference, UCVE selects entry points, traverses the graph, and reranks retrieved requirements using spectral structure.The reranking scores all same-domain requirements globally, lifting lexically dissimilar nodes in relevant neighborhoods and suppressing surface-similar nodes in irrelevant clusters.Source: §3.3 · p. 5, §3.3.1 · p. 5
3.3.2 Spectral Reranking
- Because the RL-weighted graph is dense, k-NN sparsification retains only each node’s strongest edges before Laplacian decomposition.The bottom Laplacian eigenvectors form a smoothness basis; the number of retained neighbors and eigenvectors is selected per domain on validation data.Source: §3.3.2 · pp. 5–6
- The spectral projection acts as a low-pass filter that lifts nodes with low cosine similarity in relevant communities and suppresses high-similarity nodes in irrelevant clusters.The reranked score blends projected and raw similarity with α=0.8; RL-trained edges reshape communities toward requirement co-relevance patterns.Source: §3.3.2 · p. 6
4 Results
Across three compliance domains, CRG + Spectral achieves the highest recall at matched retrieval budgets, while its learned graph structure and spectral reranking contribute jointly to gains. The results also show domain-specific robustness and validation challenges, including lower end-to-end recall than retrieval ceilings and ESRS precision collapse.
4.1.1 Datasets
- The evaluation covers OPP-115 privacy policies, CLAUDETTE consumer fairness, and 82 ESRS sustainability disclosure requirements.The base graphs contain 148, 577, and 595 edges, respectively.Source: §4.1.1 · p. 6
4.1.2 Metrics
- Matched-K recall measures how well methods order requirements at the same retrieval budget, while document inputs range from full policies to individual clauses or report pages.K is 14 for OPP-115, 16 for CLAUDETTE, and 17 for ESRS.Source: §4.1.2 · p. 7
4.2 Retrieval Performance
- CRG + Spectral achieves the highest recall across all three domains, exceeding Leiden by +2.6% on OPP-115 and +1.9% on CLAUDETTE.Baselines are evaluated at matched-K.Source: §4.2 · p. 7, Table 1 · p. 7
4.3 Graph Structure
- Learned edge weights differ by domain: OPP-115 emphasizes hierarchical metadata, CLAUDETTE metadata weights are near-zero, and ESRS uses a fixed category boost of 0.500.The ESRS category boost was fixed rather than RL-learned because of near-zero training signal.Source: §4.3 · p. 8
- RL edge weighting and spectral reranking work conditionally together: spectral reranking on the RL-weighted graph yields +3.4% lift on OPP-115, +4.1% on CLAUDETTE, and 3.2% on ESRS.Neither component alone is sufficient; BFS on the RL-weighted graph underperforms spectral reranking by −8.2% on OPP-115 and −5.4% on CLAUDETTE at matched-K.Source: §4.3 · p. 8
4.4 Robustness
- Sensitivity analysis finds stable configurations, with k=3 optimal for OPP-115 and CLAUDETTE and a wider k=5 neighborhood optimal for ESRS.The framework uses domain-specific hyperparameters selected on validation data.Source: §4.4 · pp. 8–9
4.5 Full Pipeline Validation
- The retrieval-to-validation recall gap is about 25% on OPP-115, 4–5% on CLAUDETTE, and 23% on ESRS; ESRS precision falls to 0.204 amid an approximately 3% positive rate.ESRS precision collapse varies across pillars and is concentrated in conceptually broad requirements.Source: §4.5 · p. 9, Table 3 · p. 9
5 Conclusion
UCVE reframes compliance validation as structured retrieval over a Contextual Requirement Graph whose edges encode reinforcement-learning-learned co-relevance among obligations. The conclusion emphasizes that spectral reranking and learned edge weights work conditionally together, with evaluation across three domains outperforming retrieval and graph-based baselines.
5 Conclusion
- RL edge weights reshape the graph Laplacian’s community structure for low-pass filtering, and neither component alone achieves the same gains.Source: §5 · p. 9
- Spectral reranking generalizes across all three domains without architectural changes and outperforms retrieval and graph-based baselines under a document-level recall protocol.The authors state that, to their knowledge, no prior compliance validation system has adopted this protocol.Source: §5 · p. 9
- The authors suggest spectral retrieval utility depends on edge weights encoding task-relevant co-occurrence, not graph topology alone.They identify multi-document validation as a natural next step.Source: §5 · p. 9
Limitations
The evaluation highlights limits in graph efficiency, scalability, and comparison coverage. The evaluated requirement spaces are small, and larger spaces face compounding challenges from sparse reward signals and validation imbalance.
Limitations
- CRG has poor path efficiency on dense graphs, with scores of approximately 0.18 on OPP-115, 0.12 on CLAUDETTE, and 0.10 on ESRS.Recall_positive nevertheless remains substantially above flat cosine retrieval; tuned hyperparameters yield up to 4.37% additional lift.Source: Limitations · p. 10
- The three evaluated domains span only 24 to 82 requirement nodes, and larger spaces require more RL training signal while tending to have sparser positive labels per node.On ESRS, near-zero graph-construction signal required a fixed category boost instead of data-driven edge learning.Source: Limitations · p. 10
- The authors identify sparse rewards in large obligation spaces as a future-work challenge, with reward shaping, curriculum learning, or semi-supervised edge initialization as possible directions.Source: Limitations · p. 10
- The study omits an LLM-based query-expansion baseline; under adaptive stopping, CLAUDETTE differences are not statistically significant, and heat-kernel diffusion marginally exceeds CRG + Spectral on OPP-115 under unmodified stopping.Source: Limitations · p. 10
A Graph Construction
The CRG uses cosine-similarity initialization and combines reinforcement-learning-weighted graph structure with spectral reranking. Component analysis identifies their combination as the strongest and most consistent condition, with learned edge weights shaping the structure spectral projection exploits.
A Graph Construction
- The CRG starts from all-mpnet-base-v2 pairwise cosine similarities, retaining edges at similarity ≥0.5 and producing dense base graphs.The initial graphs contain 148 OPP-115 edges, 577 CLAUDETTE edges, and 595 ESRS edges.Source: A Graph Construction · p. 13
- The combination of RL-weighted graph structure and spectral reranking achieves the strongest, most consistent performance in the component analysis.Table 5 compares graph type × retrieval method under matched-K; spectral conditions use natural stopping.Source: Table 5 · p. 13, A Graph Construction · p. 13
- RL edge weights reshape the Laplacian’s community structure into a form spectral projection can exploit; neither component succeeds independently.Source: A Graph Construction · p. 13
B Spectral Community Analysis
The learned graph forms domain-specific spectral communities, with clearer category alignment in CLAUDETTE and ESG-pillar structure in ESRS. Edge-type ablations show that the structural signal contributing to recall differs across domains.
B Spectral Community Analysis
- CLAUDETTE’s five unfair-clause types form separate spectral clusters, and this direct category alignment accompanies its largest recall lift.The five types are arbitration, unilateral change, content removal, limitation of liability, and unilateral termination.Source: B Spectral Community Analysis · p. 14
- OPP-115’s largest community groups broad-scope data practices, while other communities capture specialized attributes and rare topics.The broad-practice nodes have mean cosine similarity 0.635, versus 0.469 across communities; reranking treats them as a co-activation group.Source: B Spectral Community Analysis · pp. 14–15
- 3.2% recall lift accompanies ESRS communities aligned with environmental, social, and governance boundaries, despite a weaker spectral gap than OPP-115.Social requirements split into three subclusters, while governance nodes fragment across three communities.Source: B Spectral Community Analysis · p. 15
B.1 Structural Signal
- Removing same_category edges costs 2.7% recall on OPP-115 and 2.6% on ESRS; CLAUDETTE’s three dataset-specific edge types cost 1.2% collectively when removed.OPP-115 also loses 0.6% when same_sub_attribute edges are removed; CLAUDETTE’s individual edge types contribute negligible lift.Source: §B.1 · p. 15
C RL Edge Weight Learning Details
- The six metadata-edge weights are learned through epsilon-greedy BFS traversal, with a fixed cosine-similarity weight of 1.0.Training runs for 30 epochs at learning rate 0.05, with epsilon decaying from 0.15 to 0.02; traversal rewards depend on reaching ground-truth nodes or their vicinity.Source: C RL Edge Weight Learning Details · p. 15
D.1 GSAEPS
- GSAEPS selects traversal entry nodes using direct relevance, statistical standing, and structural position rather than relying only on the highest cosine-similarity node.Source: §D.1 · p. 15
D.2 Traversal Strategies
Traversal strategies combine graph structure with query similarity, while the graph-document alignment baseline shows domain-dependent precision–recall failures. Entry-point choice and stopping behavior affect retrieval outcomes.
D.2 Traversal Strategies
- Performance is stable across broad keig and α ranges but drops sharply at kNN = 1, where over-sparsification disconnects the graph.Source: Figure 3 · p. 16
- At matched visit budgets of about 18 nodes, GSAEPS improves OPP-115 recall by 1.7 percentage points over cosine-only entry.Random entry’s highest recall comes from a low sweep threshold that effectively reduces it to flat cosine retrieval.Source: §D.2 · p. 16, Table 9 · p. 16
- The graph-scoring alternative scores all same-domain nodes in one pass, making it more suited to smaller, denser graphs with informative structural signals.It combines query similarity and graph structure in a seven-dimensional feature vector.Source: §D.2 · pp. 16–17
D.3 Graph-Document Alignment
- The graph-document alignment baseline shows domain-dependent precision–recall collapse: it overpredicts violations on CLAUDETTE and ESRS, but underpredicts them on OPP-115.The authors attribute both patterns to insufficient discriminative signal in the document-derived context graph; CRG avoids this instability across the three domains.Source: §D.3 · p. 17
D.4 Adaptive Stopping Robustness Check
Under independently calibrated adaptive stopping, CRG + Spectral’s relative performance varies by domain: it leads on OPP-115 and ESRS against several baselines, while CLAUDETTE differences are not significant. The cross-encoder requires many more retrieved nodes on ESRS to match CRG + Spectral’s recall.
D.4 Adaptive Stopping Robustness Check
- On ESRS, CRG + Spectral outperforms flat cosine (mean ∆= −0.053) and the cross-encoder (mean ∆= −0.287); its heat-kernel gap (mean ∆= −0.016) is numerical, not statistically confirmed.The table reports matched-K recall differences, average K needed to match CRG + Spectral recall, and production-rule and tuned recall.Source: §D.4 · p. 17, Table 13 · p. 18
- On OPP-115, CRG + Spectral significantly outperforms all three baselines at matched K, but heat-kernel diffusion reaches 0.795 recall versus CRG + Spectral’s 0.791 under the unmodified production rule.Source: §D.4 · p. 17, Table 13 · p. 18
- On CLAUDETTE, matched-K differences between CRG + Spectral and the baselines are not statistically significant.All reported confidence intervals cross zero.Source: §D.4 · p. 17, Table 13 · p. 18
- The cross-encoder requires 44.9 retrieved nodes on ESRS to match CRG + Spectral’s recall at K=17, compared with being the strongest baseline on 24-node OPP-115.The authors describe this as a degradation with graph size, consistent with difficulty exploiting co-relevance without direct lexical overlap.Source: §D.4 · p. 17
E Encoder Sensitivity
CRG + Spectral has the strongest or tied-strongest recall on OPP-115 and ESRS-82 under both tested encoders. CLAUDETTE has greater cross-method variance, consistent with its dense graph structure.
E Encoder Sensitivity
- CRG + Spectral achieves the strongest or tied-strongest recall on OPP-115 and ESRS-82 under both encoders.Its relative gains over flat cosine are consistent across encoders, so the contribution is not specific to all-mpnet-base-v2.Source: E Encoder Sensitivity · p. 18
F.1 LLM Prompt Templates
- The prompt template defines a positive classification as text that exhibits, satisfies, or violates a requirement, and a negative one as absence or non-applicability.Source: §F.1 · p. 18
F.2 Full Pipeline Results - Llama 3.1 8B
The section reports Llama-3.1-8B end-to-end validation with graph-derived CRG context and describes how to interpret the recall gap. It also outlines an ESRS per-pillar precision and recall breakdown for two model sizes.
F.2 Full Pipeline Results - Llama 3.1 8B
- †∆Rec↑ measures the gap between strategy recall and the retrieval ceiling; values closer to zero are better, and negative values indicate recall below the ceiling.The gap reflects validation errors on retrieved nodes, not retrieval failure.Source: §F.2 · p. 19
- Table 14 covers end-to-end validation performance with graph-derived CRG context using Llama-3.1-8B.Source: Table 14 · p. 19
F.3 ESRS Per-Pillar Breakdown
- Table 15 presents ESRS per-pillar precision and recall for Llama-3.3-70B and Llama-3.1-8B, with approximate positive-instance support and E/S/G categories.Source: Table 15 · p. 19