PaperlayerView exact revision on arXiv

Source-linked AI summary

LexLattice: Multilingual Extractive Summarization via Neural Cellular Automata on Document Hierarchies

Sujay Uday Rittikar, Sheela Ramanna

arXiv:2609.27032v1cs.CLcs.LGcs.NE

TL;DR

Legal summaries need to remain faithful and traceable, but paragraph-wise ranking gives limited attention to evidence distributed across document structure. LexLattice consolidates paragraph salience over a hierarchy-based semantic lattice before selection, and reports state-of-the-art ROUGE across all 24 EUR-Lex-Sum languages.

  • Problem

    Legal acts are lengthy and multilingual, and extractive summarization is attractive because verbatim provisions remain traceable to their source while avoiding invented legal content.

    Source: §1 · p. 1

  • Method

    LexLattice maps a legal act’s hierarchy to a two-dimensional semantic lattice and applies a masked 2D NCA to consolidate evidence before paragraph selection.

    Source: §1 · p. 2

  • Results

    LexLattice attains state-of-the-art ROUGE across all 24 EUR-Lex-Sum languages, surpassing far larger instruction-tuned baselines.

    Source: §8 · p. 11

  • Takeaways & Limitations

    Explicit consolidation over document structure is presented as a compact substitute for scale in multilingual legal summarization.

    Source: §8 · p. 11

    The rule-based parser presupposes structurally marked text, and comparisons for most languages rely only on extractive LexRank baselines.

    Source: §9 · p. 11

Abstract

from arXiv · show

Faithfulness is a central concern in legal text summarization, which motivates extractive approaches that select verbatim content traceable to its source. Such methods typically rank paragraphs or other structural units in isolation, yet give little attention to consolidating evidence that is distributed across, and shares salience between, distant parts of a document. We introduce LexLattice, an extractive summarizer that reifies a legal act's hierarchy as a two-dimensional semantic lattice and consolidates over it with a masked 2D neural cellular automata before selection. LexLattice attains state-of-the-art ROUGE across all 24 languages of EUR-Lex-Sum in both multilingual and cross-lingual settings, surpassing instruction-tuned baselines with billions of parameters, despite concentrating all trainable capacity in a 1.8M parameter consolidator over a frozen multilingual encoder. A consolidator trained only on high-resource languages further transfers to unseen languages with near-lossless retention (0.99), indicating that the model operates on language-agnostic semantic geometry rather than surface form. Our results position explicit consolidation over document structure as a compact and traceable alternative to scale for multilingual legal summarization.

1 Introduction

Legal summarization must handle long, multilingual acts while preserving faithful, traceable content; LexLattice uses document hierarchy to consolidate evidence before selecting paragraphs. The paper reports best ROUGE among evaluated systems across its multilingual and cross-lingual settings.

1 Introduction

  • Legal acts are long and multilingual, and extractive summaries can preserve faithfulness by keeping selected provisions traceable to their source.Flat sentence scoring in reading order discards the hierarchy that signals whether passages state obligations, provide background, or list technical detail.Source: §1 · p. 1
  • LexLattice + RLOO achieves the best ROUGE among evaluated systems in multilingual and all-pairs cross-lingual EUR-Lex-Sum settings, with 1.8M trainable parameters.The comparison includes recent LLM and SLM baselines; BERTScore-F1 remains competitive.Source: §1 · p. 2
  • LexLattice maps sections to lattice rows and paragraph positions to columns, then applies a masked 2D NCA to consolidate salience before paragraph selection.Its parser and frozen mT5-base encoder feed a shared local update applied eight times; the trainable components contain about 1.8M parameters.Source: §1 · p. 2, Figure 1 · p. 2

2 Related Works

Prior legal summarization work spans extractive baselines, extract-then-abstract pipelines, and legal-domain sequence-to-sequence models. Related work also covers entity-level faithfulness and instruction-tuned small language models.

2 Related Works

  • EUR-Lex-Sum provides multilingual LexRank baselines and an English-to-Spanish track with translate-then-summarize pipelines; other work combines extraction with abstractive generation.The cited extract-then-abstract systems use a LexLM extractor followed by LongT5, Pegasus, or a QLoRA-tuned Llama-3 decoder.Source: §2 · p. 2
  • Other studies probe legal-domain sequence-to-sequence models and analyze entity-level faithfulness, distinguishing expert-introduced entities from spurious hallucinations.The passage identifies LexT5 and benchmarks it against Longformer and PRIMERA variants.Source: §2 · p. 2
  • EUR-Lex-Sum statistics report per-language word, section, and paragraph counts, with 12 high-resource languages used for training and 12 low-resource languages for zero-shot evaluation.Validation and test splits contain 187 and 188 act-summary pairs, respectively, for every language.Source: Table 1 · p. 3
  • The cited small-language-model study reports that instruction tuning alone outperforms task fine-tuning under LLM-as-a-judge evaluations.The comparison covers six EUR-Lex-Sum languages and benchmarks small instruction-tuned models against large language models.Source: §2 · p. 3

3 Dataset

The study uses the full released EUR-Lex-Sum partitions across 24 EU languages, recovering document hierarchy from plain text. It selects whole paragraphs, while long paragraphs may be chunked internally for encoding.

3 Dataset

  • EUR-Lex-Sum contains legal acts and reference summaries for all 24 official EU languages, and the study uses the released train, validation, and test partitions in full.Because the dataset has no structural markup, document structure must be recovered from text.Source: §3 · p. 3

3.1 Rule-based hierarchy parsing

  • A deterministic, convention-based parser identifies preambles, recitals, articles, and annexes as structural units, using multilingual header patterns and line-length checks.Chapter, section, and title headings separate paragraph flow; empty sections are discarded.Source: §3.1 · p. 3
  • Paragraphs are the selection units; those longer than 256 mT5 tokens are split into chunks for encoding and mean-pooled back into one paragraph vector.The model still selects and receives supervision on whole paragraphs, not chunks.Source: §3.1 · pp. 3–4
  • The lattice visualization places recovered sections in rows and paragraph positions in columns, with occupied cells holding paragraph embeddings and grey cells marking padding.The example shows an occupied 14×12 region of the full 48×32 lattice.Source: Figure 2 · p. 4

4 Preliminaries

LexLattice represents document hierarchy on a fixed semantic grid, trains against an ordered greedy extractive oracle, and scores paragraphs after masked recurrent consolidation. A capacity-matched 1D control isolates the effect of the lattice’s structural axis.

4.1 The Semantic Lattice

  • The 48×32 semantic lattice places sections in rows and paragraph positions within sections in columns, so neighboring cells represent adjacent paragraphs or structurally parallel positions.Paragraph vectors are tiled into occupied cells; out-of-grid paragraphs are clamped and merged, and masked cells do not participate in updates.Source: §4.1 · p. 4

4.2 Supervision: a Greedy Extractive Oracle

  • The greedy oracle adds the remaining paragraph with the largest marginal gain in unigram-plus-bigram ROUGE F1, stopping at non-positive gain or 40 selections.Its ordered selection sequence supervises the warm-start stage; the cap removes only the lowest-gain tail when it binds.Source: §4.2 · pp. 4–5

4.3 Model: LexLattice

  • A frozen mT5-base encoder supplies paragraph states, while a masked 2D NCA applies the same local update for eight steps to propagate salience across lattice neighbors.The final projection is zero-initialized, padding is masked at every step, and the consolidator plus readout have about 1.80M trainable parameters.Source: §4.3 · p. 5
  • The capacity-matched 1D-NCA control keeps the architecture and training setup fixed while replacing lattice updates with width-3 convolutions over paragraphs in reading order.The stated design makes lattice geometry the difference between the two systems.Source: §4.3 · p. 5
  • A residual readout scores each paragraph from its initial and consolidated states, then greedily selects by score to the reference-summary word budget and emits selections in document order.The readout receives the concatenation of the initial and final paragraph states.Source: §4.3 · p. 5

5 Methodology

Training proceeds in two stages: a listwise warm-start fits an oracle paragraph order, then reinforcement fine-tuning tests whether directly optimizing extract quality adds value.

5.1 Stage 1: Listwise Warm-Start

  • The warm-start minimizes the Plackett–Luce negative log-likelihood of the oracle paragraph sequence as a listwise ranking objective.The distribution draws paragraphs without replacement with probability proportional to exp(score).Source: §5.1 · p. 5
  • The model scores all N paragraphs, while the oracle target constrains only the first K and leaves the remaining order free.Source: §5.1 · p. 5
  • Binary cross-entropy on oracle membership anchors the absolute score scale, resolving the listwise loss’s invariance to constant shifts.The un-tuned λ = 0.3 keeps the ranking term dominant, so membership classification refines rather than overrides the ordering.Source: §5.1 · p. 6

5.2 Stage 2: Reinforcement Fine-Tuning

  • Reinforcement fine-tuning maximizes reward for extracts sampled without replacement to the word budget, then assembled in document order and truncated as at evaluation.Stage 1’s oracle-order agreement is a proxy; Stage 2 optimizes the emitted extract’s quality directly.Source: §5.2 · p. 6
  • Language-specific reward standardization prevents high-scoring languages from dominating the update.Source: §5.2 · p. 6
  • Four rollouts per document provide leave-one-out baselines for critic-free, unbiased advantages in the policy-gradient estimate.The gradient differentiates the Plackett–Luce log-likelihood through the readout scores; reward is the mean of three ROUGE F1 variants.Source: §5.2 · p. 6

6 Experiments

Experiments use cached frozen-encoder outputs and run training and evaluation on a single workstation.

6 Experiments

  • All runs used seed 0 on one workstation with an AMD Ryzen Threadripper 5975WX CPU and three NVIDIA RTX A6000 GPUs.Source: §6 · p. 6
  • Frozen encoder outputs were cached once, so warm-start, reinforcement-learning, transfer, and ablation runs proceeded without the encoder in the loop.Source: §6 · p. 6
  • RLOO required approximately 8.4 GPU-hours, while warm-start converged within roughly one epoch.Source: §6 · p. 6

7 Results and Discussion

LexLattice improves on the capacity-matched 1D control across languages and transfers to unseen languages with near-lossless retention. Judge-based evaluation and results across the remaining languages also support the structural approach, while salience visualizations show selection concentrating in contiguous lattice regions.

7.1 Multilingual Analysis

  • Across six languages, LexLattice+RLOO leads ROUGE-1, ROUGE-2, and ROUGE-L over baselines up to 8B parameters with 1.8M trainable parameters; LexLattice gains 1.7–2.0 ROUGE-1 and up to 2.7 ROUGE-2 over 1D-NCA.The ROUGE-1 and ROUGE-2 improvements over 1D-NCA hold in every language.Source: §7.1 · pp. 6, 8, Table 2 · p. 7

7.2 Cross-Lingual Transfer

  • Held-out-language retention averaged 0.993 and never fell below 0.974; English-only LexLattice beat translation and LexRank baselines in Spanish, coming within 1.8 ROUGE-1 of the Spanish-supervised model.Five of the 12 held-out languages matched or exceeded their jointly supervised counterparts.Source: §7.2 · p. 8

7.3 LLM-as-a-Judge Evaluation

  • On all 24 languages, LexLattice gains 7.8 LDP-F1 points (+17%) over 1D-NCA, mostly through coverage (+7.1); RLOO adds 1.3 points in relevance.The reported improvements are significant under a per-document paired Wilcoxon test.Source: §7.3 · pp. 8–9, Table 5 · p. 9

7.4 Coverage Across the Remaining Languages

  • Across the remaining 18 languages, LexLattice+RLOO improves over LexRank in every language by 19.0 ROUGE-1 points on average (11.8–24.6).LexRank is the only published external baseline available for these languages.Source: Figure 3 · p. 9, §7.4 · p. 9

7.5 Complete Results Across All Languages

  • Across the 18 additional jointly trained languages, LexLattice or its RLOO variant achieves the best ROUGE-1 and ROUGE-2 in every language; 1D-NCA is competitive only in Bulgarian.Source: Table 6 · p. 10, §7.5 · p. 10

7.6 Visualizing NCA Salience Evolution

  • In a representative English document, salience sharpens over eight NCA steps, and selected paragraphs cluster in contiguous lattice regions rather than isolated cells.The visualization describes rows as document sections and columns as paragraph positions within each section; padding cells are grey.Source: §7.6 · pp. 10–11, Figure 4 · p. 11

8 Conclusion and Future Work

LexLattice consolidates a legal act’s hierarchy on a 2D semantic lattice with a masked 2D NCA before selection. It reports state-of-the-art ROUGE across 24 EUR-Lex-Sum languages and proposes extending the approach to other hierarchical corpora and generator-coupled summaries.

8 Conclusion and Future Work

  • LexLattice attains state-of-the-art ROUGE across all 24 EUR-Lex-Sum languages with a 1.8M-parameter consolidator over a frozen multilingual encoder.The conclusion describes explicit consolidation over document structure as a compact substitute for scale.Source: §8 · p. 11
  • Its learned dynamics sharpen diffuse salience onto contiguous, structurally coherent lattice regions.The conclusion identifies this as evidence that the model exploits document hierarchy.Source: §8 · p. 11
  • Proposed extensions apply the lattice to other long hierarchical corpora, including legal and non-legal texts, and pair traceable extracts with an abstractive generator.The proposed generator coupling aims to add fluency while preserving auditability.Source: §8 · p. 11

9 Limitations

LexLattice has limitations tied to structural parsing, extractive output, and the available comparison baselines.

9 Limitations

  • A rule-based parser presupposes structurally marked text, and performance would degrade on documents without explicit sectioning.Source: §9 · p. 11
  • Extractive summaries are bounded by the source and a fixed length budget, capping coverage and foregoing abstraction’s fluency.Source: §9 · p. 11
  • For most languages, the only published baseline is extractive LexRank, limiting the external baselines available for comparison.Source: §9 · p. 11

Ethical Statement

The work uses a public corpus without personal or private data and aims to broaden access to law, while cautioning that summaries are not legal advice and may be incomplete or less reliable for the most resource-scarce languages.

Ethical Statement

  • EUR-Lex-Sum contains no personal or private data, and the work uses it in accordance with its license.Source: Ethical Statement · p. 11
  • The stated aim is to broaden access to law across EU official languages, particularly for speakers of lower-resourced languages underserved by English-centric systems.Source: Ethical Statement · p. 11
  • Automatic legal summaries are not legal advice or substitutes for authoritative instruments; traceability does not guarantee completeness or faithfulness.Source: Ethical Statement · p. 11
  • Quality is weaker for the most resource-scarce languages, where reliance should be correspondingly cautious.Source: Ethical Statement · p. 11
  • Training a small consolidator over a frozen encoder carries modest computational and energy cost relative to large generative summarizers.Source: Ethical Statement · p. 11
Loading 2609.27032v1…