PaperlayerView exact revision on arXiv

Source-linked AI summary

SinBrief: A Hybrid Framework for Abstractive Text Summarisation of Sinhala Legal Documents

Minduli Lasandi, Nevidu Jayatilleke

arXiv:2609.32397v1cs.CL

TL;DR

Sinhala legal summarisation faces limited annotated resources and a largely unexplored research setting. SINBRIEF combines domain-aware word graphs with sentence scoring to produce abstractive summaries, which show lower lexical overlap than extractive baselines while maintaining factual consistency and semantic similarity.

  • Problem

    Sinhala legal summarisation remains largely unexplored amid scarce annotated datasets and domain-specific corpora, while legal texts are lengthy and complex to navigate.

    Source: §1 · p. 1

  • Method

    SINBRIEF combines domain-aware word graph construction with weakly supervised sentence scoring to generate abstractive Sinhala legal summaries without human-annotated training data.

    Source: §1 · p. 1

  • Results

    SINBRIEF models achieved Coverage of 0.957–0.959 and Density of 2.924–3.125 versus extractive baselines’ Coverage of 1.000 and Density of 7.605–8.153; Llama 3.1 had the highest SummaC score (0.752).

    Source: §4.3 · p. 7

  • Takeaways & Limitations

    SINBRIEF produces more abstractive summaries while maintaining semantic faithfulness compared with the evaluated extractive and abstractive baselines.

    Source: §5 · p. 8, §4.3 · p. 8

    Native Sinhala-speaking authors manually identified grammatical errors, and evaluation relied solely on quantitative metrics, leaving human evaluation needed in future work.

    Source: Limitations · p. 8

Abstract

from arXiv · show

Legal document summarisation in low-resource languages presents significant challenges due to the scarcity of annotated data and the complexity of domain-specific terminology. This paper presents SinBrief, a hybrid abstractive summarisation framework for Sinhala legal documents that does not require human-annotated training data. The proposed framework combines domain-aware word graph construction with neural sentence scoring to generate abstractive summaries from Sinhala legal text. Five sentence scoring models are evaluated within the framework: mBert, Llama 3.1, Falcon 7B, Laser, and a continually pre-trained Llama model domain-adapted to Sinhala legal text. The framework is evaluated on a Sinhala legal corpus using reference-free metrics, including Coverage, Density, Compression Ratio, SummaC, and Self-BertScore. Experimental results demonstrate that SinBrief produces summaries with lower lexical overlap than extractive baselines while maintaining factual consistency, demonstrating the viability of hybrid, largely annotation-free abstractive summarisation for low-resource legal NLP tasks.

1 Introduction

Sinhala legal texts are difficult to navigate, while automated summarisation and annotated resources for low-resource Sinhala legal NLP remain limited. SINBRIEF addresses this gap with a largely annotation-free hybrid framework that combines domain-aware word graphs and sentence scoring to generate abstractive summaries.

1 Introduction

  • Sinhala legal records are predominantly written in Sinhala, but automated summarisation tools are absent, leaving practitioners, researchers, and citizens to navigate them manually.Legal texts are lengthy, complex, and linguistically dense, making relevant information time-consuming and cognitively demanding to extract.Source: §1 · p. 1
  • Abstractive summarisation generates novel sentences that capture source meaning, whereas extractive methods select sentences directly from the source.The introduction describes semantic compression and readability as benefits relevant to legal texts, where extraction may retain redundancy and structural complexity.Source: §1 · p. 1
  • Low-resource abstractive summarisation faces scarce annotated datasets and domain-specific corpora, and Sinhala legal summarisation remains largely unexplored.The paper identifies a need for methods that minimise reliance on human annotation in such settings.Source: §1 · p. 1
  • SINBRIEF combines domain-aware word graph construction and sentence scoring to generate abstractive summaries without human-annotated training data.Its core summarisation pipeline operates without supervision, while scoring uses weak supervision through constraint-guided label construction.Source: §1 · pp. 1–2

2 Existing Work

Prior work spans extractive, abstractive, supervised, and unsupervised summarisation, including graph-based approaches and legal-domain systems. SINBRIEF builds on BenSumm’s graph-based pipeline while adding legal keyword weighting and a weakly supervised neural scorer for Sinhala legal documents.

2 Existing Work

  • Summarisation research spans extractive, abstractive, supervised, and unsupervised methods across languages and domains.The section frames this body of work as a broad field of methodological development.Source: §2 · p. 2, §2.3 · p. 2

2.1 BenSumm

  • BenSumm combines sentence clustering, word-graph construction, and candidate-path generation, while SINBRIEF adds legal keyword weighting, a weakly supervised neural scorer, and continual pre-training of Llama 3.1 8B.BenSumm generates abstractive summaries without parallel training data; SINBRIEF replaces its fixed ranking strategy with a trained binary classifier for legal sentence quality.Source: §2.1 · p. 2

2.2 Summarisation of Legal Texts

  • Legal summarisation work includes transformer and large-language-model approaches, rhetorical-role-based extraction and abstraction, and systems for patent, tax-law, and legal-judgment summaries.The cited examples include multilingual outputs, issue-based summaries, and domain generalisation across patent fields.Source: §2.2 · p. 2

2.3 Unsupervised Summarisation Approaches

  • Unsupervised summarisation ranges from graph-based sentence fusion and semantic-graph methods to neural models using document representations, facets, semantic units, and clustering.The reviewed approaches include both abstractive and extractive systems across document and multi-document settings.Source: §2.3 · pp. 2–3

3 Methodology

SINBRIEF processes Sinhala legal documents through sentence clustering, domain-aware word graphs, candidate generation, and weakly supervised scoring. Its methodology describes the dataset and scorer training, model choices, candidate selection, and final summary construction.

3.1 Data

  • The SinhaLegal corpus contains 1,206 Acts and Bills, partitioned into 964 training documents and 242 test documents.The test set was held out for final evaluation; scorer training and validation used a further split of the training data.Source: §3.1 · p. 3, §3.5.3 · p. 5

3.2 Sentence Extraction and Clustering

  • Sentence representations are grouped by average-linkage agglomerative clustering with cosine similarity, using at most three clusters and fewer for documents with fewer than 15 sentences.The cluster count is k = min(3, ⌊N/5⌋), where N is the number of extracted sentences.Source: §3.2 · p. 3

3.3 Word Graph Construction

  • Each cluster’s directed word graph connects consecutive content-word and POS-tag nodes, with START and END nodes marking sentence boundaries.Edges involving predefined Sinhala legal keywords receive weight 2.0; other edges receive weight 1.0.Source: §3.3 · p. 4

3.4 Abstractive Candidate Generation

  • Weighted random walks generate up to eight 8–25-word candidates per cluster, favoring paths containing legal keywords.Candidates are filtered out if they are shorter than 20 characters, have less than 60% unique words, or repeat a word at least three times; sparse graphs have a fallback candidate-generation procedure.Source: §3.4 · p. 4

3.5.1 Model Selection

  • Four candidate sentence scorers—mBert, Llama 3.1 8B, Falcon 7B, and Laser—were initially evaluated; Llama 3.1 8B produced the highest-quality summaries and was selected for Sinhala-legal continual pre-training, adding a fifth scorer.Continual pre-training used the 964-document SinhaLegal training set; the resulting model did not surpass base Llama 3.1 in scoring performance.Source: §3.5.1 · pp. 4–5, §3.5.2 · p. 5

3.5.3 Training sentence scorers

  • Weakly supervised binary scorer training used 38,920 labelled sentence samples constructed from positive legal sentences and synthetically corrupted negatives.Positive examples included document openings and closings, sentences containing legal keywords, and selected sentences over 50 characters; each was corrupted using one of four strategies to create a negative example.Source: §3.5.3 · p. 5
  • Across all five scorers, Llama 3.1 achieved the highest validation accuracy at 90.05%, followed by CPT Llama (88.37%), Falcon 7B (84.96%), mBert (83.83%), and Laser (77.25%).Source: §3.5.3 · pp. 5–6

3.6 Candidate Selection

  • The top two candidates per cluster are selected, yielding up to six sentences across three clusters to preserve coverage of distinct document themes.The selected text is subject to a 300–500-character target, with sentence addition or truncation used to control length.Source: §3.6 · p. 6, §3.7 · p. 6

4 Evaluation

Evaluation on held-out Sinhala legal documents compares five SINBRIEF scoring configurations with five baselines using reference-free metrics. SINBRIEF generates less extractive summaries; Llama 3.1 scores highest on factual consistency while the framework maintains competitive semantic similarity.

4 Evaluation

  • The held-out test set contained 242 legal documents (20% of the corpus); five sentence-scoring configurations were compared with five baseline systems.Source: §4 · p. 6

4.1 Selection of Evaluation Metrics

  • Because SinhaLegal lacks reference summaries, evaluation uses reference-free Coverage, Density, Compression Ratio, SummaC, and Self-BertScore.Coverage and Density assess copied spans; SummaC assesses entailment, while Self-BertScore measures semantic similarity to the source.Source: §4.1 · pp. 6–7

4.2 Baseline Systems

  • Baselines included extractive Lead-3, TextRank, and PacSum, plus abstractive fine-tuned mT5 and BenSumm adapted with mBert.Zero-shot mT5 and mBart outputs contained special tokens, fragmented Sinhala, and mixed-script characters, rendering them unusable.Source: §4.2 · p. 7

4.3 Results and Discussion

  • Llama 3.1 achieved the highest SummaC score (0.752), ahead of all baselines; TextRank led the baselines at 0.648.Because long, morphologically complex Sinhala legal sentences challenge sentence-level NLI verification, the authors treat these SummaC scores as relative indicators.Source: §4.3 · pp. 7–8
  • Coverage 0.957–0.959 and Density 2.924–3.125 for SINBRIEF were below extractive baselines (1.000; 7.605–8.153) and abstractive baselines (0.855–0.913; 3.989–4.205).The authors interpret the lower Coverage and Density as less lexical overlap and greater novelty than any baseline.Source: §4.3 · p. 7, Table 1 · p. 8
  • SINBRIEF models scored 0.798–0.809 on Self-BertScore, compared with 0.802 for fine-tuned mT5 and 0.793 for BenSumm, approaching Lead-3 (0.850) and TextRank (0.859).The authors report this as semantic faithfulness despite lower lexical overlap.Source: §4.3 · p. 8

5 Conclusion

SINBRIEF combines domain-aware word graphs and sentence scoring to produce abstractive Sinhala legal summaries without annotated training data. It generates more abstractive summaries while maintaining semantic faithfulness, but continual pre-training did not improve scoring over base Llama 3.1.

5 Conclusion

  • SINBRIEF combines domain-aware word graph construction with sentence scoring to generate abstractive Sinhala legal summaries without annotated training data.Source: §5 · p. 8
  • Continual pre-training Llama 3.1 on Sinhala legal text produced a domain-adapted model but did not surpass base Llama 3.1 in sentence-scoring performance.Source: §5 · p. 8
  • Future work includes grammatical constraints for fluency, human evaluation with native Sinhala speakers, and extension to other low-resource languages.Source: §5 · p. 8

Limitations

The framework’s evaluation and generation have several acknowledged constraints, including grammatical errors, stochastic outputs, and limited coverage of ranking quality and legal-writing diversity.

Limitations

  • Generated summaries contain grammatical errors linked to random walks without grammatical constraints, and evaluation relies solely on quantitative metrics.The authors identify human evaluation as future work.Source: Limitations · p. 8
  • Random walks can produce different summaries across runs; computational constraints prevented multi-seed evaluation, and validation accuracy measures classification rather than ranking quality.Ranking graph-generated candidates is difficult to evaluate directly in this setting.Source: Limitations · p. 8
  • Frequent Sinling tokenisation and POS-tagging errors reduce graph-construction quality.Compression Ratio reflects the fixed length constraint rather than inherent model compression ability.Source: Limitations · p. 8
  • Without ablation studies, the separate contributions of domain-aware edge weighting and the legal keyword list remain unquantified; evaluation covers 1,206 Sinhala legal documents.That corpus may not capture the full diversity of legal writing styles.Source: Limitations · p. 9

A Continual Pre-Training of Llama 3.1

Llama 3.1 8B underwent continual pre-training on Sinhala legal text using a causal language-modelling objective, with parameter-efficient LoRA fine-tuning and a three-epoch training schedule.

A Continual Pre-Training of Llama 3.1

  • The documents were tokenised and segmented into 512-token chunks, and the causal language-modelling objective predicts each next token from preceding context.This describes the pre-training objective.Source: A Continual Pre-Training of Llama 3.1 · pp. 11–12
  • Because full fine-tuning was computationally infeasible, Llama 3.1 8B was loaded in 4-bit NF4 and adapted with LoRA adapters across major projection layers.The adapters used rank r=16 and scaling factor α=32, training a small fraction of the model’s parameters.Source: A Continual Pre-Training of Llama 3.1 · p. 12
  • Training ran for 3 epochs with batch size 4, AdamW, a 2 × 10−5 learning rate, and 100-step linear warmup.After continual pre-training, the LoRA weights were merged into the base model and saved in bfloat16 precision.Source: A Continual Pre-Training of Llama 3.1 · p. 12

B Training curves

All five sentence scorers show decreasing training loss across three epochs; Llama 3.1 has the lowest reported loss, while Laser has the highest.

B Training curves

  • All five models show decreasing training loss across three epochs, with Llama 3.1 lowest throughout and Laser highest throughout.The reported curves show no signs of divergence or instability.Source: Figure 6 · p. 12, B Training curves · p. 12
  • Llama 3.1 training loss falls from 0.281 in epoch 1 to 0.2259 by epoch 3, alongside the strongest validation accuracy of 90.05%.CPT Llama falls from 0.3118 to 0.2792, which the authors relate to continual pre-training on Sinhala legal text.Source: B Training curves · p. 12
  • mBert training loss decreases from 0.506 to 0.3744, while Falcon 7B decreases from 0.6424 to 0.4248.Laser decreases from 0.6322 to 0.5145, alongside the lower validation accuracy of 77.25%.Source: B Training curves · p. 12

C Final Summaries

The five SINBRIEF configurations produce different summaries from the same underlying candidate-generation pipeline. The examples also show persistent grammatical and fluency problems in the generated Sinhala text.

C Final Summaries

  • The five configurations share sentence extraction, clustering, word-graph construction, and candidate generation, but select different sentences.The paper says models with higher SummaC scores, Llama 3.1 and Falcon 7B, are expected to select candidates with stronger factual alignment; this is an expectation, not a reported outcome.Source: C Final Summaries · p. 12
  • Across all five summaries, grammatical errors, limited fluency, and reduced naturalness of Sinhala text are observable.The paper identifies these as limitations of graph-based random-walk generation.Source: C Final Summaries · p. 12

D.1 Word Frequency Analysis

Corpus-wide frequency analysis supports the relevance of the 19 selected Sinhala legal keywords across 1,206 documents, while showing substantial variation in their frequency. The keyword list began with a legal terminology reference and was extended through corpus inspection.

D.1 Word Frequency Analysis

  • පනත and නීතිය appear in over half of the documents, illustrating the broad presence of some selected terms.Source: §D.1 · p. 12
  • Despite its lower frequency, පැමිණිලිකරැ remains relevant to litigation, while තීන්දුව is associated with sentencing contexts.Source: §D.1 · p. 12
  • The keyword list was initially drawn from a Sinhala legal terminology reference and extended through manual corpus inspection.Source: §D.1 · p. 13
  • 24,293 occurrences for පනත; the other 18 keywords collectively account for thousands, while පැමිණිලිකරැ records 39.The analysis counts document frequency and total occurrences across the corpus.Source: Table 3 · p. 13, §D.1 · p. 13
  • All 19 keywords appear across a substantial portion of the 1,206-document corpus, supporting their domain relevance.The results are presented as a corpus frequency analysis.Source: §D.1 · pp. 12–13, Table 3 · p. 13

D.2 Human Validation

A legal-domain expert reviewed the selected keywords and confirmed the relevance of most to Sinhala legal discourse. Two terms were more strongly associated with constitutional contexts, but were retained based on their corpus frequency.

D.2 Human Validation

  • The expert confirmed the relevance of most selected keywords, while identifying බලය and ෙකාන්ෙද්සි as more strongly associated with constitutional than broader legal contexts.Source: §D.2 · p. 13
  • The expert review and corpus frequency analysis together support the keywords’ relevance to Sinhala legal text and their suitability for legal-domain adaptation.Source: §D.2 · p. 13

Related papers

Loading 2609.32397v1…