PaperlayerView exact revision on arXiv

Source-linked AI summary

Statutory Article Retrieval from Arabic Judicial Narratives: Family Law and Law of Evidence in Saudi Arabia

Zarah Shibli, Areej Alhothali, Lulwah Alharigy

arXiv:2610.38336v1cs.CLcs.AI

TL;DR

Long Arabic judicial narratives must be matched to concise statutory rules, and the effectiveness of retrieval for this setting is underexamined. The paper evaluates retrieval methods on Saudi Family Law and Law of Evidence articles, including a Decompose-and-Refine approach. DaR retrieves Evidence articles where non-decomposed configurations do not do so well, and the same retriever is best on both laws under DaR.

  • Problem

    Long judicial narratives contain procedural history and legally irrelevant detail, while statutory articles express concise rules, and article-level retrieval for long, cross-law Saudi Arabic narratives remains underexamined.

    Source: §I · p. 1

  • Method

    The paper evaluates sparse, dense, and hybrid retrieval, per-law restriction, and an adaptation of Decompose-and-Refine for article-level retrieval.

    Source: §I · p. 1, C. Retrieval Configurations · p. 3

  • Results

    Under Decompose-and-Refine, the same retriever achieves the best MRR@10 on both laws, while no non-decomposed configuration retrieves Evidence articles well.

    Source: §IV · p. 4, §V · p. 7

  • Takeaways & Limitations

    Whole-corpus scores can hide weak Evidence retrieval, and Family and Evidence scores are not directly comparable because their pools, references per case, and scored case counts differ.

    Source: §IV · p. 4, B. E3: Per-Law Restriction and RRF… · p. 5

    The benchmark is synthetic, limited to two Saudi laws and divorce-establishment scenarios, and uses citation-derived labels that may omit applicable uncited provisions.

    Source: C. E4: Decompose-and-Refine (RQ3) · p. 6

Abstract

from arXiv · show

Reliable legal information retrieval must ground legal reasoning in authoritative statutory sources. In Arabic judicial settings, long factual narratives must be mapped to concise articles that express abstract legal rules, creating a narrative-to-legal-rule abstraction gap. Using a benchmark of 200 synthetic Saudi Arabic judicial cases and 381 articles from the Family Law and the Law of Evidence, we compare word-level and char-4 BM25, zero-shot dense encoders, per-law retrieval, dense-sparse reciprocal rank fusion (RRF), and a proposed Decompose-and-Refine (DaR) pipeline. In DaR, a large language model splits each narrative into disputed legal issues and generates, for each issue, three retrieval channels: a statute-style paraphrase, legal keywords, and a candidate taxonomy chapter. On the whole corpus, the instructed Qwen3-Embedding-8B dense encoder reaches an MRR@10 of 0.849, the best result of any non-decomposed system. Under per-law scoring, this advantage reflects Family Law only: restricted to Evidence articles, the same encoder drops to an MRR@10 of 0.087, and fusion with BM25 raises it only to 0.207. DaR changes this picture, raising Evidence MRR@10 to 0.696 and Family MRR@10 to 0.864. Without decomposition, the two laws favor different strategies; under DaR, the same retriever achieves the best MRR@10 on both.

I. Introduction

The paper examines article-level retrieval for long Saudi Arabic judicial narratives, where detailed facts must be matched to concise statutory rules. It evaluates sparse, dense, hybrid, and decomposed retrieval, addressing whether the two laws need different strategies.

I. Introduction

  • Judicial narratives contain procedural history and legally irrelevant detail, while statutory articles state concise rules, conditions, and consequences.The authors call this mismatch the narrative-to-legal-rule abstraction gap.Source: §I · p. 1
  • The study evaluates sparse, dense, and hybrid article retrieval on Saudi Arabic narratives and adapts Decompose-and-Refine to long narratives.Its contributions include sixteen zero-shot encoders and evidence that the two laws behave differently under the same retrievers.Source: §I · p. 1

II. Related Work

  • Prior Saudi legal NLP work targets tasks such as classification, summarization, and judgment prediction, not article-level retrieval for Family Law or divorce-establishment narratives.The authors note that existing Saudi studies do not target this retrieval task.Source: §II · p. 1
  • Earlier Arabic legal retrieval studies use short queries within one topical domain, leaving long, cross-law Saudi judicial narratives underexplored.The narratives average 265 words and may cite two laws.Source: §II · p. 2
  • The paper adapts Decompose-and-Refine to long narratives by splitting them into disputed issues and using three per-issue retrieval channels fused by RRF.The channels are paraphrastic, lexical, and taxonomy-prior.Source: §II · p. 2

A. Dataset and Corpus

The benchmark uses 200 synthetic Arabic judicial narratives about divorce-establishment scenarios and a 381-article Saudi law corpus. Its long Fact fields must be matched to much shorter statutory articles.

A. Dataset and Corpus

  • 200 synthetic Arabic judicial narratives cover explicit, implicit, and conditional divorce-establishment scenarios, using the source corpus without regenerating cases or citations.The source corpus validated the cases through legal-expert review.Source: A. Dataset and Corpus · p. 2
  • Queries use each case’s Fact field, while citation labels come from Reasoning and Judgment, keeping query and label text disjoint.The formulaic Preface is excluded from the query.Source: A. Dataset and Corpus · p. 2
  • Fact fields average 265 words versus about 41 words per article, a roughly 6.5:1 length gap that requires mapping substantial detail to a concise rule.Articles average about 41 words and are single-topic.Source: A. Dataset and Corpus · p. 2

B. Problem Formulation

The task ranks statutory articles for each case query and evaluates the ranking against the articles cited for that case. Each article remains an intact retrieval unit.

B. Problem Formulation

  • The corpus contains 381 articles and 200 case queries, with each query defined as the case’s Fact field and retrieval evaluated at a cutoff K.The returned list is ranked by article position.Source: B. Problem Formulation · p. 2
  • Ground-truth labels are the articles cited by each case, so retrieval is treated as a multi-label task.A case can cite multiple articles.Source: B. Problem Formulation · p. 2
  • Each article is one retrieval unit without chunking, preserving its conditions, exceptions, and consequences; dense representations prepend the law and chapter context.The representation uses law name, chapter title, and article body.Source: B. Problem Formulation · p. 2

C. Retrieval Configurations

The retrieval configurations progress from lexical and dense baselines to per-law restriction, rank fusion, and Decompose-and-Refine. DaR generates multiple queries per disputed issue and merges their retrieved rankings into a case-level list.

C. Retrieval Configurations

  • The study compares word-level BM25 and char-4 BM25 after Arabic normalization, alongside zero-shot dense encoders that encode each Fact field as one query.The dense queries and article embeddings are L2-normalized and scored by exact cosine similarity, without fine-tuning.Source: C. Retrieval Configurations · p. 2
  • The enhancement experiments test per-law restriction and reciprocal rank fusion, including lexical-only, dense–sparse, and four-way whole-corpus combinations.RRF combines rankings without training; the dense–sparse pairs use instructed 8B or 4B encoders with word BM25.Source: C. Retrieval Configurations · pp. 2–3
  • Per-law restriction limits retrieval to one law’s articles at a time, avoiding dilution by articles from the other codification and establishing the DaR comparison baseline.The pools contain 252 Family articles or 129 Evidence articles.Source: C. Retrieval Configurations · p. 3
  • DaR adapts Decompose-and-Refine to long narratives by identifying legally distinct disputes independently for each law route.A constrained LLM returns one JSON entry per dispute, with retrieval channels for each issue.Source: C. Retrieval Configurations · p. 3
  • For each issue, DaR retrieves a statute-style provision paraphrase, legal keywords, and a candidate chapter from a fixed taxonomy.The provision channel paraphrases the governing rule; the keywords channel requests 5–10 terms, and the candidate channel supplies a chapter and issue label.Source: C. Retrieval Configurations · p. 3
  • The LLM produces query formulations rather than retrieving or ranking articles, using a fixed JSON schema for issue entries.The prompt has separate law-route versions with route-specific persona, issue type, and chapter taxonomy.Source: C. Retrieval Configurations · p. 3
  • Each issue’s three rankings are fused by RRF, then article scores are merged across issues by taking each article’s highest fusion score.Articles remain in contention if retrieved as relevant to at least one issue; rankings are truncated at K.Source: Figure 1 · p. 3, C. Retrieval Configurations · p. 4
  • The decomposition model is Gemini 3.6 Flash at temperature 0, selected for speed and cost efficiency across two calls per case.In a pilot comparison, it recovered a similar set of retrieval-relevant issues to larger models with lower latency.Source: C. Retrieval Configurations · p. 4

D. Embedding Models and Evaluation

The evaluation compares retrieval systems separately for Family Law and the Law of Evidence, using only cases that cite the law being scored. It reports ranking and retrieval metrics across multiple cutoffs and carries the strongest instructed Qwen3 encoders into DaR.

D. Embedding Models and Evaluation

  • Metrics are macro-averaged over 200 Family cases and 161 Evidence cases, with each system evaluated on identical case sets within each law.Whole-corpus rankings are filtered by law; E3a and E4 retrieve only from the law under evaluation.Source: D. Embedding Models and Evaluation · p. 4
  • Sixteen zero-shot encoders span Arabic-specific, multilingual contrastive, proprietary API, and decoder-based embedding paradigms.Only the two strongest E2 encoders, instructed Qwen3-Embedding-4B and 8B, are carried into DaR.Source: D. Embedding Models and Evaluation · p. 4
  • E1–E4 are evaluated at K=1,3,5,7,10,20,50 using Precision, Recall, F1, Hit Rate, and MRR, macro-averaged over eligible cases separately for Family Law and Law of Evidence.Source: D. Embedding Models and Evaluation · p. 4

IV. Results and Discussion

Per-law scoring reveals that whole-corpus performance conceals weak Evidence retrieval. DaR is the only reported configuration that retrieves Evidence articles well, while the two laws favor different strategies without decomposition.

IV. Results and Discussion

  • DaR is the only configuration reported to retrieve Evidence articles well; without decomposition, no method does.The study uses MRR@10 as its main ranking metric and reports R@10 and sometimes R@20 alongside it.Source: §IV · p. 4
  • Whole-corpus scores hide an Evidence retrieval failure, motivating separate evaluation of the two laws.Family metrics average 200 cases, while Evidence metrics average the 161 cases that cite an Evidence article.Source: §IV · p. 4
  • Without decomposition, the two laws behave differently under retrieval, whereas DaR retrieves articles well for both.Source: §IV · p. 4

A. E1–E2: Sparse and Dense Retrieval (RQ1)

On the whole 381-article corpus, instructed Qwen3-Embedding-8B leads the zero-shot encoders, but the apparent advantage is concentrated in Family Law. Evidence scores remain near zero for the encoders.

A. E1–E2: Sparse and Dense Retrieval (RQ1)

B. E3: Per-Law Restriction and RRF (RQ1, RQ2)

Whole-corpus fusion does not surpass the best dense encoder, while per-law restriction reveals different retrieval patterns for Family and Evidence. Fusion is therefore not a global default, and cross-law scores are not directly comparable.

B. E3: Per-Law Restriction and RRF (RQ1, RQ2)

  • 0.740 MRR@10 is the best whole-corpus RRF result, below instructed 8B at 0.849; lexical-only fusion also trails word BM25, 0.258 vs. 0.361.Four-way fusion matches instructed 8B on R@20 (0.634 vs. 0.635), but not R@10 or Hit@10.Source: B. E3: Per-Law Restriction and RRF… · p. 4, Table III · p. 5
  • 0.850 MRR@10 makes a single dense encoder best on Family, while Evidence's best non-decomposed MRR@10 is 0.207 for 8B-plus-word-BM25 fusion.On Evidence, the 4B encoder has the highest recall: R@10 0.300 and R@20 0.507.Source: B. E3: Per-Law Restriction and RRF… · p. 4
  • Fusion's effect depends on the law and metric, and Family and Evidence scores are not like-for-like because their pools, references per case, and scored-case counts differ.Source: B. E3: Per-Law Restriction and RRF… · p. 5

C. E4: Decompose-and-Refine (RQ3)

DaR substantially improves Evidence retrieval and slightly raises the best Family MRR@10, while its effects vary by retriever. Under DaR, char-4 BM25 is best on both laws, but the gains cannot be attributed to decomposition alone.

C. E4: Decompose-and-Refine (RQ3)

  • DaR with the 4B encoder has the highest Family R@10 (0.778) and the highest Evidence R@20 (0.923).DaR’s Family R@10 of 0.726 remains below the best baseline’s 0.764, while the 4B encoder reaches 0.778.Source: C. E4: Decompose-and-Refine (RQ3) · p. 5
  • The gains are attributed to the DaR pipeline as a whole, not decomposition alone, because its query generation, issue splitting, and taxonomy prior were not isolated.The authors identify channel-level ablations and an undecomposed control as needed to isolate the components’ contributions.Source: C. E4: Decompose-and-Refine (RQ3) · pp. 5–6
  • Evidence DaR with char-4 BM25 reaches MRR@10 0.696 and R@10 0.752; Family reaches MRR@10 0.864 versus the best baseline’s 0.850.Evidence gains are about 3.4 and 2.5 times the best non-decomposed MRR@10 and R@10 values, respectively; Family R@10 is 0.726 versus the best baseline’s 0.764.Source: C. E4: Decompose-and-Refine (RQ3) · p. 5, Table V · p. 6
  • In one case, DaR retrieves all three reference Family articles at K=10 versus two for baseline, and surfaces the digital-evidence article at rank 28 of 50.Neither path ranks that Evidence article in its top 10, and the baseline does not retrieve it within its top 50; the single case cannot establish significance.Source: C. E4: Decompose-and-Refine (RQ3) · p. 6, Figure 3 · p. 6
  • The benchmark uses synthetic narratives, covers two Saudi laws and divorce-establishment scenarios, and uses only one decomposition run per case.The authors note that synthetic cases may not capture real judicial-record variability and that run-to-run variance is unknown.Source: C. E4: Decompose-and-Refine (RQ3) · p. 6
  • With the retriever held fixed, DaR helps lexical systems on both laws and dense systems on Evidence, but lowers dense Family MRR@10 by about 0.12–0.13.Dense Family R@10 changes by at most 0.021, so the decline reflects first-correct-article ranking rather than coverage; char-4 BM25 is best on both laws under DaR.Source: C. E4: Decompose-and-Refine (RQ3) · p. 5, Table VII · p. 6

V. Conclusion

The conclusion finds that dense retrieval’s overall strength reflects Family Law, while Evidence remains difficult without decomposition. DaR improves Evidence retrieval and yields a small increase in the best Family MRR@10, with synthetic data and limited legal scope constraining the findings.

V. Conclusion

  • The strongest dense encoder’s overall result reflects Family Law; Evidence retrieval remains weak even after fusion with BM25, while DaR substantially improves Evidence and slightly raises the best Family MRR@10.Without decomposition, the two laws favor different strategies; under DaR, the same retriever is best on both.Source: §V · pp. 6–7
  • The benchmark’s limitations include synthetic cases, citation-derived labels, only two laws and divorce-establishment scenarios, and single-run decomposition.The Evidence-route cost on cases without Evidence citations is also unmeasured.Source: §V · p. 7
  • Future work will expand the benchmark with more cases and Saudi laws, and repeat decomposition runs to assess variance.Source: §V · p. 7

Related papers

Loading 2610.38336v1…