PaperlayerView exact revision on arXiv

Source-linked AI summary

GreenLeaf Law Embed Tiny: A Compact Embedding Model for Legal Domain Retrieval

Surya Saka

arXiv:2608.24936v1cs.LGcs.AIcs.CL

TL;DR

Legal retrieval needs accurate embeddings that handle specialized semantics while remaining efficient and transparent for privacy-sensitive deployment. The paper trains a 0.6B-parameter model with distillation, legal-domain fine-tuning, hard negative mining, and curated data, achieving competitive legal benchmark performance. Its results support domain-specific training as a way to improve specialized retrieval with a compact model.

  • Problem

    Legal retrieval requires specialized handling, but existing options include opaque proprietary systems and large models that demand substantial computational resources.

  • Method

    The paper combines two-stage teacher-to-student distillation, legal-domain fine-tuning with hard negative mining, a curated 3.4M-pair corpus, and quantization-aware inference.

  • Results

    75.11% on MLEB is achieved by the 0.6B-parameter model, with competitive performance on selected legal retrieval tasks.

  • Takeaways & Limitations

    Domain-specific techniques, including high-quality data, hard negative mining, and architectural adaptations, can improve performance for specialized legal retrieval applications.

  • Takeaways & Limitations

    Larger models maintain advantages in certain legal retrieval tasks.

Abstract

from arXiv · show

We present GreenLeaf Law Embed Tiny, a 0.6B parameter embedding model for legal domain retrieval. GreenLeaf-Tiny achieves 75.11% on the Massive Legal Embedding Benchmark (MLEB) and 64.38% on MTEB(Law, v1),demonstrating competitive performance among models under 1B parameters. Our approach combines a two-stage training pipeline that first distills knowledge from a larger teacher model into a compact student architecture, then applies domain-specific fine-tuning with hard negative mining; a carefully curated dataset of 3.4 million query-passage pairs, including 150,000 human-curated samples across diverse legal jurisdictions; and an efficient inference architecture supporting multiple quantization levels (BF16, INT8, binary) enabling deployment in resource-constrained environments. We provide detailed analysis of our training methodology, architectural choices, and comprehensive evaluation across legal retrieval tasks. Our results demonstrate that domain-specific training with high-quality data can improve performance for specialized domain applications

I. INTRODUCTION

The paper targets accurate, transparent, and efficient legal retrieval by introducing a compact embedding model trained with domain-specific data and a two-stage pipeline.

  • Motivation: Legal retrieval is difficult because documents contain hierarchical structures, specialized terminology, and intricate cross-references that general-purpose models struggle to capture.The paper also emphasizes the consequences of missed precedents or misinterpreted statutes.
  • Motivation: Proprietary legal embedding systems can be accurate but opaque, leaving training data, methodology, reproducibility, and privacy concerns unresolved.These concerns are especially relevant when legal documents are confidential.
  • Results: 75.11% on MLEB is achieved by the 0.6B-parameter model, with domain-specific training producing competitive performance on selected legal retrieval tasks.The reported findings also include a +9.24-point gain from distillation followed by fine-tuning over direct fine-tuning alone.
  • Approach: The paper introduces a two-stage pipeline combining teacher-to-student knowledge distillation with legal-domain fine-tuning and hard negative mining.The pipeline is adapted to legal characteristics including hierarchical document structure and citation networks.
  • Data: 3.4M query-passage pairs, including 150K human-curated samples, form the JudicialMind Legal Corpus for legal retrieval training.The corpus analysis covers composition, quality filtering, and jurisdiction balancing across 35 languages and 40+ jurisdictions.
  • Deployment: BF16, INT8, and binary inference support deployment across hardware constraints with documented accuracy-efficiency trade-offs.The reported INT8 setting reduces memory footprint by 4× with a -0.3% performance degradation.

II. RELATED WORK

Prior embedding research established efficient transformer retrieval and large-scale general-purpose training, while legal retrieval exposed a need for compact, transparent, domain-specialized models.

  • General Embeddings: Word2vec, GloVe, and transformer encoders established progressively richer text representations, while Sentence-BERT enabled efficient bi-encoder retrieval.Bi-encoders support pre-computed embeddings, unlike more expensive cross-encoder interactions.
  • General Embeddings: E5, BGE, and GTE use massive text-pair training with strategies including contrastive learning, instruction tuning, and synthetic data generation.MTEB became a standard benchmark, but its general-domain focus limits conclusions for specialized domains.
  • Legal Retrieval: Legal retrieval combines long documents, complex cross-references, and specialized terminology, limiting lexical methods for semantic matching and paraphrase detection.BM25 remains competitive for exact-match queries but does not address these semantic cases well.
  • Legal Retrieval: Legal embedding efforts include Law2Vec, voyage-law-2, and MLEB, which evaluates caselaw, contracts, and regulations across jurisdictions.The cited approaches span adapted word embeddings, specialized commercial models, and legal-specific benchmarking.
  • Research Gap: A gap remains for compact, efficient legal embedding models suitable for privacy-sensitive deployment rather than closed-source or computationally demanding systems.This boundary motivates the paper’s focus on a smaller legal embedding model.
  • Distillation: Earlier distillation work showed that bi-encoders could retain 95%+ of teacher performance with 5–10× fewer parameters, mainly on general-domain tasks.The paper extends this direction toward specialized legal semantics.
  • Distillation: The paper differs by combining distillation with subsequent domain fine-tuning, legal-specific architecture, and hard negative mining during adaptation.This combination is presented as distinct from using distillation alone.

D. Hard Negative Mining

The training procedure uses legal-aware hard negatives, curated and filtered data, and decontamination steps to improve retrieval discrimination while limiting evaluation overlap.

  • Hard Negative Mining: Hard negatives resemble positives while remaining semantically distinct, improving representation quality and discrimination in contrastive retrieval training.Prior systems mined such negatives with the model being trained to create a curriculum.
  • Hard Negative Mining: Legal-specific mining targets jurisdiction, temporal, and doctrinal confusion by selecting similar terminology from distinct legal contexts.The negatives come from different jurisdictions, time periods, or related but distinct doctrines.
  • Corpus: The training corpus contains 3.4 million legal query-passage pairs covering multiple query types, legal domains, difficulty levels, jurisdictions, and 35 languages.Its sources include case law, statutes, contracts, and regulatory text.
  • Data Controls: Decontamination removes exact matches and near-duplicates, separates bar-exam sources by jurisdiction and time, and applies a temporal cutoff.Near-duplicates are identified using MinHash with 128 permutations and a Jaccard threshold of 0.8.
  • Data Controls: Filtering removes low-similarity pairs, deduplicates 12% of initial pairs, and balances document lengths through stratified sampling.The reported target distribution is 25% short, 35% medium, and 40% long documents.
  • Human Curation: 150,000 human-curated pairs were produced by legal professionals and verified by licensed attorneys across 50+ practice areas and 25 jurisdictions.The two-stage verification achieved 94% inter-annotator agreement.
  • Human Curation: Ablation studies report measurable value from the human-curated subset despite its small share of the training corpus.The paper attributes breadth and precision to combining large-scale data with human curation.

B. Data Characteristics

The corpus and training design span varied passage lengths and use a two-stage knowledge-transfer process, with distillation preserving much of teacher performance.

  • Corpus Characteristics: Legal passage lengths show modes near 256 tokens for statutory references, 512 tokens for contractual clauses, and above 1024 tokens for judicial opinions.These modes reflect distinct document types in legal retrieval.
  • Knowledge Distillation: Embedding matching and similarity preservation are used during distillation so the compact student captures the teacher’s general knowledge.The distillation stage transfers semantic understanding from the large teacher model.
  • Corpus Characteristics: Figure 1 depicts the distribution of passage lengths in the training corpus.The supplied figure text identifies the distribution but does not provide a visual comparison outcome.
  • Training Pipeline: Figure 2 presents a two-stage pipeline in which a large teacher transfers general knowledge before legal adaptation with hard negative mining.This sequence is designed to maximize knowledge transfer while adapting to legal-domain specifics.
  • Knowledge Distillation: 94.2% of teacher performance on general-domain benchmarks is retained after the distillation process.This figure concerns the distilled student before the later domain adaptation stage.

C. Domain Fine-Tuning

The legal adaptation stage combines contrastive learning with jurisdiction-aware hard negatives and domain-specific document encoding. The compact transformer supports flexible precision for deployment trade-offs.

  • Domain fine-tuning: Hard negative mining selects documents from the same jurisdiction, legal domain, and time period to improve fine-grained discrimination.The second training stage applies contrastive learning to the 3.4M-pair legal corpus.
  • Domain fine-tuning: Hierarchical encoding splits documents exceeding 512 tokens into overlapping chunks, then combines chunk representations with attention-weighted pooling.Citation context and jurisdiction embeddings add legal-domain information to passage representations.
  • Architecture: The 0.6B-parameter encoder uses 28 transformer layers, 1024 hidden dimensions, 16 attention heads, and a 2048-token context length.Jurisdiction and temporal embeddings provide additional domain-specific inductive biases.
  • Inference optimization: BF16, INT8, and binary inference provide documented accuracy-efficiency trade-offs for deployment across hardware constraints.The inference architecture is presented as a complete pipeline with flexible quantization options.

VI. EVALUATION

The evaluation covers legal retrieval benchmarks spanning caselaw, contracts, and regulatory search, with GreenLeaf-Tiny showing competitive category-level performance. Kanon 2 Embedder leads the reported categories.

  • Evaluation setup: MLEB includes 10 datasets covering caselaw retrieval, contract analysis, and regulatory search across 6 jurisdictions, using NDCG@10 as the primary metric.The evaluation also includes MTEB(Law, v1) and compares general-domain and legal-specific baselines.
  • Main results: 82.15% in contracts and 85.73% in regulation are reported for GreenLeaf-Tiny on MLEB category evaluations.These results are described as competitive performance in the contracts and regulation categories.
  • Main results: Kanon 2 Embedder leads all reported categories, including 91.5% in regulation.The cited evaluation passage specifically highlights its strength in regulation.

D. Per-Task Analysis

Per-task results are strongest on structured regulatory text and weakest on multilingual, long-document case retrieval. Ablations attribute gains to the two-stage pipeline, legal-aware negatives, and selected data components.

  • Per-task results: 93.43% on gdpr-holdings and 91.89% on uk-legislative-long-titles are the strongest reported per-task results.Both tasks involve structured regulatory text.
  • Per-task results: 41.27% on echr-retrieval is the weakest reported result, involving multilingual European Court of Human Rights cases and long documents.The passage identifies complex multilingual aspects and document length as characteristics of this task.
  • Training-stage ablation: +9.24 points over direct fine-tuning result from combining the two training stages.The ablation study reports that both stages contribute to the final performance.
  • Negative-sampling ablation: +3.59 points over random hard negatives and +6.28 points over random sampling result from legal-aware hard negatives.The comparison concerns alternative negative-sampling strategies.
  • Data-component ablation: +8.2 points from human-curated data despite its 4.4% share of total data; citation context adds +2.68 points and jurisdiction embeddings +1.75 points.These gains are reported when the human-curated data is combined with the primary dataset.

F. Quantization Impact

Quantization trades small retrieval-performance losses for lower memory use and faster inference. INT8 offers a modest-loss option, while binary quantization targets more aggressive compression.

  • Precision trade-offs: INT8 quantization incurs only -0.31% performance loss on MLEB while providing an accuracy-efficiency trade-off.The broader precision comparison evaluates performance across benchmarks.
  • Precision trade-offs: Binary quantization retains 97.9% of full-precision performance.Binary precision is included in the deployment trade-off comparison.
  • Multilingual performance: Cross-lingual transfer is strongest between legal systems with shared origins: Romance languages reach ∼90% of English performance, versus 80–82% for East Asian languages.The passage attributes the difference to linguistic distance and differing legal traditions.

A. Production Deployment Patterns

GreenLeaf-Tiny supports cloud, on-premises, edge, and software-stack deployments through compact sizing, quantization, multiple interfaces, and flexible retrieval architectures. A binary-first, BF16/INT8-reranking pipeline preserves 94% of single-stage accuracy while reducing computational requirements by 8×.

  • Deployment environments: Cloud, on-premises, and edge deployments are supported through compact sizing and flexible quantization.The model is positioned for cloud platforms, confidential local infrastructure, and offline field-device search.
  • Cloud deployment: A single NVIDIA T4 serves 2,400 queries per second at BF16 or 6,700 queries per second at INT8.The stateless architecture supports horizontal scaling across GPU clusters without coordination overhead.
  • Edge deployment: Binary quantization enables on-device legal search with 128 bytes per embedding for mobile devices and laptops.The passage connects this capability to searching case law and statutes without network connectivity.
  • Integration interfaces: A SentenceTransformers-compatible API and REST API support integration with legal research and document-management systems.The model also provides fixed-dimension, L2-normalized outputs for vector-database search.
  • Two-stage retrieval: Binary candidate generation followed by BF16 or INT8 reranking is recommended for balancing retrieval efficiency and accuracy.The first stage retrieves 100-1000 candidates with high recall before precise ranking on the reduced set.
  • Two-stage retrieval: 94% of single-stage accuracy is achieved while computational requirements decrease by 8×.This architecture is described as particularly valuable for large-scale legal research platforms processing millions of queries daily.

D. Training Infrastructure

The paper reports a 420-GPU-hour training pipeline and attributes compact-model performance to high-quality supervision, hard negatives, and legal-domain architectural adaptations. Larger models nevertheless retain advantages on some legal retrieval and reasoning tasks, while the model adds interpretability mechanisms.

  • Training infrastructure: 420 GPU hours on 8× H100 infrastructure complete the training pipeline.
  • Performance factors: 150K human-curated pairs provide high-precision supervision within the legal retrieval training data.The paper identifies data quality as one of three contributors to compact-model performance.
  • Performance factors: Hard negative mining targets fine-grained distinctions between highly similar legal documents.The paper presents hard negatives as a factor contributing to performance on legal retrieval tasks.
  • Performance factors: Jurisdiction embeddings and citation-aware processing provide domain-specific architectural inductive biases.
  • Scale comparison: Kanon 2 Embedder (1.8B) outperforms GreenLeaf-Tiny by 6.79 points on MLEB, while Dinghy Law 8B leads by 8.20 points on MTEB(Law).The paper associates scale advantages particularly with caselaw retrieval requiring complex inference.
  • Benchmark comparisons: GreenLeaf-Tiny achieves 75.11% on MLEB versus 79.6% for Voyage-law-2 and 70.8% for OpenAI text-embedding-3-large.The reported comparisons show different outcomes against commercial legal-specialized and general-purpose embedding models.
  • Interpretability: Attention visualization and similar-training-example retrieval provide mechanisms for inspecting model behavior and explaining retrieved documents.The paper connects these mechanisms to transparency for legal decision-making.

D. Ethical Considerations and Responsible Deployment

GreenLeaf-Tiny emphasizes privacy-preserving local deployment, bias mitigation, and human oversight for high-stakes legal use. The paper also identifies longer-context, multilingual, and continual-learning extensions as future directions, while noting that cross-lingual performance remains incomplete.

  • Privacy: Compact size enables on-premises deployment without transmitting confidential legal documents to external APIs.Quantization further supports edge deployment on local devices.
  • Bias mitigation: The training corpus is analyzed for overrepresentation across jurisdictions, legal traditions, and demographic perspectives.Human-curated data includes diversity requirements across legal systems, practice areas, and geographic regions.
  • Responsible deployment: Human oversight is recommended for high-stakes applications such as litigation strategy and legal advice generation.The model is intended to augment rather than replace professional legal judgment.
  • Future directions: Sparse attention could extend context lengths to complete judicial opinions without hierarchical encoding.The paper also proposes multimodal extensions incorporating headings, citations, and footnotes.
  • Future directions: Continual learning could incorporate new legal developments without full retraining, while federated learning could preserve proprietary work-product confidentiality.
  • Multilingual scope: German and French legal text reaches 91% of English performance, but low-resource-language retrieval requires further improvement.The paper identifies cross-lingual legal retrieval as an active research direction.
  • Scope and implications: The conclusion links domain-specific techniques to specialized domains where privacy, cost, or latency constraints favor compact models.Future work includes multilingual improvement and continual learning for evolving legal doctrine.
Loading 2608.24936v1…