PaperlayerView exact revision on arXiv

Source-linked AI summary

LexDrafter: Terminology Drafting for Legislative Documents using Retrieval Augmented Generation

Ashish Chouhan, Michael Gertz

arXiv:2403.16295v1cs.CL

TL;DR

Different legal acts may define the same term differently, and the authors identify no tool for drafting legal definitions and terminology in context. LexDrafter builds definition elements from existing acts and uses RAG to identify or generate definitions; its evaluation reports high BERTScore but lower BLEU word overlap.

  • Problem

    Different legal acts may assign different definitions to the same term, and the authors report no tool for drafting definitions and terminology in the context of legal documents.

    Source: §1 · p. 1

  • Method

    LexDrafter extracts definition elements from EUR-Lex acts and uses RAG to retrieve relevant fragments and identify existing definitions or generate new ones for selected terms.

    Source: §1 · p. 2, §3 · p. 3, §4.2 · p. 6

  • Results

    Generated definitions show high BERTScore and lower 1-to-4-gram BLEU scores, indicating high semantic similarity but low word overlap with ground-truth definitions.

    Source: §5.2 · p. 7, §6 · p. 8

  • Takeaways & Limitations

    The framework aims to support harmonized legal definitions by identifying existing definitions or generating a new definition for a user-selected term.

    Source: §6 · p. 8

Abstract

from arXiv · show

With the increase in legislative documents at the EU, the number of new terms and their definitions is increasing as well. As per the Joint Practical Guide of the European Parliament, the Council and the Commission, terms used in legal documents shall be consistent, and identical concepts shall be expressed without departing from their meaning in ordinary, legal, or technical language. Thus, while drafting a new legislative document, having a framework that provides insights about existing definitions and helps define new terms based on a document's context will support such harmonized legal definitions across different regulations and thus avoid ambiguities. In this paper, we present LexDrafter, a framework that assists in drafting Definitions articles for legislative documents using retrieval augmented generation (RAG) and existing term definitions present in different legislative documents. For this, definition elements are built by extracting definitions from existing documents. Using definition elements and RAG, a Definitions article can be suggested on demand for a legislative document that is being drafted. We demonstrate and evaluate the functionality of LexDrafter using a collection of EU documents from the energy domain. The code for LexDrafter framework is available at https://github.com/achouhan93/LexDrafter.

1. Introduction

EU legal drafting must maintain consistent terminology, yet different acts may define the same term differently. LexDrafter addresses the need for support in drafting Definitions articles by retrieving existing definitions or generating new ones with RAG.

1. Introduction

  • EUR-Lex contained around 24,000 regulatory documents in 2022, of which 15% were legislative documents.Legislative documents include regulations, directives, decisions, recommendations, and opinions.Source: §1 · p. 1
  • Different legal acts may assign different definitions to the same term, while drafting legal documents requires manual effort and domain knowledge.The Joint Practical Guide and a steady increase in documents containing definitions motivate support for drafting definitions.Source: §1 · p. 1, Figure 1 · p. 1
  • The authors report that, to their knowledge, no tool helps draft definitions and terminology in the context of legal documents.They contrast this gap with tools for contract drafting and analysis and generative drafting of other documents.Source: §1 · p. 2
  • LexDrafter uses retrieval augmented generation to assist with drafting Definitions articles, retrieving relevant text fragments and generating definitions with large language models.The framework aims to harmonize definitions, reduce human errors, and reduce the time spent manually browsing corpora for consistency.Source: §1 · p. 2

2. Related Work

Prior work covers legal-definition extraction and legal document drafting, but the paper focuses on drafting Definitions articles for EUR-Lex legislative acts. Earlier extraction work uses linguistic or rule-based approaches, while document drafting includes templates, knowledge-based assembly, and RAG.

2.1. Extraction of Legal Definitions

  • Ferneda et al. (2012) and Hwang et al. (2018) used segmentation and POS tagging to extract legal terms, while Waltl et al. (2017) used rules to extract definitions and semantic information.The cited work reports that manual verification remains necessary for false positives and that definition extraction requires legal-theory work.Source: §2.1 · p. 2
  • Sai et al. (2023) extracted definitions and semantic relations from European legal acts using punctuation to identify terms and POS tags to extract explanations.Their analysis focuses on the Definitions article of a legal act.Source: §2.1 · p. 2

2.2. Document Drafting

  • Document drafting research includes template-based tools, Akoma Ntoso and rule-based assembly, and RAG applications for medical and news documents.The cited examples include Hotdocs, DocuPlanner, Virtual Court Action, ToXgene, FreeMarker, and work by Markey et al. (2024) and Ranade and Joshi (2024).Source: §2.2 · pp. 2–3
  • Legal text generation research is limited relative to traditional legal NLP, and contract clause generation is a prominent task.Lam et al. (2023) combine traditional and generative AI and evaluate clauses through retrieval of similar clauses and content similarity analysis.Source: §2.2 · p. 3
  • LexDrafter targets terminology drafting for EUR-Lex acts, extracting definitions and resolving external references to create definition elements for RAG-based identification or generation.This differs from prior work focused on extracting definitions and their semantic relations.Source: §2.2 · p. 3

3. Document Model and Definition Elements

LexDrafter models legal documents as ordered hierarchies of sections and builds definition elements from fragments that define legal terms. Each element records the term, its explanation, and any references to other definition elements.

3. Document Model and Definition Elements

  • The framework represents each legal document as an ordered sequence of sections, which may be decomposed into paragraphs or sentences while preserving element order and position.Definition fragments are sequences of tokens within a section, typically a sentence or sequence of sentences.Source: §3 · p. 3
  • A definition element is a tuple containing a unique identifier, the defined term, its explanation, and a possibly empty list of references to other definition elements.The term may be a single word or phrase, and the explanation is its definition.Source: §3 · p. 3
  • When an explanation uniquely defines a term, its reference list is empty; otherwise, the list contains the IDs of the referenced definition elements.Figure 2 depicts the related corpus-building workflow, including definition extraction and citation resolution.Source: §3 · p. 3, Figure 2 · p. 4

4. LexDrafter Framework

LexDrafter builds document and definition corpora from EUR-Lex acts, then helps users find existing definitions or generate new ones from contextual fragments. Terms needing definitions are selected by the user.

4. LexDrafter Framework

  • LexDrafter supports drafting when a document’s terminology article is missing; users select terms, and the system finds existing definitions or generates new ones.Automatic identification of terms needing definitions is not part of this work because it requires an understanding of legal theory.Source: §4 · p. 4

4.1. Creating Document and Definition Corpus

  • The first workflow preprocesses EUR-Lex acts into a document corpus and extracts definitions from Definitions articles into a definition corpus.DefExtract recognizes patterns specified in the Joint Practical Guide; definition elements can store references to cited definitions.Source: §4.1 · pp. 4–5

4.2. Identifying and Generating Definitions

  • TermRetriever shows a single existing definition, ranks multiple definitions higher when their EuroVoc descriptors resemble the draft document, or enables generation if none is found.Users decide which definition to cite when multiple definitions are retrieved.Source: §4.2 · p. 6
  • For new definitions, LexDrafter uses lexical search rather than dense embeddings to retrieve top-k fragments, reducing noise from fragments with merely similar terms.It ranks retrieved fragments by term frequency and passes the selected fragments with the term to an LLM generator.Source: §4.2 · p. 6
  • The generation prompt requests a clear, context-specific definition of 25 to 45 words, near the corpus average of approximately 32 words.The prompt supplies the term and retrieved sentences and requires JSON output.Source: §4.2 · pp. 6–7

5. Experiments

In the Energy-domain evaluation, Vicuna marginally outperformed LLaMA-2 across three metrics. Generated definitions showed lower word overlap but higher semantic similarity to ground-truth definitions.

5. Experiments

  • The Energy-domain corpus contained 539 processed HTML legal acts and 1,330 definition fragments covering 1,007 legal terms.Of these terms, 787 had one definition element and 220 had multiple definition elements across 543 elements.Source: §5 · p. 7

5.1. Large Language Models (LLMs)

  • Vicuna-7b-v1.5 and LLaMA-2-7b were run as memory-efficient 16-bit quantized models with a 4096-token context length.The evaluation used a Google Colab V100 GPU and set temperature to 0.2, top_k to 20, top_p to 0.6, and repetition penalty to 1.2.Source: §5.1 · p. 7

5.2. Results

  • The evaluation compared generated definitions with human-written ground-truth definitions using BLEU, BERTScore, and BLEURT.BLEU measures n-gram overlap, BERTScore semantic similarity, and BLEURT features including overlap, semantic similarity, and fluency.Source: §5.2 · p. 7
  • BLEU scores were lower because generated definitions differed from ground truth in styling and vocabulary, while BERTScores were higher, indicating semantic similarity and content coverage.BLEURT scores differed marginally; the authors attribute lower scores to BLEURT’s training on non-legal text and report that legal-domain fine-tuning is ongoing.Source: §5.2 · pp. 7–8
  • Vicuna marginally outperformed LLaMA-2 on BLEU, BERTScore, and BLEURT in definition generation.Table 1 reports n-gram overlap, semantic similarity, and measures including fluency.Source: §5.2 · p. 8, Table 1 · p. 8

6. Conclusions and Ongoing Work

LexDrafter combines definition extraction and retrieval with RAG-based generation to support Definitions articles in EU legal acts. Its evaluation found high semantic similarity but low word overlap, and the authors identify broader-domain coverage and automatic term detection as ongoing work.

6. Conclusions and Ongoing Work

  • LexDrafter uses RAG and existing legal-act definitions to identify definitions or generate new ones for user-selected terms.The framework aims to harmonize legal definitions across acts, including cases where definitions are outliers.Source: §6 · p. 8
  • Generated definitions showed high BERTScore but lower 1- to 4-gram BLEU, indicating high semantic similarity alongside low word overlap with ground truth.The evaluation compared generated definitions with ground-truth definitions for selected terms.Source: §6 · p. 8
  • Ongoing work includes automatically identifying terms needing definitions and expanding coverage to legal acts across all EUR-Lex domains.The stated aim is to support cross-domain harmonization by automating Definitions-article drafting.Source: §6 · p. 8

Broader Impact & Ethical Issues

The authors report no clear ethical concerns with releasing LexDrafter’s code or with the legal-act data used in the work.

Broader Impact & Ethical Issues

  • The authors report no clear privacy or harmful-text concerns for the reviewed EU legal acts and note that EUR-Lex data is licensed under CC BY 4.0.They state that the acts have undergone review within EU institutions and that they see no ethical concerns in releasing the framework code.Source: Broader Impact & Ethical Issues · p. 8

A. Examples

The examples compare original legal definitions with LexDrafter generations, showing outputs that range from close paraphrases to definitions with substantially different scope or framing.

A. Examples

  • For “rules,” the generated definition recasts the original Supply Agency rules for balancing demand and supply as procedures and guidelines for regular, equitable supply.Source: A. Examples · p. 10
  • For “generic ecodesign requirement,” the generated definition shifts from a requirement based on a product’s overall ecological profile to a method for improving environmental performance.Source: A. Examples · p. 10
  • For “refrigerating appliance,” the generated definition replaces the original cabinet and compartment description with a heat-transfer device that may store food or drinks.Source: A. Examples · pp. 10–11
  • For “abandoned land,” the generated definition broadens the original from previously cultivated land stopped due to constraints to land unused or neglected for an extended period for several reasons.Source: A. Examples · p. 11
  • For “adequacy,” the original defines ability of in-feeds to meet area load, while the generated definition describes matching electricity supply and demand in a control area.Source: A. Examples · p. 11
  • For “aggregates,” the original concerns national-level data on energy products and energy-system characteristics, while the generated definition describes quantities used as raw materials and related energy measures.Source: A. Examples · p. 11
Loading 2403.16295v1…