Source-linked AI summary
Figurative Justice: Detecting metaphors in Hindi judgements with qualitative assessment and transformers
Bhumika Bhattacharyya, Shouvik Kumar Guha, Indranil Dutta
TL;DR
Hindi legal metaphor detection addresses a sparse research area shaped by limited annotated corpora and language-specific interpretation challenges. The paper constructs the HiLeMe corpus from Hindi judicial data, annotates metaphors with MIPVU, and fine-tunes mBERT, achieving 71% accuracy while showing weaker metaphor-class recall and F1.
Problem
Automatic metaphor detection remains limited in legal discourse and under-studied languages because metaphor interpretation is complex, nuanced, and language-specific.
Method
The paper creates HiLeMe from Hindi Legal Data Corpus judgments, applies MIPVU annotation, and fine-tunes multilingual BERT for metaphor detection.
Results
71% accuracy was achieved with mBert transformers, while metaphor-class precision was 0.5077, recall was 0.3267, and F1-score was 0.3976.
Takeaways & Limitations
The study provides a first-pass solution for automated metaphor identification in Hindi legal data.
Takeaways & Limitations
Metaphorical understanding is not language agnostic, limiting direct transfer of interpretations across languages.
Abstract
from arXiv · showhide
Metaphors are figurative use of words for conceptual mapping. Metaphor detection in the legal context has been crucial as metaphors are persuasive juridical means of creating legal meaning and concepts resulting in significant consequences. Metaphorical framing in legal discourse by judges, lawyers, and legislators brings about real-time implications upon individuals and influences judicial decision-making, argumentation and interpretation of laws. This is crucial in Human Rights infringement cases where language determines severity of punishment, public perception and judicial outcomes. While automatic metaphor detection in major languages like English, Spanish, Polish, Lithuanian have aided in understanding inherent intentions of metaphorical use of language, there is no such attempt in low-resource languages like Hindi. The dearth of annotated legal corpora in Hindi makes it difficult to develop NLP models and detect metaphors in judicial proceedings. In the Indian context, Convolutional Neural Networks (CNNs) have been used for classification of bail judgements, however there are no existing models designed for metaphor detection. We present a Hindi Legal Metaphor Corpus (HiLeMe) by isolating judgements from Hindi Legal Data Corpus (HLDC). Legal experts annotated HiLeMe to classify metaphorical constructions using the MIPVU schema. We downstreamed an mBERT on Hindi legal metaphor detection task. We built a transformer-based architecture for metaphor detection that are known to outperform traditional models in legal classification tasks. This model provides insights into the judicial psyche for decoding judicial decisions. Our research contributes to advancing automated models in legal discourse in low-resource languages like Hindi and envisages adoption into 22 Indian schedule languages.
1 Introduction
Metaphors structure legal meaning and persuasion, but their interpretation is language- and domain-sensitive. The paper motivates computational detection in Hindi legal discourse and outlines its corpus and modeling approach.
- Conceptual metaphor theory links surface metaphors to underlying conceptual structures, while MIPVU provides a framework for identifying metaphorically used linguistic terms.
- Metaphors help explain opaque legal concepts through analogy but can also polarize opinion and influence legal outcomes.
- Metaphorical interpretation is not language agnostic, so meanings established in one language may not transfer directly to another.
- The paper reviews metaphor research, automatic detection, and its approach to annotating Hindi legal metaphors and training a transformer model.
2 Qualitative assessments of metaphorical usage in legal discourse
Legal metaphors are embedded in doctrine, adjudicative reasoning, institutional imagery, and persuasive framing. They can clarify abstract concepts while also shaping normative judgments, concealing judicial agency, and affecting vulnerable groups.
- Metaphors can harden into doctrine when their figurative status is forgotten, potentially obscuring the policy rationale behind legal rules.
- Persuasive legal writing uses metaphors at doctrinal and legal-method levels, including “marketplace of ideas,” “balancing,” and “piercing the corporate veil.”
- Metaphors such as “balancing” make normative judgments appear like measurements, although legal interests have no objective physical weight.
- The “finding” metaphor portrays judges as passively discovering pre-existing law, obscuring their creative role in developing new interpretations.
- Legal metaphors create conceptual structures for abstract domains, including law as a person, architecture, system, or physical balance.
- Metaphorical framing can encode assumptions about hierarchy, gender, and race, subtly tilting adjudication against vulnerable groups.
3 Complexities of automatic detection of Metaphors
Automatic metaphor detection remains difficult because metaphor interpretation is nuanced and language-specific, with limited coverage beyond widely studied languages and legal classification tasks. The paper responds by applying MIPVU annotation and multilingual BERT to Hindi legal data.
- Automatic metaphor detection has achieved limited success because metaphor interpretation involves complexity, nuance, and language-specific constraints.
- Most prior approaches target languages such as English, Russian, Spanish, and Farsi, while research on lesser-known languages remains sparse.
- Legal-domain transformer studies have focused mainly on judgment classification, bail prediction, and summarization rather than metaphor detection.
- The paper uses MIPVU to annotate HLDC and fine-tunes a multilingual BERT model for Hindi legal metaphor detection.
4 Model pipeline and our approach
The paper develops a supervised Hindi legal metaphor-detection pipeline by annotating HLDC judgments with MIPVU and constructing the HiLeMe corpus. The resulting dataset combines cleaned sentence-level labels with expert-identified metaphorical lexical units.
- Hindi legal sentences were extracted from HLDC, which contains district-court judgments from 71 Uttar Pradesh districts.
- The dataset supporting the study is available through an Open Science Framework repository.
- Six legal experts independently applied MIPVU annotations by comparing contextual and basic lexical-unit meanings and recording metaphor decisions, rationales, and confidence.
- The pipeline cleaned and standardized annotations, normalized missing lexical units, and converted annotator decisions into sentence-level 0/1 metaphor labels.
- The final HiLeMe dataset contains 7137 sentences and 162196 tokens, with experts identifying 2022 sentences containing metaphorical lexical units.Sentences average 22.73 tokens, and non-metaphor instances outnumber metaphor instances.
5 Model Architecture and Training
The model pipeline fine-tunes multilingual BERT on sentence-level MIPVU-derived labels for Hindi legal metaphor detection. On the closed test set, it achieved 71.99% accuracy but lower metaphor-class recall and F1.
- The approach fine-tunes a BERT-base-multilingual-cased model trained across 104 languages, including Hindi, for sentence-level metaphor decisions.
- Training uses MIPVU-derived sentence labels with stratified 80-10-10 train, validation, and test splits, maximum sequence length 128, AdamW, learning rate 2e-5, batch size 16, and three epochs.
- 71.99% overall accuracy was achieved on the closed test set after the final training epoch.
- For the metaphor class, precision was 0.5077, recall was 0.3267, and F1-score was 0.3976.These results were presented as indicating conservative detection of subtler metaphorical expressions.
- The pipeline comprises HiLeMe annotation, structure correction, lexical-unit processing, POS tagging, dataset splitting, and mBERT training.The figure labels POS tagging with Stanza Hindi and specifies the 80/10/10 dataset split.
6 Results
The fine-tuned mBERT model detected metaphors in Hindi legal text with 71.99% overall accuracy, but metaphor-class recall and F1-score remained limited. Error analysis highlights class imbalance and examines false positives, false negatives, and misclassified sentences.
- The results indicate substantial room for improving metaphor identification and reducing false positives.The reported metrics suggest that the classifier identifies clear metaphorical cues but still struggles with metaphor instances and false positives.
- A confusion matrix breaks model outcomes into true positives, true negatives, false positives, and false negatives for error analysis.These categories provide a visual breakdown of the types of errors made by the model.
- The corpus contains 5115 No Metaphor sentences versus 2022 Metaphor sentences, creating class imbalance that may confound detection.Potential responses include oversampling metaphors, undersampling non-metaphors, or applying class weights during training.
- A normalized confusion matrix represents each row as proportions, enabling comparison between Metaphor and No Metaphor classes despite corpus imbalance.Normalization changes the display from raw counts to row-wise proportions.
- Examples of false positives and false negatives provide specific instances of model misclassification for further inspection.False positives are non-metaphorical sentences predicted as metaphors, while false negatives are actual metaphors predicted as non-metaphorical.
- The paper presents HiLeMe as a first-pass solution, achieving about 71% accuracy while underscoring linguistic complexity in Hindi legal discourse.The discussion identifies metaphor ubiquity and jurisprudential discretion as continuing challenges for metaphor detection.