Source-linked AI summary
Named Entity Recognition using Sliding Window Approach
Hariom Ingle, Ronit Ghode, Ishwari Gondkar, Jidnyasa Harad, Ravindra Murumkar, Raviraj Joshi
TL;DR
Transformer input limits make sentence-level NER models difficult to apply directly to long documents, where truncation or chunking can impair entity coverage. This paper extends a frozen MahaNER-BERT model with overlapping-window inference and merged predictions, without retraining or architectural changes. Macro F1 degradation remains below one percentage point across the tested configurations.
Problem
Transformer models have a fixed input-token limit, and truncation or non-overlapping chunking can lose content or fragment entities in long documents.
Source: §1 · p. 1, §2.3 · p. 2Method
The framework processes a document in overlapping windows with a frozen sentence-level NER model, then merges window predictions into a document-level annotation.
Source: §1 · p. 1, §3.1 · p. 3, §3.4 · p. 4Results
Takeaways & Limitations
Overlapping-window inference extends sentence-level NER to document-level use without additional training or architectural modifications.
Source: §1 · p. 1, §6 · p. 7The aggregation strategy retains the most recently processed window’s prediction, while confidence-based, voting, or probability-fusion alternatives may improve boundary consistency.
Source: §6.5 · p. 8
Abstract
from arXiv · showhide
Named Entity Recognition (NER) is a core NLP task, but transformer-based sentence-level models struggle with long documents because of fixed input-length limits: truncation drops content, and non-overlapping chunking fragments entities at segment boundaries. We introduce an inference-only pipeline that extends a frozen NER model, MahaNER-BERT, fine-tuned on the MahaNER corpus, to document-level prediction via overlapping sliding windows that are merged into a single annotation, without any retraining or architectural change. We evaluate the pipeline on six document-level corpora built from the MahaNER test set using two strategies: Normal Repeat, which duplicates sentence sequences to extend length while preserving contextual continuity, and Random Repeat, which concatenates distinct sequences to produce longer, heterogeneous inputs, each instantiated at three length levels, across several sliding-window configurations. The model retains a macro F1-score of up to 0.8902, with variation staying below one percentage point regardless of document length or construction strategy. Compared with the conventional non-windowed approach, the sliding-window pipeline avoids the boundary-fragmentation errors introduced by non-overlapping segmentation, yielding consistently higher and more stable document-level F1-scores.
1 Introduction
Transformer input limits make long-document NER difficult: truncation loses content and non-overlapping chunks can split entities. The paper proposes overlapping-window inference and reports stable macro F1 across document lengths and construction strategies.
1 Introduction
- Transformer input limits constrain document processing, while truncation loses content and non-overlapping chunking can split multi-token entities.The paper notes that these workarounds can be especially costly in low-resource settings where retraining is difficult.Source: §1 · p. 1
- Overlapping windows merged into one annotation extend sentence-level NER to documents without retraining or changing the model architecture.The method is presented as a training-free, inference-time extension.Source: §1 · pp. 1–2
- The six evaluation corpora vary document length using Normal Repeat and Random Repeat construction strategies.The strategies are instantiated at three length levels.Source: §1 · p. 2
- Below one percentage point of macro F1 degradation occurs across the tested configurations, supporting the pipeline’s viability as a deployment strategy.Macro F1 remains close to 0.89 regardless of document length or construction strategy.Source: §1 · pp. 1–2
2 Related Work
The related work traces NER from rules and statistical methods to neural and transformer models, then describes the challenges of applying sentence-level models to long documents. It identifies limited controlled study of sliding-window NER, particularly for low-resource Indian languages.
2.1 Named Entity Recognition
- NER moved from handcrafted rules and statistical sequence models toward neural methods that reduce reliance on manual feature engineering.BiLSTM–CRF models and contextual embeddings such as ELMo are described as steps in this progression.Source: §2.1 · p. 2
2.2 Transformer-Based Named Entity Recognition
- Transformer NER models perform well on sentence-level inputs, but their maximum sequence length limits direct application to documents.The passage also notes Marathi models developed for sentence-level NER.Source: §2.2 · p. 2
2.3 Document-Level Named Entity Recognition
- Truncation loses document content, while non-overlapping chunking can fragment entities near segment boundaries.Specialized long-context architectures and hierarchical models are among the approaches described for this problem.Source: §2.3 · pp. 2–3
2.4 Inference-Time Processing for Long Documents
- Sliding-window inference has been used in other NLP tasks, but systematic study of its effects on document-level NER remains limited, especially for low-resource Indian languages.Prior investigations rarely vary window size, overlap, and document construction under controlled conditions.Source: §2.4 · p. 3
2.5 Research Gap
- The paper addresses this gap with a training-free sliding-window framework and six MahaNER-derived corpora, reporting high recognition performance across document lengths and contextual diversity.The approach retains the pretrained Marathi NER model without additional fine-tuning or architectural changes.Source: §2.5 · p. 3
3 Methodology
The methodology extends sentence-level NER to document-level prediction by processing long documents in overlapping token windows and merging their predictions. The model remains frozen, and the evaluation varies corpus construction and window configurations.
3 Methodology
- The framework performs document-level inference by dividing long documents into input-length-compliant windows, processing each independently, and merging their predictions without retraining or architectural changes.The procedure is described as training-free and requiring no additional optimization.Source: §3 · p. 3
3.1 Overview
- The five-stage overview runs from tokenization and window generation through independent inference and prediction aggregation to a final document annotation.Overlapping windows address the fixed sequence-length constraint of sentence-level transformer models.Source: §3.1 · p. 3
3.2 Base NER Model
- MahaNER-BERT is fine-tuned on MahaNER sentence-level inputs and then frozen for all document-level experiments.No further training, fine-tuning, or architectural modification is used in evaluation.Source: §3.2 · pp. 3–4
3.3 Sliding Window Inference
- The document is divided into fixed-length windows separated by a stride, so adjacent windows overlap and predictions can be merged across the full input.Overlapping windows evaluate boundary-near tokens multiple times, reducing inconsistencies caused by artificial segmentation.Source: §3.3 · p. 4
3.4 Prediction Aggregation
- For tokens predicted in multiple windows, the aggregation procedure retains the label from the most recently processed window, adding no neural-network computations.The strategy avoids confidence estimation or ensemble techniques and is described as having negligible computational overhead.Source: §3.4 · p. 4
3.5 Long-Document Corpus Construction
- The six evaluation datasets include Normal Repeat variants that duplicate sentence sequences and Random Repeat variants that concatenate different sequences.Normal Repeat preserves local continuity; Random Repeat introduces heterogeneous contextual transitions.Source: §3.5 · p. 4
3.6 Window Configurations
- Four window–stride configurations maintain an overlap ratio of approximately 25% for comparisons across window sizes.The configurations are selected to investigate contextual coverage and recognition performance.Source: §3.6 · p. 5
4 Experimental Setup
The experiments evaluate a frozen MahaNER-BERT model on the official MahaNER test split and six derived document-level corpora. They vary window configurations while keeping the model, tokenizer, and evaluation procedure fixed.
4 Experimental Setup
- The experimental setup keeps model parameters unchanged, so reported performance differences arise from document-level inference rather than retraining or architectural modifications.The same pretrained model is used throughout the experiments.Source: §4 · p. 5
4.1 Dataset and Evaluation Protocol
- The model is fine-tuned using the official MahaNER training split and evaluated on the official test split, including six derived document-level corpora.The original entity annotations are preserved in the derived corpora.Source: §4.1 · p. 5
4.2 Implementation Details
- The implementation uses Python, PyTorch, Hugging Face Transformers, NumPy, Pandas, and scikit-learn while maintaining the same checkpoint, tokenizer, software environment, and evaluation protocol.The passage attributes observed differences to the sliding-window inference strategy under these fixed conditions.Source: §4.2 · p. 5
4.3 Sliding Window Configurations
- Four window–stride configurations maintain approximately 25% overlap, enabling comparison across contextual window sizes.Smaller windows resemble training sentence lengths; larger windows offer broader context and require fewer inference passes.Source: §4.3 · pp. 5–6
4.4 Evaluation Metrics
- Macro Precision, Macro Recall, and Macro F1-score are computed over merged document-level predictions, with equal weight assigned to each entity category.The paper describes macro averaging as suitable for low-resource NER where class imbalance is common.Source: §4.4 · p. 6
4.5 Experimental Protocol
- Each of the six datasets is tested with all four configurations while the model and evaluation procedure stay fixed, isolating changes in window size and stride.The protocol is designed to study document length, contextual diversity, and configuration effects without retraining or parameter tuning.Source: §4.5 · p. 6
5 Results
Across six dataset variants and four window configurations, Macro F1 remains stable as document length and contextual diversity increase. Window size has little effect, while Normal Repeat variants slightly outperform Random Repeat variants.
5.1 Overall Performance
- 0.8902 Macro F1 on Normal Repeat 8x (W=20, S=15) and 0.8789 on Random Repeat 8x (W=80, S=60) yield a 0.0113 spread across 24 combinations.Table 4 summarizes all six variants and four configurations.Source: §5.1 · p. 6, Table 4 · p. 7
5.2 Analysis
- Increasing window size from 20 to 80 tokens reduces Macro F1 by less than 0.003 for any single dataset variant.The drop is below 0.012 across all settings combined.Source: §5.2 · p. 6
- Normal Repeat variants consistently outperform Random Repeat counterparts by a small margin at every window configuration.The paper attributes the difference to preserved contextual continuity in repeated documents and more challenging heterogeneous transitions in Random Repeat variants.Source: §5.2 · pp. 6–7
- F1-scores for the 4x, 8x, and 10x variants stay within a narrow band as document length increases within either construction strategy.The results support extending sentence-level NER to document-level inference without additional training or architectural modification.Source: §5.2 · p. 7
6 Discussion
The discussion presents overlapping-window inference as a way to extend sentence-level NER without retraining or architectural changes. Results favor small windows and remain stable across document types and lengths, while the authors identify aggregation and evaluation boundaries for future work.
6 Discussion
- Across six datasets and four window–stride configurations, the framework maintains high Macro Precision, Macro Recall, and Macro F1-scores.Overlapping inference is described as preserving contextual information while maintaining the original model's prediction quality.Source: §6 · p. 7
6.1 Effect of Sliding Window Inference
- Overlapping windows evaluate tokens near segment boundaries multiple times, reducing the likelihood of fragmented entity predictions.The framework reuses existing pretrained models without parameter changes and is compatible with sentence-level NER models used for low-resource languages.Source: §6.1 · p. 7
6.2 Influence of Window Size
- The smallest configuration (20, 15) consistently achieves the highest Macro F1-score, while larger windows provide no measurable improvement and slightly reduce Macro Recall.The authors suggest smaller windows may better match the relatively short inputs used to optimize the pretrained model.Source: §6.2 · p. 7
6.3 Performance on Different Document Types
- Normal Repeat datasets generally achieve slightly higher Macro F1-scores, while Random Repeat datasets remain reliable despite abrupt contextual transitions.Random Repeat concatenates unrelated sentence sequences, making its document structure more challenging than sentence-level training inputs.Source: §6.3 · pp. 7–8
6.4 Practical Implications
- Inference-only operation requires no additional training data, model parameters, or specialized transformer architecture, supporting use with existing sentence-level NER systems.The paper highlights potential value for low-resource Marathi and possible deployment across long-document applications and other fixed-length transformer NER models.Source: §6.4 · p. 8
6.5 Limitations and Future Work
- The current aggregation keeps predictions from the most recently processed window; the evaluation uses synthetic MahaNER-derived documents, and future work includes adaptive windows, confidence-aware aggregation, and multilingual benchmarks.Naturally occurring long-document evaluation is identified as a source of additional evidence needed about real-world effectiveness.Source: §6.5 · p. 8
7 Conclusion
The paper concludes that overlapping sliding windows extend sentence-level NER models to document-level inference without retraining or architectural changes. Across six MahaNER-derived corpora, performance remains high across lengths and contextual diversity, with the smallest window configuration performing best.
7 Conclusion
- The framework partitions long documents into overlapping token windows, processes them independently with a pretrained Marathi NER model, and aggregates predictions into document-level annotations.The method requires neither additional fine-tuning nor architectural modification.Source: §7 · p. 8
- Across six MahaNER-derived corpora and four window–stride configurations, the framework maintains high Macro Precision, Macro Recall, and Macro F1-score, with the smallest window configuration performing best.The authors describe the approach as computationally efficient and deployable with existing transformer-based NER models without retraining.Source: §7 · pp. 8–9