PaperlayerView exact revision on arXiv

Source-linked AI summary

Named Entity Recognition and Classification on Historical Documents: A Survey

Maud Ehrmann, Ahmed Hamdi, Elvys Linhares Pontes, Matteo Romanello, Antoine Doucet

arXiv:2109.11406v1cs.CLcs.LG

TL;DR

Historical NER must support semantic indexing of increasingly accessible historical collections despite noisy, changing, and diverse inputs. This survey synthesizes the field’s challenges, resources, and approaches, finding that neural and diachronic transfer-learning developments have improved performance while OCR/OLR noise and historical spelling remain major error sources.

  • Problem

    Historical NER lacks sufficient annotated resources and must handle diverse documents, noisy OCR/OLR input, and historical spelling variations.

  • Method

    The survey reviews twenty years of historical NER research, inventories resources, and examines approaches adapted to historical settings.

  • Results

    Neural approaches using diachronic transfer learning helped increase NER performance on historical material.

  • Takeaways & Limitations

    Historical NER has become a cornerstone of semantic indexing, with research momentum driven by machine-readable texts and deep learning advances.

  • Takeaways & Limitations

    Resource sparsity is likely to persist across the broad spectrum of historical domains, languages, document types, and periods.

Abstract

from arXiv · show

After decades of massive digitisation, an unprecedented amount of historical documents is available in digital format, along with their machine-readable texts. While this represents a major step forward with respect to preservation and accessibility, it also opens up new opportunities in terms of content mining and the next fundamental challenge is to develop appropriate technologies to efficiently search, retrieve and explore information from this 'big data of the past'. Among semantic indexing opportunities, the recognition and classification of named entities are in great demand among humanities scholars. Yet, named entity recognition (NER) systems are heavily challenged with diverse, historical and noisy inputs. In this survey, we present the array of challenges posed by historical documents to NER, inventory existing resources, describe the main approaches deployed so far, and identify key priorities for future developments.

1 INTRODUCTION

Historical digitisation creates vast machine-readable collections whose effective exploration depends on information extraction, especially named entity processing. Historical NER is valuable for indexing and downstream analysis but is challenged by heterogeneous domains, noisy inputs, changing language, and limited resources.

  • Motivation: Digitisation has produced extensive machine-readable historical collections, creating opportunities for computational search, retrieval, and exploration.The paper frames this as the next challenge after preservation and accessibility.
  • Applications: Named entities are prevalent in humanities searches: 80% of Gallica queries contain a proper name, while geographical and person names dominate searches across digital libraries.The cited evidence concerns historical and cultural-heritage collections rather than contemporary web search logs.
  • Motivation: Named entity processing is a crucial early step for extracting, linking, and representing information from digitised collections.NER supports semantic indexing and precedes entity linking across multilingual and heterogeneous collections.
  • Applications: Historical NER supports document indexing, information extraction, biography reconstruction, event detection, network analysis, and collection cross-linking.These uses extend beyond retrieval to analysis, visualisation, and entity linking through authority files and knowledge bases.
  • Challenges: Historical NER rarely matches contemporary well-edited English news performance because documents combine domain heterogeneity, noisy input, language change, and resource scarcity.These challenges are presented as interacting characteristics of historical material, not as a single failure source.

2 FRAMING OF THE SURVEY

The survey maps historical NER by defining its scope, reviewing prior work, and examining challenges, resources, and adaptation strategies. It covers mainly textual documents produced or published through 1979 and uses a literature search centred on formally evaluated studies.

  • Objectives: The survey asks which challenges, resources, and strategies shape NER on historical documents and which system components require adaptation.It also examines the variety of domains and uses of named entity processing in historical documents.
  • Document scope: Historical document scope is difficult to define because time, digital origin, writing type, material condition, and language provide no unambiguous boundary.The authors describe definitions based on these criteria as ultimately involving subjective decisions.
  • Document scope: The survey defines a historical document as a mainly textual document produced or published up to 1979, regardless of topic, genre, style, or acquisition method.Although the scope is broad, the reviewed NER work mainly concerns printed documents.
  • Methodology: The literature compilation combined journal and conference archive scans, search-engine discovery, and citation chaining, generally retaining publications with formal evaluation.Searches used combinations of NER and historical-document keywords in Google Scholar and Semantic Scholar.
  • Related surveys and audience: Earlier NER surveys covered general methods, specific domains or languages, and digital humanities, whereas this survey focuses on adapting NER to historical documents.Its target audience includes NLP and digital-humanities researchers, practitioners, and humanities scholars.

3 BACKGROUND

The background section introduces named entity processing, modern NER, required resources, and the principles underlying NER techniques before turning to historical documents.

  • Background: The section establishes general foundations in named entity processing, modern NER, resources, and NER techniques before discussing historical applications.

3.1 NE processing in general

Named entity processing comprises increasingly complex tasks that detect and categorize entities, link mentions to knowledge bases, and extract relations. The survey also notes that named entities lack a strict universal definition and excludes temporal analysis from its main scope.

  • Task definition: Named entity processing includes recognition and classification, disambiguation or linking, and relation extraction as progressively more complex tasks.Recognition detects entities and assigns predefined semantic categories; linking connects mentions to unique knowledge-base references; relation extraction discovers relations between entities.
  • Task definition: Recognition and classification identifies entities acting as rigid designators and categorizes them according to predefined semantic classes.
  • Named entities: Named entities are heterogeneous lexical units, mostly proper names and definite descriptions, that refer autonomously to entities of interest in context.The paper adopts criteria involving linguistic and application context rather than a strict definition.
  • Scope boundaries: The survey excludes temporal information processing and only occasionally mentions related work on geotagging.

3.2 NER in a nutshell

NER is a sequence-labelling task that assigns named-entity class labels to token sequences by learning patterns from labelled examples and textual features. Systems are evaluated primarily with precision, recall, and F-measure, with finer-grained measures accounting for partial matches and error types.

  • NER assigns named-entity class labels to token sequences by learning identification and classification patterns from labelled examples.The task infers labels for new, unseen token sequences from word–label correspondences and distinctive features.
  • IOB tagging marks each token as beginning, inside, or outside an entity of a particular class.Systems use this representation to extract features for learning NER models.
  • NER inputs are enriched with word-, context-, and document-level features, including morphological clues such as capitalization and word length.
  • Statistical models combine features because individual linguistic cues can be absent, ambiguous, or unreliable.Traditional machine learning relies on developer-specified feature engineering, whereas deep learning learns representations.
  • Precision, recall, and F-measure are the main evaluation measures, while finer-grained metrics address partial matches, incorrect types, and differing error weights.

3.3 NER resource types

NER development depends on four resource types: typologies, lexicons and knowledge bases, embeddings, and labelled or unlabelled textual data. They define entity categories, supply entity information, provide learned representations, and support training, evaluation, or language modelling.

  • NER systems rely on typologies, lexicons and knowledge bases, embeddings, and labelled or unlabelled textual data.Each resource type plays a specific role in system development.
  • Typologies define the semantic categories, scope, and organisation of entities considered by a system.They may be multipurpose or domain-specific and may vary in hierarchical structure.
  • Lexicons and knowledge bases provide lexical and encyclopedic entity information for recognition, classification, and disambiguation.
  • Word embeddings represent word meaning as dense, low-dimensional vectors learned from word distributions in running text.Their self-supervised learning from unlabelled data supports generic machine-learning representations.
  • Static embeddings assign one context-independent representation to each word, whereas language models learn representations from preceding sequences.The latter addresses the contextual-information shortcoming of traditional embeddings.
  • Labelled corpora provide learning examples and evaluation references, while unlabelled text supports embedding and language-model acquisition.

3.4 NER methods

NER methods comprise rule-based, feature-based machine-learning, and neural-based families. The field has moved toward deep learning architectures that manage context and exploit powerful embeddings or language models, with transfer learning especially relevant to low-resource and historical settings.

  • NER systems are developed using rule-based, feature-based machine-learning, and neural-based algorithms.
  • Rule-based methods manually encode observed regularities in grammars and commonly depend on linguistic preprocessing and external language resources.
  • Feature-based methods learn statistical models from manually selected features once annotated corpora are available.Examples include support vector machines, decision trees, hidden Markov models, maximum entropy models, and CRFs.
  • Deep-learning NER uses multilayer neural networks that learn representations with multiple levels of abstraction.
  • Successful deep-learning NER depends on architectures that manage context and input representations that exploit embeddings or language models.
  • SENNA achieved 89.86% F-score on the CoNLL-03 English corpus using an end-to-end CNN architecture with trained type-level word embeddings.
  • Transformers eliminate recurrence and convolutions, using self-attention to retrieve relevant information from large contexts.
  • Pre-trained embeddings and language models support transfer learning, which is particularly useful for low-resource languages and out-of-domain historical texts.

4 CHALLENGES

Historical NER is challenged by broad variation in domains, document types, languages, periods, noisy transcription, changing language and naming conventions, entity drift, and limited resources. Reported evidence shows substantial OCR-related performance losses, while diachronic effects remain less studied and historical resources require adaptation.

  • The historical variety space: Historical NER spans diverse document types, languages, periods, domains, and topics, creating a broad variety space for systems and applications.
  • Character recognition: OCR and HTR produce illegible words, tokenization errors, and out-of-vocabulary forms that machines handle less reliably than human readers.
  • Character recognition: 87% to 63% F-score for person entities was the drop reported between good and poor OCR quality.
  • Character recognition: 30 percentage points of F-score were lost when character error rate increased from 7% to 20% in a BiLSTM-based NER evaluation.
  • Character recognition: Entity noise of 0.1 already severely harmed entity prediction and could halve system performance.
  • Language and naming dynamics: Historical spelling and naming conventions vary across time, including changes in orthography, titles, name structures, and entity composition.Entity-name structure could also predict a document’s period in one study.
  • Entity and context drifts: Entity drift causes places, professions, and major entity types to fade and emerge over time, limiting generalisation to unseen mentions.
  • Entity and context drifts: Diachronic evaluations found non-linear F-score variations over roughly 200 years, with further research needed to separate time-related effects.

5 RESOURCES FOR HISTORICAL NER

The survey inventories historical NER resources across annotation schemes, corpora, and languages, finding that available coverage remains limited and uneven. The reviewed corpora span news, literature, and other domains, with news the best-equipped domain.

  • Typologies and guidelines: Historical NER guidelines often require adaptations because modern high-level schemes do not necessarily cover historical documents.Modern schemes commonly use the Person, Organisation, and Location triad, whereas historical material may require different coverage.
  • Overview: The survey inventories 17 openly released historical NE-annotated corpora across news, literature, and other document groups.Descriptions are organised by document type and account for typology, language, period, size, and licensing.
  • Overview: Most corpora are monolingual, while the inventory supports eleven living languages and two dead languages.Less than a third of the corpora include documents in two or more languages.
  • News: Historical newspaper collections are the best-equipped domain, with labelled corpora available in Czech, Dutch, English, French, and German.The news group draws on historical newspaper collections and provides the broadest language coverage among the surveyed domains.
  • Literature(s): Literary resources are more heterogeneous, ranging from ancient texts to computational collections of novels and other literary documents.They include resources for Latin, Coptic, Portuguese, German, travel writing, and literary place names.

5.3 Language representations

Historical language representations include diachronic word embeddings, character-level contextual representations, and word-level language models trained on large historical collections. These resources cover multiple languages and periods, and some models improve reported evaluation results or support diachronic study.

  • Training resources: Historical embeddings and language models are trained on large diachronic collections such as Europeana Newspaper, Trove, Digi, and impresso.Large textual corpora are needed for distributional representations to be effective.
  • Word embeddings: Dynamic embeddings train static representations on separate time bins and align them to represent semantic change over time.They can also feed neural architectures for other NLP tasks.
  • Contextualised embeddings: Historical character-level language-model embeddings are available for German, French, and English.Examples include Flair representations trained on historical newspaper material and auxiliary resources released for HIPE-2020.
  • Word-level language models: Latin BERT was trained on 640 million tokens spanning 22 centuries and achieved state-of-the-art results on POS tagging and word sense disambiguation.The model used material from the Perseus Digital Library, Latin Wikipedia, and the Internet Archive.
  • Word-level language models: German ELECTRA models outperformed BERT on two historical NE datasets, improving reported state-of-the-art results.Historical BERT models are also available for French and nineteenth-century English, including time-sliced models for diachronic study.

6 APPROACHES TO HISTORICAL NER

The survey organises historical NER research by rule-based, traditional machine-learning, and deep-learning approaches. It emphasises implementation and experimental settings while treating challenge-specific strategies separately.

  • Approach types: Historical NER work is reviewed by approach type: rule-based, traditional machine learning, and deep learning.The organisation follows the main methodological families used for historical documents.
  • Scope: The approach overview emphasises implementation choices and experimental settings for historical NER methods.Strategies for addressing particular historical-document challenges are discussed separately.
  • Evaluation context: The survey notes that research was conducted mainly in individual projects without a shared gold standard up to the review period.This context shapes how historical NER approaches and their evaluations are compared.

6.1 Rule-based approaches

Rule-based systems form an established historical NER approach, using modular rules, gazetteers, variant matching, and document-level evidence across varied materials. They remain interpretable and data-efficient but require linguistic expertise and struggle with noisy historical input.

  • Characteristics: Rule-based systems do not require training data and are readily interpretable, but designing their rules requires time and expertise.These systems were among the earliest approaches applied to historical documents.
  • Evaluation: A Swedish literary system reached a best overall F-measure of 89% using gazetteers, finite-state grammars, similarity matching, and document-level evidence.Its modules were evaluated incrementally across eight entity types.
  • Evaluation: A modern Finnish rule-based system applied without adaptation to poor-OCR newspapers achieved F-scores ranging from 30% to 45%.The corpus had word-level OCR correctness of approximately 70%–75%.
  • System design: Historical rule-based systems commonly combine gazetteer lookup, incremental rule application, and variant matching.They are modular and were adapted to domain- and language-specific peculiarities.
  • Challenges: Noisy historical input creates difficulties that require normalisation rules and additional linguistic knowledge.OCR errors, marginalia, unusual spelling, and historical variation complicate rule application.
  • Coverage: Rule-based systems were applied across diverse document types, domains, languages, and historical periods.Examples include court trials, newspapers, literary classics, ancient documentary texts, and nineteenth-century novels.

6.2 Traditional Machine Learning Approaches

Traditional machine-learning NER systems enabled extensive experimentation on historical documents, but performance varied widely and generally remained below contemporary benchmarks. In-domain training usually improved results, while pretrained systems particularly struggled with recall and organisation entities.

  • Applying existing models: 47–54% F-scores were obtained on Holocaust testimonies and 32–36% on soldiers’ letters when four off-the-shelf systems recognised Person, Location, and Organisation entities.Correcting OCR manually did not significantly improve the results.
  • Applying existing models: Major error sources included naming variation, metonymy, and insufficient background knowledge, especially for organisations.The examples included warships named after people and organisation-related errors.
  • Applying existing models: Pretrained NER models were inadequate for historical texts, with recall consistently more affected and Location usually the most robust entity type.Baseline studies reported diverse performances across systems, documents, and settings.
  • Training models: Traditional systems were trained and evaluated across diverse historical materials, including newspapers, parish registers, medieval charters, and wartime bulletins.Reported settings differed in entity types, transcription quality, language, time period, and training data.
  • Conclusion on traditional machine learning approaches: Traditional machine-learning NER systems typically achieved F-scores around 60–70% on historical documents, below the roughly 90% often obtained on contemporary material.CRF-based systems were the most commonly used, especially the Stanford CRF classifier.
  • Training models: Custom training generally produced higher performance when the training material matched the target domain, although an in-domain model did not always outperform a default model.Historical NER results were higher with in-domain training overall, but one newspaper study found both custom and default models near 75% for Person and Location.

6.3 Deep Learning Approaches

Deep-learning approaches for historical NER explore input representations and transfer learning to address noisy, shifting, and resource-scarce settings. Across studies, BiLSTM-CRF models, contextual and stacked embeddings, sub-word information, and careful preprocessing generally improve performance, although results remain difficult to compare across settings.

  • Research focus: Historical NER deep-learning research primarily varies character, sub-word, and word representations together with transfer-learning strategies.Studies examine static and contextualised embeddings, learned from in-domain or out-of-domain data and at type or token level.
  • Static embeddings: 89.7% F-score was achieved by a BiLSTM-CRF configuration using in-domain word embeddings fine-tuned during training and a CRF prediction layer.Character-level embeddings added a smaller positive contribution, with BiLSTM learning them better than CNN in that study.
  • Comparative performance: BiLSTM-CRF models generally outperform traditional CRFs when enough training data or transfer learning is available, but not consistently in small-data settings.In one synthesis, BiLSTM and CRF perform similarly with little data, whereas BiLSTM-CRF gains an advantage with sufficient data or transfer learning.
  • Contextual representations: 65% F-score was obtained by the best fastText-plus-BERT combination on coarse literal NER in a smaller-data historical setting.The reviewed experiments also report strong results for contextual and stacked representations across historical corpora.
  • Representation design: Character and sub-word information helps address out-of-vocabulary words, historical spelling variation, and OCR errors, while contextualised and stacked embeddings generally outperform static ones.The review identifies fastText or Flair combined with BERT as an effective mixture of morphological and lexical information.
  • Practical considerations: Careful preprocessing, including word de-hyphenation and sentence segmentation, appears to be a key factor, while transferability across historical settings remains a promising goal.The survey cautions that heterogeneous datasets and sensitive experimental settings often prevent direct comparison or reproducibility.

7 STRATEGIES TO DEAL WITH SPECIFIC CHALLENGES

The survey organizes responses to historical NER challenges around noisy input, language change, and limited resources. Strategies either adapt the input, adapt the NER system, or reduce data requirements through transfer and active learning.

  • Dealing with noisy input: Historical NER addresses OCR/OLR noise through input correction or systems designed to process noisy text directly.OCR/OLR errors include character misrecognition, truncated or connected word parts, spurious hyphenation, and mixed textual segments.
  • Dealing with noisy input: OCR/OLR post-correction reconstructs word forms and linguistically motivated token sequences, but depends heavily on document-specific layout and typographic conventions.Erroneous punctuation and lost distinctions between soft and hard hyphens complicate implementation.
  • Dealing with noisy input: Character-, subword-, and character-level language representations help neural systems recognize misspelled entities, while subword tokenization represents out-of-vocabulary words.These methods remain less effective for some substitutions and deletions because they can produce uncommon subword units.
  • Dealing with dynamics of language: Historical language change involves spelling variation, evolving naming conventions, and entity drift, with no single clear-cut solution.Possible responses include adapting lookup conditions and using temporally close embeddings or language models.
  • Dealing with the lack of resources: Resource scarcity concerns inappropriate typologies and insufficient labelled or unlabelled historical corpora.Typology design requires linguistic, historical, and knowledge-representation expertise, while interoperability prevents resources from becoming isolated silos.
  • Dealing with the lack of resources: Transfer learning, active learning, and data augmentation are proposed to reduce the amount of historical training data required.Active learning selects relevant examples for annotation, whereas data augmentation transforms training instances without changing their labels.

8 CONCLUSIONS AND OUTLOOK

The survey finds that historical NER has advanced substantially, with neural approaches reaching performance close to contemporary-text systems. It identifies transferability, robustness, comparability, finer-grained recognition, and resource sharing as priorities for further progress.

  • Conclusions and outlook: The survey covers two decades of historical NER challenges, resources, and approaches adapted to historical settings.Its challenges include document and domain variety, noisy input, language dynamics, and resource scarcity.
  • Conclusions and outlook: Neural systems raise historical NER F-scores from 60-70% on average for rule-based and traditional ML systems to 80% for the best neural systems.The review reports that historical NER can now handle historical and noisy inputs with performance almost competing with contemporary texts.
  • Transferability: Transferability across historical time periods and domains remains a major challenge, especially for cultural heritage institutions.The authors encourage systematic further experimentation with transfer learning.
  • Robustness: OCR/OLR noise and historical spellings remain the main error sources, motivating more systematic assessment of which noise types are most detrimental.Such assessment is intended to support more focused strategies for handling noise.
  • System comparability: Varied domains, periods, languages, and evaluation corpora hinder comparison, increasing the importance of gold standards and shared historical tasks.The survey presents shared evaluation as a way to enable system comparison and drive progress.
  • Finer-grained historical NER: Historical NER should model finer-grained phenomena such as nested entities, entity-name composition, and metonymy.These interests extend beyond recognition of the main entity types.
  • Resource sharing: Resource sharing should include large-scale historical text, derived language models, typologies, annotation guidelines, and training material, with interoperability.The survey links recent deep-learning advances to the availability of large-scale textual data.
  • Conclusions and outlook: Large-scale historical NER should account for personal-data protection and incorporate humanities scholarship knowledge and needs through interdisciplinary work.These considerations are presented as important facets of work on historical NER systems.
Loading 2109.11406v1…