PaperlayerView exact revision on arXiv

Source-linked AI summary

Correction as Annotation: Bootstrapping a Dependency Parser for Documentary Medieval Latin

Gabriel H. Pizzorno

arXiv:2609.06266v1cs.CLcs.DL

TL;DR

Historical-text NLP faces limited suitable training data and substantial linguistic divergence from available resources. This study builds in-domain training data through iterative correction of model annotations, and reports that the resulting model outperforms the baseline and models trained on nearly two orders of magnitude more text.

  • Problem

    Historical texts pose NLP challenges through linguistic idiosyncrasies, limited suitable training data, and divergence from classical and modern varieties.

    Source: §1 · p. 1

  • Method

    The study measures iterative in-domain training with sentence batches sampled independently of model state, without active-learning selection.

    Source: §7.5.1 · p. 31

  • Results

    Tagging and morphological scores increase from 57–80% to 92–98%, while syntactic scores increase from 17–65% to 82–95% relative to ITTB.

    Source: §8 · pp. 32–33

  • Takeaways & Limitations

    In this study, 33.2 hours of correction produced a parser that outperforms models trained on nearly two orders of magnitude more text.

    Source: §8 · p. 33

    The reported efficiency depends on formulaic structure; less repetitive corpora would be expected to plateau earlier and at higher residual error.

    Source: §7.4 · p. 31

Abstract

from arXiv · show

Medieval documentary sources remain inadequately served by existing natural language processing tools. None of the five readily available Latin treebank models attains usable performance on a collection of 160 inventories compiled in Marseille between 1258 and 1446. The best labelled attachment score is 0.62 and the best morphology-aware score is 0.24. Performance does not correlate with either genre or period proximity. To address this shortfall, in-domain training data was generated as a by-product of using these inadequate models. In each of nine iterations, a model pre-annotated 200 sentences; an expert corrected the annotations; and the corrected sentences were used to train the subsequent model, with batches sampled independently of model state, without active-learning selection. Thirty-three hours of annotation effort over 1,804 sentences increased universal part-of-speech accuracy from 0.80 to 0.98 and labelled attachment from 0.48 to 0.92, outperforming all baselines on the reported metrics while using 97% less training data than the largest one of them. Annotator effort declined from 54% of tokens to a plateau of 14-18%, an operational progress metric that requires no separate gold standard and can serve as a stopping criterion.

1. Introduction

Historical documentary texts pose challenges for NLP because their vocabulary, orthography, and syntax can diverge from existing resources. This paper presents an iterative human-correction approach and illustrates it on late-medieval Marseille inventories, where standard models perform poorly.

1. Introduction

  • Historical NLP is challenged by linguistic idiosyncrasies, limited training data, and divergence from classical and modern language varieties (Piotrowski 2012; Kestemont, de Pauw, et al. 2017).Specialized documentary corpora also feature domain vocabularies, non-standard orthographies, and syntax unlike that represented in literary treebanks (Crane 2006; McGillivray, Poibeau, and Ruiz 2020).Source: §1 · p. 1
  • The paper introduces iterative training that pairs existing-model preprocessing with expert correction to adapt models to highly idiosyncratic historical corpora.The corrected annotations are used in a bootstrapping loop intended to improve performance while maintaining annotation quality.Source: §1 · p. 2
  • The case study uses late-medieval Marseille inventories whose legal-administrative context combines Medieval Latin, vernacular Occitan, and formulaic descriptions.These linguistic features distinguish the inventories from varieties represented in existing Latin treebanks.Source: §1 · p. 2
  • 47–57% accuracy requires a token’s complete morphological analysis to be correct, while 15–24% requires attachment, label, and morphology to be correct simultaneously.The paper reports substantial improvement from its human-in-the-loop methodology while requiring less manual annotation than traditional treebank construction.Source: §1 · p. 2
  • The paper’s contributions include analyzing specialized historical-corpus challenges, presenting an adaptation methodology, and offering a documentary-source case study with practical guidance.The authors frame the method for corpora with extreme domain distance from available resources.Source: §1 · p. 2

2. Related Work

The related work spans domain adaptation, human-in-the-loop learning, historical-language processing, and treebank construction. This paper positions iterative, curated correction as an approach tailored to documentary corpora far from available resources.

2. Related Work

  • The paper situates its contribution at the intersection of domain adaptation, human-in-the-loop learning, and historical-language processing.It also connects the work to treebank creation and adaptation.Source: §2 · p. 2

2.1. Domain Adaptation for NLP

  • Domain-adaptation methods include invariant representations, fine-tuning, and transfer learning, but typically presuppose source–target similarity that specialized historical corpora may lack (Blitzer, McDonald, and Pereira 2006; Howard and Ruder 2018; Pan and Q. Yang 2010).The literature also explores adversarial adaptation, multitask learning, character-level normalization, and transformer adaptation.Source: §2.1 · pp. 2–3

2.2. Human-in-the-Loop Learning

  • Human-in-the-loop work integrates human expertise into learning, while active-learning strategies select informative examples using uncertainty, committee disagreement, or information-theoretic criteria (Holzinger 2016; D. D. Lewis and Gale 1994; Seung, Opper, and Sompolinsky 1992; MacKay 1992).Such methods have been applied to NLP tasks including parsing and machine translation (Tomanek and Hahn 2009; Hwa, Resnik, Weinberg, Cabezas, et al. 2005).Source: §2.2 · p. 3

2.3. Historical Language Processing

  • Historical-language processing has matured beyond normalization and spelling standardization to complex tasks such as parsing and semantic analysis, with historical treebanks crucial to advancing the field.Source: §2.3 · p. 3

2.4. Treebank Creation and Adaptation

  • Prior treebank-construction strategies include projection, bootstrapping, and cross-lingual transfer, while Universal Dependencies supports standardized cross-lingual development (Hwa, Resnik, Weinberg, and Kolak 2002; McClosky, Charniak, and M. Johnson 2006; McDonald, Petrov, and Hall 2011; Nivre, de Marneffe, Ginter, Goldberg, et al. 2016).This paper instead proposes curated human corrections in an iterative loop tailored to corpora at extreme distance from available resources, yielding gains on morphological and syntactic tasks.Source: §2.4 · pp. 3–4

3. The DALME-Marseille Corpus

DALME-Marseille is a collection of late-medieval inventories whose repeated administrative structures coexist with multilingual, elliptical, and lexically unstable language. Those features make the corpus both structurally patterned and difficult for standard NLP systems.

3.1. Coverage and Provenance

  • DALME-Marseille comprises 160 Marseille inventories from 1258 to 1446, drawn from varied legal and archival contexts and spanning diverse social and economic groups.The collection includes court, hospital, religious-foundation, and notarial records, among other settings.Source: §3.1 · pp. 4–5

3.2. Linguistic Features

  • The inventories are working documents combining Medieval Latin, specialized technical vocabulary, local Occitan, and formulaic legal expressions.Their macrostructure uses conventional openings, spatially ordered goods lists, and formulaic attestations, with recurring templates for objects, quantities, and attributes.Source: §3.2 · p. 5

3.3. Challenges for Automated Processing

  • Automated processing faces four consequential obstacles arising from the corpus’s distinctive language and structure.The following challenges include code-switching, ellipsis, formulaic repetition, and orthographic instability.Source: §3.3 · p. 5

3.3.1. Multilingual Code-Switching

  • 36.5% of sentences contain both Latin and Occitan material, with vernacular object and material terms embedded in Latin grammatical frames.In the gold standard, 8.7% of non-punctuation words are identifiable as Occitan and 88.6% as Latin; proper names account for a further 2.6%.Source: §3.3.1 · pp. 5–6

3.3.2. Ellipsis and the Enumerative Frame

  • 88.8% of sentences begin with the enumerative marker item, and 84.3% of those contain no verb, departing from predicate-centered literary Latin syntax.Descriptive phrases also omit copulas and often determiners.Source: §3.3.2 · p. 6

3.3.3. Formulaic Repetition

  • The ten most common part-of-speech patterns account for 44.1% of sentences, and the most common marker–numeral–noun pattern occurs 1,439 times.This repetition can make a small annotated sample informative, while requiring models to handle variable domain vocabulary within recurring frames.Source: §3.3.3 · p. 6

3.3.4. Orthographic Instability and the Lexicon

  • 68% of distinct DALME-Marseille lemmata are out-of-vocabulary relative to the largest available Latin treebank.Scribal spelling varies within the corpus and stable non-classical forms also occur, complicating exact-form lookup and any normalization without an explicit target standard.Source: §3.3.4 · pp. 6–7

4. Baseline Performance Analysis

Five models trained on existing Latin treebanks were tested on the same 200 manually annotated DALME-Marseille sentences. The analysis compares their task-specific performance and examines how the corpus differs from the training resources.

4.3. Baseline Results

  • Best baseline LAS was 0.62 and best MLAS was 0.24; no pre-trained model performed adequately across the full annotation pipeline.MLAS requires attachment, label, and morphology to be correct together.Source: §4.3 · pp. 8–9, §4.2 · p. 8, Table 2 · p. 8

4.4. Lexicon and Evaluation Independence

  • 95.5% to 90.2%: exact-lemma lookup coverage fell after gold-only lexicon entries were excluded to prevent evaluation contamination.The excluded entries were attested only in gold-standard sentences.Source: §4.4 · p. 10

4.5. Does a Newer Release Close the Gap?

  • The current release's best out-of-the-box result was LLCT at 0.77 UPoS and 0.54 LAS, below the earlier package's 0.85 UPoS and 0.62 LAS.The comparison used the same gold standard and no domain lexicon.Source: §4.5 · p. 10, Table 3 · p. 10

4.6. Genre and Period Do Not Predict Transfer

  • Genre or period proximity poorly predicted transfer: UDante outperformed the closest-genre LLCT on seven of eleven metrics; lexicon overlap and structural profile are more informative starting-model criteria.Source: §4.6 · p. 11

4.7.1. Structural and Syntactic Complexity

  • 6.87 words per sentence: DALME-Marseille is shorter than the comparison treebanks, whose mean sentence lengths range from 11.00 to 27.40 words.The paper relates the shorter sentences to formulaic enumerative frames rather than extended literary or theological argumentation.Source: §4.7.1 · p. 11

4.7.2. Length and Dependency Structure

  • r=0.848: DALME-Marseille has the strongest sentence-length/dependency-depth coupling, but is close to PROIEL at 0.821; its mean dependency distance is lowest at 1.89 words.The paper links predictable structure to rapid acquisition of the enumerative frame.Source: §4.7.2 · p. 12

4.7.3. Morphological Distribution Patterns

  • 13.7:1: DALME-Marseille's nominal-to-verb ratio far exceeds other resources, and its extreme nominal dominance contributes to baseline parsing degradation.Source: §4.7.3 · pp. 12–13, Figure 2 · p. 13

4.8. Implications for Model Performance

  • A few hundred in-domain sentences can cover much of the formulaic structural variation, whereas lexical innovation in core object vocabulary has no analogous compensation from out-of-domain data.Source: §4.8 · p. 13

5. Method

The method creates in-domain training data by correcting model pre-annotations rather than annotating from scratch. Corrected batches are accumulated in an iterative training loop.

5. Method

  • Machine-produced annotations are corrected rather than created de novo, with edits declining as models improve.Models are specialized to this corpus rather than preserving out-of-domain performance; one annotator's corrections make within-person drift the concern.Source: §5 · p. 14, §5.1 · p. 14

5.1. Methodological Framework

  • Each iteration randomly samples 200 sentences, pre-annotates and corrects them, then adds the corrections to the accumulated pool for fresh model training.The first batch uses the ITTB baseline for pre-annotation; later batches use the preceding iteration's model.Source: §5.1 · pp. 14–15
  • Evaluation does not guide selection: each next batch is independent of the preceding model, isolating gains from added in-domain data but leaving active-learning efficiency untested.Source: §5.1 · p. 15

5.2. Corpus Preparation

The corpus was deterministically prepared, then sampled in disjoint batches without model-guided selection; each batch was pre-annotated by a baseline or the preceding iteration’s model before expert correction.

5.2. Corpus Preparation

  • Deterministic preparation normalized Roman numerals and their NumForm features and regularized condam’s lemma to quondam, but preserved surface spellings.The 65 surface occurrences of condam remained alongside 27 of quondam; the preprocessing code was released for reproducibility.Source: §5.2 · p. 15

5.3. Sampling and the Annotated Subset

  • Each regular 200-sentence batch used fixed quotas of 136 short, 46 medium, and 18 long sentences, sampled without replacement and stratified only by sentence length.Batches were not selected by date, document type, lexical rarity, uncertainty, disagreement, or estimated information gain.Source: §5.3 · p. 15
  • 1,804 sentences and 14,516 tokens were annotated across nine batches, representing 26% of corpus sentences and 27% of tokens.The initial 204-sentence batch was a minor exception to the later sampling procedure.Source: §5.3 · p. 15

5.4. Pre-annotation

  • The first batch used ITTB as pre-annotator; later batches used the preceding iteration’s model, with ITTB chosen for its strongest combined morphological measure (All Tags, 0.57).Pre-annotation used the full Stanza pipeline, with a domain-lexicon lookup before the sequence-to-sequence lemmatizer.Source: §5.4 · p. 16

5.5. Correction and Annotation Guidelines

Corrections proceeded from morphology to syntax, guided by consistent, corpus-specific conventions for recurring ambiguities and structures that standard guidelines do not uniquely resolve.

Correction workflow

  • The annotator corrected morphology first, then dependency heads and labels, using the token tags to constrain plausible syntactic relations.The syntactic pass required adjudicating non-local grammatical relationships.Source: §5.5 · p. 16

Annotation principles

  • Project-specific conventions addressed recurring cases left open by UD, which account for roughly one third of the final model’s residual errors.Consistency across repeated structures was prioritized over a theoretically maximal analysis.Source: §5.5 · pp. 16–17

Annotation conventions

  • Surface spellings were preserved while lemmata were regularized, and Occitan lexemes in Latin clauses followed Latin morphosyntax, leaving code-switching out of the CoNLL-U tree.The lexicon’s language labels compensate for that syntactic-layer trade-off.Source: §5.5 · p. 17
  • Material genitives attach as nominal modifiers, adjectival material descriptions as adjectival modifiers, and multi-word names are flat unless inflection signals internal structure.The genitive-versus-adjectival distinction remains a source of final-model errors.Source: §5.5 · p. 17
  • In verbless item clauses, the enumerated object is the head, item is a discourse marker, and quantifiers attach to the object; no null predicate is posited.Annotation was limited to the basic dependency layer, with no enhanced dependencies.Source: §5.5 · p. 17

5.6. Annotation Consistency

Repeated sequences show high within-annotator stability, though this is not inter-annotator agreement; model iterations were trained on growing data pools using a fixed, reproducible setup.

5.6. Annotation Consistency

  • 99.30% UPoS, 100% head-attachment, and 99.65% dependency-relation agreement were observed across repeated word sequences in different rounds.The two sequences annotated differently involved analytic choices, not errors.Source: §5.6 · pp. 17–18, Table 5 · p. 18
  • These agreement figures measure one annotator’s stability over time, not agreement between annotators, and repeated sequences are a favorable test case that bounds corpus-wide consistency.Near-duplicate comparisons approximately doubled the sample while retaining high agreement.Source: §5.6 · p. 18

5.7. Training and Data Partitioning

  • The annotated pool grew from 204 to 1,804 sentences, with each iteration shuffling and splitting the cumulative batches into training, development, and held-out portions.Because Stanza split the training portion again for development, the effective training set was 72% of the pooled sentences.Source: §5.7 · p. 19
  • Models were retrained from scratch with fixed configuration, so the learning trajectory combines the effects of adding data and retraining rather than isolating each batch’s contribution.The curve is best read as a dataset-size curve; no cross-validation was performed.Source: §5.7 · p. 19

5.8. Evaluation Design

Evaluation used an independently annotated fixed gold standard and complementary metrics for morphology, syntax, annotation effort, and model validation.

5.8.1. The Gold Standard

  • The fixed gold standard comprised 200 independently sampled and annotated sentences, evaluated alongside morphological, syntactic, and annotation-effort measures.It contained 1,441 tokens and 1,241 non-punctuation words.Source: §5.8.1 · p. 19, §5.8 · p. 19

5.8.2. Morphological Metrics

  • Morphological evaluation scored tagging and features at token level, lemmatization by exact match, and out-of-vocabulary rate against distinct gold-standard lemmata.Lemma scores were reported separately because lemmata came predominantly from the domain lexicon.Source: §5.8.2 · p. 20

5.8.3. Syntactic Metrics

  • Syntactic scores followed CoNLL 2018 conventions, while enhanced dependency scores were omitted because neither gold nor model outputs contained an enhanced layer.Construction-specific comparisons used heuristic classes, some of which were small and yielded noisy estimates.Source: §5.8.3 · p. 20

5.8.4. Annotation Effort

  • Annotation effort was measured by elapsed correction time and by the proportion of pre-annotated tokens whose head or dependency label changed.The token-level rate was recoverable retrospectively and correlated closely with elapsed-time measurements.Source: §5.8.4 · p. 20

5.8.5. Model Validation

  • Each model was evaluated on the same fixed gold standard, and iteration improvements were assessed with 10,000-sample sentence-level bootstrap resampling and paired t-tests.The two procedures were used to reduce spurious confidence from a modest evaluation set and to cross-check results.Source: §5.8.5 · pp. 20–21

5.9. Stopping

The experiment stopped after nine rounds based on the annotator’s judgment, without a formal convergence criterion. Retrospectively, correction rate and annotation time tracked evaluation plateaus and offer possible stopping signals for future projects.

5.9. Stopping

  • The experiment stopped after nine rounds when the annotator judged that correction was no longer substantially teaching the model; the correction-rate plateau after round four supported that judgment only retrospectively.The stopping-point analysis describes where returns diminished but did not guide this experiment.Source: §5.9 · p. 21
  • Correction rate flattened when evaluation metrics began to plateau and requires no held-out gold standard, making it a prospective stopping signal for projects that cannot afford one.Annotation time per sentence behaved similarly; both signals are recommended based on their behavior here.Source: §5.9 · p. 21

6. Results

Across nine iterations, annotation improved tagging and parsing, and the final model outperformed the treebank baselines on reported metrics with substantially less training data. The section also examines early tagging instability, vocabulary coverage, code-switching, and construction-type evaluation.

6.1.1. Morphological Performance

  • UPoS accuracy rose from 0.743 at s1 to 0.977 at s9, while All Tags increased from 0.570 at baseline to 0.915 at s9.Lemmata stayed essentially flat, from 0.948 at s1 to 0.953 at s9, with no significant change; tagging progress included plateaus and discrete gains.Source: §6.1.1 · pp. 21–22, Figure 4 · p. 22

6.1.2. Syntactic Performance

  • UAS increased from 0.648 to 0.945, LAS from 0.482 to 0.917, and MLAS from 0.244 to 0.822, a 237% relative improvement.The reported improvements were statistically significant except for Lemmata; final-round UAS and LAS dipped slightly, with swings corresponding to roughly ten tokens on the 1,441-token gold standard.Source: Table 7 · p. 22, §6.1.2 · pp. 22–23

6.3.1. An Early Dip in Tagging

  • Adpositions accounted for approximately 85% of the 8.3-point UPoS dip at s2, while nouns and numerals improved from the first iteration.The analysis attributes the remaining effect to probability redistribution under a small, skewed training sample; vector rebuilds coincided with later step changes, so those gains cannot be attributed solely to annotation.Source: §6.3.1 · pp. 23–24, Figure 5 · p. 25

6.2. Cross-Treebank Performance Comparison

  • The final model outperformed every baseline on every reported metric using 14,516 in-domain tokens versus ITTB’s 450,554—approximately 97% less training data.Table 8 compares the final model with each treebank baseline across the reported metrics.Source: §6.2 · p. 23, Table 8 · p. 24

6.3.2. Vocabulary Coverage

  • The out-of-vocabulary rate fell from 68.0% at baseline to 13.3% at s9, a 54.7-point reduction, while s1 accounted for more than half the total decline.At s9, out-of-vocabulary lemmata remained less accurate than in-vocabulary items (86.5% versus 97.3%), and Occitan made up 42.9% of the remaining uncovered lemmata.Source: §6.3.2 · p. 25

6.3.3. Code-Switching

  • The gap between Latin-only and mixed-language sentences narrowed on UPoS, UAS, and LAS, but mixed sentences remained roughly four points harder on parsing at s9.The UPoS gap narrowed from 3.39 to 0.88 points, the UAS gap from 7.93 to 3.91, and the LAS gap from 6.34 to 3.64.Source: §6.3.3 · pp. 25–26

6.3.4. Construction Types

  • The 200 gold-standard sentences were classified by construction type using surface heuristics, and accuracy was measured for each class across the model chain.The classes included simple versus complex object descriptions, single- versus multiple-object listings, and code-switching.Source: §6.3.4 · p. 26

UPoS LAS

The results show gains across construction types, while annotation effort falls and the final rounds yield uneven returns. Residual errors increasingly reflect lexical gaps and genuine ambiguity rather than failures to learn the corpus’s recurring structure.

UPoS LAS

  • Code-switching gained 19.0 UPoS and 45.1 LAS points; vernacular-interference constructions improved most on both metrics, approaching simple enumerations in tagging within four points.Complex single-object descriptions gained 18.0 UPoS and 37.4 LAS points; simple enumerations reached 0.989 LAS.Source: Table 9 · p. 26, UPoS LAS · p. 26
  • Improvements are measured against ITTB, whose strength varies by construction; two multi-object classes have only one or two sentences, making their gains unstable.Source: UPoS LAS · p. 26

6.4.1. Annotation Time

  • Correction time fell from 2.57 to 0.50 minutes per sentence, while corrected tokens declined from 54.3% to 14–18% from s5 onward.The correction rate stabilized rather than continuing to fall.Source: §6.4.1 · p. 26, Table 10 · p. 27

6.4.2. Returns to Annotation Effort

  • LAS returns were uneven: s6 and s8 were most productive per hour, while s8 accounted for the final LAS gain and 98% of MLAS improvement.LAS was slightly higher at s8 than s9 (0.924 versus 0.917), a difference within expected sampling variance.Source: §6.4.2 · p. 27, Figure 6 · p. 27
  • For this formulaic corpus, most attainable gains accrued within roughly 1,200 sentences and 27 hours; these noisy results do not establish a general stopping rule.The final third of effort yielded refinement rather than transformation.Source: §6.4.2 · pp. 27–28
  • The 33.2-hour correction chain moved LAS from 0.482 to 0.917, but cross-project effort comparisons are unreliable across heterogeneous units and annotation schemes.The reported rate is 1.11 minutes per sentence for a full morphosyntactic correction pass; shallower schemes should be faster and annotation from scratch considerably slower.Source: §6.4.2 · p. 28

6.5. Qualitative Analysis of Model Learning

  • At s9, 174 of 1,441 gold-standard tokens had an error in UPoS, lemma, head, or relation; including XPoS and morphology raises this to 226 tokens.Errors overlap across categories.Source: §6.5 · p. 28
  • Lemmatization errors chiefly involve vernacular or technical vocabulary absent from the domain lexicon, while syntactic errors often reflect genuine ambiguity in names and trailing qualifications.Fifty-one of 69 word forms incorrectly lemmatized at s1 remained incorrect at s9; approximately one third of residual errors could plausibly be resolved differently by another annotator.Source: §6.5 · pp. 28–29

7. Discussion

The discussion limits generalization to corpora with suitable structure and a usable starting model, and distinguishes what iterative correction can learn from what it cannot. It also specifies the study’s scope, constraints, and proposed directions for further testing.

7.1. Generalization to Other Corpora

  • The procedure’s external generalizability is limited by its single-corpus evaluation; formulaic administrative genres and asymmetric language mixing are plausible settings, not universal cases.A usable existing treebank model is a precondition; a 200-sentence batch and one correction round can test whether pre-annotation is worthwhile.Source: §7.1 · p. 29
  • The procedure depends on philological and syntactic expertise, and orthographic instability calls for a lexicon rather than expecting iteration to resolve lemmatization.Coordination across several people can undermine consistency.Source: §7.1 · pp. 29–30

7.2. The Limits of Purely Computational Approaches

  • The baseline evaluation used five Latin treebanks without further tuning; it did not test fine-tuning, ensembling, or synthetic data.The target gap is primarily lexical and structural, and reliable generation of historical variants is unavailable for augmentation.Source: §7.2 · p. 30
  • Iterative correction teaches recurrent syntax more readily than lexical coverage: learned syntactic judgments generalize across recurring constructions, but token-specific lexical knowledge does not.Residual lemmatization errors require external lexical resources and editorial effort.Source: §7.2 · p. 30

7.3. Adequacy for Scholarly Use

  • For corpus linguistics, 0.98 UPoS and 0.92 LAS support systematic querying of syntactic and morphological patterns, but not claims about individual sentences.Source: §7.3 · pp. 30–31

7.4. Limitations and Constraints

  • Interpretation is constrained by length-stratified sampling without uncertainty selection, one annotator, no formal stopping criterion, retraining from scratch, and embedding-set changes.The learning curve therefore conflates data accumulation with restart effects and includes an additional source of variance.Source: §7.4 · p. 31
  • Transfer is limited by dependence on formulaic structure, the corpus’s asymmetric code-switching, and treating a single-city tradition spanning under two centuries as effectively synchronic.Less repetitive corpora may plateau earlier at higher residual error; the code-switching gains should not be generalized to all multilingual materials.Source: §7.4 · p. 31

7.5.1. Technical and Methodological Improvements

  • Proposed methodological extensions include active-learning comparisons, joint multitask training, transformer encoders, lexicon expansion, and interface support for recurring forms.Active-learning tests could hold other conditions fixed; transformer effects on annotation demand versus performance ceiling remain open.Source: §7.5.1 · pp. 31–32

8. Conclusion

The study concludes that iterative human correction yields usable models for this documentary corpus, and identifies correction rate as a practical progress signal. It also makes the annotated data, code, and iteration models available for reuse and verification.

8. Conclusion

  • Relative to ITTB, tagging and morphological scores rise from 57–80% to 92–98%, while syntactic scores rise from 17–65% to 82–95%.Lemmatization is excluded because it is chiefly supplied by the domain lexicon rather than learned by the iterative procedure.Source: §8 · pp. 32–33

8.2. Data and Code Availability

  • The released corpus, gold standard, lexicon, code, and nine iteration models support reuse and independent verification of the analyses.The 1,804 corrected sentences and 200-sentence gold standard are released in CoNLL-U format.Source: §8.1 · p. 33, §8.2 · p. 33

8.1. Materials for Reuse

  • Correction rate per round can be tracked without a gold standard and, in this study, aligns closely with evaluation metrics as a pragmatic stopping signal.The annotation rate is also offered as a replicable planning benchmark when reported with its annotation scheme.Source: §8.1 · p. 33
Loading 2609.06266v1…