PaperlayerView exact revision on arXiv

Source-linked AI summary

OCR-Based Field Extraction for Archaeological Pottery Metadata: The CENTURIA Dataset

Gissu Valentina Naghavi, Dominik Hagmann, Martin Kampel, Irene Ballester

arXiv:2608.30616v1cs.CV

TL;DR

Handwritten pottery annotations are important for archaeological analysis but remain difficult to process computationally. This paper introduces CENTURIA and benchmarks OCR, extraction, and adaptation strategies, finding that LoRA fine-tuning on 57 samples reduces transcription error below 1.5 % and raises overall field accuracy above 87 %.

  • Problem

    Handwritten archaeological annotations remain inaccessible to computational analysis, while existing methods do not adequately address scattered annotations and domain-specific vocabulary.

  • Method

    The paper introduces CENTURIA, a 507-record Carnuntum dataset, and evaluates five OCR models, adaptation strategies, and field-extraction methods.

  • Results

    LoRA fine-tuning on 57 samples reduces transcription error below 1.5 % and lifts overall field accuracy above 87 %, while cross-pipeline agreement auto-accepts 56.4 % of documents at 95.5 % precision.

  • Takeaways & Limitations

    A small expert-validated fine-tuning set can convert handwritten pottery documentation into structured metadata for archaeological database use.

  • Takeaways & Limitations

    Remaining failures include domain-specific token omissions, digit errors, unresolved percentage signs, and unfiltered struck-through text.

Abstract

from arXiv · show

Pottery is a primary source for reconstructing the chronological and economic dimensions of past societies. Archaeologists often document ceramic finds through technical drawings and handwritten metadata. This metadata is critical for dating, provenance attribution, and cross-site comparison, but remains inaccessible to computational analysis, requiring manual transcription of every record. We investigate whether state-of-the-art document analysis models can address this task, and introduce CENTURIA, a dataset of 507 pottery records from the Roman site of Carnuntum, providing transcriptions, bounding boxes, and structured field-level labels across seven metadata categories. Benchmarking five OCR models reveals a substantial domain gap: zero-shot transcription error reaches 15-32% SpACER-M, far exceeding rates on printed archival documents, with domain-specific fields recovered in fewer than 3% of cases. LoRA fine-tuning on just 57 samples, reflecting a realistic archival annotation budget, closes this gap, reducing transcription error to below 1.5% and recovering overall field-level accuracy above 87%. Our results show that a small expert-validated fine-tuning set suffices to convert handwritten pottery documentation into structured, searchable metadata ready for archaeological databases.

1 Computer Vision Lab, TU Wien, Vienna, Austria

The paper addresses the conversion of analogue pottery documentation into machine-readable metadata and introduces CENTURIA for systematic benchmarking.

  • Handwritten annotations in archaeological records remain inaccessible to computational analysis.
  • CENTURIA benchmarks automated conversion of archaeological documentation into machine-readable metadata.
  • The dataset targets handwritten text recognition and information extraction in pottery documentation.

1 Introduction

Archaeological pottery records contain valuable handwritten information, but their scattered annotations and specialised terminology resist direct computational analysis. The paper introduces CENTURIA and evaluates OCR adaptation strategies, showing that small-sample LoRA fine-tuning substantially improves transcription and field extraction.

  • Pottery documentation supports reconstructing past societies, but handwritten annotations carrying excavation and typological information require intensive manual transcription.
  • Existing HTR benchmarks do not reflect archaeological archives with spatially scattered annotations and specialised vocabulary.
  • Key Information Extraction maps transcribed text onto semantic fields needed for archive-scale retrieval and aggregation.
  • 15–32 % zero-shot transcription errors fall below 1.5 % after LoRA fine-tuning on 57 samples, while overall field-level accuracy exceeds 87 %.
  • CENTURIA provides 507 expert-validated samples for benchmarking handwritten archaeological record transcription and field extraction.
  • 56.4 % of scans are auto-accepted at 95.5 % field precision, rising to 98.7 % for fields confirmed by all four pipelines.
  • The complete dataset, fine-tuned checkpoints, and code are released under CC-BY-4.0.

2 Related Work

Prior archaeological document-analysis work focuses mainly on visual recognition or printed records, while existing HTR and KIE benchmarks do not represent handwritten archaeological documentation. CENTURIA addresses this gap with a benchmark for scattered annotations, technical terminology, and structured extraction.

  • Archaeological computer vision has addressed artefact recognition, geometric extraction, and printed-document metadata, but not a complete handwritten-record solution.
  • Existing HTR and KIE methods span specialised systems, detection-recognition pipelines, full-page VLMs, and structured schema mapping.
  • CENTURIA document crops include high-quality, low-contrast, crossed-out, and drawing-overlapping handwriting conditions.
  • Transferability to handwritten archaeological archives with scattered annotations and no canonical reading order remains unaddressed.
  • Existing heritage benchmarks cover administrative, genealogical, ecclesiastical, and census records rather than archaeological field documentation.

3 CENTURIA Dataset

CENTURIA is a manually annotated dataset of 507 analogue pottery records from Carnuntum, designed to support reproducible recognition and structured metadata extraction. Its records capture handwritten text, technical drawings, and seven-category field annotations across diverse archival conditions.

  • Dataset composition: 507 annotated scans from Carnuntum include transcriptions, text and drawing bounding boxes, and structured field-level values.The dataset covers analogue pottery records from the Roman site and uses a predefined train/test split of 57 and 450 documents.
  • Dataset composition: The source archive contains approximately 70,000 records, while CENTURIA samples documents from six campaigns spanning 1976–2017.Sampling captures variation in handwriting style, scribal conventions, annotation density, and document condition.
  • Annotation process: All 507 documents required manual correction, averaging 70 seconds per record; 9.47% also required expert consultation because the scan alone was ambiguous.Field annotations were verified by a Carnuntum site specialist, with corrections in 3.75% of documents.
  • Field annotations: Each transcription is mapped to seven semantic categories, with representative fields including pottery form and artefact category.The schema includes optional fields and assigns unrecognised text to an others category.

4 Method

The method converts a scanned pottery record into structured metadata through two stages: handwritten text recognition followed by field extraction. It evaluates multiple OCR model paradigms and applies post-processing before mapping transcription output to fields.

  • Pipeline: The pipeline first recognizes handwritten text from a single-page scan, then extracts structured metadata fields from the transcription.The transcription is post-processed to normalize formatting artefacts before extraction.
  • OCR models: The benchmark spans TrOCR, olmOCR, olmOCR2, Florence2, and LightOnOCR across detection-recognition, full-page VLM, generalist, and lightweight model paradigms.All models are open-source to support reproducible and cost-free deployment for heritage practitioners.

LoRA fine-tuning*

The method compares OCR architectures and adaptation strategies before extracting seven metadata categories with either deterministic rules or an LLM. Its design separates recognition quality from downstream field-mapping behavior.

  • OCR model paradigms: TrOCR detects text regions with CRAFT and recognizes each region independently, whereas full-page models process the entire document image.TrOCR therefore depends directly on detection quality and lacks full-document context.
  • OCR model paradigms: LightOnOCR is a 1B-parameter end-to-end full-image VLM positioned at the lightweight end of the baseline model spectrum.It uses a ViT encoder, spatial patch merging, and a Qwen3 decoder without task prompts at inference.
  • Adaptation strategies: A domain gap in archaeological vocabulary and annotation conventions motivates zero-shot instruction prompting, few-shot prompting, and LoRA fine-tuning.Few-shot prompting uses 1, 5, or 8 annotated examples, while LoRA adapts weights using 57 training samples.
  • Field extraction: Field extraction maps each processed transcription to seven semantic categories and produces field-value pairs whose values may be null.Because field boundaries are unmarked, the mapping relies entirely on textual patterns and domain vocabulary.
  • Field extraction: REGEX extraction matches categorical values against controlled vocabularies and open-domain values against field-specific patterns.Matched substrings are removed to prevent duplicate assignments, and unmatched text is assigned to others.
  • Field extraction: The LLM extractor prompts unmodified Qwen3-8B with the cleaned transcription and schema, returning a structured JSON object of field-value pairs.It uses eight corrected transcription examples, which perform slightly better than corrupted examples, particularly for publication type.

5 Evaluation

The evaluation measures transcription, field extraction, and efficiency across five OCR models, then tests domain adaptation and extraction pipelines. Zero-shot performance is poor, while LoRA fine-tuning on 57 samples substantially improves transcription and field-level recovery.

  • Evaluation setup: Five OCR models are evaluated for transcription quality, while olmOCR2 and LightOnOCR are further evaluated for adaptation, field extraction, confidence estimation, and processing time.The evaluation separates transcription from downstream extraction and records timing for scalability.
  • Baseline performance: SpACER-M reaches 15.58–32.35% in zero-shot transcription, while the strongest baseline, olmOCR2, reaches 30.55% EMR and 57.43% ANLS.These results indicate that zero-shot outputs rarely recover field values verbatim.
  • Domain adaptation: 15.58% → 6.77% SpACER-M error follows zero-shot instruction prompting for olmOCR2, but few-shot prompting causes LightOnOCR error to reach 48.71% at 1 shot and 100% at 5 shots.LoRA fine-tuning outperforms prompting, reducing SpACER-M error below 2% for both models with 57 samples.
  • Field-level performance: ∼90% overall EMR is achieved after LoRA fine-tuning, whereas baseline models recover measurements in 65–80% of cases and pottery form in 26–37%.Artefact Category and Publication Type have near-zero baseline EMR because OCR noise corrupts abbreviations and domain tokens.
  • Extraction accuracy: 91.73% vs. 88.30% overall extraction accuracy under REGEX favors fine-tuned olmOCR2 over fine-tuned LightOnOCR.The largest reported gaps occur on Publication Type and Surface Treatment, at 14.91 and 20.37 percentage points.
  • Confidence and review: 98.7% of fields are correct when all four fine-tuned model–extractor combinations agree, while partial-agreement rules auto-accept 56.4% of documents at 95.5% precision.Manual review is concentrated on scans with persistent disagreement.

6 Limitations and Future Work

CENTURIA and its pipeline show promising performance, but important limitations remain in failure handling, validation, human evaluation, and transfer beyond Carnuntum. Future work must address unresolved field errors, agreement-based reliability, real-world workflow costs, and cross-archive generalisation.

  • Remaining failure modes: Short domain-specific tokens and digit-level structured-field errors remain systematic failure modes after fine-tuning.Percentage radius measurements remain unresolved because no pipeline reliably transcribes the % sign.
  • Remaining failure modes: Crossed-out text is transcribed alongside corrections, introducing noise into downstream field extraction.
  • Quality validation and human evaluation: 98.7 % accuracy on unanimously confirmed fields and 95.5 % precision on 56.4 % auto-accepted scans are conditional on agreement-selected subsets.The pipelines are also not fully independent, so agreement may overestimate reliability.
  • Quality validation and human evaluation: Time savings and error-correction effort with domain experts in real archival settings remain unevaluated.The paper plans to formalise a full human-in-the-loop workflow.
  • Transferability and scope: Cross-archive generalisation depends on each archive’s recording conventions, script, and language, and performance scaling with annotation effort remains open.CENTURIA provides evidence only for within-archive diversity across six Carnuntum campaigns.
  • Transferability and scope: Restricted access to archaeological collections makes cross-archive evaluation non-trivial.

7 Conclusion

The conclusion presents CENTURIA as a benchmark showing that transcription quality, rather than extraction method, governs end-to-end performance on handwritten pottery documentation. Fine-tuning and agreement-based triage make structured archival processing practical within the Carnuntum collection.

  • CENTURIA contains 507 annotated handwritten pottery records from Carnuntum and benchmarks five OCR models for HTR and structured field extraction.
  • 15–32 % SpACER-M error in zero-shot models falls below 1.5 % after LoRA fine-tuning on 57 samples.
  • Overall field accuracy rises above 87 %, after which structured metadata recovery follows from either REGEX or LLM-based extraction.
  • 56.4 % of documents are auto-accepted at 95.5 % field precision, rising to 98.7 % where all four pipelines agree.This concentrates manual effort on disagreements.
  • A small expert-validated annotation effort is enough to begin converting Carnuntum’s handwritten pottery documentation into searchable, structured metadata.

A Dataset Details

CENTURIA samples handwritten pottery documentation from six Carnuntum campaigns and represents variation in handwriting, conventions, notation, and document structure. Its annotations preserve spatial, textual, and semantic information for structured analysis.

  • The dataset draws on six Carnuntum excavation and field-walking campaigns conducted between 1976 and 2017.Sampling covers eight strata and captures variation across campaigns and documentation periods.
  • Campaigns differ in find-identifier conventions and measurement notation despite sharing a common annotation schema.
  • Bounding boxes follow left-to-right, top-to-bottom reading order, with splits for heavily slanted or irregular handwriting.Non-overlapping boxes reduce character duplication in spatial metrics such as SpACER.
  • Transcriptions preserve capitalisation, spacing, special characters, and abbreviations exactly as written.Measurements use a single space between value and unit, such as 12 cm.
  • Field labels are assigned by semantic content across seven categories rather than by spatial position.The schema includes excavation, pottery form, measurement, artefact category, publication type, and surface treatment fields among its categories.
  • The data are distributed as transcription and field-annotation JSON files containing text, bounding boxes, reading order, drawing indices, and structured field dictionaries.

B Implementation Details

The implementation benchmarks five OCR models with standardized inference and post-processing, then adapts olmOCR2 and LightOnOCR using LoRA, domain prompts, and few-shot examples. Field extraction uses a schema-constrained Qwen3 setup.

  • Experiments run on NVIDIA GeForce RTX 3090 GPUs with model-specific numerical precision and batch size 1 for VLMs.TrOCR batches all EasyOCR-detected text regions within each image.
  • Five OCR checkpoints are evaluated: TrOCR, olmOCR, olmOCR2, Florence2, and LightOnOCR.
  • Post-processing removes LaTeX markup, Markdown image references, hyperlinks, URLs, and appended model commentary.
  • For olmOCR, transcription comes from the natural_text JSON field; TrOCR and Florence2 segments are concatenated in reading order before cleaning.
  • olmOCR2 and LightOnOCR are fine-tuned for two epochs on n = 57 training samples using LoRA.LoRA ranks are r = 16 for olmOCR2 and r = 8 for LightOnOCR.
  • A domain-context prompt supplies vocabulary for site abbreviations, identifiers, vessel forms, ware types, measurements, surface treatments, and publication abbreviations.
  • Few-shot OCR examples are stratified across all six campaigns to provide representative coverage.The combined strategy applies the domain prompt in every conversation turn, including eight demonstration turns.
  • Qwen3-8B extracts fields greedily under a schema-constrained system prompt with twelve hard rules, field definitions, a fixed JSON template, and eight examples.

C.1 Per-field transcription analysis

Field-level performance varies substantially across OCR models and adaptation strategies, with transcription quality driving downstream extraction accuracy. LoRA fine-tuning improves robustness, while degraded scans and OCR noise remain the main sources of errors.

  • Per-field performance: 91.67% measurement EMR is reached by both LoRA fine-tuned models, while olmOCR records only 8.6% pottery-form EMR at baseline.The baseline olmOCR model omits pottery form in 384 of 450 images; its measurement ANLS is 64.4%.
  • Adaptation strategies: LoRA fine-tuning dominates prompting strategies, whereas few-shot prompting reduces olmOCR2 error but causes LightOnOCR repetition failures.LightOnOCR reaches 48.71% SpACER-M error at 1 shot and 100% at 5 shots; olmOCR2 reaches 5.38% at 5 shots.
  • Error sources: 60% of REGEX errors and 71% of LLM errors are shared by both models, indicating that OCR noise outweighs extractor choice.Digit substitutions and omitted rare domain tokens recur across crate numbers, publication years, measurements, and specialized vocabulary.
  • Error sources: 77% of LightOnOCR’s REGEX-unique errors and 73% of its LLM-unique errors account for most model-specific failures.Publication Type is LightOnOCR’s dominant weakness, including truncated inventory references and cut-off multi-reference entries.
  • Qualitative examples: On degraded scans, all four extraction combinations assign fields to the correct categories, but OCR value errors persist and one spacing error causes LightOnOCR FN extraction failure.Examples include lost decimal points, corrupted percentages, and REGEX/LLM failures caused by spacing artefacts.
  • Efficiency and output format: 1.42 s versus 2.95 s per document makes fine-tuned LightOnOCR faster than baseline, alongside an 86% reduction in output length.Average output falls from 509 to 72 characters, producing concise label-like transcriptions; olmOCR2 shows no speed improvement.
Loading 2608.30616v1…