Source-linked AI summary

Source-preserving alignment for robust evidence localization in scientific PDFS

Zihao Liu, Wei Yang, Zixiao Dong, Chenshu Li, Longzhang Liu, Tao Tan, Hong Xie

arXiv:2609.35588v1cs.AI

TL;DR

Scientific evidence strings can differ from PDF text because of layout, encoding, and fragmentation, making precise localization difficult. The paper aligns normalized text while preserving source-character provenance, then renders matched page geometry. It reports substantially better localization than text-search and precomputed bounding-box baselines, especially for long evidence strings.

  • Problem

    Evidence strings from logical formats must be located in PDF text despite line breaks, Unicode variants, superscripts, citation markers, and fragmented items.

    Source: §1 · p. 1
  • Method

    The framework normalizes text for matching while preserving links from normalized characters to source items and offsets for precise localization.

    Source: §1 · p. 1, §3.2 · p. 2
  • Results

    The framework substantially improves localization over text-search and precomputed-BBOX baselines, especially for long evidence strings, with normalization and approximate token alignment both contributing.

    Source: §4.5 · p. 3, §6 · p. 4
  • Takeaways & Limitations

    Reliable evidence verification requires robust matching and precise localization within a shared source-preserving alignment representation.

    Source: §6 · p. 4

    The current implementation estimates character geometry from item width, so variable-width fonts can yield loose boundaries.

    Source: §5 · p. 4

Abstract

from arXiv · show

Scientific information-extraction systems often return a claim with an evidence string, which users must locate in the original PDF. This is challenging because the extracted evidence and PDF text layer are different representations: line wrapping, Unicode variants, superscripts, citation markers, and fragmented items alter text sequences and geometry. We present a source-preserving alignment framework: normalize text for robust matching while preserving provenance for accurate localization. It aligns evidence with normalized page text, maps matches back to source-character spans, and renders only their geometry. When exact alignment fails, line-break-aware token alignment recovers supported spans while excluding unmatched noise. Experiments on 1,020 chemistry papers show that the framework achieves a 92.6\% quote-level automatic localization rate, compared with 43.6\% for text search and 19.1\% for a precomputed bounding-box baseline. Component ablation confirms distinct contributions from normalization and approximate token alignment, while human verification assesses the visual correctness of returned highlights. Overall, these results demonstrate that reliable evidence verification requires robust matching and precise localization within a shared source-preserving alignment representation.

1. INTRODUCTION

Evidence from scientific information-extraction systems must be located on the original PDF page despite differences between logical text and PDF text. The paper proposes preserving source-character links through normalization so matching can produce precise page highlights.

1. INTRODUCTION

  • Evidence strings must be matched to PDF text despite line breaks, Unicode variants, superscripts, citation markers, and fragmented items.These differences arise from the PDF text model and character encoding.Source: §1 · p. 1
  • Text search is sensitive to normalization differences, while precomputed bounding boxes cannot handle unexpected text and are sensitive to layout or encoding changes.Neither baseline preserves the character-level link to source geometry needed for accurate highlighting.Source: §1 · p. 1
  • The framework aligns normalized text, source characters, and page geometry, mapping exact or approximate matches to source spans for rendering.Its central principle is to normalize for matching while preserving provenance for localization.Source: §1 · p. 1

2. RELATED WORK

Prior work provides scientific text, extraction tools, and methods for handling textual errors, but these do not by themselves ensure traceable visual source spans. Evidence localization additionally projects matched text through source provenance to page geometry.

2. RELATED WORK

  • PDF libraries expose page strings, text items, and geometry, while neural parsers improve structured extraction without guaranteeing traceable visual source spans.Standard search is fragile to extraction-order, encoding, and layout differences; bounding-box indexes are limited by text-to-box correspondence.Source: §2 · p. 1
  • Scientific PDFs fragment or encode equations, formulas, Greek letters, superscripts, and reference markers nonstandardly, amid substantial layout variation.A robust locator therefore needs tolerant comparison together with faithful recovery of source characters and geometry.Source: §2 · pp. 1–2
  • Approximate matching, sequence alignment, and LCS identify correspondences despite textual errors or gaps [16–18].This paper's token-level LCS fallback projects supported tokens through a provenance map to source characters and page geometry.Source: §2 · p. 2

3. PROPOSED METHOD

The method normalizes quote and page text for matching while preserving links to original text items and character offsets. It tries exact matching first, then uses token-level alignment for disrupted text, and renders only the supported source geometry.

3.1. Deterministic Matching Procedure

  • Fixed rules search for an exact occurrence first, then enumerate candidate windows for fallback matching when exact matching fails.The fallback is accepted only at score ≥0.8; otherwise, no fallback match is returned.Source: §3.1 · p. 2

3.2. Normalization and Provenance Mapping

  • Normalization unifies Unicode and dash variants, repairs malformed encoding, maps scientific symbols, and removes layout-only separators.Each normalized character retains its source item and character offset, allowing matches to project back to source intervals; original page text is not rewritten.Source: §3.2 · p. 2

3.3. Exact Matching and Character-Range Clipping

  • For exact matches, the method renders only matched character intervals and clips the transformed rectangle to the page viewport.Horizontal bounds are estimated proportionally from item width, preventing a quote ending mid-item from highlighting the following sentence.Source: §3.3 · p. 2

3.4. Line-Break and Noisy-Text Alignment

  • When exact matching fails, token windows at several lengths are compared using LCS; only aligned tokens are highlighted, excluding unaligned markers.Alphabetic line-end hyphenation is merged when fragments form a word, while formula-like strings such as “ZSM-5” are preserved.Source: §3.4 · p. 2

3.5. Geometric Rendering

  • Rendering transforms text-rectangle corners and merges nearby fragments only on the same rendered line.This avoids merging across columns, headers, or unrelated text regions.Source: §3.5 · p. 2

4. EXPERIMENTS AND ANALYSIS

The experiments use a paired corpus of chemistry-paper PDFs and Markdown evidence files and compare the proposed framework with direct PDF search and an offline bounding-box index.

4.1. Dataset

  • The benchmark contains 1,020 chemistry papers and 18,580 quote instances, averaging 18.2 quotes per paper.Only papers with both PDF and Markdown files were retained, so the benchmark evaluates localization rather than upstream claim extraction.Source: §4.1 · p. 2
  • The corpus includes varied quote lengths, multi-column pages, chemical formulas, figure references, and noisy text layers.These features expose ordinary search failures and boundary errors after apparently successful matches.Source: §4.1 · p. 2

4.2. Comparison Methods

  • The comparison evaluates the proposed source-preserving method against PyMuPDF direct lookup and an offline page-text and bounding-box index.The proposed method combines source-preserving normalization, alignment, source-span recovery, and geometry-aware rendering.Source: §4.2 · p. 3

4.3. Evaluation Metrics and Human Verification Protocol

The protocol distinguishes candidate availability from visual correctness: automatic localization counts returned regions, while manual review assesses the highlights. The proposed method leads the automatic comparison.

4.3. Evaluation Metrics and Human Verification Protocol

  • Manual verification covers 18,572 quotes per method, with Correct requiring fully accurate highlighting and Incorrect including boundary excess or omission.The remaining eight of 18,580 quotes have no manual labels.Source: §4.3 · p. 3

4.4. Overall Performance

  • 92.6% automatic localization rate for the proposed method exceeds 43.6% for text search and 19.1% for precomputed BBOX.Automatic localization measures whether a method returns at least one candidate region for a quote.Source: §4.4 · p. 3, §4.3 · p. 3

4.5. Component Ablation

Component ablations distinguish gains from normalization, approximate alignment, and character-range rendering. Across evidence lengths, the proposed method remains ahead of both baselines for longer quotes, despite a short-quote exception.

4.5. Component Ablation

  • Normalization provides a larger localization-rate gain than approximate alignment, which further recovers disrupted token sequences.The ablation removes each component separately on the same 1,020 papers and 18,580 quotes.Source: §4.5 · p. 3, Figure 2 · p. 3
  • Whole-item rendering adds 62.8 source characters per quote on average, while character-range clipping introduces no whole-item excess.This rendering-only comparison holds matched source spans fixed across 17,213 successfully localized quotes and does not change localization decisions.Source: §4.5 · p. 3

4.6. Robustness to Evidence Length

  • The proposed method peaks at 81–160 characters, then declines but remains substantially ahead of both baselines for longer quotes.BBOX slightly exceeds the proposed method in the shortest group.Source: §4.6 · p. 3, Figure 3 · p. 4
  • Longer evidence can cross text items and line breaks or contain marker insertions and encoding variation; provenance mapping excludes unmatched content from highlights.PyMuPDF peaks at 41–80 characters before declining, while BBOX declines across all groups.Source: §4.6 · pp. 3–4

4.7. Human Evaluation

Six laboratory evaluators assessed localization outputs against evidence in the original PDFs. The proposed method has the highest Correct rate, while both baselines have larger Not found shares.

4.7. Human Evaluation

  • Six evaluators randomly assessed quote–method outputs by comparing each highlight with its evidence in the original PDF.Correct required exact location and boundaries; any positional error, excess, or omission was Incorrect, and no usable localization was Not found.Source: §4.7 · p. 4
  • The proposed method achieves the highest Correct rate, while both baselines have larger Not found shares.Figure 4 summarizes the three-category results.Source: §4.7 · p. 4

4.8. Error Analysis

Remaining errors stem from layout and geometry approximations or extraction failures, while source-character provenance enables tighter highlight boundaries. The framework assumes a usable PDF text layer.

  • Residual errors mainly reflect disrupted extraction order in complex layouts, approximate character widths, or extraction errors beyond the normalization rules and LCS threshold.Source: §4.8 · p. 4
  • The framework assumes that the PDF has a usable text layer.Source: §4.8 · p. 4
  • Character-level provenance crops highlights at the final evidence character, excluding unsupported text that whole-item highlighting would include.Source: §4.8 · p. 4

5. LIMITATIONS AND FUTURE WORK

The implementation estimates character geometry from item width, which can loosen highlight boundaries with variable-width fonts; proposed future work targets geometry and large-scale verification.

5. LIMITATIONS AND FUTURE WORK

  • Variable-width fonts can produce loose highlight boundaries because character geometry is estimated from item width.Source: §5 · p. 4
  • Glyph-level metrics or native text-layer positioning could address the character-geometry limitation.Source: §5 · p. 4
  • Layout-aware analysis, confidence calibration, and lightweight correction are proposed to improve reading order and support large-scale verification.Source: §5 · p. 4

6. CONCLUSION

The conclusion presents a source-preserving framework for robust matching and precise highlights. It reports gains over text-search and precomputed-BBOX baselines, with both normalization and approximate token alignment contributing.

6. CONCLUSION

  • The framework maps normalized evidence through source characters to page geometry for robust matching and precise highlights.Source: §6 · p. 4
  • The method substantially improves over text-search and precomputed-BBOX baselines, especially for long evidence strings.Source: §6 · p. 4
  • Ablation results show that both normalization and approximate token alignment are necessary for the reported gain.Source: §6 · p. 4
Loading 2609.35588v1…