PaperlayerView exact revision on arXiv

Source-linked AI summary

P2TPO: Structure-Aligned Page-to-Token Credit Assignment for Full-Page Document Parsing

Yong Zu, Ninghua Yang, Zezhong Tan, Xinhong Ma, Ziqiang Dong, Yang Yang

arXiv:2610.21979v1cs.LGcs.CLcs.CV

TL;DR

Full-page document parsing must preserve heterogeneous content and structure, but page-level feedback does not identify which elements differ in quality or where their errors occur. P2TPO aligns reference elements and allocates local token credit alongside page feedback. It outperforms evaluated response-level baselines across the reported benchmark splits, with additional settings showing broad gains.

  • Problem

    Page-level advantages do not identify element-specific quality differences or local errors and omissions in otherwise accurate full-page outputs.

    Source: §1 · pp. 1–2

  • Method

    P2TPO aligns generated spans to reference elements, completes missing and imperfect-consensus outcomes with UAP, and redistributes matched-block credit across aligned spans with SCR.

    Source: §1 · p. 2

  • Results

    P2TPO outperforms four response-level baselines on all 13 primary splits across six benchmarks in five-seed mean score.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Element-indexed contrasts retain local quality differences that fixed-weight additive page aggregation can erase.

    Source: §1 · p. 2

    Alignment and quality scores depend on annotation accuracy and serialization conventions, and experiments do not evaluate code or figures as distinct element types.

    Source: §6 · p. 9

Abstract

from arXiv · show

End-to-end full-page document parsing must preserve the content and structure of heterogeneous elements, yet otherwise accurate outputs can contain local errors and omissions. A single page-level advantage does not explicitly identify which elements differ in quality across sampled outputs or where their errors occur. We introduce Page-to-Token Policy Optimization (P2TPO), a structure-aligned credit-assignment framework that operates on full-page policy trajectories. Reference-element alignment first associates block-level quality contrasts with generated spans. Unified Anchor Pricing (UAP) incorporates matched and missing element outcomes and provides feedback relative to a fixed quality ceiling when sampled outputs receive identical but imperfect scores for a reference element. Within matched blocks, Span Credit Redistribution (SCR) redistributes existing block credit according to aligned correct and erroneous spans, using boundary proxies for within-block omissions while preserving the total block-derived advantage across its tokens. Under a fixed-weight additive page utility, we show that element-indexed contrasts retain local quality differences that page-level aggregation can cancel out. In the main experimental comparison, P2TPO outperforms all evaluated policy optimization baselines in seed-averaged scores on each of the six document parsing benchmarks. Additional experiments demonstrate gains across model scales and initializations.

1. Introduction

Full-page document parsing must preserve accurate text and structure across heterogeneous elements, but local errors and omissions can be obscured by page-level feedback. P2TPO aligns reference elements and assigns local credit while retaining page-level feedback; its reported comparisons show gains across document-parsing benchmarks.

1. Introduction

  • Full-page parsing transcribes digital documents, scans, and photographs into structured sequences, where dense content and capture conditions challenge accuracy and completeness.Errors can be local, such as a corrupted formula, omitted words, or a missing element.Source: §1 · p. 1
  • Page-level rewards can assign the same response advantage to every token even when sampled outputs have similar scores but opposite element-level errors.For example, one output may transcribe a formula correctly but omit a table, while another recovers the table but corrupts the formula.Source: §1 · p. 1
  • The feedback problem is to preserve reference-element identity, complete comparisons for missing or tied outcomes, and control credit placement within generated blocks.Centering equally imperfect scores diminishes their relative signal, and an omitted element has no generated span.Source: §1 · p. 2
  • P2TPO aligns outputs to reference elements, uses UAP for missing and imperfect-consensus outcomes, and applies SCR to redistribute matched-block credit while preserving its sign and total mass.Whole-element omissions route through existing boundary tokens, and the final update combines local feedback with a shared page term.Source: §1 · p. 2, Figure 2 · p. 3
  • P2TPO outperforms four response-level baselines on all 13 primary splits across six benchmarks, with additional model and initialization settings supporting its generality.The comparisons use a shared supervised checkpoint, matched update budgets, and five-seed runs; P2TPO has the highest mean in 37 of 39 split-setting comparisons across three settings.Source: §1 · p. 2

2. Method

P2TPO assigns token-level coefficients to sampled full-page outputs while retaining autoregressive page context and shared page feedback. Its pipeline aligns reference elements, completes their outcome comparisons, redistributes eligible matched-block credit, and combines local terms with page feedback.

2. Method

  • P2TPO assigns each generated token a coefficient from reference-aligned local feedback and a shared page term, while retaining full-page autoregressive conditioning.Matched-element coefficients act on generated blocks; whole-element omissions use a separate boundary-token route that bypasses SCR.Source: §2 · p. 3

2.1. Reference-element comparisons

  • Content-based sequence alignment indexes each matched prediction by its reference block, preserving distinct instances of the same category across rollouts even when earlier elements are omitted.Predictions and references are parsed into ordered text, formula, and table blocks.Source: §2.1 · p. 3

2.2. UAP: complete outcomes and retain quality deficits

  • UAP completes each reference element’s outcomes by assigning zero to missing matches and uses ceiling-gap feedback when outcome dispersion is negligible.For positive ceiling-gap weight, fully matched imperfect ties and all-missing groups receive negative coefficients, while a ceiling-quality tie receives zero.Source: §2.2 · p. 4

2.3. SCR: redistribute credit under a conserved budget

  • SCR redistributes eligible matched-block coefficients toward edit-aligned errors for negative credit and correct positions for positive credit, under a conserved block budget.Quality gating limits redistribution to high-scoring, type-matched blocks; within-block deletions are assigned to nearby generated positions, while whole-element omissions use UAP’s separate boundary route.Source: §2.3 · p. 4

2.4. Page-to-token composition

  • The page term is shared across generated positions, while local terms vary by reference identity and within-block location; a mixing weight balances these feedback scopes.Tokens outside local supports retain the page term.Source: §2.4 · pp. 4–5

3. Related Work

Related work spans document-parsing reinforcement learning, group-relative reward optimization, and structured local credit assignment. P2TPO focuses on comparisons between individual reference-block instances in full-page rollouts.

3. Related Work

  • Document-parsing systems use reinforcement-post-training rewards for transcription, layout, reading order, validity, and structural fidelity, while table- and element-specific approaches tailor prediction units and rewards to local content.P2TPO studies how feedback for individual reference elements reaches tokens within a full-page rollout.Source: §3 · p. 5
  • Group-relative methods construct response advantages without a learned critic, while later approaches revise sampling, clipping, reduction, or reward-dimension normalization.UAP unifies missing outcomes and imperfect reward consensus within reference-element comparisons and routes feedback to generated tokens.Source: §3 · p. 5
  • Process supervision and trajectory methods provide structured credit, while CSPO routes table-wide functional-attribute advantages to corresponding tokens alongside global feedback.P2TPO instead compares individual reference-block instances, preserving differences between same-type elements and supporting omission feedback and intra-block redistribution.Source: §3 · p. 5

4. Experiments

Across six document-parsing benchmarks, P2TPO leads the evaluated response-level baselines in the primary setting. Component comparisons, additional model settings, and sensitivity tests examine where its gains hold and how its components contribute.

4.1. Experimental setup

  • Experiments use six benchmarks and 13 splits; the primary comparison reports five-seed mean ± standard deviation after 100 RL updates from a shared SFT initialization.The primary model is Qwen3.5-2B trained on 700K pages; additional initializations, scales, and a backbone are also evaluated.Source: §4 · p. 5, §4.1 · pp. 5–6, Table 1 · p. 6

4.2. Cross-benchmark performance

  • P2TPO scores 93.12 ± 0.23 on OmniDocBench v1.6, 0.74 points above the strongest response-level baseline, and exceeds that baseline on all 12 external splits.Margins range from 0.19 points on DocPTBench Dewarp to 1.56 points on FinixDocBench; gains also hold across all five Real5-OmniDocBench capture conditions.Source: §4.2 · p. 6, Table 1 · p. 6

4.3. Component ablations and credit diagnostics

  • P2BPO exceeds the strongest response-level baseline on 12 of 13 splits, while adding SCR improves 11 of 13 splits with UAP but only two without UAP.Full P2TPO has the highest mean among the four component configurations on ten splits; these descriptive five-seed comparisons support complementarity between feedback construction and token allocation.Source: §4.3 · pp. 6–7
  • Full P2TPO exceeds the UAP consensus-feedback-disabled variant on eight of 13 splits and the omission-routing-disabled variant on 11; its observed across-seed standard deviations are lower than the former on all 13.These interventions assess UAP’s feedback and delivery rules.Source: §4.3 · p. 7
  • Aligned SCR exceeds randomized support on 12 of 13 splits and circular shift on ten, supporting alignment-informed placement beyond nonuniform weighting alone.Random support and circular shift exceed uniform allocation on two and three splits, respectively.Source: §4.3 · p. 7, Table 2 · p. 7
  • On OmniDocBench, adding UAP to P2BPO lowers observed element omissions from 1.09% to 1.01% and local omissions from 6.72% to 5.59%.Full P2TPO has the lowest observed omission rates and best formula, table, and reading-order metrics among the four component configurations; P2BPO + UAP has the lowest Text ED.Source: §4.3 · p. 7, Table 3 · p. 7

4.4. Generality across initialization, scale, and backbone

  • Across three additional model and initialization settings, P2TPO has the highest mean in 37 of 39 split-setting comparisons, including all 13 with Qwen3.5-4B/50K, and beats GSPO in all 39.The tested settings cover different initial policies, model scales, and backbones.Source: §4.4 · p. 8

4.5. Hyperparameter sensitivity and extended training

  • Across one-at-a-time hyperparameter sweeps, configurations stay within 0.95 points of the default on each split and beat the strongest response-level baseline on at least 11 of 13 splits.In the 500-update study, P2TPO has the highest mean on all 13 splits at update 500; Figure 4 reports five-run means and ±1 SD bands.Source: §4.5 · p. 8, Figure 4 · p. 8

5. Theoretical Analysis

The analysis shows that aggregating reference-block scores into groups or a page can erase local quality contrasts. Retaining reference-block identity preserves distinctions lost by these non-invertible mappings, while the appendix establishes UAP feedback routing and SCR budget-conservation conditions.

What is lost in aggregation?

  • The analysis considers common matched rollouts, centers block scores across rollouts, and combines grouped scores using fixed positive normalized weights.A matrix pools K block scores into C nonempty groups before an additive page utility is formed.Source: §5 · p. 9
  • When 1 < C < K, grouping and page aggregation are non-invertible and can cancel opposing block or group contrasts.Retaining reference-block identity preserves distinctions these coarser mappings can erase before normalization.Source: §5 · p. 9

Theoretical results

  • UAP activates and routes feedback for omitted or uniformly imperfect outcomes.The analysis establishes SCR source-block budget conservation and conditions on redistribution strength and mask precision for directing more credit to intended targets.Source: §5 · p. 9

6. Discussion and Limitations

P2TPO’s element-level feedback depends on ground-truth references, annotation accuracy, and serialization conventions. The framework’s current scope and span-level allocation leave distinct element types and finer token-level credit as areas for future work.

Limitations

  • Reference-based alignment and quality scores depend on annotation accuracy and serialization conventions, making reliable references and alignment important conditions for use.Annotation noise, repeated text, and alternative valid serializations can make element matching and token labels ambiguous.Source: §6 · p. 9

Scope

  • Experiments evaluate text, formulas, and tables, not code or figures as distinct element types.Extending the framework to code and figures requires suitable representations, alignment procedures, and quality measures.Source: §6 · p. 9

Credit allocation

  • SCR allocates credit between aligned correct and erroneous token regions, using a shared redistribution weight within each region.Finer localization of individual token contributions remains future work, with its optimization effects still to be evaluated.Source: §6 · pp. 9–10

7. Conclusion

P2TPO combines reference-element identity, UAP outcome feedback, and budget-preserving SCR for full-page parser post-training. It exceeds four response-level baselines across all 13 primary splits on six benchmarks, with additional experiments supporting applicability across settings.

Overall results

  • P2TPO exceeds four response-level baselines in five-seed mean score on all 13 primary splits across six benchmarks.The reported gain over the strongest baseline on OmniDocBench v1.6 is 0.74 points.Source: §7 · p. 10

Method and analysis

  • P2TPO preserves reference-element identity, completes missing and imperfect-consensus feedback with UAP, and localizes matched-block credit through budget-preserving SCR.The analysis characterizes local quality contrasts that fixed additive aggregation can erase; additional settings and ablations support applicability and component complementarity.Source: §7 · p. 10

AI Use Statement

The supplementary material covers related reward interfaces, credit-construction algorithms, experimental protocols, and further results. It also reports manuscript AI assistance and author verification practices.

AI Use Statement

  • The authors used generative AI for literature discovery, drafting and language refinement, figure and table review, illustrative material, and mathematical proofs, while retaining responsibility for verified final content.They state that AI was not used to formulate research questions, develop core ideas, or design methods.Source: AI Use Statement · p. 10

Appendix overview

  • The supplementary material presents reward-interface comparisons, method details and algorithms, experimental protocols and results, theoretical analysis, and qualitative case studies.Its sections are ordered accordingly.Source: Appendix overview · p. 14

A. Supplementary Related Work and Reward Interfaces

  • The reward-interface appendix distinguishes full-page transcription from type-specific recognition targets and classifies optimizer signals by how rewards reach generated tokens.Its comparison includes response aggregation, functional-attribute routing, and reference-block credit assignment; CSPO-inspired table scoring is proposed as an extension, not evaluated in reported experiments.Source: §A.1 · p. 15, §A.2 · p. 15, Table 5 · p. 15

A.3. Scope of reported parsing recipes

  • The reported parsing recipes have differing scopes: document question-answering methods optimize answers and reasoning rather than page transcriptions.The recipe descriptions also distinguish particular parsing tasks and auxiliary tasks from full-page parsing targets.Source: §A.3 · p. 16

B.2.2. Span credit redistribution

  • SCR selects error spans for negative coefficients and correct spans for positive coefficients, while unit-mean weights preserve the block’s signed and absolute credit budgets.Its mask construction uses edit-based comparisons; whole-element omissions follow UAP’s separate boundary route.Source: §B.1 · p. 16, §B.2.2 · p. 17

B. Credit Construction and Policy Update Algorithms

  • Algorithm 1 combines page-reward advantages with reference-indexed local contributions, routing matched-block coefficients across generated spans and missing-element credit through boundary-token weights.UAP compares matched scores with missing outcomes represented as zero; SCR redistributes eligible matched-block credit, and local terms combine with a shared page coefficient.Source: §B.2.1 · p. 17

B.2.3. Policy objective and update

  • The policy update uses a shared clipped PPO objective, averaging over active response tokens while holding advantages and old-policy probabilities fixed during differentiation.Component variants use the same loss with their corresponding token coefficients.Source: §B.2.3 · p. 18

C. Experimental Details and Supplementary Results

  • Experiments use converted full-page Markdown data, distinct RL training runs for seed summaries, and shared training and evaluation settings within each comparison.The appendix specifies training-record selection, a common parsing prompt, comparison methods, component controls, and experimental-series configurations.Source: §C.1.1 · p. 19, §C.1.2 · p. 19, §C.1.3 · p. 19

C.1.5. Benchmark protocols and score conventions

The appendix specifies benchmark scoring and evaluates how UAP, SCR, and related settings affect parsing results. Full P2TPO has higher means than UAP variants on most tested splits, while SCR placement effects vary by split.

C.1.5. Benchmark protocols and score conventions

  • The six-benchmark evaluation uses a shared OmniDocBench v1.6 engine for five benchmarks and FinixDocBench’s official Markdown evaluator for the sixth.DocPTBench is evaluated through the shared engine; Real5 and Wild reuse OmniDocBench source pages and assess capture robustness.Source: §C.1.5 · pp. 20–21, Table 8 · p. 21

C.2. Component ablations

  • The ablations compare full P2TPO with versions removing UAP or SCR; the no-UAP setting also removes outcome completion, consensus feedback, and omission routing.UAP interventions use five paired seeds with the mixing weight fixed, while SCR controls retain UAP and page feedback.Source: §C.2 · p. 22, §C.2.1 · p. 22, §C.2.2 · p. 22

C.2.1. Internal UAP interventions

  • Full P2TPO has higher means than the no-consensus variant on 8 of 13 splits and than the no-omission-routing variant on 11 of 13.The comparisons support benefits across most tested splits, with split-dependent effects.Source: §C.2.1 · p. 22

C.2.2. SCR allocation controls

  • Aligned SCR’s largest gains over random support are +1.00 on PureDocBench Real, +0.89 on FinixDocBench, and +0.84 on PureDocBench Clean.Circular shifts exceed aligned SCR on some splits, while uniform allocation leads on Real5-OmniDocBench Scanning and DocPTBench Dewarp.Source: §C.2.2 · p. 23

C.3. Credit-assignment diagnostics

  • Figure 5’s heatmaps show token-wise advantages on common frozen responses, with token position, page-reward ranges, and advantage sign and magnitude encoded.The panels include shorter and longer responses, variation across rollouts, and within-response sign changes.Source: Figure 5 · p. 23, §C.3 · p. 23

C.4. Generality across initialization, scale, and backbone

  • Across additional initialization, scale, and backbone settings, GDPO has the higher mean in only two reported comparisons: 76.47 versus 76.31 and 76.57 versus 76.51.These are Qwen3.5-2B/50K on DocPTBench SynReal and Qwen3-VL-2B-Instruct/700K on PureDocBench Digital, respectively.Source: §C.4 · pp. 23–24

C.5. Robustness to hyperparameter choices

The sensitivity study varies P2TPO’s imperfect-consensus, span-concentration, and page–local mixing parameters one at a time. Across the tested ranges, score changes remain bounded and cross-benchmark gains persist.

C.5. Robustness to hyperparameter choices

  • The one-at-a-time sweeps vary λ, β, and ω while holding the other two at default values; bars show five-seed means and standard-deviation error bars.The default setting is (λ, β, ω) = (0.1, 0.5, 0.2).Source: §C.5 · p. 24, Figure 6 · p. 24
  • Every tested mean remains within 0.95 points of default, and each of eight configurations improves on pre-RL scores on every split and beats the strongest response-level baseline on at least 11 of 13 splits.Maximum differences across tested values are at most 1.10 points for λ and 0.94 points for either β or ω.Source: §C.5 · p. 24, Figure 6 · p. 24

C.6. Extended training over 500 updates

Under a 500-update budget, P2TPO retains strong comparative performance, leading the reported final checkpoint scores across the additional splits. The appendix also reports compute overhead and theoretical properties of the credit construction.

C.6. Extended training over 500 updates

  • The 500-update comparison uses a shared Qwen3.5-2B checkpoint SFT-trained on 50K pages, 64,000 RL records, and five runs per method.The study evaluates OmniDocBench v1.6 every ten updates and the other 12 splits every 50.Source: §C.6 · pp. 24–25
  • At update 500, P2TPO scores 91.49 on OmniDocBench v1.6, 1.21 points above Init, and has the highest mean at 45 of 50 evaluated checkpoints.Its early- and late-window means are 91.44 and 91.24; the trajectory is nonmonotonic, peaking at 91.72 at update 180.Source: §C.6 · pp. 25–26, Table 11 · p. 26
  • At update 500, P2TPO has the highest mean on all 12 additional splits, with margins of 0.25–1.94 points over the strongest response-level baseline.It leads at 78 of 120 evaluated split–checkpoint pairs and ties for the highest mean at two more; across the 12 splits, its update-500 mean exceeds update 100 on seven and falls on five.Source: §C.6 · p. 26, Table 12 · p. 27

C.7. Computational cost

  • P2TPO’s credit assignment takes 1.215 s/update versus 0.707 s/update for GRPO, adding 0.51 s/update, or about 0.19% of per-update wall time.Generation and actor-update timing distributions are comparable to GRPO.Source: §C.7 · p. 28

D. Supplementary Theoretical Analysis

  • The supplementary analysis establishes properties for aggregation loss, feedback restored by UAP, and conditions under which SCR increases allocation to effective targets.The analysis holds alignments, token supports, and masks fixed and uses common matched rollouts with a fixed additive utility.Source: D. Supplementary Theoretical Analysis · p. 28

D.1. What is lost in aggregation?

Under a fixed-weight additive page utility, aggregation can erase block-level quality distinctions; UAP and SCR address missing or imperfect feedback and its allocation within matched blocks.

D.1. What is lost in aggregation?

  • Block, element-group, and page routing form a strict hierarchy of information loss: pooled contrasts cannot generally recover finer distinctions, even for scores in [0, 1].The stated hierarchy concerns pre-standardization scores and applies to page rewards with the specified fixed additive form.Source: §D.1 · pp. 28–29

D.2. How does UAP restore missing feedback?

  • UAP completes reference-wise comparisons with missing outcomes and uses a quality ceiling to preserve negative feedback for uniformly imperfect ties and all-missing groups.For low-dispersion outcomes, the coefficient is −λ(c−μ), vanishing at ceiling quality; missing-element coefficients are routed to generated tokens.Source: §D.2 · pp. 29–30

D.3. When does SCR direct feedback to effective targets?

  • SCR’s cap-and-rebate operation produces nonnegative token weights that preserve the matched block’s signed and absolute credit budgets.The source block must be eligible for redistribution; disabling SCR or using empty or full masks leaves allocation uniform.Source: §D.3 · p. 30, The target’s share of absolute… · pp. 30–31

The target’s share of absolute coefficient mass satisfies

  • SCR allocates more absolute coefficient mass to a target than uniform routing exactly when redistribution is active and mask precision exceeds target prevalence.For an imperfect matched tie, UAP activates the block credit that SCR can localize; the increase is proportional to Lλ(c−r)η(ϱ−q).Source: The target’s share of absolute… · pp. 30–31

E. Qualitative Case Studies

The qualitative case studies compare selected GRPO and P2TPO outputs after 100 updates, illustrating output differences without isolating individual credit-assignment components.

E. Qualitative Case Studies

  • The selected cases use per-page edit distances, for which lower values are better, alongside source pages, reference excerpts, and generated excerpts.Ellipses mark abridgment, and italic annotations identify omitted content rather than generated text.Source: E. Qualitative Case Studies · p. 32
  • Figure 7 compares heading and paragraph transcription in a clinical article from OmniDocBench v1.6.The figure presents a qualitative comparison; the supplied caption does not state a winner.Source: Figure 7 · p. 32
  • One reported case states that P2TPO retains the passage.Source: E. Qualitative Case Studies · p. 33
  • Figure 8 compares text coverage on a multi-column magazine page, while Figure 9 compares text coverage and superscript notation in a chemistry article.Both examples are from OmniDocBench v1.6.Source: Figure 8 · p. 33, Figure 9 · p. 34

Related papers

Loading 2610.21979v1…