PaperlayerView exact revision on arXiv

Source-linked AI summary

AcroMELD: Recovering Interactive PDF Forms with Structure-Aware Graph Set Transformers

Samuel Abramov

arXiv:2608.22338v1cs.CV

TL;DR

Missing interactive widgets make visually form-like PDFs difficult to fill, motivating recovery of their geometry and type. AcroMELD combines visual and PDF-structural evidence in a graph-set detector and passes the registered native gate, while strict external results reveal transfer and rare-class limitations.

  • Problem

    Visually form-like PDFs may lack interactive widgets, and detecting missing fields is difficult because evidence is overlapping, PDF structure is incomplete, and pages can be dense.

  • Method

    AcroMELD combines a high-resolution visual transformer, label-free PDF primitives, heterogeneous queries, graph-set layers, same-field relations, and containment-aligned localization quality.

  • Results

    Native containment micro-F1 reached 0.9344 internally and 0.8477 externally, passing FFGBT-v8 by 0.0186, while strict external performance trailed CommonForms-L.

  • Takeaways & Limitations

    The result supports the registered operational gate while showing that containment-based success does not establish a general detector ranking.

  • Takeaways & Limitations

    The corpus is not shareable for exact independent replication, and strict external evaluation exposed domain-transfer and rare-class limitations.

Abstract

from arXiv · show

Interactive PDF form fields are often absent from documents that visually resemble forms, leaving users unable to enter data without printing or external editing tools. Detecting the missing widgets is difficult because a field may be indicated by several overlapping cues, born-digital PDFs expose useful but incomplete drawing structure, and dense pages can contain hundreds of fields. We introduce AcroMELD (AcroForm Multi-source Evidence Linking Decoder), a 39.4M-parameter detector that combines a high-resolution visual transformer with label-free PDF primitives. Its 896-query set comprises 384 visual proposals, 384 structure-seeded proposals, and 128 learned recovery queries. Four graph-set layers exchange information over geometry-biased sparse neighborhoods and cross-attend to PDF structure. A learned same-field relation links co-referent candidates, while a localization-quality head is trained on the containment-aware overlap used by the downstream recovery decision. We define a hash-bound evaluation protocol with disjoint development, calibration, internal-test, and quarantined external-holdout roles. The sealed, single-seed candidate reaches native containment micro-$F_1$ 0.9344 on the internal test and 0.8477 on the one-shot external holdout (95% PDF-cluster bootstrap interval [0.8339, 0.8605]). This passes the registered historical FFGBT-v8 reference by 0.0186 absolute $F_1$. Under the stricter external adapter, however, performance is 0.7786 IoU-$0.5$ $F_1$ and 0.2900 COCO mAP, below a locally evaluated CommonForms-L reference; the signature class receives no prediction at the selected threshold. Thus the result supports the registered operational gate while exposing substantial domain and rare-class limitations.

1 Introduction

AcroMELD addresses missing interactive PDF widgets by combining visual and incomplete PDF-structural evidence in a high-capacity graph-set detector, with hash-bound evaluation and containment-aligned localization.

  • Motivation: Missing AcroForm widgets can leave visually form-like PDFs unable to support direct data entry, motivating geometric and type recovery.The paper narrows recovery to localizing and classifying missing field controls rather than full accessibility conformance.
  • Motivation: PDF primitives complement visual evidence because rasterization discards spatial cues, while rectangles, text, lines, paths, and images can be incomplete or misleading.Scanned documents may expose only an unavailable structure sentinel, and decorative graphics can resemble field evidence.
  • Motivation: Dense pages create a duplicate-prone set-prediction problem, including a sealed page with 849 valid widgets beyond a conventional Top-300 cap.Aggressive non-maximum suppression is unsafe when neighboring controls touch or nest.
  • AcroMELD: AcroMELD uses 384 visual proposals, 384 structure-seeded proposals, and 128 learned recovery queries in one exclusive 896-node set.The set is refined through four sparse, geometry-biased graph layers with cross-attention to PDF primitives.
  • AcroMELD: A supervised same-field relation links co-referent candidates, while localization confidence is trained against the containment-aware recovery decision.These mechanisms support inference-time consolidation while retaining strict localization safeguards.
  • Evaluation: The protocol hash-binds data, code, configuration, checkpoints, operating points, and reports, but the single frozen trajectory supports system-level evidence rather than component attribution.Internal testing followed candidate and operating-point freeze, then a one-shot external gate was executed.

2 Related Work

AcroMELD differs from document-understanding models and prior form detection by targeting precise, variable-size widget sets while retaining high-resolution visual and PDF-structural evidence.

  • Form detection: CommonForms treats field detection as three-class object detection, whereas FUNSD connects and semantically interprets text entities.AcroMELD adopts the CommonForms taxonomy while adding low-level PDF primitives.
  • Multimodal document models: LayoutLM and DocFormer encode text, geometry, and visual information for semantic document predictions rather than precise, high-cardinality widget sets.AcroMELD uses structure as conditional geometric evidence rather than as a document-language backbone.
  • Set prediction: DETR motivates direct set prediction with bipartite matching, while Deformable DETR reduces attention cost through sparse sampling.AcroMELD changes the downstream problem from a single visual proposal source to a heterogeneous query set.
  • Graph-set decoding: AcroMELD defines a 32-neighbor spatial graph and separately supervised same-field embeddings that participate directly in representative selection, unlike generic NMS.The relation mechanism supports duplicate removal across query origins.
  • Localization: Localization confidence addresses ranking misalignment between classification probability and box quality, while dual reporting guards against containment accepting undersized boxes.The paper reports both containment-based recovery metrics and strict IoU metrics.

3 Problem Definition and Protocol

The task predicts an unordered set of widget boxes and coarse types from raster images plus optional label-free PDF structure, using containment-aware native matching alongside strict diagnostics and leakage-resistant role separation.

  • Field-recovery task: The model receives raster image I and optional structure tokens S representing PDF primitives with normalized boxes, descriptors, text bytes, and availability flags.Existing AcroForm annotations and field types are withheld; failed extraction or scanned pages yield an unavailable page sentinel.
  • Field-recovery task: The target is an unordered set of axis-aligned widget boxes with coarse classes Text, Choice, and Signature.Predictions support candidate widget instantiation rather than semantic field completion.
  • Field-recovery task: Recovery means geometric and type recovery only, excluding field names, descriptions, values, tab order, required-state semantics, and accessibility associations.The paper explicitly distinguishes this scope from automatic accessibility-standard conformance.
  • Decision geometry: Native evaluation accepts class-consistent matches when ov(b, g) ≥ 0.3, using descending-score greedy one-to-one matching within each page and class.The containment-aware criterion can accept nested or undersized boxes, so native and strict numbers are not interchangeable.
  • Decision geometry: Strict diagnostics use IoU-0.5 metrics, COCO AP over IoU thresholds 0.50:0.05:0.95, and a stable class-global Top-300 cap per page.Native evaluation may retain all 896 queries.
  • Evaluation protocol: Whole document groups occupy disjoint training, development, calibration, and test roles, with development selecting checkpoints and calibration selecting thresholds and representative parameters.The candidate and operating point were frozen and digest-bound before internal-test inference, while the external corpus remained quarantined until validation.

4 AcroMELD

AcroMELD combines visual proposals, PDF-structure evidence, and learned recovery queries in one graph-set decoder. Geometry-aware refinement, same-field linking, and containment-aligned quality scoring consolidate candidates into an exclusive prediction set.

  • Heterogeneous query set: AcroMELD forms an 896-query set from 384 visual proposals, 384 structure-seeded proposals, and 128 learned recovery queries.The visual branch produces proposal features and boxes, while the structure branch selects eligible primitive tokens or uses learned fallbacks.
  • Architecture: The raster and label-free PDF structure remain independent evidence channels until their heterogeneous queries enter one graph-set decoder.All structure tokens provide cross-attention context, while the decoder produces one exclusive prediction set.
  • Graph refinement: Each graph layer uses 32 nearest query centers for sparse self-attention, then cross-attends from every query to all encoded structure tokens.The sparse aggregation stores O(Qk) edges, although the current exact nearest-neighbor search computes an O(Q^2) distance matrix.
  • Graph refinement: Four graph layers iteratively update valid normalized boxes and change the graph neighborhood used by subsequent layers.Each layer emits auxiliary class, box, and quality predictions while refining geometry in inverse-sigmoid coordinates.
  • Objectives and selection: The decoder trains class, box, generalized-IoU, quality, and relation objectives, with quality supervision aligned to containment-aware localization.A supervised same-field relation links overlapping candidates, and class-agnostic suppression consolidates candidates that may disagree on type.
  • Objectives and selection: Class-thresholded candidates undergo stable greedy selection with IoU- or relation-based suppression and an 896-field page cap.Candidates below a raw score floor of 0.01 are discarded before calibrated class thresholds and representative selection are applied.

5 Data and Training

The study trains on a large internal corpus combining real PDFs with synthetic pages, while preserving visual-only operation when structure is unavailable. Training uses sealed data roles and a single registered 33-epoch run whose convergence and stability remain unresolved.

  • Data: The internal corpus contains 35,388 PDFs, 119,418 pages, and 2,550,287 spatially valid widgets.Invalid or off-page boxes were excluded, including 2,295 Text, 77 Choice, and 798 Signature widgets.
  • Data: The densest real training page contains 849 fields, below the model’s 896-query capacity, and 23,607 empty training pages are retained.Structure is available for 91,637 pages; pages without it, including all synthetic pages, use the visual-only path.
  • Data: Synthetic data contributes 15,785 pages from 5,000 generated PDFs, with widgets disabled and no PDF primitive tokens.Real signature-bearing pages receive threefold sampling weight, while synthetic pages remain at unit weight.
  • Training procedure: Training preserves page geometry through shared raster, box, and structure transforms while disabling augmentations that could alter form semantics.Structure is dropped on 12% of pages and 5% of non-page tokens are independently dropped during training.
  • Optimization: The registered run ended after 33 epochs, with the final best EMA checkpoint frozen as the candidate while the selection metric was still improving.All reported results come from one run, so convergence is unresolved and run-to-run stability is unmeasured.
  • Model scale: The full model has 39,375,424 trainable parameters, with 83.4% in the visual branch and 16.6% in the structure encoder and graph-set decoder.Parameter transfer from the published ECDet-L checkpoint covers 99.9969% of compatible visual parameters.

6 Evaluation Design

Evaluation separates development, calibration, internal testing, and a quarantined external holdout, with adapter-specific metrics and a registered native decision reference. The design also acknowledges that the single frozen run cannot isolate component effects and that strict and native references are not jointly rankable.

  • Calibration: Calibration searches representative-selection settings and class thresholds only after the checkpoint and terminal phase are fixed.The registered link-threshold grid contains six values, and thresholds are selected from 981 descending values between 0.99 and 0.01.
  • Calibration: The selected operating point uses Text, Choice, and Signature thresholds of 0.769, 0.812, and 0.990, with (τnms, τlink) = (1.000, 0.950).The tie-break selects the least-aggressive representative settings among cells sharing the same aggregate native calibration F1.
  • Evaluation protocol: Internal-test inference begins only after the candidate and operating point are sealed, and the internal test is used for neither checkpoint selection nor calibration.The report includes native and strict IoU-0.5 metrics, COCO localization metrics, and classwise results, but not native containment AP.
  • External holdout: The external holdout contains 1,996 documents, 6,843 pages, and 116,548 native widgets, including 1,469 Signature widgets.It includes 1,439 target-empty pages and 388 pages without vector primitives; no page exceeds 300 fields.
  • Decision reference: FFGBT-v8 is the registered native pass/fail reference, reporting micro-F1 0.8290 on the identical external corpus and native adapter.Its classwise F1 values are 0.810 for Text, 0.891 for Choice, and 0.121 for Signature; paired uncertainty cannot be estimated because predictions were not retained.
  • Adapter interpretation: Native containment and strict IoU use different annotations, matching rules, and caps, so AcroMELD is compared with each reference only under its corresponding adapter.The locally evaluated CommonForms-L/FFDNet-L reference reports micro-F1 0.8014 at IoU 0.5 and COCO mAP 0.3410.
  • Scope and limitations: The study evaluates one registered training trajectory without retrained component ablations or post-freeze interventions removing structure or link-based selection.Naturally occurring density and vector-availability strata are confounded by document source and visual characteristics.

7 Results

The frozen candidate improved throughout the 33-epoch development run and passed the registered external native-containment gate, but strict localization and signature transfer remained substantially weaker.

  • Training trajectory and candidate selection: 0.8956 native development micro-F1 was reached at epoch 33, alongside 0.8564 strict IoU-0.5 micro-F1 and 0.7243 strict COCO mAP.The final record was the selected EMA checkpoint, and native development F1 had improved for five consecutive epochs at termination.
  • Calibration and post-freeze evaluation: 0.9269 selected native calibration micro-F1 determined thresholds (0.769, 0.812, 0.990) and decoder settings (τnms, τlink) = (1.000, 0.950).All 40 decoder-grid cells with τlink < 1 tied on aggregate calibration micro-F1, so the registered tie-break selected the least-aggressive tied point.
  • Calibration and post-freeze evaluation: 0.8477 external native micro-F1 exceeded FFGBT-v8’s 0.8290 by 0.0186 and passed the registered one-shot gate.The descriptive 10,000-resample PDF-cluster bootstrap interval was [0.8339, 0.8605], but it was not a confidence interval for the candidate–baseline margin.
  • Calibration and post-freeze evaluation: 0.0707 was the external native F1 gain from calibrated thresholds over the fixed-0.3 control for the same candidate and representative settings.The fixed-0.3 control reached 0.7770, while FFGBT-v8’s threshold-tuning equivalence could not be assessed from aggregate results alone.
  • Strict localization diagnostics: 0.7786 strict external IoU-0.5 micro-F1 and 0.2900 strict COCO mAP were below CommonForms-L references of 0.8014 and 0.3410.These strict values used 116,542 annotations and a Top-300 cap, placing them outside the native gate comparison.
  • Class behavior and descriptive strata: 0.0023 external strict Signature AP accompanied zero selected-threshold signature predictions, while lowering class thresholds raised signature F1 only to 0.0677.The calibrated threshold directly produced zero selected predictions; the fixed-threshold and strict-AP results indicate thresholding alone cannot explain the transfer failure.

8 Discussion and Limitations

The registered native gate is passed, but stricter evaluation and external transfer reveal substantial limitations in localization precision, rare-class recall, component attribution, and deployment scope.

  • Evaluation boundaries: 0.7770 fixed-threshold external F1 fell below FFGBT-v8, while strict evaluation also trailed the local CommonForms-L reference.The registered native result is specific to the declared adapter and calibration procedure.
  • Generalization: 0.0867 internal-to-external F1 gap shows that the internal test is not a reliable proxy for this external domain.The experiment cannot distinguish provenance concentration from synthetic-signature rendering effects.
  • Component attribution: No controlled structure-removal or capacity-matched retraining isolates contributions from structure seeds, free queries, relation supervision, or containment-quality supervision.The frozen single-seed trajectory supports the complete system rather than any individual component.
  • Capacity and deployment: The external holdout never exceeds 300 fields per page, so the benefit of 896 queries over a Top-300 cap cannot be observed externally.Pages above 896 fields exceed the training contract and are truncated during decoding.
  • Data limitations: Native widget supervision may be inaccurate, duplicated, inconsistent with visible cues, and biased toward documents that were already interactive.The corpus is not shareable, limiting independent replication of exact training.
  • Metric limitations: Containment overlap can give perfect scores to undersized boxes, so strict IoU losses and metrics remain necessary safeguards.Native bootstrap intervals describe only native matching and do not quantify strict performance or the candidate–baseline margin.
  • Operational boundary: False positives occurred on 16.82% of target-empty pages, requiring abstention and human review in remediation workflows.Detected boxes and coarse classes do not establish accessibility or semantic correctness.

9 Conclusion

AcroMELD combines complementary proposal sources in an 896-query graph-set decoder and binds its reported results to a frozen, hash-based experimental protocol. It passes the registered native gate, while strict external metrics and signature results expose important limitations.

  • Conclusion: AcroMELD recovers missing PDF form-widget geometry and coarse type with a structure-aware graph-set transformer.The method targets interactive widgets absent from visually form-like PDFs.
  • Method: The decoder learns containment-aligned localization quality and same-field links while preserving the declared operational evaluation semantics.The experimental protocol binds capacity, adapter semantics, calibration, and artifact lineage to the result.
  • Results: 0.9344 internal and 0.8477 external native micro-F1 passed the historical FFGBT-v8 gate by 0.0186.Calibration fixed one operating point before either test role was opened.
  • Limitations: 0.7786 strict external IoU-0.5 F1 and 0.2900 COCO mAP trailed the local CommonForms-L reference, with no signature prediction surviving the selected threshold.Further runs and controlled ablations are needed to determine which architectural components account for the system-level result.

A.3 Complete development trajectory

The development trajectory distinguishes a diagnostic fixed-threshold evaluation from a separate calibration-selected re-evaluation of the same frozen candidate.

  • Complete development trajectory: The diagnostic decoder used class thresholds of 0.3 and (τnms, τlink) = (0.90, 0.80).This was the registered trajectory setting.
  • Complete development trajectory: The calibration-selected development F1 is not directly comparable with the trajectory’s final value 0.8956.Both evaluations use the same frozen candidate but different operating parameters.

A.4 Decoder controls

Calibration selected a near-unsuppressed decoder operating point after tied NMS/link configurations were resolved by the registered tie-break rule.

  • Decoder controls: Forty NMS/link cells attained the same aggregate calibration F1, and the least-aggressive tied corner was selected.The selected parameters were (τnms, τlink) = (1.000, 0.950).
  • Decoder controls: The near-unsuppressed setting still removes identical boxes at IoU 1 or candidates with sufficiently confident links.The search covered 48 NMS/link cells and 981 class-threshold values per cell.

B.1 Hash-bound final record

The final record binds the frozen candidate to its metric history, operating point, run identities, reports, prediction shards, and verification artifacts through SHA-256 digests.

  • Final record: The hash-bound record includes a final completion record tying the frozen candidate to its full metric history and selected operating point.The record also includes a table of 33 development evaluations with precision, recall, class AP, loss, phase, and evaluation time retained in CSV.
  • Artifact identities: The complete SHA-256 identities distinguish the sealed result from later reruns or manuscript revisions.Recorded artifacts include the candidate, metrics, operating point, training records, run identities, external report, completion marker, post-freeze analysis, and prediction shards.
  • Verification: The repository manifest covers the compact artifacts in the evidence snapshot and verifies cross-artifact bindings before generating tables or figures.The post-freeze analysis is descriptive only and validates source PDFs and both prediction shards.
  • Availability: The repository, trained weights, and source PDF corpus are private, while verification artifacts may be made available for hash audit on request.The corpus contains third-party documents with heterogeneous redistribution rights.
Loading 2608.22338v1…