PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Instruction-Driven Editing: Source-Grounded Problem Discovery with User-Governed Repair for Scientific Posters

Xingda Lyu, Honglin Lu, Xinye Luo, Shiqi Yang

arXiv:2609.01813v1cs.HC

TL;DR

Interactive editors often assume users already know what to change, leaving a gap when they recognize a poster problem without an actionable instruction. PROS surfaces source-grounded candidate problems while reserving repair-goal adoption and commitment for users, then evaluates this workflow with PROS-Bench. The results show strong local target resolution but divergence between resolution and realized benefit, supporting separate evaluation of discovery, repair, and outcome.

  • Problem

    Existing editing workflows generally begin with an articulated request, although users may recognize that a scientific poster is not working without knowing a localized intervention.

  • Method

    PROS combines user-invoked, source-grounded diagnosis with user acceptance, native-object PPTX editing, validation, reversible preview, and staged repair.

  • Results

    87.6% operator-verified target resolution coexists with decline on 14.8% of assessable accepted targets, revealing a gap between local closure and realized benefit.

  • Takeaways & Limitations

    Intelligent editors can support problem discovery before a concrete edit request exists without taking authority over consequential change.

  • Takeaways & Limitations

    The evaluation does not establish whether diagnosis-guided refinement reduces articulation effort, improves experienced control, or is preferred to direct editing.

Abstract

from arXiv · show

Interactive editors usually assume that users already know what to change. Yet an important interaction state comes earlier: a user may recognize that an artifact is not working without knowing what intervention to request. We call this the articulation gap. We introduce PROS (Proactive Refinement Of Scientific Posters), which separates epistemic initiative from behavioral authority: the system can surface source-grounded candidate problems, while users decide which become repair goals and whether resulting changes are committed. Accepted issues hand off to native-object PPTX editing with validation and reversible preview. We also introduce PROS-Bench, a source-linked collection of 120 papers and 320 editable PPTX posters, including a 120-poster matched primary core and a separate conference representation challenge. On the primary core, PROS achieves a mean VLM-rated stage-balanced diagnosis quality score of 67.2 on a 0-100 scale and 87.6% operator-verified target resolution among accepted diagnoses. Temporally blinded automated scoring yields a +22.7-point paper-macro accepted-target uplift, yet 14.8% of assessable accepted targets decline. This divergence shows why problem discovery, local resolution, and realized outcome should be evaluated separately. More broadly, intelligent editors can support problem discovery before a concrete edit request exists without taking authority over consequential change.

1 Introduction

PROS addresses the articulation gap by surfacing source-grounded problems before users can state an edit, while keeping repair-goal adoption and commitment under user control. The paper introduces a governed workflow and benchmark that separately evaluate discovery, repair, and outcome.

  • Problem: The articulation gap is the distance between recognizing that a poster is not working and formulating a localized, actionable intervention.This differs from instruction disambiguation because an adequate edit instruction may not yet exist.
  • Approach: PROS separates problem discovery from consequential change: users invoke diagnosis, inspect candidate issues, accept repair goals, preview changes, and decide whether to commit.Direct requests and diagnosis-guided refinement share the same source-grounded artifact state and repair substrate.
  • Approach: The system combines a replaceable reasoning model with deterministic detector and artifact services for editable-scene construction, typed operations, PPTX execution, and validation.Representation policy permits coordinated reflow for reliable structure but conservative local intervention for uncertain geometry.
  • Research questions: The study asks how well PROS executes articulated requests, discovers candidate issues, repairs accepted diagnoses, and improves accepted target conditions.It also examines losses across discovery, governance, repair, and outcome, plus variation across provenance and representation conditions.
  • Evaluation: PROS-Bench contains 120 source papers and 320 editable posters, with a matched primary core, a separate conference representation challenge, and capability measures spanning execution through realized outcome.The evaluation preserves source-paper, editable-PPTX, and rendered-poster relationships.

2 Related Work

Prior work covers poster generation, structured editing, and computational critique, but these strands generally address separate capabilities. PROS positions its contribution at their intersection: diagnosis before an edit instruction, source-grounded repair, user authority, and persistent editable artifacts.

  • Poster generation: Poster resources and systems have advanced generation, layout understanding, reading-order modeling, and paper-conditioned poster creation.Examples include paired paper–poster datasets, layout annotations, editable PPTX generation, and hierarchical content/layout refinement.
  • Instruction-driven editing: Instruction-driven editors increasingly support executable changes over presentations and layered documents, but typically begin from a user-specified request.APEX is the closest poster-specific precursor and evaluates human-refined instructions over paper–poster pairs.
  • Computational critique: Computational-critique systems expose design problems, provide mixed-initiative feedback, or support user-selected remediation across visual and interface artifacts.Related systems include embedded critics, layout exploration tools, visualization repair, poster feedback, and multi-perspective critique.
  • Positioning: PROS differs by connecting diagnosis before a concrete instruction with explicit authority boundaries and persistent editable artifacts, rather than claiming each component is individually unprecedented.PROS-Bench measures the resulting discovery–governance–repair–outcome trajectory.
  • Grounding and evaluation: The workflow combines source-grounded reasoning with deterministic artifact services because retrieved evidence can reduce unsupported content, while internal self-critique does not guarantee artifact preservation.Automated multimodal judgment remains useful for scale but is subject to known design-evaluation biases.

3 Design Principles and Authority Boundaries

PROS treats underarticulated editing as a user-controlled interaction state rather than permission for autonomous modification. Its design bounds scope, separates reasoning from artifact services, and uses inspectable, reversible operations.

  • Design principles: PROS exposes issue class, target, rationale, and source evidence while allowing users to dismiss, select, revise, accept, or reject proposed changes.A proposal never becomes a committed change without explicit user action.
  • Design principles: Typed operations, temporary copies, validation, and candidate preview block detectable invalid targets, unsafe geometry, ownership drift, and failed execution before commitment.These mechanisms implement reversible, failure-contained commitment.
  • Scope: PROS targets one-slide editable PPTX scientific posters and does not claim to infer latent author intent, create novel scientific figures, or perfectly recover edit semantics from flattened PDFs.The diagnosis taxonomy is an operational core rather than a complete ontology of poster quality.

4 The PROS Workflow

The PROS workflow maintains a semantically structured, representation-aware PPTX state and supports both direct editing and diagnosis-guided refinement. Diagnosis proceeds through ordered structural, scientific-communication, and spatial-composition stages before validated, reversible repair.

  • Artifact state: PROS separates model reasoning from deterministic artifact services while maintaining source text, working PPTX, rendered preview, object inventory, scene structure, layout profile, and temporary candidates.The reasoning backbone is replaceable, while parsing, execution, policy guards, and evaluation remain fixed across configurations.
  • Artifact state: The inventory infers semantic roles and relations from native object properties, geometry, neighborhood, and section ownership, linking user-facing labels to native shape identifiers.Relations include ownership, column and vertical ordering, overlap, and title/content/figure companionship.
  • Interaction paths: Direct editing accepts an explicit request, optionally localized through a clicked semantic label, whereas diagnosis-guided refinement proposes issues before an edit operation is specified.Accepted diagnoses become editable repair requests rather than immediate modifications.
  • Diagnostic layers: Diagnosis runs in order from structural validity to scientific communication to spatial composition, with the structural stage rechecked after edits and cleared stages locked.The ordering places spatial balancing after communication-level changes have stabilized.
  • Diagnostic layers: Scientific-content diagnoses use extracted source evidence but do not claim exhaustive fact-checking; users inspect proposed text and E3 externally verifies accepted-target resolution.The workflow therefore claims evidence-conditioned planning and governed review rather than automatic scientific verification.
  • Repair and validation: Candidate actions execute on temporary PPTX artifacts, with transaction-snapshot target resolution, re-rendering, representation-aware reflow, fail-closed freeform checks, and individual retries for failed batches.Invalid candidates are not enabled for acceptance, and evaluation focuses on outcome constraints and collateral damage rather than one exact action trace.

5 PROS-Bench

PROS-Bench links source papers, editable PPTX posters, renders, and evaluation assets to study source-grounded refinement across distinct poster representations. Its primary study preserves provenance and auditability while separating discovery, repair, and realized outcome.

  • Benchmark Composition: PROS-Bench links source papers, editable PPTX posters, and renders while preserving native objects, semantic relations, provenance, and stable identifiers.The benchmark is designed to test native-object targeting and artifact preservation rather than image-only editing.
  • Benchmark Composition: The benchmark contains 320 editable posters: 160 in 40 four-way source-matched groups and 160 in 80 generated pairs.The primary E1–E4 study uses 120 posters from the 40 matched papers, while conference variants form a separate representation challenge.
  • Benchmark Composition: The primary repair core comprises 120 posters from Paper2Poster, PosterGen, and expert-authored variants across 40 source papers.Conference posters are retained as a separate challenge because mature design and heterogeneous editability create different repair conditions.
  • Release Structure and Auditability: All artifacts receive stable identifiers, hashes, render checks, object inventories, slide-dimension checks, and provenance validation.The release preserves diagnosis text, governance decisions, repair requests, outcomes, evidence manifests, and available before/final artifacts.
  • Evaluation Assets: E2 scores predicted issues, E3 links accepted diagnoses to repairs and target-resolution outcomes, and E4 compares original and final accepted-target conditions.This design keeps discovery quality, governed repair, and realized target uplift distinct and does not claim holistic aesthetics or publication readiness.

6 Evaluation

PROS evaluates an interactive workflow through separate measures for explicit execution, diagnosis, accepted-issue repair, and realized outcome. The study uses trained operators, paper-clustered uncertainty, and blinded automated judgments rather than a composite score.

  • Evaluation Design: PROS evaluates workflow behavior across explicit-request execution, problem discovery, accepted-issue repair, and realized accepted-target outcome.The evaluation treats these as separate capability-specific quantities rather than combining them into one score.
  • Evaluation Design: The primary study exhausts 120 matched posters from 40 source papers, with 720 E1 instructions and a four-backbone sensitivity analysis on eight papers.The sensitivity analysis varies the reasoning backbone while fixing posters, tasks, artifact services, stopping policy, and judges.
  • E2 Problem Discovery: E2 permits abstention and scores diagnosis correctness, grounding, target specificity, and actionability from 0–4 using stage-conditioned evidence.Structural items use geometry and renders, Scientific items add the source paper, and Spatial items require visible layout relations.
  • E3 Accepted-Issue Repair: E3 measures verified target-resolution success only for operator-accepted diagnoses linked through stable issue IDs.A stage–provenance cell with no accepted issue is treated as not applicable rather than 0%.
  • E4 Realized Outcome: E4 measures stage-balanced accepted-target uplift between randomized opaque A/B render states, not holistic aesthetics or the isolated causal effect of one repair operation.The automated judge receives the target, repair direction, and source evidence for Scientific items without temporal labels.
  • Statistics and Reproducibility: The source paper is the clustering unit, with 20,000 paper-level bootstrap resamples and paper-clustered 95% confidence intervals.Automated judgments remain separate from operator-verified outcomes, and unavailable or non-applicable cases are not imputed.

7 Results

PROS separates explicit-request execution, diagnosis, accepted-issue repair, and realized target outcome across the primary core. Results show strong execution and conditional repair, but diagnosis and final benefit remain distinct challenges, especially under limited repair opportunity and uncertain representation.

  • Primary Findings: 83.3% verified E1 success was achieved on explicit-request execution, with Spatial tasks lower than Structural and Scientific tasks.Verified E1 success was 83.3% (600/720), while Structural, Scientific, and Spatial tasks achieved 89.6%, 85.8%, and 74.6%, respectively.
  • Primary Findings: 67.2/100 stage-balanced diagnosis quality was assigned across 1109 recorded diagnoses, excluding 9 cannot-assess items.The score measures surfaced diagnosis quality rather than recall or completeness.
  • Primary Findings: 87.6% verified target-resolution success was obtained for 876 accepted issues, distinguishing accepted-issue repair from diagnosis quality and yield.All accepted diagnoses had complete E3 outcomes, but proposal yield varied by stage and provenance.
  • Primary Findings: A 22.7-point paper-macro uplift raised accepted-target quality from 65.0 to 87.7, while 67.1% improved, 18.2% tied, and 14.8% declined.The temporally blinded VLM evaluation scored 865 accepted repair targets across 118 applicable posters.
  • Result Synthesis: The results support separate capability-specific reporting because verified local resolution does not guarantee improved final artifacts.The E1–E4 decomposition keeps execution, discovery, accepted repair, and realized outcome distinct; conference challenge artifacts were excluded from primary estimates.
  • Separate Conference Representation Challenge: The conference challenge produced 39.8/100 E2 quality, 80.6% immediate E3 resolution, and a -6.1-point E4 change whose interval crossed zero.Limited repair headroom and uncertain recovered structure constrain interpretation, while PDF-to-PPTX conversion confounds starting-design maturity and representation recovery.
  • Exploratory Portability: Backbone sensitivity on a fixed eight-paper subset yielded E2 values from 51.5–80.2/100, E3 from 73.6–91.7%, and E4 from +0.5 to +23.3 points.These small-sample estimates indicate exploratory portability rather than a stable model ranking.

8 Discussion

PROS broadens the user’s attention through source-grounded candidate diagnoses while preserving user authority over goals and commitment. Its results motivate evaluating proposal, governance, repair, and after-state outcome separately, with proactivity calibrated to repair opportunity and representation confidence.

  • Before Instruction: Initiative Without Authority: A diagnosis remains a candidate rather than a command: acceptance establishes the repair goal, and preview separates execution from commitment.This governed handoff keeps initiative distinct from authority over consequential artifact changes.
  • Before Instruction: Initiative Without Authority: PROS extends this initiative-without-authority framing to already shared editable artifacts, but the paper does not evaluate slide decks, reports, or structured figures.Scientific posters provide the evaluated setting because they connect an external evidence source with an executable representation.
  • Discovery, Closure, and Realized Benefit Are Different Capabilities: Figure 3 separates capability measures, paper-level uplift, and target-level improved, tied, and declined cases rather than presenting a composite score.The panels distinguish verified local resolution from realized final-state benefit.
  • Discovery, Closure, and Realized Benefit Are Different Capabilities: 87.6% verified target resolution coexists with 14.8% decline among assessable accepted targets, showing that immediate closure is necessary but not sufficient evidence of benefit.A proactive editor can identify the wrong problem, fail to resolve a useful one, or resolve a local condition without improving the resulting artifact.
  • Evaluation and Interface Design: Evaluation and interface design should preserve the path from evidence-backed proposal through governance and execution to validated after-state outcome.This keeps proposal yield, diagnosis quality, governance, repair success, and final uplift interpretable as separate quantities.
  • Proactivity Must Be Calibrated to Opportunity and Representation: More diagnoses are not necessarily better because starting artifacts differ in repair headroom and safe intervention opportunities.A reached-no-issue state or rejection can reflect warranted restraint, but may also reflect missed issues, weak grounding, duplication, infeasibility, or trajectory risk.
  • Proactivity Must Be Calibrated to Opportunity and Representation: PROS contracts authority when representation confidence is low, especially for global rearrangement, because native and PDF-derived PPTX structures can encode relations imperfectly.The separate conference challenge exposes this editability boundary without pooling unlike opportunity conditions.

9 Limitations and Future Work

The evaluation establishes artifact-centered capability but leaves user effort, control, preference, and broader poster-quality coverage unresolved. Representation recovery, limited issue classes, and model/judge dependence define important boundaries for interpretation.

  • Future Work: A controlled user study is needed to assess articulation effort, perceived control, and preference against direct editing.The current evaluation measures workflow capability and accepted-target outcomes rather than user experience.
  • Scope Boundaries: The three layers and six issue classes are an operational core, not a complete theory of poster quality.Accessibility, figure validity, narrative flow, audience adaptation, multilingual content, and venue-specific constraints remain incomplete.
  • Evidence Boundaries: Qualitative repair traces illustrate successful local repairs but do not establish general effectiveness across poster conditions.The traces show different repair opportunities for PosterGen and Paper2Poster, including cases with no actionable Structural or Spatial issue.
  • Scope Boundaries: Representation quality and repair opportunity vary across native PPTX, PDF-converted artifacts, and shared-template expert posters.PDF conversion cannot recover authoring intent, while the shared template does not capture natural expert-design diversity.
  • Evaluation Boundaries: The primary evaluation uses one reasoning backbone and one multimodal judge, while the four-backbone analysis covers only eight papers.The sensitivity analysis indicates bounded portability rather than stable ranking or identical-target comparison, and independent human judgment remains future work.

10 Responsible Deployment and Ethical Considerations

Responsible deployment must protect sensitive poster content, preserve provenance, and keep authors responsible for scientific decisions. Because system suggestions can shape communication choices, deployment should expose evidence and uncertainty while retaining dismissal and commitment control.

  • Data Governance: Scientific posters may contain unpublished research, copyrighted figures, personal information, or sensitive data requiring careful handling.Deployment guidance includes session isolation, minimized retention, disclosure of external model access, and permission-aware redistribution.
  • Data Governance: Public benchmark releases should preserve paper/poster provenance and license status, while restricted evaluation assets should not be redistributed when necessary.These requirements address both source ownership and the conditions under which artifacts can be shared.
  • Evaluation Ethics: Operator outcomes are not independent human-participant or external-expert evidence, and later study-author checks were neither independent nor blinded.Future external validation or user studies must complete the applicable ethics review and report its determination.
  • Human Responsibility: System-proposed diagnoses should expose supporting evidence and uncertainty, permit dismissal, and keep final scientific responsibility with the author.Suggestions can shape authors’ communication choices, so they should not be presented as objective truth.
  • Human Responsibility: PROS separates proactive problem discovery from autonomous modification by leaving issue acceptance and commitment to the user.This governance boundary is central to the system’s deployment stance.

A Cross-Model Sensitivity and Exploratory Patterns

Cross-model and representation analyses are exploratory and expose substantial variation in diagnosis quality, agreement, issue class, and repair opportunity. The conference subset is a separate representation challenge, so its outcomes are not directly interchangeable with the primary core.

  • Cross-Model Sensitivity: The fixed-subset sensitivity analysis holds eight papers and 24 starting posters constant but gives each backbone its own diagnosis and accepted-target stream.Accordingly, it measures end-to-end sensitivity rather than controlled pairwise model effects.
  • Diagnosis Yield: Claude emits 368 diagnoses at 51.5/100 E2 quality, whereas Sol emits 167 at 80.2/100.Across pooled sensitivity records, target specificity is 3.76/4 while actionability is 1.78/4, indicating that proposal volume and diagnosis quality diverge.
  • Cross-Model Sensitivity: Agreement rises with diagnosis quality and E3 success, but E4 uplift is non-monotonic across one through four backbones.Mean E2/E3 values are 28.8/100 and 55.1% for one-backbone groups versus 84.2/100 and 87.5% for four-backbone groups; E4 means are -7.8, -10.0, +1.2, and +18.0 points.
  • Issue-Class Patterns: PosterGen improves under every backbone by +21.9 to +35.0 paper-macro points, while accepted-target means vary by issue class.Missing-content and whitespace average +15.2 and +31.2 points; redundancy and collision average -21.5 and -14.8 points, respectively.
  • Conference Representation Challenge: The conference challenge evaluates 40 acquired posters separately because PDF conversion and native-PPTX representation jointly affect repair opportunity.It is not a fourth exchangeable primary-core provenance.
  • Conference Representation Challenge: In the conference challenge, 62 of 360 diagnoses are accepted, all Structural, and 50 of 62 accepted targets are immediately verified as resolved.E1 succeeds on 96/240 instructions, while accepted diagnoses contain no Scientific or Spatial cases.
  • Conference Representation Challenge: Conference E3 is conditional on accepted targets, and E4 excludes unlocalizable targets and posters without accepted targets.The reported confidence intervals use 20,000 source-paper bootstrap resamples.
  • Conference Representation Challenge: Across 52 assessable conference targets, 19.2% improve, 61.5% tie, and 19.2% decline.The paper-macro target condition changes from 82.9 to 76.7, a -6.1-point change with a 95% CI of [-19.3, 6.3], which crosses zero.

D Evaluation Protocol, Rubrics, and Aggregation

The evaluation separates diagnosis, governance, repair execution, and realized outcome through stage-conditioned rubrics and target-conditioned blinded scoring. Aggregation preserves opportunity, uncertainty, provenance, and representation boundaries instead of imputing unavailable cases.

  • D Evaluation Protocol, Rubrics, and Aggregation: Operators invoke diagnosis, review candidate execution, apply a frozen acceptance policy, and export the final PPTX.Trajectories stop after two no-new-issue calls, 12 diagnosis rounds, or eight accepted and committed repairs.
  • D Evaluation Protocol, Rubrics, and Aggregation: E2 scores correctness, grounding, target specificity, and actionability independently on 0–4 scales with stage-specific evidence requirements.Structural diagnoses require native geometry and render agreement; Scientific diagnoses use poster and source paper; Spatial diagnoses require visible layout relations.
  • D Evaluation Protocol, Rubrics, and Aggregation: Mean E2 quality is computed by stage within poster and then equally averaged across available stages.Unvisited stages are not-reached, reached stages without issues are reached-no-issue, and cannot-assess items are excluded rather than imputed.
  • D Evaluation Protocol, Rubrics, and Aggregation: E4 scores opaque same-resolution before/after states without temporal labels using criteria matched to Structural, Scientific, or Spatial targets.Missing required evidence is cannot-assess, and extra text or movement receives no credit without target-specific resolution.
  • D Evaluation Protocol, Rubrics, and Aggregation: Target uplift is aggregated within stage, poster, variant, and source paper, while posters without accepted diagnoses remain not applicable.Positive, tied, and negative target rates are retained alongside before/after scores and paper-macro uplift.
  • E Reproducibility and Evidence Boundaries: The analysis uses source-paper bootstrap resampling and records identifiers, configurations, prompts, outputs, decisions, outcomes, judge responses, and artifact links.These records support reproducibility while preserving paper-level dependence across poster variants.
  • E Reproducibility and Evidence Boundaries: The conference challenge retains diagnosis yield, abstention, grounding failure, representation issues, and repair outcomes separately from headline estimates.Low intervention yield is not labeled failure without independent evidence that an actionable issue was missed or ungrounded.
  • F Complete Claim Boundaries: The primary evidence uses one backbone and one multimodal judge, while cross-backbone comparisons share detectors, services, prompts, and the judge.Thus, the sensitivity analysis measures end-to-end portability rather than identical-target model ranking, and independent human validity remains unestablished.
Loading 2609.01813v1…