PaperlayerView exact revision on arXiv

Source-linked AI summary

RINI: Seeing the Prior Is Not Enough

Hongyi Du, Tianyi Zhang, Heng Wang, Zhelun Gao, Yimei Liu, Ambrose Luo, Annie Hao, Jiayan Ni, Jiawei Han, Jiaxuan You

arXiv:2609.33284v1cs.AI

TL;DR

The paper asks whether supplying an earlier paper corrects proposals’ claims about who established a method, since a proposal can cite prior work while still claiming to introduce its technique. RINI links contribution claims to evidence, checks whether a proposed remaining distinction is already covered, and applies local edits to the proposal. After seed balancing, RINI successfully repairs 72.2% of proposals versus 39.1% for Retrieve-and-Revise, an improvement of 33.0 percentage points with the same decisive prior and target relation.

  • Problem

    The paper asks whether supplying an earlier paper corrects proposals’ claims about who established a method, since a proposal can cite prior work while still claiming to introduce its technique.

    Source: §1 · p. 1

  • Method

    RINI links contribution claims to evidence, checks whether a proposed remaining distinction is already covered, and applies local edits to the proposal.

    Source: §5 · p. 6, §5.3 · p. 7

  • Results

    After seed balancing, RINI successfully repairs 72.2% of proposals versus 39.1% for Retrieve-and-Revise, an improvement of 33.0 percentage points with the same decisive prior and target relation.

    Source: §7.1 · p. 8

  • Takeaways & Limitations

    On matched proposals requiring correction, RINI’s improved repair accompanies retention of the original research question and technical method while correcting scientific credit.

    Source: §9 · pp. 9–10

Abstract

from arXiv · show

A research proposal can describe an established mechanism correctly while claiming to introduce it. We study whether providing the earlier paper corrects such contribution claims. Three controlled experiments compare proposals generated with a contribution-bearing prior and a same-topic control. Providing the prior yields no clear aggregate reduction in unsupported novelty. Human analysis of 175 interpretable exposed proposals finds that 137 recognize the prior's relevance, but 61 correctly attribute the established contribution. Of 71 proposed remaining distinctions, 37 are covered by the same prior. We introduce Research Idea Novelty Inspection (RINI), which audits contribution claims against evidence, checks the remaining distinction, and applies local revisions. Five human annotators evaluate 1,080 original-revision pairs across three methods. On the same 240 originals judged to require correction, successful repair is 11.7% for Self-Revision, 39.1% for Retrieve-and-Revise, and 72.2% for RINI, with research tasks weighted equally. The improvement over same-evidence direct revision is 33.0 percentage points. The revised proposals retain their research questions and technical methods. These results motivate explicit contribution attribution when using literature to generate and revise research proposals.

1 INTRODUCTION

A proposal can accurately describe an established mechanism yet claim it as new; unsupported novelty is excess originality beyond what available evidence supports. The paper examines whether supplying prior work corrects these claims and introduces RINI to audit and revise contribution positioning.

1 INTRODUCTION

  • Unsupported novelty occurs when a proposal claims more originality than available evidence supports, rather than simply using a mechanism that is not new.The proposed correction preserves the research question and usable technical content while changing the claimed contribution.Source: §1 · p. 1
  • Providing the contribution-bearing prior yields no clear aggregate reduction in unsupported novelty, while proposals may acknowledge relevant work yet retain incorrect ownership claims.The authors also analyze attribution, claim narrowing, and whether proposed remaining distinctions are already covered by the same prior.Source: §1 · pp. 1–2
  • RINI links contribution claims to evidence, checks whether a proposed remaining distinction survives, and makes local revisions.The paper evaluates its revisions against self-revision and direct revision with the same added evidence.Source: §1 · pp. 1–2

2 RELATED WORK

Related work covers literature-grounded idea generation, novelty and evidence assessment, and revision with feedback. The paper distinguishes citation support from the separate question of who established a method.

Literature-grounded scientific ideation

  • Scientific idea-generation systems use literature to develop research problems, methods, and experiments, including iterative refinement and multi-agent hypothesis work.Examples include SciMON (Wang et al., 2024), ResearchAgent (Baek et al., 2025), The AI Scientist, and the AI co-scientist (Lu et al., 2024; Gottweis et al., 2026).Source: §2 · p. 2

Uncredited reuse and novelty assessment

Using evidence that is already available

Citations, evidence use, and scientific credit

  • Citation correctness and faithfulness do not establish scientific credit: a proposal can cite a paper beside a technical description while still claiming to introduce the technique.The diagnosis separates acknowledgment, contribution ownership, and novelty claims, and checks whether the same prior establishes a claimed extension.Source: §2 · pp. 2–3

Claim verification and evidence localization

  • Claim–evidence methods motivate explicit evidence links, but contribution assessment also needs the attribution around a technical statement.Describing a mechanism as adopted versus newly introduced makes different contribution claims (SciFact: Wadden et al., 2020; FActScore: Min et al., 2023; Minimal Evidence Groups: Li et al., 2025).Source: §2 · p. 3

Revision with feedback and external evidence

  • Revision work uses self-generated feedback or retrieved evidence, while this study compares direct revision with and without added evidence against RINI’s attribution and extension checks.The direct-revision comparison gives both methods the same evidence and prior-work reference (Self-Refine: Madaan et al., 2023; RARR: Gao et al., 2023a).Source: §2 · p. 3

3 CONSTRUCTING AND MEASURING THE EXPOSURE STUDY

The exposure studies construct matched cases and compare proposals generated with decisive-prior versus same-topic control evidence. Their machine outcome is the raw total excess of asserted over evidence-supported novelty across extracted claims.

3.1 CASE CONSTRUCTION AND MATCHED GENERATION

  • The selected cases define a target as the central technical operation, paired with a decisive prior that establishes it and a same-topic control that does not.Human curation and cross-review produced 62 evidence bundles; a 32-seed pool yielded 30 Study 1 seeds, and later studies added cases.Source: §3.1 · p. 3, Table 1 · p. 3
  • Matched H/E proposals differ in whether they receive a same-topic control or decisive-prior excerpt, with other literature, instructions, excerpt budget, position, and generator settings matched.Study 3 contains 180 matched H/E pairs, comprising 360 proposals for diagnosis and revision.Source: §3.1 · p. 3

3.2 MEASURING UNSUPPORTED NOVELTY

  • Unsupported novelty is measured by summing each extracted claim’s positive excess of asserted novelty over what the prior evidence supports.Each novelty level ranges from zero to four; the study retains the raw Gap score for controlled exposure analyses.Source: §3.2 · pp. 3–4

3.3 EFFECT OF PROVIDING THE PRIOR

  • The per-round and combined intervals include zero, so providing the contribution-bearing prior yields no clear aggregate decrease in measured unsupported novelty.The combined estimate retains all 90 observations and resamples by seed identity; Table 2 presents the exposure estimates.Source: §3.3 · p. 4, Table 2 · p. 4

4 WHAT CHANGES AFTER THE PRIOR IS SHOWN?

The human analysis distinguishes recognizing a prior from crediting it and examines whether shifted contribution claims survive comparison with that same prior. It also shows why aggregate novelty scores can diverge from changes to a target contribution.

4 WHAT CHANGES AFTER THE PRIOR IS SHOWN?

  • The analysis examines whether exposed proposals recognize the prior, correctly attribute its contribution, contract the target claim, or shift the claim to a concrete distinction.Five damaged or unjudgeable outputs leave 175 interpretable exposed proposals for analysis.Source: §4 · p. 4

4.1 RECOGNITION DOES NOT IMPLY CORRECT ATTRIBUTION

  • Among 175 interpretable exposed proposals, 137 recognize the prior’s relation to the target, but only 61 correctly attribute the covered contribution; 69 contract their target claim.Of 71 proposed remainders, 37 are covered by the same prior, 24 survive, and 10 are unresolved; 23 meet the criteria for valid relocation.Source: §4.1 · pp. 4–5, Table 3 · p. 5

4.2 WHY THE AGGREGATE SCORE CAN HIDE THE TARGET CHANGE

  • Target contraction and aggregate Gap decrease agree in 91 of 175 pairs, a seed-balanced agreement rate of 52.1%.They can diverge when proposals introduce unsupported novelty elsewhere or reduce another claim while retaining ownership of the covered target.Source: §4.2 · p. 5
  • The contribution-level diagnosis identifies revision operations: locate the claimed contribution, establish what the prior covers, and check whether any remaining distinction survives.Section 5 implements these operations in RINI.Source: §4.2 · p. 5

5 RINI: CONTRIBUTION AUDIT AND LOCAL REPAIR

RINI links contribution claims to prior evidence, checks whether the claimed remainder survives that same prior, and then makes local edits. Its audit separates contribution decisions from changes to the proposal text.

5 RINI: CONTRIBUTION AUDIT AND LOCAL REPAIR

  • RINI takes a proposal, target contribution, and decisive-prior excerpt, then audits claims and checks any remainder before local edits.The intended edits leave the research question, technical mechanism, and evaluation plan intact.Source: §5 · p. 6

5.1 AUDIT THE CONTRIBUTION IN ITS ATTRIBUTION CONTEXT

  • RINI retains each relevant statement's attribution context and checks both whether the proposal claims it and whether the prior establishes it.A positive coverage judgment must link to an exact passage from the supplied prior.Source: §5.1 · p. 6

5.2 CHECK WHAT REMAINS AFTER THE PRIOR IS CREDITED

  • If the same prior already establishes a proposed remainder, the revision must credit or withdraw that claim rather than relocate the novelty claim.Unresolved coverage remains a question, implementation choice, or experiment; only comparison-supported distinctions are retained as differences.Source: §5.2 · p. 6

5.3 REPAIR ONLY THE AFFECTED TEXT

  • After the audit, RINI proposes at most four exact-span replacements to reattribute or narrow covered claims, remove unsupported relocations, or turn unresolved distinctions into questions.A validator checks source spans and locality constraints; if reconstruction fails, the original proposal is retained.Source: §5.3 · p. 7

6 REVISION BENCHMARK AND HUMAN EVALUATION

The benchmark evaluates three revision policies on a frozen set of proposals, with human annotation and a matched comparison designed to assess repair. The matched comparison uses originals judged to need correction under all three evaluations.

6 REVISION BENCHMARK AND HUMAN EVALUATION

  • The benchmark revises 360 frozen proposals under three policies, producing 1,080 original–revision items.The 360 proposals come from Study 3.Source: §6 · p. 7

6.1 THREE REVISION POLICIES

  • Self-Revision gets no decisive prior, Retrieve-and-Revise revises directly with it, and RINI uses the same prior plus an audit, remainder check, and bounded edits.Table 4 covers the three policies on the same 360 frozen proposals.Source: §6.1 · p. 7, Table 4 · p. 7

6.2 HUMAN EVALUATION

  • Five annotators assessed all 1,080 items, judging contribution revision need, attribution, unsupported novelty, new errors, and preservation of the question, method, and evaluation plan.A separate 90-item pilot informed the final rubric.Source: §6.2 · p. 7

6.3 MATCHED COMPARISON

  • The primary matched comparison uses the 240 originals judged to require contribution revision in all three evaluations, covering all 30 Study 3 seeds.Rates are averaged within seed and weighted equally across seeds; intervals use paired whole-seed bootstrap resampling.Source: §6.3 · p. 8

7 REPAIR RESULTS

RINI repairs contribution problems more often than the two revision baselines in the matched comparison and leaves fewer proposals needing further revision across all sources. The reported outcomes also include content preservation, new errors, and variation across seeds.

7.1 RINI IMPROVES CONTRIBUTION REPAIR

  • Correct target attribution is 75.3% under RINI, 53.2% under Retrieve-and-Revise, and 12.0% under Self-Revision; still requiring revision are 25.9%, 60.9%, and 87.2%, respectively.Source: §7.1 · p. 8
  • 72.2% of common-flaw proposals are successfully repaired by RINI, versus 39.1% for Retrieve-and-Revise and 11.7% for Self-Revision; the same-evidence advantage is 33.0 percentage points.The 95% paired whole-seed bootstrap interval for the improvement over Retrieve-and-Revise is [24.5, 41.4].Source: Table 5 · p. 8, §7.1 · p. 8

7.2 FINAL CONTRIBUTION STATE ACROSS ALL 360 SOURCES

  • Across all 360 sources, 279/360 RINI outputs (77.5%) need no further revision, versus 163/360 Retrieve-and-Revise (45.3%) and 124/360 Self-Revision (34.4%).RINI–RR is +32.2 percentage points (95% CI [24.2, 39.7]).Source: §7.2 · p. 8
  • Research questions are preserved in 359 of 360 outputs under every policy; technical methods are preserved in 358 RINI, 344 Retrieve-and-Revise, and 359 Self-Revision outputs.Source: §7.2 · p. 8
  • Annotators record new contribution errors in 16 Self-Revision outputs, 29 Retrieve-and-Revise outputs, and zero RINI outputs.Source: §7.2 · p. 8
  • Across 30 research seeds, 26 favor RINI, three tie, and one favors direct revision.Source: §7.2 · p. 8

8 CASE STUDY: CORRECTING A COLBERT-BASED PROPOSAL

In the evaluated ColBERT-based proposal, RINI credits the prior's covered mechanism and reframes performance claims as questions for evaluation. The repair is judged successful while preserving the proposal's research content.

8 CASE STUDY: CORRECTING A COLBERT-BASED PROPOSAL

  • The supplied ColBERT prior already establishes the proposal's contextualized token representations and late-interaction MaxSim scoring, so the audit identifies the mechanism as covered.Source: §8 · p. 9
  • RINI makes four local edits, including crediting ColBERT and reframing performance claims as evaluation questions; annotators judge the repair successful with the question, method, and plan preserved.No new contribution error is introduced.Source: Figure 3 · p. 9, §8 · p. 9

9 DISCUSSION, FUTURE WORK, AND CONCLUSION

RINI links contribution claims to evidence, checks what distinction remains, and makes local edits. On matched proposals requiring correction, it improves successful repair over direct revision while preserving research questions and technical methods.

9 DISCUSSION, FUTURE WORK, AND CONCLUSION

  • RINI makes attribution explicit, checks the remaining distinction, and translates those decisions into local edits.Crediting the established retrieval trigger can still leave a concrete experiment on controller settings.Source: §9 · p. 9

Future work

  • Contribution auditing could extend to evidence distributed across multiple papers and support interactive workflows that refine hypotheses, assess distinctions, and guide experiments.Source: §9 · p. 9

Conclusion

  • 72.2% successful repair with RINI versus 39.1% with direct same-evidence revision, while preserving research questions and technical methods.The comparison uses matched proposals judged to require correction.Source: §9 · p. 10

ETHICS STATEMENT

  • The study evaluates model-generated scientific text and attribution of prior work; human annotators provided proposal and evidence judgments, with no sensitive or identifiable personal data collected.Annotators participated voluntarily without compensation, gave verbal consent, and are represented by anonymous identifiers.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The appendices specify the research-case construction, matched generation, scoring, diagnosis, revision, and human evaluation procedures.They include task instructions, model settings, scoring rules, complete categorical results, and paired seed-level estimators.Source: REPRODUCIBILITY STATEMENT · p. 11

AI USE STATEMENT

  • Generative AI tools assisted benchmark construction and experimental pipeline development, while model-based experimental components are described in the methods and appendices.The authors reviewed the design, implementation, outputs, analyses, citations, and manuscript, and take responsibility for the submission.Source: AI USE STATEMENT · p. 11

A STUDY ACCOUNTING AND WORKFLOW

A.1 SAMPLE COUNTS AND ANALYSIS UNITS

This section explains how study samples, literature evidence, cohorts, and novelty measurements were constructed, including the sample units and review procedures. It also defines how proposal claims and unsupported-novelty scores are counted and describes the matched W/H/E exposure design and its machine-scored H–E contrast.

A.1 SAMPLE COUNTS AND ANALYSIS UNITS

  • 30 seeds × 3 generators × 2 repeats produced 180 H/E pairs, 360 originals, and 1,080 repair items; the primary repair cohort contains 240 originals.The same 30 seeds are the resampling units, and seed-balanced rates give each seed equal weight.Source: §A.1 · p. 14, §B.5 · p. 20

B.1 FROZEN LITERATURE AND THE MEANING OF A CUTOFF

  • The frozen retrieval corpus contains 50,932 arXiv papers from cs.CL, cs.LG, and cs.CV (2018–2023), with BM25 and SPECTER2 rankings fused for retrieval.Each case records its date restriction, exact paper version, and evidence span; full text was fetched only for papers considered during evidence curation.Source: §B.1 · p. 15

B.3 CONSTRUCTION OF THE INITIAL SEED COHORT

  • Review retained 314 of 376 candidate seeds, and fixed-key sampling selected 30 seeds from a 32-seed pool for Study 1.SPECTER2 similarity of at least 0.85 triggered deduplication; human adjudication retained 18 of 19 split-judgment seeds after revision and rejected one.Source: §B.3.1 · p. 16, §B.3.5 · p. 17

B.3.2 ANNOTATING SEED SUITABILITY

  • Seed review assessed coherence, specificity, auditability, method–problem fit, duplication, and neutral wording, without asking reviewers to establish novelty or find a decisive prior.A candidate could be accepted, revised with a concrete change, or rejected with a reason.Source: §B.3.2 · p. 16

B.3.3 ASSEMBLING AND ANNOTATING REFERENCE LITERATURE

  • Curation paired a decisive prior establishing the seed’s central operation with a relevant same-topic control that did not establish it.Cross-reviewers checked coverage under relevant assumptions and challenged overlooked distinctions, including cases where shared triggers differed in consequential constraints or evaluation settings.Source: §B.3.3 · pp. 16–17, §B.3.4 · p. 17

B.3.6 STUDY 2 EXPANSION AND CROSS-STUDY OVERLAP

  • Studies 1 and 2 contain 41 distinct seed identities, while all three studies contain 90 study–seed observations and 71 distinct seeds.Study 2 selected 30 seeds from an expanded pool, including 19 from the original pool and 11 machine-curated additions; Study 3 used a separate 30-seed evidence-first cohort.Source: §B.3.6 · p. 17

B.4 STUDY 3 MACHINE EVIDENCE LOCATION AND SEMANTIC VETTING

  • Study 3 included all 30 selected seeds after machine evidence checks, with 30 accepted decisive-prior verdicts and 24 control verdicts at topicality four plus six at topicality three.Decisive evidence had to cap the central technical claim; controls had to be topical, non-covering, and near-decisiveness zero.Source: §B.4 · p. 18

C UNSUPPORTED NOVELTY MEASUREMENT

  • The extraction ledger retained 4,493 core claims across two machine views, totaling 8,986 records, without enforcing the prompt’s eight-claim limit.Novelty levels count positive responses across four cumulative thresholds; the asserted view judges what the proposal claims, while the supportable view judges what the displayed prior permits.Source: §C.1 · p. 20, §C.2 · pp. 20–21, §C.2.1 · p. 21

C.2.2 SUPPORTABLE VIEW

  • Reviewers assess contribution coverage, assumptions, and surviving technical distinctions; diagnosis and repair separately record attribution, remaining errors, and content preservation.A decisive reference may establish that a mechanism is known while leaving a setting, implementation, or evaluation question unresolved.Source: §C.2.2 · p. 21, §C.6 · p. 22

C.3 WORKED CALCULATION

  • In the constructed example, raw Gap is four rungs and UNB density is 0.25; threshold-crossing decomposition and word normalization are distinct quantities.The example also reports two of three claims with positive gaps and density of 0.75 claims per 100 words; density divides by proposal length.Source: §C.3 · p. 22, Table 12 · p. 22, §C.4 · p. 22

C.5 OBSERVED CLAIM COUNTS AND LENGTH SENSITIVITY

  • 481 of 540 proposals retained eight claims; median and interquartile range were both eight in every arm and overall.The overall histogram also included 49 proposals with more than eight claims and one with an empty inventory.Source: §C.5 · p. 22, Table 13 · p. 22

D.1 TREATMENT CONSTRUCTION

  • Each seed receives one focal document—neutral filler W, same-topic non-covering H, or decisive prior E—while the seed and proposal instruction remain identical across arms.Document count, excerpt budget, focal position, and surrounding materials are matched within generation blocks.Source: §D.1 · p. 23, Table 14 · p. 23

D.2 POTENTIAL OUTCOMES AND THE REPORTED MACHINE ESTIMAND

  • Study 3’s H–E estimator equally weights 30 seed means, averaging six matched contrasts per seed; positive raw-Gap contrasts indicate reduced unsupported novelty with the decisive prior.The machine outcome uses extracted claims and two machine-scored threshold views; human diagnosis and repair retain separate categorical endpoints.Source: §D.2 · p. 23, §D.8 · p. 24

D.3 WHY H–E IS PRIMARY

  • H–E isolates decisive contribution coverage against relevant non-covering evidence, while W–E also changes whether the focal slot contains seed-relevant topical information.The W–E contrast decomposes into W–H and H–E; executed contexts used W, H, and E in the same focal slot with matched word budgets.Source: §D.3 · p. 23

D.4 RANDOMIZATION AND INFORMATION SEPARATION

  • Focal position and API execution order were randomized, and separate calls prevented one arm’s proposal or evaluation feedback from entering another arm’s context.The causal contrast requires matched inputs and a common outcome reference; tool availability, instructions, decoding settings, and output budgets were fixed within blocks.Source: §D.4 · p. 24, §D.6 · p. 24

D.5 REPLICATION STRUCTURE AND CUTOFF STRATA

  • Study 3 produced 540 generation records—180 per condition—in 180 complete seed–model–repeat blocks; only complete matched blocks enter the exposure analysis.It used 30 seeds, three generator families, and two repeats; failed or missing calls cannot be replaced by another model’s output or treated as low Gap.Source: §D.5 · p. 24, §D.7 · p. 24

D.9 ANALYSIS AND UNCERTAINTY

The combined exposure analysis retains shared seed identities when pooling across studies, and human repair evaluation used a separate pilot-informed rubric. The supplied passages describe the aggregation and preparation procedures, but not a combined estimate’s numerical value.

D.10 COMBINING STUDIES WITH SHARED SEED IDENTITIES

  • The combined analysis retains all 90 study–seed observations and clusters resampling by seed identity, keeping both observations together for shared seeds.The three rounds contain 71 unique seeds.Source: §D.10 · p. 25
  • A second aggregation averages the two observations for each shared seed before equally weighting all 71 distinct seeds.Both aggregations retain observed outcomes and account for overlap between the first two studies.Source: §D.10 · p. 25

E.1 GENERAL TASK RULES AND PARTICIPANT REPORTING

  • Repair annotators label only information displayed in their assigned task and do not infer hidden treatment or another panel’s judgment.Ambiguity is recorded rather than resolved by inventing evidence.Source: §E.1 · p. 25

E HUMAN PREPARATION AND QUALIFICATION

  • The human repair rubric was established after a separate colleague piloted 90 items and discussed the pilot results.The formal repair task is distinct from this preparation and qualification material.Source: §E.1 · p. 25

E.2 PREPARATION PACKAGES AND TASK SEQUENCE

The appendices document participant preparation, proposal generation, repair-benchmark construction, scoring, and result linkage. The supplied passages emphasize qualification rules, matched source conditions, and the shared 240-source repair cohort.

E.2.1 KNOWN-PERTURBATION QUALIFICATION

  • Qualification compares constructed claim pairs with known assertion ordering; five recorded returns passed the eligibility thresholds.The task measures which wording asserts a stronger unsupported claim, not whether the contribution is scientifically novel.Source: §E.2.1 · p. 26

E.3 RELATION TO THE COMPLETED REPAIR EVALUATION

  • The repair task received 1,080 formal annotations, following a separate 90-item pilot that informed the final rubric.Five annotators each received 216 shuffled items, with one annotation per item.Source: §E.3 · p. 26

F.1 TWO INTERVENTIONS AND THEIR DISTINCT UNITS

  • Each repair method is evaluated on frozen source proposals, with 360 sources and 1,080 completed policy conditions in the executed batch.The three policies are Self-Revision, Retrieve-and-Revise, and RINI.Source: §F.1 · p. 27, §F.2 · p. 27

F.3 STRUCTURAL GATE AND GENERATION DISPOSITIONS

  • RINI’s local-edit reconstructions are structurally checked; rejected reconstructions retain the original proposal rather than replacing it.The batch records 291 accepted reconstructions and 67 rejections.Source: §F.3 · p. 27, §G.5 · p. 32

F.4 COMMON-FLAW SUBSET AND FULL-SET REPORTING

  • All three methods label the same 240 originals as needing contribution revision, the common denominator for their primary comparison.The full-set endpoint separately reports final contribution-revision need for all 360 sources.Source: §F.4 · p. 27

G.6 MEASUREMENT AND EVALUATION WORKFLOW

  • The machine-scoring workflow extracts auditable claims, scores asserted and supportable novelty separately, then aggregates claim gaps into seed-level exposure estimates.Study 3 retained 4,493 claims from 540 extraction records; 8,986 judge records parsed.Source: §G.4 · p. 31, §G.6 · p. 32

G.7 RECORD LINKAGE AND DEPENDENCY CHECKS

  • The linked records retain source, seed, policy, generation disposition, and output identities for outcome and preservation analyses.Machine outcomes and the diagnostic dataset are stored separately from final repair labels.Source: §F.6 · p. 28, §G.7 · p. 32

H.1 CONTROLLED-EXPOSURE RESULTS

The appendices report controlled-exposure estimates and extensive repair-outcome checks. Seed-balanced comparisons favor RINI over direct revision on the common-flaw cohort, while sensitivity analyses examine label disagreement and final-text judgments.

H.1 CONTROLLED-EXPOSURE RESULTS

  • −0.133333 is the Study 3 exposure estimate, with a seed-level Student-t 95% interval of [−0.740337, 0.473670].The main text reports the controlled-exposure results and a combined estimate with seed-clustered uncertainty.Source: §H.1 · pp. 32–33

H.2 ALL RETURNED REPAIR OUTCOMES

  • On the 255-source RINI–RR pairwise common-flaw cohort, successful repair is 73.4% versus 39.7%, a +33.7-point difference.The primary three-method comparison instead uses the shared 240-source cohort.Source: §H.2 · p. 33

H.3 FULL-SET OUTCOMES AND SEED-LEVEL VARIATION

  • The full-set outcome figure shows the complete distributions for 360 sources per method and the within-seed direction of repair comparisons.Its seed-level panel reports the RINI-favoring direction on the common-flaw cohort.Source: Figure 4 · p. 34
  • +33.0 percentage points is the mean seed contrast for RINI versus Retrieve-and-Revise successful repair; 26 seeds favor RINI, three tie, and one favors RR.Each seed contributes 6–11 common-flaw sources.Source: §H.3 · p. 34, Figure 4 · p. 34

H.7 SENSITIVITY TO DISAGREEMENT IN ORIGINAL-TEXT JUDGMENTS

  • After excluding 88 sources with RR–RINI disagreement on original labels, the seed-balanced final-state advantage remains +31.2 points, with 95% CI [23.2, 39.0].The recorded final labels are held fixed, so the check does not rule out shared annotation error or systematic final-output evaluation differences.Source: §H.7 · p. 37

H.8 INDEPENDENT FINAL-TEXT-ONLY REANNOTATION

  • Under conservative source-level aggregation, 24/30 RINI finals and 19/30 RR finals need no further contribution revision, a descriptive +16.7-point difference.This final-text-only check does not condition on method-specific judgments of the unchanged original text.Source: §H.8 · p. 38

H.9 RINI GATE DISPOSITION JOINED TO FINAL EVALUATION

  • Among the 1,080 items, none of the exactly unchanged texts receives a successful-repair label, and no successful repair also receives a positive new-contribution-error label.There are 69 exactly unchanged RINI texts, compared with one RR and ten Self-Revision texts.Source: §H.9 · p. 38

I CONTRIBUTION DIAGNOSIS AND REVISION PROMPTS

This section defines how prior-work contribution claims are diagnosed and operationalized through evidence-linked audits and local edits. It also specifies annotation outputs and criteria for judging repair and preservation, and describes how formal annotations were collected.

I.2 FINAL H/E CONTRIBUTION-DIAGNOSIS INSTRUCTIONS

  • The diagnosis judged Proposal A against prior D and used same-seed Proposal B only to compare claims about target X.Annotators separately assessed acknowledgment, attribution, contraction versus B, a concrete remainder Y and its coverage by D, misreading, and non-response; citations alone did not count as substantive acknowledgment or attribution.Source: §I.2 · p. 40

I.3 DIAGNOSTIC FIELD COUNTS

  • Among 175 interpretable exposed proposals, 137 acknowledged the prior’s relevance, but only 61 correctly attributed its established contribution.The export contained 180 matched pairs; five damaged proposals were excluded from the interpretable set.Source: §I.3 · p. 41
  • 37 of 71 proposed remainders were covered by the same prior, 24 survived, and 10 were unresolved; valid relocation occurred in 23 of 165 eligible pairs.Valid relocation required correct attribution, contraction, a surviving remainder, and no misreading; its seed-balanced rate was 13.8333%.Source: §I.3 · p. 41

I.6 THREE METHODS BEGIN FROM THE SAME ORIGINAL

  • Self-Revision received no prior, while Retrieve-and-Revise and RINI received the same D excerpt and explicit D–X relation; only RINI required an audit before revision.The primary comparison used the same 240 originals requiring correction under all three method-specific evaluations.Source: §I.6 · pp. 41–42, §I.8 · p. 42, §I.9 · p. 42

I.10 RINI STEP 1: CONTRIBUTION-LEVEL AUDIT

  • RINI first audits atomic contribution claims against D, then requests at most four exact-span local edits that correct covered novelty while preserving the proposal’s mechanism and evaluation plan.The audit treats missing evidence in the supplied excerpt as uncertain, not proof of novelty; unresolved distinctions should be phrased as testable possibilities.Source: §I.10 · p. 43, §I.11 · p. 43

I.12 RECONSTRUCTION CHECKS AND OBSERVED DISPOSITIONS

  • The reconstruction gate rejected 67 RINI edits and retained their source text; each direct-revision condition produced 359 proposals and one source-degenerate no-op.RINI recorded 359 audits and one degenerate source without a substantive audit.Source: §I.12 · p. 44, Table 31 · p. 44

J.1 TASK AND EVIDENCE

  • Five annotators each judged 216 shuffled items once, using the original, revision, target, and prior evidence while blinded to method and source identities.The task evaluated whether the revision distinguishes established prior work from a defensible research question across the whole proposal.Source: §J.1 · pp. 44–45

J.2 THREE STRICT BOUNDARIES

  • Human repair judgments required clear attribution of established work, rejected novelty relocated to content already covered by D, and assessed preservation of the research question, method, and evaluation plan separately.An unresolved Y could pass when framed as a testable choice rather than certified novelty; peripheral technical imperfections were recorded separately unless they supported the contribution claim.Source: §J.2 · p. 45, §J.3 · p. 45, §J.4 · p. 46

J.7 JSONL OUTPUT CONTRACT

  • The output contract records novelty judgments, attribution, remaining distinctions, technical issues, and preservation of the research question and methods.It also records evaluation-plan preservation and a final repair-outcome category.Source: §J.7 · p. 47
  • An honestly bounded remaining distinction may still count as successful repair, but a new substantive contribution error does not.An already acceptable original cannot count as a newly successful repair; when no prior contribution is present, the distinction is not applicable and its text is empty.Source: §J.7 · p. 47

J.8 COMPLETE CONSTRUCTED OUTPUT EXAMPLE

  • The constructed JSONL example illustrates how the record captures prior attribution, a remaining distinction, novelty status, and preservation judgments.It is explicitly illustrative, excluded from study counts, and its quotations are not from a real paper.Source: §J.8 · p. 47

J.9 FORMAL ANNOTATION RECORD

  • Five formal annotators labeled 1,080 shuffled items once each after the rubric was established through a 90-item pilot and discussion.The records retain categorical outcomes and evidence fields used in the reported analyses.Source: §J.9 · p. 48

J.10 INDEPENDENT FINAL-TEXT-ONLY ROBUSTNESS ANNOTATION

The appendix defines a final-text-only review of whether proposals need substantive contribution correction, alongside human preparation protocols for seed vetting, evidence curation, and novelty-burden qualification.

J.10 INDEPENDENT FINAL-TEXT-ONLY ROBUSTNESS ANNOTATION

  • Reviewers judge each complete final proposal against supplied prior evidence, using yes, no, or unclear for whether substantive contribution revision remains necessary.A no judgment is limited to the supplied evidence and does not certify global novelty.Source: §J.10 · pp. 48–49

K.1 NEUTRAL SEED VETTING

  • Seed vetting asks reviewers to assess scope, specificity, auditability, source support, duplication, and generation neutrality, with decision rules governing accept, revise, or reject.Reviewers judge the seed as written without searching for prior work or assessing novelty.Source: §K.1 · p. 51

K.2 EVIDENCE CURATION: FIRST ISSUED ROUND

  • In the first evidence-curation round, reviewers select a decisive prior covering the seed’s central technical move and a tempting same-topic hard negative that does not cover it.They provide verbatim supporting spans and rationales, plus the required control documents; they exclude rows lacking defensible evidence for both roles.Source: §K.2 · p. 52

K.3 EVIDENCE CURATION: SECOND ISSUED ROUND

  • The second evidence-curation round repeats the same decisive-prior and hard-negative selection protocol, requiring verbatim spans and the specified control documents.Candidates are shuffled, and inclusion requires both a defensible decisive prior and hard negative.Source: §K.3 · pp. 53–54

K.4 KNOWN-PERTURBATION QUALIFICATION

  • Known-perturbation qualification asks reviewers which of two claims carries more unsupported novelty burden, while allowing equal or unclear judgments.Reviewers judge assertion strength rather than whether the contribution is actually novel, and should not force a direction when the text cannot settle it.Source: §K.4 · pp. 54–55
Loading 2609.33284v1…