ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomes

1.1 Problem and scope

• The suite makes Paper-Search and Scoop-Check standalone skills alongside IdeaSpark, which composes grounding, bottleneck diagnosis, generation, collision checking, audit, and idea-card packaging.The evaluation is scoped to idea generation and does not test implementation success, human peer review, or program-committee selection.Source: §1.1 · p. 4

1.2 Motivation

• ResearchStudio-Idea addresses the missing middle between retrieved evidence and actionable research opportunities by structuring how an LLM identifies bottlenecks and audits a direction against prior work.Its ideation patterns specify applicable gaps, method construction, differentiation, credibility evidence, and failure modes.Source: §1.2 · pp. 4–5

1.3 Approach overview

• IdeaSpark mines outcome-labeled conference papers into strategy signatures, clusters them, and induces operational cards that guide generation and audit against failure modes and prior-art threats.The cards record research situations, differentiating mechanisms, evidence expectations, and failure modes; generation and audit use the same corpus-derived objects.Source: §1.3 · pp. 5–6

1.4 Main empirical findings

• The pipeline induces 15 higher-level ideation patterns from 31 sub-patterns; this is an operational granularity, not a canonical ontology.The count comes from one Opus 4.7 induction call; inter-prompt, inter-seed, and inter-model stability remain future work.Source: §1.4 · p. 6
• Rejected papers map into the existing 15-pattern vocabulary, while pattern acceptance and impact profiles vary by domain-pattern cell.Rejected work therefore informs weak instantiations and boundary cases, while domain statistics serve as audit context rather than a deterministic generation prior.Source: §1.4 · p. 6
• Papers commonly combine patterns: k = 2 is modal across Oral, HC, and Reject papers, and 33.6% have k ≥3, motivating one-to-three pattern roles.The finding comes from a separate paper-level multi-label pass.Source: §1.4 · p. 6
• Blind automated judges find that IdeaSpark improves generated-idea quality over no-skill and generic-skill baselines, but acceptance-level claims require human review.The evaluation does not test implementation success, human peer review, or program-committee selection.Source: §1.1 · p. 4, §1.4 · p. 6

1.5 Contributions

• The outcome-grounded analysis covers 1,947 Oral, high-citation, and rejected papers, inducing 15 patterns, 31 sub-patterns, and 28 research domains.It measures pattern-level acceptance, citation, domain, conference, temporal, and multi-pattern composition profiles; cards pair success conditions from accepted work with failure modes from rejected work.Source: §1.5 · p. 6

2.5 Surveys and skill-engineering practice

• Surveys organize the broader scientific-automation landscape, while skill-engineering practices inform IdeaSpark’s two-tier design and phase boundaries.The related-work section groups literature into relevant families and defers direct comparison to §2.6; the surveys include “From Automation to Autonomy” [61] and “Towards Scientific Intelligence” [37].Source: §2 · p. 7, §2.5 · p. 8

2.1 End-to-end “AI scientist” systems

• Full-lifecycle systems automate research from idea generation through execution and writing, but documented failures motivate IdeaSpark’s narrower focus on improving ideas before execution.Prior systems include AI Scientist [29], Agent Laboratory [40], Idea2Plan [18], and Google AI co-scientist [11]; critiques identify execution drift, weak domain knowledge, and missing experimental feedback.Source: §2.1 · p. 7

2.2 Multi-agent and search-based ideation

• Multi-agent and search-based approaches use role-separated collaboration, iterative search, or learned signals to explore and refine research ideas.The surveyed benchmark reports mean novelty of 4.17/5 for decomposition-based and long-context workflows; other approaches include FlowPIE and Alien Science [4].Source: §2.2 · p. 7

2.3 Pattern induction from conference outcomes

• Related pattern-induction work mines accepted papers or literature graphs for reusable structure, while IdeaSpark’s contrast also incorporates rejected submissions.Examples include Oral-paper taxonomy annotation [26], MoRI’s motivation→innovation pairs [13], and knowledge-graph approaches such as SciMuse [14] and Graphs of Research.Source: §2.3 · pp. 7–8, §2.6 · p. 9

2.4 Novelty, evaluation, and benchmarks

• Idea-evaluation research spans novelty-collision checks, direct quality judgments, and standing benchmarks, with studies finding that novelty judgments can diverge from expert assessments and AI ideas can underperform after execution.The literature includes RINoBench, the execution study [45], IdeaBench [15], and ResearchBench [28].Source: §2.4 · p. 8

2.6 How IdeaSpark differs from current idea-generation methods

• IdeaSpark binds pattern induction, Oral/HC/Reject contrast, and retrieval-grounded generation to one non-parametric pattern card used to generate and audit a single reviewer-facing idea.The audit checks retrieved prior art and corpus-derived failure modes associated with the generating pattern, surfacing execution-level gaps rather than masking them with surface novelty.Source: §2.6 · pp. 8–9

3.2 Metadata coverage

• The collection covers ICLR, ICML, and NeurIPS papers from 2021–2025, retaining available metadata and review fields; 56 papers lack the abstracted fields needed for the embedding pool.Initial coverage includes 1,074 full abstracts, 716 parsed reviews, 682 extracted introductions, and 340 meta-reviews. Corpus IDs resolve to OpenReview records, and substantive body-text discussions receive bibliography entries.Source: §3.1 · p. 9, §3.2 · pp. 9–10, §3.3 · p. 10

3.1 Scope and labeling

• The corpus contains 1,014 Oral papers, 260 High-Cited papers, and 722 Reject papers; the 49 Oral–HC overlaps yield 1,947 unique papers.Oral represents program-committee preference, High-Cited community adoption, and Reject a concrete failure signal; the 2025 High-Cited cutoff is top 10 rather than top 30. Only 340 papers have meta-reviews despite decisions being available for all 1,947, reflecting review-access and parsing gaps.Source: §3.1 · p. 9, Table 1 · p. 10

4 Two-Stage Innovation-Signature Extraction

• The innovation signature retains eight paper-specific base fields, while only the four rewritten mechanism fields are embedded for clustering; the eight base fields remain available for extraction and later card evidence.The eight fields comprise four strategy-bearing fields and four review- and outcome-related fields.Source: §4 · p. 10

4.1 Stage 1: Eight base fields

• Stage 1 extracts eight structured fields covering innovation approach, key step, non-obviousness, trigger condition, reviewer praise and concerns, acceptance signal, and contribution type; all 1,947 papers pass.The two review-derived fields are constrained to non-empty lists of three items.Source: §4.1 · pp. 10–11

4.2 Stage 2: Four domain-agnostic rewrites

• Stage 2 rewrites the four strategy-bearing fields with generic placeholders and imperative phrasing, preserving mechanisms while dropping application detail; 1,891 of 1,947 papers have all four fields.The remaining 56 papers with at least one empty abstract field are excluded from the embedding pool.Source: §4.2 · p. 11

5.1 Embedding

• The four abstract strategy fields are concatenated into records averaging 1,185 characters, embedded in 3,072 dimensions, and L2-normalized.The fields are abstract strategy, abstract key step, abstract why non obvious, and abstract trigger condition.Source: §5.1 · p. 11

5.2 Clustering

• At min cluster size=10, HDBSCAN identifies 31 ideation sub-patterns with silhouette 0.584, while 902/1,891 (47.7%) remain unclustered; the unclustered share stays 40–49% across the sweep.The authors interpret the unclustered papers as lying between modal clusters rather than outside the strategy space; paper-level multi-label tagging covers clustered and unclustered papers alike.Source: Table 2 · p. 11, §5.2 · p. 12

5.3 Cluster inventory

• The 31-cluster inventory ranges from 13 to 86 papers per cluster (mean 31.9; median 22); automated labeling and disambiguation cards capture tactics, selection conditions, and failure modes.The cards also distinguish sibling clusters and distill lessons from Oral and Reject examples without per-paper citations.Source: §5.3 · p. 12

5.4 Per-cluster acceptance composition

• Six clusters meet the Oral-safe threshold, one meets the Reject-warn threshold, and 24 are mixed; these are informational risk signals, not retrieval or generation gates.C13 is the sole Reject-warn cluster (4/1/12, 25% Oral share among O+R); reviewers flag topology-as-prior proposals that do not show identifiable empirical value over standard alternatives.Source: §5.4 · pp. 12–13, Figure 3 · p. 13

6 Ideation-Pattern Induction

• The taxonomy maps each of 31 clusters to one primary pattern, and the resulting 15 patterns are reusable reasoning strategies rather than domain or venue categories.The induction prompt required definitions, operational signatures, and when-to-apply clauses; Opus returned the taxonomy and cluster mapping in one structured call, without edits.Source: §6 · p. 13

6.1 From definition to operational card

• Each pattern’s operational card combines success conditions, failure modes, outcome contrasts, reviewer expectations, cognitive barriers, and representative examples.Evidence-bearing fields cite paper IDs; cards support pattern fit, idea instantiation, auditing, and explanation, but are not used as acceptance-rate priors or ranking signals.Source: §6.1 · p. 15

6.2 Coverage and granularity

• The 15-pattern taxonomy covers all 989 clustered papers by primary inheritance and all 1,891 papers through multi-label tagging, with a long tail of rare patterns.The three most common primary patterns account for 384 papers; the three least common account for 48. The 31 clusters have sizes of 13–86 (mean 31.9), with only two below 15 and silhouette 0.584.Source: §6.2 · p. 16

6.3 Multi-strategy assignments

• Among clustered papers, 73.7% carry both a primary and secondary pattern, and paired papers have a 58.2% Oral rate versus 52.7% for single-pattern papers.The strongest reject-enriched pair, operator substitution + heterogeneous decomposition, falls 17.9 percentage points below the 58.4% baseline; the cluster-level view is capped at two patterns and excludes 902 unclustered papers.Source: §6.3 · p. 16

6.4 Ideation-pattern adjacency in embedding space

• High-cosine neighbors align with Oral-enriched pairings, while three patterns—Audit and Pivot, Reframe as a Solvable Object, and Substitute the Operator or Representation—serve as nearest-neighbor hubs.The two most isolated patterns are Design a Confound-Isolating Diagnostic (0.904) and Characterize a Limit, Then Surpass It (0.917); the diagnostic pattern has n=86, so isolation does not simply track small support.Source: §6.4 · p. 18

6.5 Paper-level multi-label ideation pattern assignment

• Paper-level tagging finds that 59.2% of papers use exactly two patterns, while three combinations reach Oral rates of 85.0%, 81.8%, and 80.0%.The 80.0% combination—Audit and Pivot + Characterize a Limit, Then Surpass It—is the highest-volume strongly Oral-enriched pair; class mean pattern counts are statistically indistinguishable (2.26–2.32). The skill uses 1–3 patterns per idea, with two as the default, and selects by bottleneck fit rather than combination-rate priors.Source: §6.5 · pp. 19, 21, Table 6 · p. 20

7 Acceptance and Impact Analysis

• The bias estimates are partial: cluster-level primaries cover 482 of 1,014 Oral papers and 367 of 722 Reject papers, with unclustered papers covered by §6.5 tagging.The table also reports 172 HC papers with cluster-level primaries; denominators are cluster-level totals, not full class populations.Source: §7 · p. 21, Table 7 · p. 21
• The Oral–Reject axis spans only ±2.9 pp, while the PC–community axis reaches roughly ±13 pp and is the stronger discriminating signal.The analysis describes strategy choice at the main-pattern level as contributing little to acceptance; execution at the cluster level is more discriminating.Source: Table 7 · p. 21, §7 · p. 21
• Audit and Pivot is the largest cluster-level pattern for both Oral and Reject papers, with 19.5% and 21.5% shares, respectively, and ∆OR=−2.0 pp.Its ∆OH=+13.1 pp is the taxonomy’s largest positive PC bias; the analysis attributes the separation between its 94 Oral and 79 Reject papers to execution quality rather than strategy choice.Source: §7 · p. 21

(O3) Unify Heterogeneous Inputs into One Space is the dataset’s clearest community-favored

• Unify Heterogeneous Inputs into One Space is community-favored: its HC share is 17.4% versus a 6.2% Oral share, yielding ∆OH=−11.2 pp.The passage says cross-modal-alignment papers are cited heavily but are not elevated to Oral at the same rate.Source: (O3) Unify Heterogeneous Inputs into… · p. 22
• Reframe as a Solvable Object has positive ∆OR=+2.9 pp and ∆OH=+7.1 pp, and its high cosine to Audit and Pivot supports a recipe of recasting an intractable formulation before auditing and pivoting its assumption.The reported cosine is 0.956; the passage characterizes this as the cleanest Oral signal.Source: (O3) Unify Heterogeneous Inputs into… · p. 22
• PCs tend to reward structural-insight moves while the citing community tends to reward usable-infrastructure moves, and IdeaSpark uses these statistics only for audit context, not as selection priors.The two rankings are described as nearly orthogonal; IdeaSpark selects patterns by structural fit to the bottleneck.Source: (O3) Unify Heterogeneous Inputs into… · p. 22

8.1 Domain induction

• The induced taxonomy has 28 domains, assigns at least one domain to 98.5% of papers (1,918 / 1,947), and averages 1.81 domains per paper.The taxonomy was induced from normalized paper tags clustered with UMAP+HDBSCAN and labeled by language models; the model returned 28 domains without edits.Source: §8.1 · p. 23

8.2 Domain × ideation pattern heatmap

• 100% Oral (nO=12, nR=0) for Trustworthy & Responsible ML × Characterize a Limit, Then Surpass It marks the dataset’s cleanest acceptance cell.Characterize a Limit, Then Surpass It occupies four of the five strongest cells; formally identifying a limit and proving a strict improvement gives it a reviewer-defensible cross-domain signature, while weak cells show that patterns can underperform elsewhere.Source: Figure 10 · p. 23, §8.2 · pp. 23–24

8.3 Ideation-pattern breadth: how domain-agnostic is each pattern?

• Audit and Pivot an Assumption and Reframe as a Solvable Object reach all 28 domains, while Decompose for Differentiated Treatment reaches 27.These are also the highest-volume patterns, with 1008, 527, and 529 tagged papers respectively.Source: §8.3 · pp. 24–25
• Construction-level patterns reach 22–25 domains but retain domain-modal anchors, whereas Relax Discrete Search to Continuous, Unify Heterogeneous Inputs into One Space, and Characterize a Limit, Then Surpass It reach only 9, 11, and 12.Their narrower footprints reflect prerequisites such as discrete-search-amenable problems, cross-modal alignment scaffolds, or an existing formal limit characterization; the last is nevertheless the cleanest cross-domain Oral signal observed.Source: §8.3 · p. 25
• The study omits domain-by-year pattern rates because smaller-domain cells contain only 1–3 papers, relying instead on live retrieval to assess domain-specific saturation.IdeaSpark’s Phase 2.1 saturation read uses the literature retrieved in each run rather than coarse dataset bins.Source: §8.3 · p. 25

9.1 Temporal trends

• Decompose and Delegate to Solvers rose fastest, while Decompose for Differentiated Treatment fell most; Audit and Pivot an Assumption remained highest-volume.The comparison averages 2021–2022 against 2024–2025. The skill reads pattern saturation from each run’s retrieved literature rather than using corpus-wide trends as generation priors.Source: §9.1 · pp. 25–26, Figure 12 · p. 25
• Early-period shares use a smaller base: 77 papers in 2021 and 73 in 2022, versus 537–606 papers annually in 2023–2025.Changes below 2 percentage points are not considered statistically robust; the reported risers and fallers are at least 2.5 points.Source: §9.1 · p. 26

9.2 Conference preferences

• +3.8 pp for Audit and Pivot an Assumption and +1.4 pp for Characterize a Limit, Then Surpass It distinguish ICML from other venues.ICML’s signature favors diagnostic or theoretical contributions; venue signatures appear as audit context on the Phase 4 idea card.Source: Figure 13 · p. 26, §9.2 · pp. 26–27
• +4.4 pp for Encode Structure by Construction and +3.0 pp for Liberate a Fixed Generative Component mark ICLR’s construction-side over-indexing.ICLR also under-indexes on Prove Equivalence to Unify and Audit and Pivot an Assumption.Source: Figure 13 · p. 26, §9.2 · p. 26
• +4.7 pp for Decompose for Differentiated Treatment and +3.7 pp for Manufacture the Supervisory Signal mark NeurIPS over-indexing.NeurIPS under-indexes on Design a Confound-Isolating Diagnostic; its signature is described as systems-level composition plus invented supervision.Source: Figure 13 · p. 26, §9.2 · pp. 26–27

10 Do Rejected Papers Inhabit a Different Strategy Space?

• Rejected papers cluster within the same 15-pattern taxonomy: all 13 Reject-only clusters map to existing patterns at high cosine, with no out-of-taxonomy label.The assigned cosine similarities are 0.897–0.986, and 11 of the 15 patterns receive at least one Reject-only cluster.Source: §10 · p. 27, §10.1 · p. 27

10.1 Reject-only clustering and mapping

• On 711 complete-field Reject papers, fixed UMAP/HDBSCAN produced 13 clusters and a 40.5% unclustered fraction; 11 main patterns received Reject-only mappings.The four patterns without mappings included three small-sample patterns (n ≤18 each) and one constructively encoded pattern.Source: §10.1 · p. 27

10.2 Interpretation

• Ten of 15 patterns differ by |∆| ≤10 across computations; Reject-only assigns +52 to Unify Heterogeneous Inputs into One Space and +19 to Audit and Pivot an Assumption, but fewer to three constructive patterns.The decreases are −23 for Encode Structure by Construction, −15 for Reframe as a Solvable Object, and −10 for Manufacture the Supervisory Signal.Source: §10.2 · pp. 27–28, Figure 14 · p. 28
• Reject-only clustering shifts surface forms toward audit/unify and away from constructive moves, so Phase 3.2 should check whether a selected constructive pattern has collapsed into an audit/unify lane.The proposed interpretation is that failed constructive moves can resemble audit/unify moves when stripped of their construction.Source: §10.2 · p. 28

11 Ablation: Embedding Model and Abstraction Stage

• The ablation compares the production paraphrase-strong encoder and domain-agnostic rewrites against SPECTER2 and original base fields on the same 1,891-paper subset.UMAP parameters are fixed, while HDBSCAN min cluster size is swept over 10, 15, 20, and 25.Source: §11 · p. 28

11.1 Embedding model: OpenAI vs SPECTER2

• At min cluster size 10, text-embedding-3-large yields 31 clusters and silhouette 0.584, versus SPECTER2’s 23 clusters and 0.438; SPECTER2 reaches silhouette 0.333 at size 25.The paraphrase encoder better preserves strategy-equivalence similarity, whereas SPECTER2’s citation-informed training favors topical adjacency; its unclustered fraction remains 41–54%.Source: §11.1 · pp. 28–29, Figure 15 · p. 29

11.2 Abstraction stage: silhouette is not enough

• At min cluster size 10, base fields score silhouette 0.586 versus production’s 0.584 and have 35.7% versus 47.7% unclustered, but collapse to two topic mega-clusters at sizes 15–20.The collapsed clusters have silhouette 0.884 and 0% unclustered, yet group papers by broad topic rather than reasoning move.Source: §11.2 · p. 29, Figure 15 · p. 29
• Production remains stable across coarser settings, with 12–19 clusters and silhouette 0.53–0.58; it is selected for construct validity and stability rather than maximum silhouette.Domain-agnostic rewrites remove topic anchors while tying base fields on silhouette at the chosen setting and avoiding collapse at adjacent settings.Source: §11.2 · pp. 29–30

14.1 Systems compared

• The four sources form a control ladder: bare Opus-4.8, a generic auto-built Opus skill, corpus-grounded IdeaSpark, and bare GPT-5.5.IdeaSpark, Opus-self-gen, and bare Opus-4.8 share the Opus 4.8-high backbone; the second bare model separates backbone-specific from skill effects.Source: §14.1 · p. 36

14.2 Problem seeds: 100 method-agnostic directions from ICLR 2026

• The evaluation uses 100 ICLR 2026 Oral seeds, selected after the 2021–2025 induction corpus and rewritten from titles alone to specify problems without hinting at methods.Each seed was screened for leakage and omitted method suffixes and abstract-derived bottleneck diagnoses.Source: §14.2 · pp. 36–37

14.3 Output normalization (controlling confounds)

• All sources were normalized to three-section idea cards with matched section-length ranges and interpreted formal equations, controlling verbosity and technical-style confounds.IdeaSpark’s protected author-decision annotations were stripped to avoid an asymmetric honesty tax.Source: §14.3 · p. 37

14.4 Two automated judges

• Separate blind skills score quality from first principles across problem position, method quality, and problem-fit, and novelty by the worst collision across four prior-art matching axes.Quality uses listwise ranks and requires quoted evidence for each axis; novelty levels range from 1 (all four axes match) to 5 (no overlap).Source: §14.4 · p. 37

14.5 Protocol and metrics

• Each seed’s four ideas were shuffled under random labels and judged in three independent rounds, with quality averaged across seeds and novelty across 300 judgments.Quality ranks map to scores 4 through 1; novelty uses the five-level scale, and the two axes are judged in separate passes.Source: §14.5 · pp. 37–38

14.6 Results

• IdeaSpark scores 3.87/4 on quality and ranks first on 88/100 seeds, while skill-based systems remain at medium-overlap novelty (L3).The same-backbone comparison indicates the lift comes from IdeaSpark’s corpus-grounded pattern cards and multi-phase, audit-anchored workflow, not model family or retrieval alone.Source: Table 11 · p. 38, §14.7 · pp. 38–39

14.7 Analysis

• GPT-5.5 (bare) scores 3.73 novelty but 1.00 quality with zero wins, illustrating how vague ideas can appear novel by evading prior-art collisions.The quality–novelty plane distinguishes this “novel-but-empty” regime from high-quality ideas that share framing and domain with prior work but contribute distinct mechanisms.Source: Figure 16 · p. 38, §14.7 · p. 39
• In a 5-seed pilot, retaining IdeaSpark’s author-decision annotations cost roughly one rank position; stripping them restored its lead over bare Opus-4.8.The annotations were removed before scoring because they are author-facing meta-notes that other sources do not emit.Source: §14.3 · p. 37, §14.7 · p. 39

15 Limitations

• The stated limitations mark where further evidence could extend the study’s idea-stage claims, rather than changing their scope.Each limitation is paired with a concrete next step.Source: §15 · p. 39
• Using only ICLR, ICML, and NeurIPS outcomes and reviews, with ICLR 2026 Oral evaluation seeds, bounds coverage to mainstream ML and may limit transfer.Cross-venue and cross-discipline corpora, backdated retrieval for leakage-free forward prediction, and cross-backbone judges are proposed to test and widen external validity; domain-agnostic patterns and live-literature novelty checks mitigate but do not eliminate these limits.Source: §15 · p. 39
• Automated LLM-based evaluation measures agreement with structured reviewers, not human program-committee judgment, and judges may share generators’ blind spots.A blind expert study is the planned next step before making any acceptance-level claim.Source: §15 · p. 39
• Idea-quality favors novel mechanisms over benchmark, systems, or pure-measurement contributions, while novelty alone can reward generic ideas that evade prior-art collision.The paper interprets quality only relatively across generation systems, reports novelty jointly with quality, and proposes contribution-specific rubrics and specificity-aware novelty metrics.Source: §15 · p. 39

16.1 Data card

• The corpus uses public metadata and review material, retaining author identities as metadata but not using them as model features for pattern induction or recommendation.Source: §16.1 · p. 39

16.2 Model and backend card

• The pipeline uses backend-specific LLMs for extraction, taxonomy induction, card generation, and skill phases, plus embeddings for clustering.Claude Sonnet 4.6 handles signature extraction and paper-level tagging; Claude Opus 4.7 handles taxonomy/domain induction and pattern-card generation; text-embedding-3-large clusters signatures; Claude Opus 4.8 high runs the skill and endpoint evaluation; GPT-5.5 is the cross-family baseline. Cross-backend quality and format-validity measurement remains future work.Source: §16.2 · p. 40

16.3 Skill card

• IdeaSpark is suited to literature-grounded ML directions with a stated domain, adjacent literature, constraints, and desired contribution type, but not to several broader or sensitive tasks.It is not intended for confidential literature without connector permission, broad agenda-setting without adjacent papers, pure benchmark construction, complete paper writing, automated submission production, or claims requiring private peer-review data; it cannot guarantee novelty, acceptance, feasibility, or reviewer agreement.Source: §16.3 · p. 40

17 Responsible Use

• IdeaSpark is scoped as an ideation scaffold rather than a paper-writing agent, producing one idea card with visible failure paths, falsification, and feasibility checks.This scope addresses risks including low-quality proposals, reviewer burden, shallow arXiv submissions, and overconfident novelty claims; candidates are audited against reject lessons and prior-art threats.Source: §17 · p. 40
• Rejected submissions and reviews are used only as aggregate pattern-level evidence about failed strategic moves, never to rank or profile papers, authors, or reviewers.Public deployment should include rate limits, artifact provenance, citation grounding, warnings against interpreting the system as predicting acceptance, and rules for review-text redistribution and removal requests.Source: §17 · p. 40

18 Conclusion

• A 1,947-paper analysis of ICLR, ICML, and NeurIPS outcomes from 2021–2025 induced 15 ideation patterns, 31 sub-patterns, and 28 research domains.The three-skill suite comprises standalone Paper-Search and Scoop-Check, plus IdeaSpark, whose operational card library guides bottleneck-to-move selection and candidate auditing.Source: §18 · p. 40
• At matched length and format, IdeaSpark improves judged idea quality over no-skill and generic-skill baselines while remaining competitively novel.The evaluation provides initial endpoint evidence; validation beyond the idea stage remains open.Source: §18 · p. 40
• The claim is bounded: public conference outcomes provide reusable strategic operators, but the work does not show that an LLM generates Oral papers, predicts acceptance, or replaces experimental judgment.Remaining validation includes human reviewer preference, implementation success, and transfer beyond the studied ML conference setting.Source: §18 · p. 40

Appendix

• The appendix reproduces real artifacts: an end-to-end generated card and JSON evidence trail, a reject-path run, and a pattern/sub-pattern card.The artifacts are reproduced verbatim from §14.Source: Appendix · p. 41

A End-to-end Generated Idea Card

• IdeaSpark’s rank-1 card is a verbatim standard-register output for a held-out ICLR 2026 seed on reinforcement learning for long-context reasoning.Its JSON evidence trail, including the core claim, falsification, and literature deltas, is in Appendix B.Source: A End-to-end Generated Idea Card · p. 41
• The card targets diluted learning signals in long-context RL, where GRPO spreads one answer-level score across long chains and prior methods do not assign credit by phase.It argues that decisive steps remain a small fixed portion as context grows, while LoongRL changes training data, KRPO adjusts answer-level scores, GDRO selects problems, and CoVRL changes reward sources.Source: A End-to-end Generated Idea Card · p. 41
• The proposed evaluation compares the phase-stratified method with a same-data, same-update-step GRPO baseline across context lengths and reports accuracy and decisive-phase credit concentration.The plan tests at 16,000, 32,000, 64,000, and 128,000 tokens on RULER and an extended multi-hop QA suite, and averages C, the fraction of credit magnitude on decisive phases.Source: A End-to-end Generated Idea Card · p. 41
• A rule-based splitter assigns answer tokens to plan, retrieve, reason, and recheck phases, falling back to fixed 512-token chunks when markers are absent.The phases follow the model’s observable structure rather than a learned scorer.Source: A End-to-end Generated Idea Card · p. 41
• The card derives phase credit-density from correct-versus-incorrect rollout contrasts and within-answer confidence, without an external judge or extra forward pass.It pairs correct and incorrect attempts with the longest shared phase-type prefix, marks their first divergence +1 and −1, and blends those marks with standardized phase confidence using weights β1 = 1.0 and β2 = 0.3 before softmax.Source: A End-to-end Generated Idea Card · pp. 41–42
• Phase-Stratified GRPO reallocates each rollout’s advantage across phases while preserving its original average credit.Each token in a phase receives the rollout advantage scaled by the number of phases and that phase’s credit-density; the rest of the GRPO update remains unchanged.Source: A End-to-end Generated Idea Card · p. 42

B Kill-switch and Falsification Fields

• The proposed method redistributes GRPO’s single trajectory-level advantage into verifier-free, in-group-harvested per-phase credit density to recover long-context signal diluted by uniform broadcast.The claim targets context growth to 128K without adding a reward model or critic.Source: B Kill-switch and Falsification Fields · p. 42
• The falsification experiment compares vanilla and phase-stratified GRPO at 16K, 32K, 64K, and 128K, predicting a widening accuracy gap as context length grows.It uses two checkpoints from the same base model, identical 16K KeyChain data and equal compute, evaluated on RULER and extended multi-hop QA.Source: B Kill-switch and Falsification Fields · pp. 42–43
• The falsification fields name concentration ratio C as load-bearing, predict failure when pivotal-phase labels are randomly permuted at fixed C, and require byte-identical kill-switch preservation across phases.A stripped model retaining only the minimal-pair contrast term is the positive control and should recover most of the 128K gain.Source: B Kill-switch and Falsification Fields · pp. 42–43

C Reject-path Output (Phase 3 Abandon)

• The Phase 3 audit abandoned a candidate for a non-asymptotic certificate for Sticky Track-and-Stop after detecting an exact-mechanism prior-art collision.The example run’s seed direction was essentially the title of the existing 2025 paper, so the pipeline emitted a failure artifact rather than an idea.Source: C Reject-path Output (Phase 3 Abandon) · p. 43

1. paper pointed threat — exact-mechanism collision (hard floor, unaddressable)

• The surfaced threat, Poiani, Bernasconi & Celli’s 2025 arXiv:2505.22475, already provides non-asymptotic guarantees for Track-and-Stop and Sticky Track-and-Stop.Its stated gap and deliverable match the candidate’s headline claim.Source: §1 · p. 43
• The proposed B(mu) burn-in functional is an internal proof quantity, not an external differentiator, and the collision cannot be repaired by editing a field.Any non-asymptotic bound for sticky Track-and-Stop must control the same tracking-error or burn-in term.Source: §1 · p. 43

2. gap closure reject check — triggered

• The triggered C19 and C29 checks flag a mechanical lift of a known argument and a near-identical algorithmic change that re-derives an already-certified result.The candidate holds the sticky Track-and-Stop algorithm fixed.Source: §2 · p. 43

Triggering checks

• The root cause is prior-work overlap: the seed nearly repeats the existing paper’s title, while the recipe was applied and no anti-pattern match was found.The named B(mu) residual was an internal lemma object rather than a differentiating external claim.Source: §2 · pp. 43–44

• Non-asymptotic analysis of sticky Track-and-Stop is already occupied by the 2025 paper arXiv:2505.22475, so that headline should be dropped or replaced.Source: User-side options · p. 44
• Potential residuals include finite-confidence costs of sticky forced exploration, instance-dependent lower bounds on burn-in, or structured, non-exponential-family, or contextual settings.The prior analysis supplies an upper bound on burn-in, not the proposed lower bound; each pivot needs fresh Phase 0 retrieval to confirm it is unoccupied.Source: User-side options · p. 44
• Re-run Phases 2.1 and 2.2 on a different Phase 1 gap with the threat paper in scope, then critique the result again.Four gaps were found in phase1 output.json.Source: User-side options · p. 44

Artifacts on disk

• The phase0 directory contains the literature table, results, and full-text cache artifacts.The listed files are lit table.md, lit results.json, and fulltext cache.json.Source: Artifacts on disk · p. 44
• Phase2 produced a selection output and a generated candidate that passed the citation gate.The files are phase2 select/phase2 select output.json and phase2 generate/phase2 generate output.json.Source: Artifacts on disk · p. 44
• The phase3 collision check recorded 173 hits and identified arXiv:2505.22475 as critical.The collision results are stored in phase3 collision/collision hits.json.Source: Artifacts on disk · p. 44
• The phase3 critique returned an abandon verdict at the hard floor, and no phase4 or evaluation output was created.The critique output is phase3 critique/phase3 critique output.json; evaluation/opus4 8/85 was not created because of the abandon path.Source: Artifacts on disk · p. 44

Audit and Pivot an Assumption

• A pivot must expose a load-bearing assumption, then weaken it or violate it to re-derive a guarantee or demonstrate behavior the assumption had ruled out.Removing a non-binding assumption or cosmetically reframing the prior result yields only an incremental generalization.Source: D Exemplar Ideation Pattern and… · pp. 44–45
• A pivot is unconvincing if it swaps in an equally strong or unverifiable assumption, bundles components without isolating the key move, or fails to distinguish a near-identical precedent.Rejected attempts also lose credibility when they advertise theoretical machinery that is not used in the derivation.Source: D Exemplar Ideation Pattern and… · pp. 45–47
• Accepted executions validate the pivot in a real setting, certify relaxed guarantees with tightness evidence, and show prior methods fail because of the audited assumption; rejected ones often omit these steps.The Oral-versus-Reject contrast is based on observable tendencies, not a strict law; the Oral-versus-HC comparison is also qualified by a moderate HC sample.Source: D Exemplar Ideation Pattern and… · pp. 45–46

Representational-locus sub-pattern

• The tactical sub-pattern relocates enforcement or signal from a conventional locus—such as inputs, parameters, or scalar scores—to overlooked representations such as labels, activations, distributions, or behavior.It calls for a structural account of why the new locus carries the signal, not just evidence that the relocation works.Source: D Exemplar Ideation Pattern and… · p. 48
• A relocated attack or defense must be validated against realistic access constraints or an informed adaptive adversary, and its novelty must be distinguished from prior uses of that locus.A successful demonstration against only a fixed baseline leaves robustness, stealth, or evasion claims unverified.Source: D Exemplar Ideation Pattern and… · pp. 48–49
• Examples include moving triggers to supervision or label space, using transformation-resilient intermediate-representation geometry, and treating distribution shape rather than a scalar summary as the signal.The card also highlights structural root-cause analysis as a way to turn observed failures into systematic, predictive attack design.Source: D Exemplar Ideation Pattern and… · p. 49

Reject lessons

• Relocating to loci already used in published work can collapse novelty unless differences are made explicit and measured.Examples include normalization statistics, activation-similarity losses, and semantic concept triggers.Source: D Exemplar Ideation Pattern and… · p. 49
• A defense relocation fails to remove a strong assumption if it still depends on a clean reference model or known anomaly distribution.Source: D Exemplar Ideation Pattern and… · p. 49
• Showing a new locus only against fixed, non-adaptive, or outdated baselines, without explaining why it works, can make the approach appear to stack existing components.Source: D Exemplar Ideation Pattern and… · p. 49
• Integrating off-the-shelf generators or tools into a new attack setting without a structural reason for the locus change is judged as engineering integration rather than insight.Source: D Exemplar Ideation Pattern and… · p. 49