PaperlayerView exact revision on arXiv

Source-linked AI summary

Intern-Atlas: A Methodological Evolution Graph as Research Infrastructure for AI Scientists

Yujun Wu, Dongxu Zhang, Xinchen Li, Jinhang Xu, Yiling Duan, Yumou Liu, Jiabao Pan, Qiyuan Zhu, Xuanhe Zhou, Jingxuan Wei, Siyuan Li, Jintao Chen, Conghui He, Cheng Tan

arXiv:2604.28158v2cs.AI

TL;DR

Existing scholarly infrastructure links papers through citations but does not explicitly represent how methods emerge, adapt, and build upon one another, limiting AI research agents’ access to methodological evolution. Intern-Atlas builds a typed, evidence-grounded methodological evolution graph with temporal lineage search, and shows stronger lineage recovery than baselines alongside utility for idea evaluation and generation.

  • Problem

    Existing scholarly platforms represent papers and citations without semantic method-level relationships describing methodological evolution, limiting AI agents’ ability to reconstruct evolution topologies.

  • Method

    Intern-Atlas constructs a typed heterogeneous graph of methods and causal citation relations grounded in direct evidence, then uses temporal tree search to reconstruct evolution chains.

  • Results

    Intern-Atlas more faithfully recovers expert-curated evolution chains than graph-search baselines and supports idea evaluation and generation across complementary evaluations.

  • Takeaways & Limitations

    Methodological evolution graphs can serve as a foundational data layer for automated scientific discovery.

  • Takeaways & Limitations

    Edge-type classification is imperfect, with production accuracy ranging from 70.4% to 93.0%, and the fixed taxonomy maps emerging dimensions to existing axes.

Abstract

from arXiv · show

Existing research infrastructure is fundamentally document-centric, providing citation links between papers but lacking explicit representations of methodological evolution. In particular, it does not capture the structured relationships that explain how and why research methods emerge, adapt, and build upon one another. With the rise of AI-driven research agents as a new class of consumers of scientific knowledge, this limitation becomes increasingly consequential, as such agents cannot reliably reconstruct method evolution topologies from unstructured text. We introduce Intern-Atlas, a methodological evolution graph that automatically identifies method-level entities, infers lineage relationships among methodologies, and captures the bottlenecks that drive transitions between successive innovations. Built from 1,030,314 papers spanning AI conferences, journals, and arXiv preprints, the resulting graph comprises 9,410,201 semantically typed edges, each grounded in verbatim source evidence, forming a queryable causal network of methodological development. To operationalize this structure, we further propose a self-guided temporal tree search algorithm for constructing evolution chains that trace the progression of methods over time. We evaluate the quality of the resulting graph against expert-curated ground-truth evolution chains and observe strong alignment. In addition, we demonstrate that Intern-Atlas enables downstream applications in idea evaluation and automated idea generation. We position methodological evolution graphs as a foundational data layer for the emerging automated scientific discovery.

1 Introduction

Intern-Atlas addresses the limitations of document-centric scientific infrastructure for AI research agents by explicitly representing methodological evolution as a queryable causal graph. It introduces graph construction and search methods designed to support evolution-chain reconstruction, idea evaluation, and idea generation.

  • Motivation: AI-driven research agents cannot reliably reconstruct method-evolution topologies from unstructured text, and their parametric memory underrepresents low-frequency methodological knowledge.These agents automate scientific workflows from hypothesis generation through experimentation, creating a need for machine-consumable structure.
  • Motivation: Structured knowledge infrastructure becomes especially valuable when automated systems emerge that cannot operate effectively over unstructured data.The introduction relates this pattern to the Protein Data Bank and ImageNet, which organized scientific or visual knowledge before later machine-learning advances increased their value.
  • Contribution: Intern-Atlas extracts method entities, assigns semantic types to citation edges, and grounds non-background edges in verbatim evidence with bottleneck and mechanism annotations.The resulting representation is a queryable causal topology of methodological evolution rather than a paper-level citation network.
  • Method: Methodological progress branches, so SGT-MCTS balances exploitation and exploration while enforcing temporal coherence to construct more faithful evolution trajectories.The introduction illustrates branching with Transformer-derived methods including BERT, GPT, Vision Transformer, and DETR.
  • Evaluation: The evaluation measures graph quality against expert-curated evolution chains and tests graph utility for idea evaluation and idea generation.Idea evaluation examines monotonic alignment with human tiers, while idea generation compares outputs with and without graph-derived evolutionary context.

2 Related Work

Prior work traces research evolution through citation or keyword networks, evaluates ideas with noisy human and LLM judgments, and advances toward increasingly autonomous research systems. Intern-Atlas-related work instead emphasizes semantically causal methodological structure and computable topological signals.

  • Research evolution tracing: Main Path Analysis and CiteSpace reconstruct evolution using citation routes, keyword bursts, or co-citation clusters rather than semantically causal methodological relationships.Both treat papers or keywords as atomic units and define edges through statistical co-occurrence.
  • Idea evaluation: Human idea evaluation suffers from low inter-annotator agreement and inconsistent peer reviews, while LLM novelty judgments can favor safe, highly connected concepts.The cited account attributes this LLM bias to reliance on parametric familiarity rather than structural gap reasoning.
  • Topological idea signals: HindSight, SciMuse, and IdeaBench are translated into graph weightings, disconnection ratios, and computable topological formula components.These mappings convert theoretical metrics into explicit topological signals.
  • Automated research systems: Automated research systems have progressed from template-driven pipelines toward multi-agent loops with dynamic problem-solving and integrated research-review-revision cycles.AI Scientist v1 relied heavily on hand-crafted templates, whereas AI Scientist v2 used agentic tree search and CycleResearcher closed the research-review-revision loop.

3 Method

Intern-Atlas constructs a method-centric heterogeneous graph from over one million AI papers by resolving method entities, typing citation relations, and attaching verbatim causal evidence. Its operators use localized graph contexts for lineage reconstruction, idea evaluation, and idea generation, with lineage search guided by evidence confidence and temporal plausibility.

  • Graph construction: Intern-Atlas represents papers, canonical methods, and out-of-corpus stubs in a heterogeneous graph whose edges encode methodological rather than merely bibliographic relationships.The graph uses node sets VP, VM, and VS, with methods expanded through entity resolution and aliases consolidated across names.
  • Evidence extraction: Each causal edge records quoted bottleneck, mechanism, and trade-off spans plus an LLM confidence, while bottlenecks receive one of 14 taxonomy categories.A deterministic post-checker validates the extracted evidence, and category grouping supports reasoning about unsolved problem types.
  • Graph construction: 1,030,314 AI papers from 1965–2025 are processed through entity resolution, seven-way edge typing, and structured evidence extraction.The seven relation labels are extends, improves, replaces, adapts, uses_component, compares, and background.
  • Operators over the graph: All operators localize a textual input by lookup-table matching and BM25 retrieval, then operate on the resulting papers and typed-edge context.Lineage reconstruction traverses strong-causal edges, idea evaluation scores method positions, and idea generation searches for structural gaps.
  • Lineage reconstruction: Lineage reconstruction uses a graph-aware MCTS search to produce publication-year-ordered evolution chains from exact seed matches.Its prior multiplies LLM edge confidence by a temporal-coherence function peaked at publication gaps of 1–3 years; chains are ranked using length, confidence, and rollout convergence.
  • Idea evaluation: Idea evaluation computes five dimension scores from deterministic graph statistics and aggregates them with nonlinear conjunctive penalties plus an optional one-sided LLM veto.The dimensions are Novelty, Feasibility, Significance, Validity, and Clarity; an anti-stacking statistic down-weights overstudied methods.

4 Experiment

Intern-Atlas is evaluated for graph fidelity, lineage recovery, and downstream usefulness in idea evaluation and generation. It matches survey-derived method structures closely, improves lineage search over baselines, and produces scores aligned with publication quality and expert judgments.

  • Evaluation setup: The benchmark contains 30 survey-derived evolution graphs with 2,268 nodes, 1,462 edges, and 133 evolution chains.These graphs provide references for evaluating graph construction and lineage reconstruction.
  • Lineage search: Against Beam@10, SGT-MCTS improves NR by 39.9 points, ER by 55.8 points, and CAS by 39.9 points.The comparison evaluates recovery of coherent method lineages against beam search and random walk baselines.
  • Graph construction quality: 91.0% NMR, 89.7% ERR, and 92.0% PSC show that Intern-Atlas recovers reference methods, directed evolution paths, and their semantics.NMR measures matched reference methods, ERR measures recoverable directed paths, and PSC measures semantic correctness.
  • Idea evaluation: Mean overall scores decrease from 8.48 for top-tier conference papers to 7.83 for core conferences, 6.85 for workshops, and 5.84 for rejected submissions.The same ordering holds across all five evaluation dimensions, with the largest gaps in Significance and Validity.
  • Idea evaluation: Intern-Atlas correlates more closely with expert ratings than a pure LLM-as-Judge baseline across all five evaluation dimensions.The human evaluation samples 100 idea profiles and uses ratings from 10 AI PhD researchers under the same rubric.
  • Idea generation: Intern-Atlas achieves the best Overall score at 7.20 and ranks first in Novelty at 6.37, Significance at 6.30, and Validity at 6.26.Method-evolution chains help identify recurring limitations and central research problems; Feasibility and Clarity scores are close across knowledge sources.

5 Conclusion … B Lineage Reconstruction (SGT-MCTS)

Intern-Atlas constructs a typed, evidence-grounded graph of methodological evolution from over one million AI papers and supports lineage reconstruction, evaluation, and generation. Its schema, extraction pipeline, and SGT-MCTS search make methodological trajectories queryable while preserving causal evidence and parallel branches.

  • 5 Conclusion: 1,030,314 papers yield 9,410,201 semantically typed edges with verbatim bottleneck-to-mechanism evidence, supporting SGT-MCTS lineage reconstruction and graph-grounded downstream applications.The graph promotes flat citations into a queryable causal network of methodological evolution.
  • A.1 Schema: The schema distinguishes paper nodes, canonical method entities, and metadata-only stub nodes, keeping historical lineages reachable during SGT-MCTS.Stub nodes represent cited papers outside the collection window.
  • A.1 Schema: The strong-causal subgraph uses extends, improves, replaces, and adapts for lineage traversal while excluding uses_component from methodological inheritance.Component reuse remains represented in the method-level DAG for evaluation and generation.
  • A.2 Corpus, PDF Pipeline, and Reference Resolution: The corpus spans AI conferences, journals, and preprint servers from 1965–2025, with full texts parsed into structured sections and references for downstream extraction.NOUGAT-V1.0 is primary, with GROBID-V0.8.0 used for 3.7% of degraded or scanned files.
  • A.2 Corpus, PDF Pipeline, and Reference Resolution: Method entities combine a hand-curated seed list of 247 methods with an LLM expansion pass requiring multi-paper co-occurrence and adjudication.The expansion scans full-text Method sections for proper-noun method names using capitalization and acronym heuristics.
  • A.3 Two-Phase Extraction Protocol: Extraction uses high-recall Phase 1 classification followed by Phase 2 structured record completion, with QWEN3.6-35B-A3B for production and CLAUDE-SONNECT-4.6 for audits.All calls use temperature 0 and deterministic decoding; Phase 2 records bottlenecks, mechanisms, impacts, tradeoffs, and confidence with verbatim quotations.
  • B Lineage Reconstruction (SGT-MCTS): SGT-MCTS searches the strong-causal graph bidirectionally using SGT-UCT selection, temporal filtering, confidence-weighted rollouts, and a depth cap of 5.It runs for B = 200 iterations per direction and seed, then reruns from branch points with masked covered edges and budget B/2 to surface parallel trajectories.

C Idea Evaluation · C.1 Common Retrieval Routine · C.2 Duplicate-Risk Detection

Intern-Atlas supports idea evaluation through a common context-retrieval routine and a separate novelty-only retrieval stack for duplicate-risk detection. The routine returns methods, papers, causal edges, and bottleneck records, while novelty detection uses staged dense-sparse retrieval, reranking, fusion, and threshold penalties.

  • C.1 Common Retrieval Routine: Context retrieval returns (M_x, P_x, E_x, B_x) for an idea or query.The tuple contains canonical methods, top-ranked papers, incident causal edges, and aggregated bottleneck records.
  • C.1 Common Retrieval Routine: Canonical methods M_x are resolved from the input through the alias registry A.M_x is a subset of the graph’s method vertices V_M.
  • C.1 Common Retrieval Routine: The routine retrieves K = 500 papers using a hybrid score combining alias counts over M_x with BM25 relevance on remaining content words.The selected papers form P_x, a subset of paper vertices V_P.
  • C.1 Common Retrieval Routine: Retrieved papers define E_x as non-background causal edges incident on P_x.These edges are selected from the graph edge set E.
  • C.1 Common Retrieval Routine: Bottleneck records B_x are aggregated from the retrieved causal edges through ρ(E_x).This completes the context tuple returned by the common retrieval routine.
  • C.2 Duplicate-Risk Detection: Novelty-only retrieval pools the top-20 candidates by fusing BGE-SMALL-EN-V1.5 dense cosine ranking with BM25-Okapi through Reciprocal Rank Fusion.The dense encoder uses 384-dimensional cosine similarity; BM25-Okapi uses k1 = 1.5 and b = 0.75, with k_RRF = 60.
  • C.2 Duplicate-Risk Detection: The novelty stack reranks pooled candidates with MS-MARCO-MINILM-L-12-V2 and computes fused(d, p) as an equal-weight dense-signal and cross-encoder fusion.The sparse signal contributes only during Stage 1 pooling, not the final fusion.
  • C.2 Duplicate-Risk Detection: Novelty applies penalties of −0.5, −1.5, −2.5, and −4.0 when s reaches 0.55, 0.65, 0.75, and 0.85, respectively.Here s = max_p fused(d, p), and the penalty is zero otherwise.

C.3 Per-Dimension Signal Specifications … D Evaluation Setup and Benchmarks

The scoring system combines five weighted dimensions with explicit signals for novelty, feasibility, significance, validity, and clarity, plus cross-dimensional adjustments. When no method is identified, it abstains from graph-grounded scoring because text-only signals show weak agreement with peer-review scores; optional adjudication can modify scores through bounded interventions.

  • C.3 Per-Dimension Signal Specifications: All five dimensions use base 5.0, with weights (0.20, 0.20, 0.25, 0.20, 0.15), additive contributions, and clipping to [10].The weights correspond to novelty, feasibility, significance, validity, and clarity.
  • C.3 Per-Dimension Signal Specifications: Novelty combines graph disconnection, mechanism-level Jaccard distance, a frontier-leaf bonus, and a duplicate-risk penalty.The component contributions reach +2.0, +1.5, and +0.8, while the duplicate-risk penalty ranges from −4.0 to 0.
  • C.3 Per-Dimension Signal Specifications: Feasibility uses a non-monotonic paper-count maturity curve capped at +3.5, with thresholds at 500 and 2000 papers.The thresholds are the 60th and 90th percentiles of paper counts in the released corpus; resource availability and method-set complexity provide auxiliary signals.
  • C.3 Per-Dimension Signal Specifications: Significance uses time-decayed in-degree, frontier presence, and a method-frontier regularizer that rewards niche popularity and penalizes saturation.The popularity term contributes +2.5 below 300 and decays linearly to −2.0 above 1000.
  • C.3 Per-Dimension Signal Specifications: Validity checks bottleneck grounding, ancestry consistency within depth 4, and type-weighted edge density, while clarity evaluates recognition, specificity, structural completeness, and length adequacy.Clarity favors method sets of size 2 or 3, penalizes sizes at least 6, and targets descriptions of 20–200 words.
  • C.4 Cross-Dimensional Regularizer Ωcross: Cross-dimensional priors apply additive adjustments when conjunctive conditions over post-red-flag scores hold, making one dimension more or less informative alongside another.These four empirical priors comprise Ωcross.
  • C.5 Graceful Fallback and Optional Adjudication: If no method is identified, the pipeline returns population prior 6.5 and abstains from graph-grounded scoring; text-only signals achieve Spearman ρ = 0.08 (p = 0.31) against peer-review scores.This fallback was evaluated on a 147-idea pilot from NeurIPS-2024 submissions.
  • C.5 Graceful Fallback and Optional Adjudication: The optional adjudicator uses two LLM calls to modify scores via bounds, restoring 0%, 60%, or 90% of the duplicate-risk penalty for duplicate, related, or unrelated candidates.The strongest verdict among the top-3 retrieved candidates determines the restoration rate.

D.1 Method-evaluation Benchmark and Metrics · D.2 Lineage-Search Baselines

The evaluation uses a manually audited benchmark derived from 30 high-impact surveys, with separate metrics for static graph coverage and lineage reconstruction. SGT-MCTS is compared with beam search and random walk under matched graph, seed, depth, and computational-budget conditions.

  • D.1 Method-evaluation Benchmark and Metrics: The benchmark draws on 30 high-impact survey papers selected for authority, impact, classic status, subfield diversity, and methodcentricity.The surveys serve as proxies for expert consensus on methodological evolution and prioritize recognized venues, citation, taxonomy use, diverse AI subfields, and methodcentric content.
  • D.1 Method-evaluation Benchmark and Metrics: An LLM extracts methods, directed evolution relations, and narration-based chains, after which domain researchers manually audit and verify retained relations.Auditors remove hallucinated methods, correct relation directions, merge duplicate names, and check relations against original survey text or figures.
  • D.1 Method-evaluation Benchmark and Metrics: Static graph coverage reports Node Match Ratio, Edge Reachable Ratio, and Path Semantic Correctness for matching methods, recovering directed paths, and preserving evolution semantics.Shortest-hop statistics are also reported, although the supplied passage does not specify their values or further details.
  • D.1 Method-evaluation Benchmark and Metrics: Lineage reconstruction reports Node Recall, Edge Recall, and Chain Alignment Score for recovering reference methods, adjacent transitions, and chain ordering.Each search method starts from the same seed and returns a candidate chain for every reference chain.
  • D.2 Lineage-Search Baselines: SGT-MCTS is compared with two standard graph-search baselines on the identical knowledge graph.The comparison evaluates lineage reconstruction capability against beam search and random walk.
  • D.2 Lineage-Search Baselines: All lineage-search algorithms use identical query seeds and the same maximum depth and computational budget.These controls ensure differences are assessed under matched search conditions.
  • D.2 Lineage-Search Baselines: Beam Search retains the top-k partial paths using local heuristics, with evaluated beam widths k ∈{1, 5, 10}.It provides a greedy semantic-search baseline lacking global rollout-based exploration and long-term planning.
  • D.2 Lineage-Search Baselines: Random Walk uniformly samples valid outgoing edges until the depth cap and repeats iterations to match SGT-MCTS’s total node-evaluation budget.It serves as a naïve exploration baseline for testing the need for heuristic guidance and value estimation; all baseline lineages use the same reference chains and metrics.

D.3 Dataset and Baselines Details for Evaluating Idea Evaluator

The idea evaluator is assessed on a 1,200-paper Strata Dataset spanning four publication strata and a 100-profile expert-rated subset. All methods use standardized Idea Profiles, with a pure Qwen3.5-397B-A17B zero-shot judge serving as the non-retrieval baseline.

  • Strata Dataset: 1,200 papers are evenly divided across four strata: top-tier conferences, core conferences, workshops, and rejected submissions.Each stratum contains 300 papers.
  • Human-Rated Subset: 100 idea profiles are rated by 10 AI PhD researchers on novelty, feasibility, significance, validity, and clarity using a 1-10 scale.Experts see only extracted idea profiles, without source papers, venue outcomes, or evaluator scores.
  • Idea Extraction Protocol: An LLM-based extractor converts each paper’s abstract, introduction, and core method sections into a standardized structured Idea Profile before scoring.The profile focuses evaluation on the research idea rather than writing style.
  • Baselines: Qwen3.5-397B-A17B provides a zero-shot pure LLM-as-Judge baseline using the same five-dimensional 1-10 rubric without retrieval or method-evolution graph context.The baseline relies strictly on internal parametric knowledge.

D.4 Experimental Details for Evaluating Idea Generation · E Case Studies

The idea-generation evaluation uses 100 structured research queries spanning six AI fields and assesses proposals through double-blind pairwise comparisons by active expert researchers. Each query specifies a domain setting, core challenge, and expected solution capabilities.

  • D.4 Experimental Details for Evaluating Idea Generation: 100 high-quality research queries were curated to standardize automated idea-generation evaluation.The full query list is provided in the supplementary material.
  • D.4 Experimental Details for Evaluating Idea Generation: Each query defines a domain setting, an existing-method bottleneck, and the target solution’s desired capabilities.The tripartite structure clarifies both evaluation constraints and objectives.
  • D.4 Experimental Details for Evaluating Idea Generation: 34% of queries concern NLP & LLMs, followed by General ML & Optimization at 19% and Computer Vision at 14%.The remaining fields are Reinforcement Learning & Agents (14%), Multimodal & Speech (13%), and Graph Learning (6%).
  • D.4 Experimental Details for Evaluating Idea Generation: The query distribution covers six major AI fields to evaluate generative generalizability across different topics.Fields include NLP & LLMs, General ML & Optimization, Computer Vision, Reinforcement Learning & Agents, Multimodal & Speech, and Graph Learning.
  • D.4 Experimental Details for Evaluating Idea Generation: Human experts were active researchers with publication records in top-tier AI venues, providing an expert-based assessment of idea quality.The panel was assembled to compute win rates and support a rigorous, unbiased evaluation.
  • D.4 Experimental Details for Evaluating Idea Generation: Experts used a strict double-blind pairwise comparison protocol to compare anonymized Intern-Atlas and baseline ideas for each query.They received the reference query alongside the anonymized proposals.

E.1 Case Study on Lineage Search

The ConvNeXt V2 case study shows that SGT-MCTS most faithfully reconstructs the ordered residual ConvNet lineage, while beam search and random walk recover incomplete subsequences. Its advantage comes from preserving multiple plausible ancestors in a high-branching architecture graph.

  • Search behavior: SGT-MCTS recovers the complete reference subsequence in the correct order, with extra Caffe and downstream-user nodes not disrupting the residual ConvNet backbone.The reference lineage proceeds through VGG, ResNet, ResNeXt, ConvNeXt, and ConvNeXt V2.
  • Search behavior: Beam search recovers the later ResNet-to-ConvNeXt V2 segment but misses VGG’s depth-scaling step and enters through an RNN/Highway-Network branch.This makes its lineage incomplete relative to the survey-level reference chain.
  • Search behavior: Random walk recovers only a shorter local subsequence around ResNet, ConvNeXt, and ConvNeXt V2, skipping both VGG and ResNeXt.It therefore captures less of the main residual ConvNet backbone than SGT-MCTS.
  • Interpretation: The case demonstrates the value of MCTS-style exploration because retaining multiple plausible ancestors preserves the ordered VGG–ResNet–ResNeXt–ConvNeXt backbone more faithfully than greedy and random baselines.All searches are seeded at ConvNeXt V2, and the graph has high branching in this architecture lineage.

F Limitations and Broader Impact

Intern-Atlas is limited by imperfect edge-type extraction, domain-specific temporal calibration, a fixed bottleneck taxonomy, deterministic evaluation tradeoffs, and incomplete coverage of rare generation modes. Its broader risks are indirect, including amplification of citation biases, with provenance evidence and a public reliability audit providing mitigation mechanisms.

  • Limitations: 70.4% (production model) to 93.0% (audit model) is the Phase-1 edge-type classification accuracy range, reflecting difficulty distinguishing fine-grained causal relations.Downstream operators use edge types for routing rather than as ground truth, so misclassifications affect ranking quality without breaking the operator interface.
  • Limitations: Temporal coherence TC is calibrated on post-2015 AI literature, so fields with different research cadences require recalibration.The evaluator uses zero trainable parameters to prioritize deterministic reproducibility and full auditability over possible learned-scorer accuracy gains.
  • Limitations: The four generation strategies cover frequent observed topological moves but omit less common modes such as theoretical unification.This limits coverage of rarer methodological transition patterns.
  • Broader impact: Downstream agents could amplify existing citation biases when allocating credit across methodological traditions.Mitigations include verbatim-grounded evidence on every non-background edge, enabling provenance audits, and public release of the reliability audit alongside the graph.
Loading 2604.28158v2…