Source-linked AI summary
Intern-S2-Preview: Scientific Agentic Foundation Model
Lei Bai, Jiaqi Cao, Chiyu Chen, Guanzhou Chen, Kai Chen, Guangran Cheng, Erfei Cui, Xuanlang Dai, Shengyuan Ding, Shangheng Du, Yanhui Duan, Yue Fan, Youqing Fang, Quan Gan, Yuanyuan Gao, Jiaye Ge, Lixin Gu, Yuzhe Gu, Qipeng Guo, Junjun He, Xin Hong, Ming Hu, Zhouqi Hua, Haian Huang, Junhao Huang, Zixian Huang, Minxi Jin, Lingkai Kong, Alexander Lam, Zehao Li, Zonglin Li, Tianhao Liang, Dahua Lin, Junyao Lin, Tianyang Lin, Zhouhan Lin, Jiangning Liu, Jin Liu, Kuikun Liu, Wenran Liu, Yifei Liu, Yuhong Liu, Yuhong Liu, Zhoumianze Liu, Ziyan Liu, Ziyu Liu, Haijun Lv, Han Lv, Chengqi Lyu, Le Ma, Ningsheng Ma, Zerun Ma, Haoyang Peng, Runyu Peng, Jifei Shan, Zixin Shang, Kou Shi, Xiang Shi, Qisheng Su, Xuerui Su, Hao Sun, Xiao Sun, Yanan Sun, Yu Sun, Huanze Tang, Yinghao Tang, Wenhui Tian, Zhongbo Tian, Bingli Wang, Haomin Wang, Jiarui Wang, Jingzhi Wang, Rui Wang, Xiquan Wang, Yi Wang, Zhecan Wang, Ziyi Wang, Zun Wang, Rubin Wei, Lianyi Wu, Wen Wu, Yue Wu, Yuhan Wu, Zhenyu Wu, Zijian Wu, Shuhao Xing, Jun Xu, Xingle Xu, Xuenan Xu, Xiangchao Yan, Ziang Yan, Bowen Yang, Danni Yang, Lin Yang, Zhiqi Yang, Qian Yao, Haochen Ye, Peng Ye, Jinhui Yin, Jiashuo Yu, Dingbo Yuan, Fei Yuan, Yuhang Zang, Bo Zhang, Chao Zhang, Chen Zhang, Hongjie Zhang, Junming Zhang, Wenlong Zhang, Wenwei Zhang, Yiming Zhang, Zhuo Zhang, Ziyang Zhang, Haiteng Zhao, Penghao Zhao, Yibo Zhao, Zhonghan Zhao, Zhihang Zhong, Bowen Zhou, Peiheng Zhou, Xin Zhou, Xinyu Zhou, Yunhua Zhou, Dongsheng Zhu, Yicheng Zou
TL;DR
Scientific discovery requires models that reason over heterogeneous evidence, use tools, and sustain adaptive progress beyond isolated question answering. Intern-S2-Preview combines multimodal scientific pretraining with unified supervised, reinforcement-learning, and agentic post-training, alongside forecasting and specialization pathways. Across scientific, multimodal, agentic, general-purpose, and time-series evaluations, Intern-S2-Preview-397B achieves competitive or leading performance in multiple settings.
Problem
Scientific discovery demands sustained reasoning and adaptive planning over heterogeneous evidence, repeated tool interaction, and long-horizon workflows rather than isolated question answering.
Method
Intern-S2-Preview combines scientific multimodal pretraining with supervised fine-tuning, scalable multitask reinforcement learning, agentic reinforcement learning, and on-policy distillation, supported by interleaved document data and rollout-efficiency techniques.
Results
Across scientific, multimodal, agentic, general-purpose, and time-series evaluations, Intern-S2-Preview-397B achieves competitive or leading results in multiple settings, including strong scientific and open-model benchmark performance.
Takeaways & Limitations
The evaluations support assessing scientific capability through executable, verifiable, and iterative workflows in addition to isolated benchmark-answer accuracy.
Abstract
from arXiv · showhide
Scientific discovery increasingly requires AI systems that can reason over scientific evidence of heterogeneous modalities, interact with scientific tools and environments, and sustain progress across long task horizons. We present Intern-S2-Preview, a series of scientific agentic foundation models designed to support multimodal scientific understanding, reasoning, generation, and long-horizon tasks. The training pipeline begins with scientific multimodal pre-training over rendered scientific documents, interleaved image-text data, and diverse scientific corpora. Starting from the pretrained checkpoint, we apply a unified post-training pipeline consisting of supervised fine-tuning, scalable multi-task reinforcement learning (RL), black- and white-box agentic RL, and on-policy distillation. This pipeline is supported by practical techniques that improve rollout and training stability and efficiency, including partial rollout with off-policy correction, adaptive length regularization, online speculative decoding, robust multi-task optimization, and trace-aware experience assembly for agentic tasks. At the architecture level, Intern-S2-Preview-397B extends time series modelling from efficient long-sequence understanding to numerical forecasting, while Memory Decoder is studied as a separate memory-augmented path for rapid scientific specialization without modifying the frozen 397B backbone. Evaluations across scientific, multimodal, agentic, and general-purpose benchmarks show that Intern-S2-Preview-397B achieves competitive or leading results in multiple settings. The time series modules improve scientific signal understanding and forecasting on SciTS, while the separate Intern-MemDec-4B extension improves the Biology-Instructions average score from 56.92 to 60.32 without modifying the frozen 397B backbone.
1. Introduction
Intern-S2-Preview targets scientific workflows that require heterogeneous evidence, specialized reasoning, tool interaction, and sustained planning rather than isolated question answering. It combines scientific multimodal pre-training with staged post-training and evaluates the resulting model across scientific, multimodal, agentic, general-purpose, and time-series tasks.
- Scientific discovery requires sustained reasoning and adaptive planning over heterogeneous evidence, with repeated interaction with tools and external environments.
- General-purpose and scientific multimodal models remain incomplete because they lack specialization for heterogeneous scientific modalities, domain protocols, verifiable tool interaction, or long-horizon agency.
- The training pipeline combines scientific multimodal pre-training, supervised fine-tuning, scalable multi-task reinforcement learning, black- and white-box agentic RL, and on-policy distillation.
- Intern-S2-Preview-397B is evaluated on scientific, multimodal, agentic, general-purpose, and time-series benchmarks, covering both static problem solving and workflow-oriented tasks.
- The evaluations indicate broad capability coverage, including competitive or leading results across multiple scientific benchmarks and measurable gains in time-series understanding and forecasting.
2. Architecture
The architecture extends scientific modeling in two directions: numerical time-series understanding and forecasting, plus modular domain specialization through external memories attached to a frozen backbone. It also preserves document structure through matched text-visual pre-training and interleaved layout-aware data.
- Memory Decoder: Memory Decoder attaches independently trained parametric memories to the frozen Intern-S2-Preview-397B backbone for domain specialization without modifying core parameters.
- Memory Decoder: A token-level router dynamically fuses next-token distributions from the frozen backbone and memory decoder using hidden-state and uncertainty features.
- Memory Decoder: The memory objective combines retrieval distillation with gold-answer supervision, balancing the retrieval teacher and SFT answer through β.
- Time series: The upgraded time-series encoder raises maximum supported input length from approximately 240,000 to 300,000 time steps, runs approximately 5∼6× faster at maximum length, and uses around 20% of prior GPU memory.
- Multimodal pre-training: Matched text-visual pre-training and layout-aware interleaved PDF sequences preserve relationships among figures, equations, tables, surrounding text, and page structure.
- Time series: A dedicated forecasting branch uses multimodal and temporal representations to condition a causal Transformer forecaster and predict flexible future horizons.
3. Pre-training
Intern-S2-Preview strengthens scientific multimodal pre-training by preserving document structure, learning from rendered pages, constructing interleaved PDF sequences, and retrieving higher-quality visual data.
- Visual Pre-training: Visual Pre-training learns directly from rendered scientific pages, preserving figures, tables, equations, and layout information that text extraction may lose.It complements conventional text pre-training on the same document corpus.
- Visual Pre-training: The visual sequence is formed by masking blank regions, raster-scanning retained visual features, projecting them into the LLM hidden space, and modeling them autoregressively.The resulting sequence has length L≤N after foreground filtering.
- Visual Pre-training: Visual Pre-training uses a contrastive next-latent prediction objective with in-batch negatives, jointly interleaved with text samples during continued pre-training.The visual encoder remains frozen while the LLM backbone, visual projection, and prediction head are optimized.
- Interleaved Text-Image Data: Interleaved PDF data links images, equations, and tables to surrounding text and layout, enabling learning of their roles in document narratives and reasoning.Pages are parsed, visual units are cropped, and sequences are organized according to layout reading order before visual-gain filtering.
- Image Retrieval Enhancement: The image retrieval pipeline builds a vector database, supports text-to-image and image-to-image retrieval, and post-processes recalled results for quality.Its stated purpose is to recall high-quality visual data and increase its training sample ratio.
4. Post-Training
Intern-S2-Preview uses a staged post-training framework that combines supervised fine-tuning, multitask reinforcement learning, agentic RL, and on-policy distillation to develop scientific and interactive capabilities. Supporting techniques address rollout efficiency, reasoning length, heterogeneous-task optimization, and long-horizon agent training.
- Pipeline: The post-training pipeline begins with supervised fine-tuning and continues through multitask RL, black-box agentic RL, and on-policy distillation.SFT establishes reasoning, response-format, scientific-generation, and tool-use behaviors before reinforcement learning and policy consolidation.
- Scalable RL: Reinforcement learning improves correctness, reasoning depth, scientific generation, and response efficiency under verifiable objectives across heterogeneous tasks.The pipeline uses partial-rollout correction, adaptive length regularization, speculative decoding, and robust multitask optimization to address scaling challenges.
- Scalable RL: Adaptive length regularization produces comparable reward curves while substantially reducing average output length, improving reasoning efficiency without sacrificing model performance.The method reweights advantages of positive responses without adding an independent reward signal.
- Scalable RL: Online speculative decoding improves rollout acceptance during training, delivering approximately 2× faster rollout generation and a 1.7× end-to-end RL speedup.Online draft adaptation and a hybrid LK objective maintain or improve acceptance as the policy evolves.
- Agentic RL: The harness × task framework converts tool-using agent executions into common RL experiences with explicit environments, observable action–observation histories, and verifier-defined outcomes.A skill-state graph composes compatible capabilities into sequences with varied horizons and complexity.
- Agentic RL: A shared training and experience-assembly path improves or recovers reward trajectories across software-engineering, general-purpose, and terminal tasks despite harness-dependent dynamics.The authors interpret the shared upward trend as evidence that the common trace, credit-assignment, and optimization stack works across harness–task compositions, not as evidence that reward scales are directly comparable.
- Policy consolidation: On-policy distillation consolidates scientific reasoning and long-horizon interaction capabilities from separately optimized expert policies into one unified model.Shared initialization and sampled-token supervision support stable optimization over long trajectories while avoiding full-vocabulary or top-k distillation overhead.
5. Evaluation
Intern-S2-Preview-397B is evaluated across scientific, multimodal, agentic, time-series, and general-purpose benchmarks, showing leading or competitive performance across these settings.
- Scientific benchmarks: Intern-S2-Preview-397B outperforms strong open- and closed-source models on Biology-Instructions (56.92), Mol-Instructions (52.37), and SciReasoner (63.97).
- Scientific benchmarks: The model achieves the best open-source results on MolecularIQ (61.49), TOMG-Bench (65.66), XLRS-Bench (51.97), and MicroVQA (68.81).
- Agentic benchmarks: On science-oriented agentic tasks, Intern-S2-Preview-397B generally surpasses DeepSeek-V4-Pro and Qwen3.5-397B, ranking second only to GLM-5.2.
- General-purpose benchmarks: On general-purpose benchmarks, it leads open-source models on MMLU-Pro (89.75), SimpleQA-Verified (69.90), MMMU-Pro (80.46), and ChartQAPro (69.65).
- Agentic benchmarks: Intern-S2-Preview-397B outperforms Qwen3.5-397B on general-purpose agentic tasks and performs comparably to Kimi-K2.7-Code.
BIOLOGY-INSTRUCTIONS
The time-series evaluation extends beyond understanding to forecasting, while the cross-domain profile compares the base model with the separate Memory Decoder extension across task families.
- Cross-domain capability profile: Figure 12 compares Biology-Instructions task results, category averages, and seven-benchmark capability profiles for Intern-S1-Pro, Intern-S2-Preview-397B, and the Memory Decoder extension.
- Time-series understanding: Intern-S2-Preview-397B achieves comparable or better time-series understanding than the trillion-parameter Intern-S1-Pro using less than half as many parameters.
- Time-series understanding: The model surpasses Intern-S1-Pro on seven of nine shared SciTS tasks, with PHU01 F1 rising from 36.8 to 66.9.
- Time-series understanding: The upgraded time-series module additionally supports radar coding-scheme and mode-and-modulation classification, which Intern-S1-Pro did not support.
- Time-series generation: Table 5 reports SciTS forecasting using MAPE and success rate, where lower MAPE and higher success rate indicate better performance.
6. Conclusion
Intern-S2-Preview-397B is presented as a broad scientific agentic foundation model spanning multimodal understanding, reasoning, generation, tool interaction, and iterative execution. Its evaluations support the design choices, while Memory Decoder enables specialization on the frozen backbone; reliability and tool integration remain future priorities.
- Intern-S2-Preview-397B covers multimodal understanding, domain-specific reasoning, scientific generation, tool interaction, and iterative execution.
- Evaluations across scientific, multimodal, agentic, general-purpose, and time-series settings support the model’s architecture, pretraining, and post-training design choices.
- Agentic evaluations indicate that scientific capability should include executable, verifiable, and iterative workflows rather than isolated benchmark-answer accuracy alone.
- Memory Decoder adds targeted scientific specialization to the frozen backbone while preserving Intern-S2-Preview-397B as the general model.
- Future work targets longer-workflow reliability, broader domain-specific memories and environments, stronger verifiers, and deeper integration with specialized scientific tools.
Author Contributions
The supplied author information states that authors are listed alphabetically by last name.
- Authors are listed in alphabetical order by their last names.