Generative AI Act II: Test Time Scaling Drives Cognition Engineering

Stage 1: Pre-training Scaling - Formation of Knowledge Islands

• Pre-training scaling forms isolated knowledge islands of physics concepts, with limited connections that constrain complex reasoning across the knowledge graph.Increased model size and training data establish foundational knowledge, but primarily innate relationships between concepts.Source: Three Scaling Phases · p. 2

Stage 2: Post-training Scaling - Knowledge Densification

• Post-training scaling densifies physics knowledge through learned connections between related concepts, enabling richer associations but not comprehensive distant-node reasoning.Fine-tuning creates moderate connectivity between previously isolated concepts while remaining limited for multi-step inference across distant knowledge nodes.Source: Three Scaling Phases · p. 2

Stage 3: Test-time Scaling - Cognitive Pathway Formation

• Test-time scaling enables complex cross-domain reasoning by forming robust multi-hop pathways between previously disconnected physics concepts during inference.Extended computation explores deeper search spaces, connecting Universal Gravitation to applications such as Orbital Motion and Falling Objects.Source: Three Scaling Phases · pp. 2–3

Conclusion

• The progression reframes AI advancement as the development of cognitive capabilities that let models navigate physical principles through principled reasoning, rather than merely accumulating data or parameters.Test-time scaling marks the transition from knowledge repositories to systems capable of deep scientific insight through sustained deliberation.Source: Three Scaling Phases · p. 3

The Practitioner’s Roadmap: How to Apply Test-Time Scaling to your Applications?

• A workflow guides the application of test-time scaling in a specific domain.The workflow is summarized in Figure 2.Source: Figure 2 · p. 4
• The roadmap is intended to support domain-specific applications of test-time scaling.Its scope is applying test-time scaling within a particular domain.Source: Figure 2 · p. 4
• Further details about the application workflow are available in the main paper.Figure 2 directs readers to the main paper for additional information.Source: Figure 2 · p. 4

• Act I LLMs used scaled data, computation, and parameters to organize knowledge and support natural-language interaction, but suffered knowledge latency, shallow reasoning, and limited thought processes.These limitations kept them primarily focused on knowledge retrieval and simple reasoning rather than human-like intelligence.Source: §1 · p. 7
• Test-time scaling inaugurates generative AI’s second act, cognition engineering, by redefining what LLMs can achieve.Source: §1 · p. 7
• Cognition engineering systematically constructs AI thinking through test-time scaling, human cognitive-pattern distillation, and AI-driven discovery such as reinforcement learning.Source: §1 · p. 7
• The paradigm targets deep cognition—complex reasoning, deliberate thought, conceptual connection, novel insight, and metacognition—through intentional training and inference interventions.Its engineering emphasis includes reinforcement learning and extending inference-time computation rather than relying on emergent capability from scale alone.Source: §1 · p. 7
• Cognition engineering shifts inference from prompt-based knowledge retrieval toward deeper test-time searches and shifts training toward thinking abilities, enabling bidirectional exchange in which AI can discover effective strategies beyond human intuition.The paper cites AlphaGo’s “Move 37” as an example of AI-discovered cognition that may enrich human understanding.Source: §1 · p. 7
• The paper develops cognition engineering from its concepts and urgency through test-time scaling, training strategies, systemic changes, applications, implications, infrastructure, implementation tutorials, and future directions.It aims to provide researchers and practitioners with a framework for generative AI’s second act.Source: §1 · pp. 7–8

2.1 Cognition

• Cognition engineering advances AI from the knowledge level toward DIKW’s wisdom level, characterized by judgment, creativity, and metacognitive abilities.The DIKW framework moves from data and information through applicable knowledge to wisdom; first-generation LLMs primarily reached the knowledge level.Source: §2.1 · p. 8, §2 · p. 8
• Wisdom-level cognition combines deep, multi-path reasoning with metacognition, creative cross-domain connections, adaptability, and abstraction.It addresses not only knowing what, but also knowing why and knowing how (Von Eckardt, 1995; N´u˜nez et al., 2019).Source: §2.1 · p. 8

2.2 Engineering Methodology

• Engineering methodology intentionally guides systems from data toward wisdom through targeted training interventions and inference optimization rather than scaling alone.The approach treats system development as organized design, construction, and optimization for specific problems.Source: §2.2 · p. 8

2.3 Cognition Engineering

• Cognition Engineering systematically constructs and optimizes AI’s ascent from knowledge to wisdom using design patterns, training strategies, and computational allocations.Its goal is deep thinking, complex reasoning, and creative problem-solving with characteristics resembling human wisdom-level cognition.Source: §2.3 · p. 9
• Unlike traditional models that imitate human outputs, cognition engineering imitates human thought processes to address wisdom-level cognitive characteristics.Traditional approaches remain primarily at the DIKW knowledge level, whereas cognition engineering directly targets thought construction.Source: §2.3 · p. 9
• Cognition engineering shifts AI from static knowledge retrieval toward dynamic, complexity-adaptive thinking and the creation of new insights through deep reasoning.Systems can adjust thinking depth and resource allocation during inference instead of only retrieving and recombining existing knowledge.Source: §2.3 · p. 9

3.1 The Necessity of Cognition Engineering

• LLMs still struggle with multi-step reasoning, reliable proofs, scientific problem-solving, and multidimensional analysis beyond what pre-training scaling alone enables (Mirzadeh et al., 2024).These tasks require decomposing problems, exploring multiple reasoning paths, and conducting deep logical analysis.Source: §3.1 · p. 9

3.2 Three Pillars

• Cognition engineering emerges from three technological pillars that collectively enable progress from knowledge management toward deeper cognitive capabilities.The pillars are richer knowledge foundations, test-time scaling, and advanced self-training.Source: §3.2 · p. 9

3.2.1 Knowledge Foundation

• Modern foundation models provide richer cognitive raw material through large-scale, curated corpora integrating scientific literature, technical documentation, mathematics, code, and specialized knowledge (Shao et al., 2024; Zhou et al., 2024b; Wang et al., 2023d; Yang et al., 2024a).Llama 2’s training scale reached 2 trillion tokens.Source: §3.2.1 · p. 9, §3.3 · p. 10

3.2.2 Test-time Scaling Foundation

• Test-time scaling reallocates inference computation through CoT, tree search, self-correction, and verification, creating a cognitive workspace for deeper reasoning (Wei et al., 2022; Yao et al., 2023a; Hao et al., 2023; Feng et al., 2023).These methods let models articulate intermediate steps, explore multiple pathways, detect errors, and refine reasoning.Source: §3.2.2 · p. 10

3.2.3 Self-Training Foundation

• Reinforcement learning with verifiable rewards enables reflection, backtracking, verification, and dynamic computation allocation, while iterative training on reasoning trajectories supports continued improvement (DeepSeek-AI et al., 2025; Gandhi et al., 2025; Yu et al., 2025a).These self-training methods can reduce reliance on expert human cognition data and support AI self-discovery.Source: §3.2.3 · p. 10

3.3 From Theory to Practice: The Road Ahead

• Test-time scaling is the most immediate practical bridge from cognition engineering’s theoretical foundations to systems that allocate inference computation for deeper reasoning.The paper frames refining these methods as a route toward AI systems that think rather than merely predict.Source: §3.3 · p. 10

4 How – Part I: Test-Time Scaling Methods

• Test-time scaling uses a search strategy M to guide generator g toward an optimal response, with computation dimensions λ determining inference cost and performance scaling.The framework measures scaling efficiency as performance relative to a computation budget and improves it by optimizing individual methods or combining methods.Source: §4 · pp. 10–11

4.1.1 Key Components

• Parallel sampling independently generates N candidate responses for one query and selects an answer through Best-of-N scoring, majority voting, or a combined strategy.Figure 4 illustrates these three selection methods.Source: §4.1.1 · pp. 11–12, Figure 4 · p. 11
• Best-of-N selects the highest-scoring candidate using external verifiers, outcome or process reward models, specialized models, or the generator’s probability distribution.Code interpreters and math proof checkers can provide direct verification, while ORM scores whole responses and PRM scores individual steps.Source: §4.1.1 · pp. 11–12
• Majority voting selects the most frequent answer and is lightweight, but it requires answer-equivalence checking and is limited on open-ended tasks or large sample sets.Universal Self-Consistency (Chen et al., 2023b) uses LLMs to select the most consistent answer, but context limits remain challenging.Source: §4.1.1 · p. 12

4.1.2 Scaling Laws

• The sampling number N is the main scaling dimension: Pass@N usually grows steadily and often log-linearly, whereas practical metrics such as Maj@1 or BoN can diverge from it.Brown et al. (2024) report a significant disparity between Pass@N and practical metrics when using ArmoRM-Llama3-8B-v0.1 (Wang et al., 2024d) as the scorer.Source: §4.1.2 · pp. 12–13, Figure 5 · p. 12
• Imperfect verifiers create false positives, impose accuracy ceilings, and can make majority-voting or scoring performance saturate or degrade as sampling increases.Code can pass unit tests while failing additional cases, and mathematical solutions can reach correct answers through incorrect reasoning; Stroebl et al. (2024) link higher false-positive rates to lower Pass@1 accuracy.Source: §4.1.2 · pp. 12–13

4.1.3 Improving Scaling Efficiency

• Query-aware sampling allocates more samples to difficult queries, while early stopping halts generation when answers converge, confidence is high, or reward and consistency scores are low.Chen et al. (2024c), DSC (Wang et al., 2024f), Speculative Rejection (Sun et al., 2024a), and ST-BoN (Wang et al., 2025e) exemplify these approaches.Source: §4.1.3 · p. 14
• Under fixed computation, larger models with fewer samples outperform on code, whereas smaller models are better for mathematics but saturate earlier; selection precision also improves with PRMs, weighted voting, or multiple verifiers.Lightman et al. (2023) find PRM superior to ORM in BoN, Sun et al. (2024b) find weighted voting superior at large sample counts, and MAV performs better than a single verifier at high total budgets.Source: §4.1.3 · p. 14

4.2.1 Key Components

• Tree search frames problem solving as exploration over tree structures, using guided algorithms and value functions to improve deliberate planning.Its components are search space, value function, and search algorithm.Source: §4.2.1 · pp. 14–16
• Search granularity ranges from tokens to steps to whole solutions, trading candidate optimality and error sensitivity against computational efficiency.Token-level search suits code tasks where minor changes can cause errors, while step-level search is most common; solution-level expansion updates whole responses through critique and revision.Source: §4.2.1 · p. 15
• Tree-search value functions use self-evaluation, specialized models, action likelihoods, self-consistency, or rollouts to estimate nodes for pruning and exploitation.Specialized evaluators include process reward models trained from human labels, Monte Carlo sampling, or ORM-derived process annotations.Source: §4.2.1 · p. 15
• BFS, DFS, and MCTS define complementary tree operations, with MCTS balancing exploration and exploitation through selection, expansion, evaluation, and backup.BFS includes beam search and A*, whereas DFS backtracks from unpromising paths; MCTS updates node values and visit counts along traversed paths.Source: §4.2.1 · p. 16

4.3 Multi-turn Correction

• The multi-turn correction taxonomy organizes methods by fine-tuning requirements and by whether self-feedback or self-refinement shares parameters with the initial generator.Table 2 summarizes these distinctions across multi-turn correction methods.Source: Table 2 · p. 17

4.2.2 Scaling Laws

• Increasing MCTS rollouts, beam size, or A* step limits improves performance, but scaling eventually saturates as sampled nodes become insufficiently diverse.Larger process-reward models can also improve performance, emphasizing value-function reliability through additional training or test-time compute.Source: §4.2.2 · p. 17

4.2.3 Improving Scaling Efficiency

• Tree-search algorithm choice is task-dependent: PG-TD finds MCTS significantly better than beam search at equal computation time, while ToolChain∗ finds A* more time-efficient for API calls.These comparisons are reported for code generation and API call tasks, respectively.Source: §4.2.3 · p. 17
• Scaling efficiency can improve by reducing value-function rollout overhead, adapting expansion breadth to node value and depth, and merging semantically redundant nodes.ALPHALLM uses a smaller rollout model, rStar-Math replaces later rollouts with a trained value function, and LiteSearch, REBASE, FETCH, and ETS improve token efficiency.Source: §4.2.3 · p. 17

4.3.1 Key Components

• Multi-turn correction iteratively generates, critiques, and refines responses, using self-feedback, external feedback, or specialized models until a stopping condition is met.The feedback stage may be omitted for direct refinement, and Self-Refine uses one language model for generation, feedback, and refinement.Source: §4.3.1 · p. 18

4.3.2 Scaling Laws

• After correction training improves critique ability, performance grows with revision steps and eventually saturates, whereas unreliable intrinsic self-correction can degrade performance as revisions increase.Reliable external feedback enables gains, but models often produce unreliable critiques and revisions in planning and reasoning without it.Source: §4.3.2 · p. 18, §4.3.1 · p. 18

4.3.3 Improving Scaling Efficiency

• Scaling efficiency in multi-turn correction requires improving feedback quality or refinement ability rather than merely adding revision steps.The paper identifies these capabilities as the main constraints on effective scaling.Source: §4.3.3 · p. 18, §4.3.2 · p. 18

4.4 Long CoT

• Long CoT extends traditional shallow, linear reasoning with more sophisticated thinking patterns and longer responses, as adopted by models such as OpenAI o1 and DeepSeek R1.CoT improves representational complexity and reasoning performance, but long CoT targets limitations in complex cognitive capabilities.Source: §4.4.1 · p. 19

4.4.1 Key Components

• Long CoT supports reflection, backtracking, verification, divergent thinking, and internal thinking, enabling models to reassess errors, explore alternatives, and reason more finely.These mechanisms are especially relevant to long-horizon planning and robust problem solving.Source: §4.4.1 · p. 19, §4.4 · p. 20

4.4.2 Scaling Laws

• Reasoning performance generally increases with token count under log-linear or post-processed scaling, but shorter responses can outperform longer ones when extra reasoning becomes convoluted or reflects harder problems.Hou et al. (2025) truncate responses and summarize them, while Muennighoff et al. (2025) control length with budget forcing.Source: §4.4.2 · p. 20

4.4.3 Improving Scaling Efficiency

• Long CoT can overthink simple problems and allocate computation uniformly across tokens, motivating prompting, heuristic, and query-aware compression methods with task-dependent length targets.Heuristic compression can harm performance when based only on GPT-4-compressed data, whereas query-aware approaches adapt reasoning length to difficulty and support SFT or DPO.Source: §4.4.3 · p. 20, §4.5 · p. 21

4.5 Comparisons of Test-Time Scaling Methods

• Test-time scaling can be optimized through self-training, query routing, model merging, intermediate-state compression, or latent-space reasoning to reduce unnecessary tokens, memory, and computation.These methods include length-penalized RL, difficulty-based routing, combining long- and short-CoT models, compressing KV-cache states, and reasoning outside language space.Source: §4.5 · p. 21

4.5 Comparisons of Test-Time Scaling Methods

• Performance rankings depend on problem difficulty and computation budget: long CoT has the highest ceiling, beam search suits complex limited-budget questions, and BoN suits simpler questions with more computation.The passages caution that absolute rankings are difficult because implementations and comparison conditions vary.Source: §4.5 · p. 21
• External scaling dimensions provide coarse budget control, whereas standard long CoT is difficult to control without truncation; S1 and L1 add response-length control through delimiters and reinforcement learning.Truncating long CoT can produce incomplete responses that significantly harm performance, while L1 improves token efficiency relative to S1.Source: §4.5 · p. 22
• Long CoT outperforms other methods in ceiling performance, adaptivity, simplicity, and cognitive behaviors, but requires additional training to elicit.Its advantages include stronger performance on difficult problems, implicit resource allocation, unified reflection and backtracking, and fewer auxiliary components.Source: §4.5 · p. 22, §4.6 · p. 23

4.6 Ensemble of Test-Time Scaling Methods

• Ensemble methods combine multiple scaling approaches or select among them by context, leveraging complementary strengths rather than allocating all computation to one method.All test-time scaling methods are compatible, and parallel sampling is the easiest to integrate with others.Source: §4.6 · p. 23, Figure 9 · p. 23
• Combining parallel sampling with tree search or multi-turn correction improves search diversity, performance, or token efficiency by coupling global sampling with independent subtrees or local revision.TreeBON reduces BoN overhead by pruning low-quality responses, while mixed sampling-and-correction schemes can outperform using more sampling or correction alone.Source: §4.6 · p. 23
• Long-CoT ensembles can outperform majority voting and single high-cost responses when voting accounts for overthinking, while parallel sampling with shorter responses can be more compute-optimal than budget-forced long-CoT scaling.High overthinking correlates with lower performance in math tasks and agentic environments.Source: §4.6 · pp. 23–24
• Tree search benefits from integrating critique and revision as actions, while long CoT can internalize branching and self-correction, though defining its search space remains an open direction.Revision-aware tree search performs better than sequential revision, and o1-mini benefits from self-correction whereas traditional LLMs perform worse.Source: §4.6 · p. 24
• Adaptive selection can match methods to query difficulty: correction or BoN for easier queries, beam search for harder ones, and mixed sampling-correction strategies for difficult queries.Model size can also inform the choice of an optimal test-time scaling strategy.Source: §4.5 · p. 21, §4.6 · p. 24

5.1 Scaling Reinforcement Learning

• Online reinforcement learning with rule-based rewards enables models to learn long-CoT reasoning and exhibit self-reflection and self-correction, known as the RL scaling phenomenon or “Aha moment.”The section summarizes this research in Table 4 and provides training recipes in Table 6.Source: §5.1 · p. 24

5.1.1 Training Algorithm

• PPO has the highest computational cost, whereas GRPO and REINFORCE++ remove the critic model and are more stable than REINFORCE; reported performance comparisons vary by reward setting.PPO and GRPO perform similarly in RLHF settings, while PPO and REINFORCE++ outperform GRPO on synthetic logic puzzles with rule-based rewards; broader evaluation remains necessary.Source: §5.1.1 · p. 26, Table 5 · p. 27

5.1.2 Reward Function

• Model-based rewards generalize across tasks but can be reward-hacked, while process rewards offer finer credit assignment yet remain difficult to construct and less advantageous in RL than in parallel sampling.Proxy rewards can rise while golden rewards saturate and decline; process rewards can also reward unnecessary correct reasoning steps.Source: §5.1.2 · pp. 27, 29
• Rule-based rewards support RL scaling but can cause language misalignment and overthinking, motivating language-consistency rewards, long-CoT initialization, and response-length penalties.Deepseek-R1 uses long-CoT fine-tuning and language-consistency rewards, while Kimi k1.5 and T1 penalize excessive or repetitive responses.Source: §5.1.2 · p. 29

5.1.3 Policy Model Selection

• RL outcomes depend strongly on the policy model: pretrained cognitive behaviors improve scaling, larger models are more token-efficient, and smaller models may need long-CoT cold starts.Qwen-2.5 improves under RL where Llama-3.2-3B plateaus, while larger models show better scaling behavior and token efficiency.Source: §5.1.3 · p. 30

5.1.4 Training Data Construction

• 57K curated prompts produce continuous accuracy and response-length scaling, whereas a 7.5K MATH set plateaus; dynamic filtering also removes prompts that are always solved or unsolved.Difficulty control targets prompts that provide useful gradients while avoiding unnecessary RL computation.Source: §5.1.4 · p. 30

5.1.5 Multi-stage Training

• Multi-stage training improves RL efficiency through long-CoT cold starts, controlled context-length expansion, and curriculum sampling, although curriculum gains can be task-dependent.DeepScaleR expands context from 8K to 16K and then 24K, while logic-RL reports limited gains from curriculum sampling on logic puzzles.Source: §5.1.5 · p. 30

5.2 Supervised Fine-tuning

• Supervised fine-tuning can elicit long-CoT behavior with simpler, more data-efficient training, especially through distillation, but students generally struggle to surpass their teachers.Synthesis methods face limited task coverage, coherence, diversity, or critique ability, whereas distillation is cost-effective and performs well relative to synthesis.Source: §5.2 · p. 31, §5.3 · p. 32

5.3 Iterative Self-reinforced Learning

• LoRA fine-tuning performs close to full-parameter fine-tuning for learning self-correction, contradicting earlier findings from GPT2-small synthesized-data experiments that LoRA could not learn the pattern.The effectiveness of LoRA remains under exploration and requires further study.Source: §5.3 · p. 32
• Long-CoT fine-tuning benefits vary substantially by base model, with models of ≤3B parameters not consistently benefiting and often preferring shorter reasoning chains.The difference is attributed to limited domain knowledge, motivating quantitative analysis of base-model characteristics.Source: §5.3 · p. 32
• SFT-based approaches are cheaper and easier than RL but rely on long-CoT teacher models, may memorize fixed patterns, and require analysis of training steps versus generalization.The paper suggests combining SFT and RL for data efficiency; small-data SFT can nevertheless improve performance across subjects and domains (Ye et al., 2025).Source: §5.3 · p. 32
• ISRL forms a self-reinforcing cycle by generating trajectories, learning from them with offline SFT or DPO, and replacing the policy with its improved iteration.The framework is designed for self-improvement without requiring expert demonstrations.Source: §5.3 · p. 32
• ISRL samples responses through parallel sampling, tree search, or multi-turn correction, then scores and selects them using verification, reward models, self-judging, search scores, or majority voting.The selected data updates the policy with SFT or DPO and may update reward models, using current, original, or accumulated previous-turn data.Source: §5.3 · p. 33
• ISRL improvement often plateaus or slightly declines after a few iterations, unlike RL scaling’s monotonic gains, partly because ISRL is more off-policy and SFT omits negative gradients.More on-policy sampling and negative-gradient updates are associated with stronger later-stage or reasoning-task performance.Source: §5.3 · p. 34

6.1 Mathematics

• Long CoT, parallel sampling, tree search, and multi-turn correction advance mathematical reasoning, with DeepSeek-R1 reaching 79.8 on AIME and formal systems enabling verifiable search and correction.Natural-language reasoning benefits from easy answer verification, while formal systems such as Lean and Isabelle provide verifiable reasoning signals; AlphaProof and AlphaGeometry combine neural methods with proof checkers.Source: §6.1 · p. 35

6.2 Code

• o1 and o3 earned gold medals at the 2024 IOI, while execution verification, self-reflection, debugging, and test generation support stronger coding models such as those reaching state-of-the-art SWE-bench performance.Coding challenges include sandboxing security risks, overthinking, false positives from incomplete tests, and alignment with real-world tasks.Source: §6.2 · p. 36

6.3 Multimodality

• Multimodal test-time scaling has progressed from distilled long CoT, search, correction, and sampling to RL-based methods that deepen visual reasoning and broaden vision-centric tasks.Unlike LLMs, VLMs must integrate visual and textual inputs; current methods often miss visual rethinking and face weak base models requiring substantial data and computation.Source: §6.3 · p. 37, §6.4 · p. 38

6.4 Agent

• Multimodal generation applies long CoT, multi-turn correction, and inference-time tree search to improve image, video, and visual generation, but remains underexplored.Open problems include architectures, non-text pretraining and instruction tuning, computational efficiency, and reward signals and infrastructure for reinforcement learning.Source: §6.4 · p. 38
• 26.6% accuracy on Humanity’s Last Exam shows Deep Research’s optimized OpenAI o3 can synthesize online sources at research-analyst level, surpassing previous SOTA models.Agents require long-horizon planning and substantial test-time computation; ReAct and Reflexion improve per-step decisions through reasoning and feedback (Yao et al., 2023b; Shinn et al., 2023).Source: §6.4 · p. 38, §6.5 · p. 39

6.5 Embodied AI

• Embodied systems use internal thoughts, failure explanations, uncertainty alignment, visual grounding, and preference learning to improve planning and low-level action generation, but unified planning-execution remains unresolved.Existing approaches often separate high-level planning from low-level control, while pure RL for embodied self-reflection and continuous internal-feedback optimization remain open challenges.Source: §6.5 · p. 39, §6.6 · p. 40

6.6 Safety

• Test-time scaling has dual safety implications: stronger long-horizon reasoning may detect risks and vulnerabilities, but could also pursue value-misaligned strategies beyond human cognitive limits.The section frames safety applications around hallucination, jailbreaks, adversarial attacks, and related risks.Source: §6.6 · p. 41
• Increasing interaction and reasoning steps at inference time offers a scalable path to safer, more robust behavior without retraining.Methods include parallel sampling, tree search, multi-turn correction, multi-agent collaboration, and long chain-of-thought reasoning.Source: §6.6 · p. 41, §6.7 · p. 42

6.7 RAG

• RAG test-time scaling strengthens multi-hop reasoning through iterative search, interleaved reasoning, and learned search strategies, but current RL methods rely mainly on short-answer rule-based rewards.IterDRAG reports a near-linear relationship between performance and effective context length; SFT, preference fine-tuning, and RL have been used to train search-reasoning systems.Source: §6.7 · p. 42
• RAG research should develop long-form reward functions and benchmarks that separately evaluate internal reasoning and external search.Existing benchmarks do not disentangle these benefits, limiting assessment of test-time scaling in complex reasoning tasks.Source: §6.8 · p. 43

6.8 Evaluation

• Allocating more inference-time computation improves LLM-based evaluation through fine-grained judging, structured chain-of-thought, multi-agent feedback, parallel sampling, and tree search.MCTS-Judge accuracy consistently improves with greater search depth and rollouts, while more reasoning tokens benefit evaluators for long-CoT models.Source: §6.8 · p. 43, Figure 18 · p. 43
• Evaluation research must improve reliability for complex, long-horizon agentic tasks while balancing the benefits of test-time scaling against evaluation speed.Future directions also include integrating evaluation frameworks with reward design in reinforcement-learning training.Source: §6.8 · p. 43

7 So What? – From Scaling to Cognitive Intelligence

• The broader implication is a new cognitive ecosystem that transforms AI development, human-AI relationships, and scientific research beyond incremental improvements to conventional assistants.The paper frames this shift as expanding collective problem-solving capacity and enabling potentially unprecedented acceleration in knowledge creation.Source: §7 · p. 43, §7.3 · p. 45, §7.4 · p. 45

7.1 Data Engineering 2.0: Cognition Data Engineering

• Cognition engineering marks a paradigm shift from training on products of human thought to engineering thought processes as valuable data.This transition establishes cognitive data engineering as a new discipline centered on cognitive processes rather than knowledge acquisition alone.Source: §7.1 · p. 43, §7 · p. 43

7.2 Reward & Environment Engineering

• Cognitive data combines human cognition projections, AI-generated trajectories, and human-AI co-creation, potentially becoming as strategically valuable as large-scale computing resources.Sources include recorded expert artifacts, tool-mediated traces, frontier expertise extraction, reward-guided exploration, self-play, trajectory filtering, human seeding, and iterative refinement.Source: §7.2 · p. 44

7.2.1 Reward Models Design

• As tasks move from Olympiad proofs toward Deep Research (OpenAI, 2025b), PaperBench (Starace et al., 2025), scientific discovery, and literary creation, reward verification becomes increasingly difficult.Reference-based assessment and criterion-based assessment provide complementary ways to construct reward signals when simple correctness metrics are insufficient (Bhandari et al., 2020; Qin et al., 2023; Yuan et al., 2021; Zheng et al., 2025; Fu et al., 2024a; Yuan et al., 2024b).Source: §7.2.1 · p. 44, §7.3 · p. 45

7.2.2 Cognitive Environment Design

• Cognitive environments span text, code, browsers, computers, and physical-world interaction, with specialized simulators and adversarial frameworks forming curricula from foundational to frontier capabilities.Examples include scientific discovery, legal reasoning, medical diagnosis, structured debate, red-teaming, and Socratic dialogue.Source: §7.2.2 · p. 45

7.3 Human-AI Cognitive Partnership

• Human-AI cognitive partnership enables bidirectional exchange, with humans contributing strategies and values while AI systems reveal blind spots, expand strategy repertoires, and generate novel frameworks.AI also amplifies working memory, exploration breadth, and reasoning externalization, supporting problem solving beyond individual human cognitive capacity.Source: §7.3 · p. 45

7.4 Research Acceleration

• Cognition systems accelerate research by generating thousands of hypotheses, detecting anomalies, selecting efficient experiments, and connecting knowledge across disciplines, periods, and formats.These capabilities may remove bottlenecks in hypothesis generation, literature integration, experimental design, and theory refinement while broadening participation in scientific discovery (Lu et al., 2024a).Source: §7.4 · pp. 45–46

8 Infrastructure

• The infrastructure discussion also highlights acceleration of long text generation and directs readers to a comprehensive survey for broader coverage.This concern is presented alongside RL and MCTS as important to cognition engineering infrastructure.Source: §8 · p. 46

8.1 RL

• RL infrastructure separates rollout generation from model updates, using dedicated inference backends for decoding and distributed training systems for parameter optimization.Rollouts generate responses and compute the values needed for training, while model updates derive losses from policy, critic, reference-model, and reward computations.Source: §8.1 · p. 46
• OpenRLHF and veRL provide actively maintained RL frameworks with distinct resource-management designs, code accessibility, and programming interfaces.OpenRLHF uses dedicated resource pools and a concise implementation, whereas veRL dynamically shares resources and offers a more centralized, programmable interface.Source: §8.1 · p. 46, Table 10 · p. 47
• Large-scale RL remains difficult: scaling to 32B parameters can require prohibitively many GPUs, while long chain-of-thought outputs cause low utilization through length variance and inference stragglers.These limitations motivate further optimization of RL frameworks for long-CoT scenarios.Source: §8.1 · p. 46

8.2 MCTS

• MCTS infrastructure has released code for LLM applications, but limited hardware- and software-aware acceleration still constrains large-scale deployment.The section organizes acceleration around speculative decoding, KV-cache management, and parallel processing.Source: §8.2 · p. 46
• MCTS acceleration uses speculative decoding, shared KV-cache states, and adaptive path parallelism to reduce rollout cost and improve resource utilization.Speculative decoding speeds SC-MCTS reasoning by an average of 52%; cache-sharing methods reduce duplication, while adaptive parallelism responds to real-time GPU memory availability.Source: §8.2 · p. 47

9 Tutorial

• The tutorial explains how reinforcement learning can unlock the long CoT ability of LLMs.Source: §9 · p. 47

9.1 Preparatory Work

• Preparation uses veRL, recommended Qwen2.5 base models, and selected mathematical datasets designed to support stable improvement and longer solutions.Recommended starting models are Qwen2.5-1.5B, Qwen2.5-3B, and Qwen2.5-7B; datasets include NuminaMath, DeepScaleR, and MATH.Source: §9.1 · p. 47

9.2 Start RL Training

• RL training adopts GRPO and provides commands, default parameters, and configurable paths for models and training or testing datasets.Source: §9.2 · p. 48

9.3 Understanding Algorithm with Code Analysis

• The implementation generates responses, computes policy and reference log probabilities and rewards, estimates advantages, then updates the critic and actor.The tutorial also covers prompt templates that elicit reasoning before boxed final answers and correctness rewards of 1 or -1.Source: §9.3 · pp. 48–49

9.4 Results

• Accuracy increases markedly and mathematical solution processes become more detailed during reinforcement learning, as shown in Figure 19.Figure 19 tracks accuracy on MATH500 and response length on the Qwen-Base model’s training set during RL.Source: §9.4 · p. 50, Figure 19 · p. 50

New architecture

• New architectures such as Mamba, linear transformers, and language diffusion models could address Transformer memory limits and improve cognitive capability and computational efficiency.Progress requires theoretical frameworks, efficient training and inference infrastructure, and large-scale pretrained foundation models.Source: §10 · pp. 50–51

Pretraining on latent thought

• Pretraining on large-scale data containing latent cognitive behaviors could strengthen test-time scaling beyond current small-scale demonstrations.Potential approaches include inferring latent thoughts from existing reasoning models and studying their effects on test-time scaling (Gandhi et al., 2025; Liu et al., 2025g,f; Zelikman et al., 2024a; Jiang et al., 2024; Ruan et al., 2025).Source: §10 · p. 51

RL scaling

• RL scaling needs rigorous, reproducible, and accessible large-scale studies that address hyperparameters, infrastructure, algorithmic efficiency, reward hacking, and expansion beyond verifiable tasks.These studies should combine empirical investigation with theoretical analysis and clarify how reward reliability relates to RL scaling (Jordan et al., 2024; Hochlehnert et al., 2025).Source: §10 · p. 51

Evaluation

• Evaluation should measure cognitive-process quality, efficiency, and generalizability alongside task performance rather than relying on word matching or isolated case studies.Proposed measures include reasoning depth, backtracking efficiency, verification quality, metacognitive awareness, and dynamic protocols developed with cognitive scientists and domain experts (DeepSeek-AI et al., 2025; Gandhi et al., 2025).Source: §10 · p. 51

Scientific discovery

• Cognition engineering could support scientific discovery by connecting knowledge domains and enabling extended deliberation for hypothesis generation, experimental design, and theory formation.Future systems should combine specialized prompting or training with scientific tools and experimental platforms to design and potentially execute validation experiments (Lu et al., 2024a).Source: §10 · p. 51

Survey perspective

• The survey organizes existing work from a test-time scaling perspective and provides detailed coverage of scaling laws, efficiency across four primary methods, method comparisons, and ensemble applications.This differs from works focused on system-2 reasoning, long CoT, meta-generation, or only partial scaling methods.Source: §11 · p. 51
• It offers more concrete discussions of RL and SFT for long-CoT techniques than previous works, which provide comparatively vague treatment.The discussion is based on recent research and specifically addresses both training approaches.Source: §11 · p. 52
• The paper covers a broader range of test-time scaling applications and provides thorough discussion and future directions for each.Source: §11 · p. 52

Position paper perspective

• As a position paper, it proposes cognition engineering to unify technologies that enhance models’ cognitive abilities and identifies directions such as pretraining on latent thought.This conceptual framework distinguishes the paper from other survey papers and frames generative AI as entering an era focused on deep cognitive capabilities.Source: §11 · p. 52

• Cognition engineering shifts AI development from knowledge accumulation toward the systematic development of thinking capabilities.Source: §12 · p. 52
• Test-time scaling and specialized training enable models to perform deep thinking, complex reasoning, and creative problem-solving.Source: §12 · p. 52
• These capabilities improve model performance across domains including mathematics and multimodal understanding.Source: §12 · p. 52
• Cognition engineering initiates a symbiotic human–AI relationship based on deep understanding and cognitive exchange.Source: §12 · p. 52