PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Post-training of Large Language Models

Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang, Jiangyue Yan, Yao Su, Zhenhan Dai, Yifeng Xie, Yihan Cao, Lichao Sun, Pan Zhou, Lifang He, Hechang Chen, Yu Zhang, Qingsong Wen, Tianming Liu, Neil Zhenqiang Gong, Jiliang Tang, Caiming Xiong, Heng Ji, Philip S. Yu, Jianfeng Gao

arXiv:2503.06072v3cs.CLcs.AI

TL;DR

LLMs remain limited in specialized reasoning, ethical alignment, and domain performance despite broad capabilities. This paper surveys PoLMs across their methods, datasets, applications, and evolution from ChatGPT to DeepSeek-R1, concluding that the field has advanced reasoning precision, domain adaptability, and ethical integrity while retaining dataset-quality and generalization challenges.

  • Problem

    Pre-trained LLMs have limited reasoning, ethical alignment, and domain-specific adaptation, while PoLM research has lacked a systematic comprehensive survey.

  • Method

    The paper conducts a structured survey of PoLM evolution, categorizing five technique families, datasets, applications, historical developments, and future directions.

  • Results

    The survey synthesizes PoLM progress from ChatGPT’s alignment strategies to DeepSeek-R1’s reasoning advances across fine-tuning, alignment, reasoning, efficiency, and integration and adaptation.

  • Takeaways & Limitations

    PoLMs have advanced reasoning precision, domain adaptability, and ethical integrity, while Large Reasoning Models and large-scale reinforcement learning define important future directions.

  • Takeaways & Limitations

    Human-labeled datasets remain constrained by scalability, annotation bias, and inconsistent annotation quality and diversity.

Abstract

from arXiv · show

The emergence of Large Language Models (LLMs) has fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific exploration. However, their pre-trained architectures often reveal limitations in specialized contexts, including restricted reasoning capacities, ethical uncertainties, and suboptimal domain-specific performance. These challenges necessitate advanced post-training language models (PoLMs) to address these shortcomings, such as OpenAI-o1/o3 and DeepSeek-R1 (collectively known as Large Reasoning Models, or LRMs). This paper presents the first comprehensive survey of PoLMs, systematically tracing their evolution across five core paradigms: Fine-tuning, which enhances task-specific accuracy; Alignment, which ensures ethical coherence and alignment with human preferences; Reasoning, which advances multi-step inference despite challenges in reward design; Efficiency, which optimizes resource utilization amidst increasing complexity; Integration and Adaptation, which extend capabilities across diverse modalities while addressing coherence issues. Charting progress from ChatGPT's alignment strategies to DeepSeek-R1's innovative reasoning advancements, we illustrate how PoLMs leverage datasets to mitigate biases, deepen reasoning capabilities, and enhance domain adaptability. Our contributions include a pioneering synthesis of PoLM evolution, a structured taxonomy categorizing techniques and datasets, and a strategic agenda emphasizing the role of LRMs in improving reasoning proficiency and domain flexibility. As the first survey of its scope, this work consolidates recent PoLM advancements and establishes a rigorous intellectual framework for future research, fostering the development of LLMs that excel in precision, ethical robustness, and versatility across scientific and societal applications.

1 Introduction

Post-training adapts general language models to specific tasks and requirements, addressing limitations in reasoning, alignment, knowledge adaptation, and domain transfer. This survey organizes the field's evolution, methods, datasets, applications, and future challenges from ChatGPT to DeepSeek.

  • 1 Introduction: Post-training refines pre-trained models for specific tasks through fine-tuning, alignment, knowledge adaptation, and reasoning improvements.These techniques collectively define Post-training Language Models (PoLMs), including GPT-4, LLaMA-3, Gemini-2.0, and Claude-3.5.
  • 1 Introduction: PoLMs focus on adapting general knowledge and capabilities to specific requirements, while pre-trained language models primarily provide general-purpose capabilities.DeepSeek-R1 illustrates this adaptation through improved reasoning, preference alignment, and domain adaptability.
  • 1 Introduction: The survey addresses a gap in systematic PoLM reviews by tracing post-training methods from ChatGPT to DeepSeek across fine-tuning, alignment, reasoning, and efficiency.It also examines methods used by DeepSeek and their adaptation to user preferences and domain-specific needs.
  • 1.1 Major Contributions: The paper presents a comprehensive historical synthesis spanning ChatGPT’s RLHF and DeepSeek-R1’s cold-start RL across five post-training technique categories.The categories are Fine-tuning, Alignment, Reasoning, Efficiency, and Integration and Adaptation.
  • 1.1 Major Contributions: Its structured taxonomy classifies post-training methods into five categories, datasets into seven types, and applications across professional, technical, and interactive domains.The framework is intended to clarify interrelationships and practical implications for navigating post-training research.
  • 1.1 Major Contributions: The survey highlights Large Reasoning Models such as o1 and DeepSeek-R1, while identifying scalability, ethical alignment, and multimodal integration as open challenges.It proposes adaptive reinforcement learning and fairness-aware optimization as research directions.

2 Overview

Post-training evolved from general pre-trained models toward task adaptation, alignment, domain specialization, multimodal integration, and computational efficiency. The overview also introduces reinforcement-learning methods used to stabilize policy updates and optimize language-model behavior from human preferences.

  • History of PoLMs: From 2020 onward, lightweight prefix-tuning and prompt-tuning adapted models by modifying inputs rather than retraining entire architectures.These methods supported multi-task flexibility while conserving computational resources.
  • History of PoLMs: Between 2022 and 2024, post-training expanded toward domain adaptation, ethical robustness, and multimodal integration.Retrieval-Augmented Generation incorporated external knowledge for specialized applications without requiring full retraining.
  • History of PoLMs: Mixture of Experts models improve computational efficiency by dynamically activating selective parameter subsets while supporting larger parameter scales.Switch-C Transformer pioneered this direction, followed by models including Mixtral and DeepSeek V2.5.
  • Policy Optimization: PPO stabilizes reinforcement-learning updates by constraining policy changes, reducing the risk of abrupt behavioral shifts during large-model fine-tuning.Its clipped objective restricts updates to a safe range, while the value function estimates expected cumulative reward.
  • Preference Optimization: RLHF optimizes a language model toward human preferences by using feedback-derived rewards, whereas DPO directly optimizes outputs from pairwise preferences without a traditional reward function.RLHF maximizes expected rewards for input-output pairs; DPO instead performs preference-based optimization directly.

3 PoLMs for Fine-Tuning

Fine-tuning adapts pretrained LLMs to specialized tasks through supervised, full-parameter, adaptive, and reinforcement-based procedures. These approaches range from curated labeled datasets and task-specific optimization to parameter-efficient prompting and reinforcement learning that improves reasoning outputs.

  • Fine-Tuning: Fine-tuning adapts pretrained LLMs to specialized tasks by optimizing parameters with labeled or task-specific datasets.It bridges general pretraining with domain-specific requirements.
  • Supervised Fine-Tuning: SFT dataset preparation includes constructing instruction–instance pairs, screening them with quality thresholds, and evaluating curated subsets for model performance.Self-Instruct expands dataset diversity, while IFD measures how effectively instructions guide expected responses.
  • Supervised Fine-Tuning: SFT adjusts a pretrained model using annotated task data and commonly minimizes cross-entropy loss against ground-truth labels.The process begins with a pretrained LLM and refines its parameters for a target application.
  • Full-Parameter Fine-Tuning: Full-parameter fine-tuning updates all model parameters, offering high precision but requiring substantial computational memory compared with LoRA or Prefix-tuning.Fine-tuning a 65-billion-parameter model may require over 100 GB of GPU memory; mixed precision and activation checkpointing reduce memory demands.
  • Adaptive Fine-Tuning: Instruction tuning converts existing NLP datasets into natural-language instructions, improving generalization across familiar and unseen tasks.Self-Instruct can generate additional instruction–output pairs, while the tuning procedure aligns model parameters with task-specific guidance.
  • Adaptive Fine-Tuning: Prompt-tuning can match full-parameter fine-tuning across many tasks while using significantly fewer trainable parameters, reducing computational and memory costs.Its effectiveness depends on the underlying model’s capacity because only a small number of input-layer parameters are modified.
  • Reinforcement Fine-Tuning: ReFT combines an SFT warm-up with reinforcement learning on the same dataset and has been reported to outperform traditional SFT without additional training data.Reward signals based on answer correctness update the policy to generate more accurate and diverse chain-of-thought annotations.

4 PoLMs for Alignment

Alignment methods guide LLM outputs toward human expectations and preferences through distinct feedback and optimization paradigms. The survey compares these methods across reward modeling, feedback, policy, and optimization dimensions.

  • Alignment targets model outputs that conform to human expectations and preferences, especially in safety-critical or user-facing applications.
  • The survey discusses RLHF, RLAIF, and DPO as major alignment paradigms with distinct advantages, challenges, and trade-offs.
  • Table 2 compares alignment methods across eight metrics covering reward models, feedback sources, reference policies, reinforcement-learning settings, and optimization modes.

4.1 Reinforcement Learning with Human Feedback

RLHF aligns language models by collecting human preferences, learning a reward model, and iteratively updating a policy. The survey covers feedback types, reward evaluation, and online, offline, and hybrid learning settings.

  • RLHF uses human preference labels or reward signals to train a reward model that guides iterative policy updates toward human preferences.
  • Human feedback is classified by dimensions including granularity, involvement level, and explicitness, with modalities offering different interpretability, scalability, and noise tolerance.
  • Primary feedback includes critiques, comparisons, and inter-temporal feedback, while supplementary feedback includes emergency stops and importance labels.
  • Reward models approximate unknown true rewards from human-labeled or compared outputs, typically using cross-entropy training on preference comparisons.
  • Reward evaluation can use distance functions, visual inspection, and curated datasets to assess equivalence, interpretability, and semantic consistency.
  • Online RLHF collects preferences from newly generated trajectories, whereas offline RLHF learns from previously collected preference-labeled trajectories; hybrid methods combine both.

4.2 Reinforcement Learning with AI Feedback

RLAIF replaces or complements human preference collection with AI-generated feedback to make alignment more scalable. Its pipeline collects AI feedback, trains a reward model, and iteratively updates the policy.

  • RLAIF uses LLM-generated feedback instead of human-generated preferences, addressing the cost, time, inconsistency, and scalability constraints of RLHF labeling.
  • RLAIF can achieve performance comparable to or superior to RLHF according to human raters and can surpass supervised fine-tuning baselines.
  • The RLAIF pipeline collects AI feedback according to predefined criteria, trains or refines a reward model, and updates the policy iteratively.
  • AI-generated feedback scales continuously across model outputs without requiring continual human intervention, alleviating the human-labeling bottleneck.

4.3 Direct Preference Optimization

DPO simplifies preference alignment by training the policy directly on human preference data rather than fitting a separate reward model. Its KL-regularized objective supports direct optimization, with variants addressing generation control, listwise preferences, and harmful outputs.

  • DPO avoids explicit reward-model fitting and Bradley–Terry dependencies by treating reward maximization as direct policy training from human preference data.
  • DPO maximizes the likelihood of preferred outputs over less-preferred outputs using preference triplets, without requiring a separate reward model.
  • DPO inherits KL-regularization from RLHF, constraining policy adaptation relative to a reference policy while retaining stated consistency guarantees under defined preference assumptions.
  • The DPO pipeline keeps a reference policy fixed while iteratively updating a target policy using human comparisons of sampled candidate responses.
  • Effective DPO depends on a robust reference policy and sufficiently diverse preference data to support adaptation and reduce narrow-task overfitting.
  • DPO variants address token-level alignment, verbosity, reference-policy removal, listwise preferences, and learning from negative or harmful outputs.

5 PoLMs for Reasoning

Post-training reasoning methods improve LLMs through self-refinement and reinforcement learning. The survey traces a progression from local error correction to large-scale RL that develops multi-step inference, including cold-start approaches that avoid supervised fine-tuning.

  • 5.1 Self-Refine for Reasoning: Self-refine methods improve reasoning by detecting and correcting errors during or after generation.The taxonomy includes intrinsic, external, fine-tuned intrinsic, and fine-tuned external approaches.
  • 5.1 Self-Refine for Reasoning: Intrinsic methods revise outputs internally, while external methods use tools, references, symbolic solvers, or intermediate-state verification.Examples include contradiction-triggered correction, step-by-step checking, external documents, and compiler feedback.
  • 5.2 Reinforcement Learning for Reasoning: Self-refinement offers quick inference gains for single-step tasks but struggles with complex, long-term multi-step reasoning.The survey presents reinforcement learning as an alternative for refining long internal chains of thought.
  • 5.2 Reinforcement Learning for Reasoning: Large-scale RL uses iterative reward feedback to develop reasoning skills for mathematical problem-solving, logical deduction, and strategic planning without relying on pre-annotated datasets.DeepSeek-R1 exemplifies this paradigm through adaptive exploration, trajectory management, and policy optimization.
  • 5.2 Reinforcement Learning for Reasoning: GRPO reduces optimization overhead while preserving robust policy updates, supporting scalable RL-based reasoning on resource-constrained systems.This positions GRPO as a practical component of DeepSeek-R1-Zero’s reasoning pipeline.
  • 5.2 Reinforcement Learning for Reasoning: DeepSeek-R1-Zero develops multi-step reasoning through pure RL from an untrained base model, bypassing supervised fine-tuning and pre-annotated data.The survey describes this cold-start approach as achieving marked improvements on mathematical and logical reasoning tasks.

6 PoLMs for Efficiency

Post-training efficiency reduces the operational cost of LLMs through compression, parameter-efficient adaptation, and knowledge distillation. The survey covers quantization, pruning, PEFT, and reasoning-oriented distillation as complementary approaches to reducing computation or trainable parameters.

  • 6 PoLMs for Efficiency: Post-training efficiency targets processing speed, memory usage, and resource consumption through model compression and related adaptation methods.The section organizes efficiency techniques around compression, parameter-efficient fine-tuning, and knowledge distillation.
  • 6.1 Model Compression: Quantization converts high-precision representations into lower-precision formats, with PTQ adjusting weights and activations after pre-training using a small calibration dataset.The survey distinguishes weight-only, weight-activation, and KV-cache quantization.
  • 6.1 Model Compression: Parameter pruning reduces model size and complexity through unstructured or structured removal of parameters.Unstructured pruning removes individual weights, whereas structured pruning removes parameter groups to improve hardware efficiency.
  • 6.2 Parameter-Efficient Fine-Tuning: PEFT freezes the LLM backbone and updates only a limited set of added or selected parameters.Its categories include additive, selective, reparameterized, and hybrid approaches; adapters use compact bottleneck layers.
  • 6.3 Knowledge Distillation: DeepSeek-R1 distills reasoning patterns from large-scale models into compact architectures, improving smaller LLMs without direct RL on them.The method preserves complex inference processes and uses teacher-generated reasoning trajectories to reduce student training requirements.

7 PoLMs for Integration and Adaptation

Integration and adaptation techniques extend LLMs across heterogeneous data types and specialized domains. The survey presents multi-modal integration as a central strategy for improving versatility across real-world applications.

  • 7 PoLMs for Integration and Adaptation: Integration and adaptation methods help LLMs process heterogeneous data, adapt to specialized domains, and combine multiple architectural strengths.The survey identifies multi-modal integration as one of three principal strategies in this area.

7.1 Multi-Modal Integration

Multi-modal integration connects non-text inputs with LLMs through modal connection methods, encoders, and large multi-modal architectures. The survey organizes these connections into projection-based, query-based, and fusion-based approaches.

  • 7.1.1 Modal Connection: Modal connection methods synthesize multi-modal data through projection-based, query-based, and fusion-based strategies.These approaches provide different mechanisms for connecting visual or other modality representations with LLMs.
  • 7.1.1 Modal Connection: Projection-based methods transform modal inputs into a unified text embedding space aligned with LLM linguistic representations.Examples include LLaMA-Adapter variants and other projection-based vision-language systems.
  • 7.1.1 Modal Connection: Query-based methods use learnable query tokens to extract structured information from diverse modalities.BLIP-2 pioneered query transformers, while later systems extended the approach to video comprehension and instruction tuning.
  • 7.1.1 Modal Connection: Fusion-based methods embed multi-modal features directly into LLM architectures to deepen cross-modal interactions during inference.Examples use cross-attention, vision-encoder outputs, visual experts, or interleaved image-text training data.
  • 7.1.2 Modal Encoders: Modal encoders compress raw inputs into compact semantic representations compatible with LLM backbones.The survey covers encoders for vision, audio, and other modalities supporting visual reasoning, audio comprehension, and 3D understanding.

7.2 Domain Adaptation

Domain adaptation transforms a source LLM into a domain-specific model through targeted adaptation, improving relevance in specialized fields while reducing reliance on extensive labeled data and resources. The section also places knowledge editing and RAG among complementary adaptation approaches.

  • Domain Adaptation: Domain adaptation applies an adaptation function to transform a source model into a domain-specific target model.The formulation is rooted in transfer learning and targets specialized domain requirements.
  • Domain Adaptation: It improves proficiency in programming and mathematical reasoning while increasing computational efficiency for specialized applications.The section contrasts these gains with general-purpose models’ difficulty handling domain terminology and reasoning paradigms.
  • Domain Adaptation: Domain adaptation can reduce dependence on extensive labeled datasets and computational resources compared with training specialized models from scratch.This makes it a central post-training strategy for targeted applications.
  • Knowledge Editing: Knowledge editing updates domain knowledge through targeted parameter adjustments while preserving prior capabilities and limiting degradation on old data.Updates may be constrained to attention or feed-forward components to reduce overhead and avoid comprehensive retraining.
  • Knowledge Editing: Knowledge editing proceeds through knowledge identification, association, and intrinsic embedding of updated facts.Representative methods detect relevant information, integrate it with existing knowledge, and embed it internally with limited performance impact.
  • Retrieval-Augmented Generation: RAG combines retrieval with generation by injecting external information, and its training methods are categorized as independent, sequential, or joint.Independent training separates modules, sequential training optimizes them in turn, and joint training optimizes retriever and generator end to end.

7.3 Model Merging

Model merging combines specialized LLMs into unified systems without extensive retraining, organizing methods into weight-level, output-level, and model-level approaches. The survey further distinguishes pre-merging preparation, during-merging optimization, and post-calibration.

  • Overview: Model merging consolidates specialized models into a unified architecture, reducing the need for extensive retraining during training and inference.It integrates single-task models into a multi-task-capable system while addressing model size and computational demands.
  • Taxonomy: The taxonomy divides merging into weight-level, output-level, and model-level techniques.These levels manipulate parameters, output distributions, or routed submodels and layers, respectively.
  • Merging Levels: Weight-level merging aggregates parameters from architecturally similar models, while output-level merging combines prediction distributions when architectures or initializations diverge.Linear parameter combination and output interpolation are representative formulations.
  • Merging Levels: Model-level merging routes submodels or layers, often through mixture-of-experts mechanisms with discrete or continuous gating.Discrete gating can reduce computation, whereas continuous gating supports smoother expert integration.
  • Pre-merging Methods: Pre-merging methods improve compatibility through linearization, architecture transformation, and weight alignment before fusion.These methods seek to reduce conflicts and interference while preserving constituent model strengths.
  • During-merging and Post-calibration: During-merging methods include basic, weighted, subspace, and routing-based merging, followed by post-calibration to correct representation bias.Basic merging is computationally efficient but can suffer task interference from unmitigated parameter interactions.

8 Datasets

Post-training datasets supply the quality, diversity, and relevance that shape LLM adaptation, spanning human-labeled, distilled, and synthetic resources. Human annotation supports alignment and instruction tuning, while distilled and synthetic data improve scale and efficiency with quality trade-offs.

  • Dataset Landscape: Dataset quality, diversity, and relevance strongly influence the effectiveness of post-training.The survey categorizes post-training datasets and analyzes their roles across training phases.
  • Human-Labeled Datasets: Human-labeled datasets provide accurate, context-sensitive signals for instruction fine-tuning and alignment.Their value comes from annotators’ task understanding and precise evaluations.
  • Human-Labeled Datasets: Flan, Sup-Natinst, and Dolly-15K provide curated instruction-response data spanning multiple tasks, languages, and application scenarios.Sup-Natinst covers 76 task types across 55 languages, while Dolly-15K contains 15,000 human-generated prompt-response pairs.
  • Human-Labeled Datasets: P3, xP3, and SHP provide human-annotated evaluations for RLHF and reward modeling, including multilingual and preference-focused resources.SHP contains 349,000 human preference annotations across 18 subject areas.
  • Dataset Limitations: Human-labeled datasets face scalability, annotation-bias, domain-coverage, and quality-consistency challenges.The survey recommends broader data and rigorous quality control to support alignment and generalization across real-world tasks.
  • Distilled and Synthetic Datasets: Distilled and synthetic datasets reduce training cost and computational demands, but synthetic data may sacrifice depth and authenticity while propagating bias.The survey presents synthetic data as scalable and privacy-preserving, while emphasizing the value of combining it with human-generated data.

9 Applications

Post-training extends LLMs into specialized and multimodal applications where general-purpose models face domain, reasoning, reliability, and bias limitations. The surveyed applications include legal, medical, financial, agentic, mathematical, coding, recommendation, speech, and video tasks.

  • Motivation: Specialized deployment exposes limitations in context length, hallucination control, reasoning proficiency, and bias, making precision and ethical alignment important.These limitations motivate post-training for real-world applications.
  • Professional Applications: Legal post-training uses curated corpora to improve models’ legal reasoning and terminology recognition, including multilingual applications.ChatLaw integrates extensive legal texts into conversational datasets, while LexiLaw and SAUL support English and Chinese.
  • Professional Applications: Medical and financial post-training supports tasks such as drug discovery, diagnosis, report synthesis, sentiment analysis, information extraction, and question answering.Domain-specific models such as FinGPT and DISC-FinLLM are refined on financial corpora.
  • Agentic Applications: Post-training supports GUI agents operating across web, personal-computing, and mobile environments.This application area develops multimodal assistants capable of executing tasks across diverse interfaces.
  • Technical Applications: Mathematical and code-focused post-training improves specialized reasoning, automated coding, debugging, documentation, and programming assistance.DeepSeekMath and Code Llama exemplify domain-focused refinement for mathematics and software development.
  • Interactive and Multimodal Applications: Post-trained models also support personalized recommendation, speech processing, and video understanding or generation.Applications include sentiment-aware recommendations, transcription and synthesis, captioning, summarization, content analysis, and text-to-video generation.

10 Open Problems and Future Directions

Future post-training research must extend reasoning while addressing scalability, ethical alignment, multimodal coherence, context-dependent trustworthiness, and creative intelligence. The survey frames these as interconnected challenges for making advanced reasoning models broadly useful and accessible.

  • Reasoning Enhancement Beyond Large-Scale RL: Reasoning models achieve strong multi-step problem-solving, but binary rewards and extensive human feedback limit effective generalization.The concern is raised for mathematical proofs and logical derivations under RLHF and GRPO-based training.
  • Scalability of Post-Training for Next-Generation LLMs: Resource-intensive RL post-training restricts access to well-funded entities and raises sustainability concerns as model complexity grows.DeepSeek-R1’s cold-start approach is cited as requiring extensive computational infrastructure.
  • Ethical Alignment and Bias Mitigation in RL-Driven Models: RL-driven alignment can reinforce dataset biases, while fairness requirements vary across cultural contexts and sensitive applications.The passage highlights healthcare diagnostics and judicial decision-making as domains where these risks matter.
  • Seamless Multi-Modal Integration for Holistic Reasoning: Future systems need unified multimodal representations and adaptive policies that coordinate text, images, and audio for holistic reasoning.The proposed direction includes shared latent spaces, dynamic modal weighting, and new multimodal benchmarks and synthetic datasets.
  • Context-Adaptive Trustworthiness Frameworks: Trustworthiness must adapt to context because safety requirements can conflict with utility demands across educational and creative tasks.The passage contrasts cautious outputs in education with freer responses in creative settings.
  • Creative Intelligence & System 2 Thinking: Advanced reasoning models still need creative intelligence for novel ideas, cross-concept synthesis, and flexible responses to unstructured problems.The survey identifies creative intelligence as an emergent frontier beyond deliberate step-by-step System 2 reasoning.

11 Conclusion

The survey synthesizes PoLM progress from ChatGPT’s alignment origins to DeepSeek-R1’s reasoning milestone and evaluates its effects across capabilities and application domains. It also identifies persistent bias, scalability, and context-dependent ethical challenges.

  • Survey scope: This first exhaustive survey traces PoLM evolution from ChatGPT’s 2018 alignment origins to DeepSeek-R1’s 2025 reasoning milestone.It organizes the literature across fine-tuning, alignment, reasoning, efficiency, and integration and adaptation.
  • Findings: PoLMs have advanced reasoning precision, domain adaptability, and ethical integrity across professional, technical, and interactive domains.The survey covers applications ranging from legal analysis to multimodal comprehension.
  • Open challenges: Persistent challenges include bias persistence, computational scalability, and ethical alignment that varies with context.The conclusion calls for integrating reasoning advances with efficiency and ethical imperatives.
Loading 2503.06072v3…