PaperlayerView exact revision on arXiv

Source-linked AI summary

RedOne 2.0: Rethinking Domain-specific LLM Post-Training in Social Networking Services

Fei Zhao, Chonggang Lu, Haofu Qian, Fangcheng Shi, Zijie Meng, Jianzhao Huang, Xu Tang, Zheyong Xie, Zheyu Ye, Zhe Xu, Yao Hu, Shaosheng Cao

arXiv:2511.07070v1cs.AIcs.LG

TL;DR

SNS’s heterogeneous, fast-changing, multilingual workloads create distribution-shift challenges, while how to structure RL-centric adaptation for efficient, stable transfer remains underexplored. RedOne 2.0 uses progressive, RL-prioritized post-training; its 4B variant surpasses a 7B counterpart by 2.41 on average, and using half RedOne’s data it achieves an 8.74 performance lift.

  • Problem

    SNS’s heterogeneous workloads, rapidly changing norms, and multilingual, culturally diverse corpora create distribution shift, while RL-centric training for efficient, stable domain transfer remains underexplored.

    Source: §1 · pp. 1–2

  • Method

    RedOne 2.0 uses three stages: exploratory SNS alignment and diagnosis, targeted SFT on diagnosed gaps, then SNS-centric RL refinement.

    Source: §1 · p. 2

  • Results

    RedOne 2.0 4B surpasses the 7B counterpart by 2.41 on average; using half RedOne’s data, it achieves an 8.74 performance lift.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors present RedOne 2.0 as a competitive, cost-effective SNS-domain baseline with robustness under distribution shift.

    Source: §1 · p. 2

    RedOne 2.0 can over-optimize engagement and stylistic appeal at the expense of essential details, as in a case that omits a key transportation-card reference.

    Source: §4.6 · p. 8

Abstract

from arXiv · show

As a key medium for human interaction and information exchange, social networking services (SNS) pose unique challenges for large language models (LLMs): heterogeneous workloads, fast-shifting norms and slang, and multilingual, culturally diverse corpora that induce sharp distribution shift. Supervised fine-tuning (SFT) can specialize models but often triggers a ``seesaw'' between in-distribution gains and out-of-distribution robustness, especially for smaller models. To address these challenges, we introduce RedOne 2.0, an SNS-oriented LLM trained with a progressive, RL-prioritized post-training paradigm designed for rapid and stable adaptation. The pipeline consist in three stages: (1) Exploratory Learning on curated SNS corpora to establish initial alignment and identify systematic weaknesses; (2) Targeted Fine-Tuning that selectively applies SFT to the diagnosed gaps while mixing a small fraction of general data to mitigate forgetting; and (3) Refinement Learning that re-applies RL with SNS-centric signals to consolidate improvements and harmonize trade-offs across tasks. Across various tasks spanning three categories, our 4B scale model delivers an average improvements about 2.41 over the 7B sub-optimal baseline. Additionally, RedOne 2.0 achieves average performance lift about 8.74 from the base model with less than half the data required by SFT-centric method RedOne, evidencing superior data efficiency and stability at compact scales. Overall, RedOne 2.0 establishes a competitive, cost-effective baseline for domain-specific LLMs in SNS scenario, advancing capability without sacrificing robustness.

1 Introduction

SNS workloads, rapidly changing norms, and multilingual audiences create distribution-shift challenges for LLMs, while SFT-heavy adaptation can trade in-domain gains for weaker out-of-distribution generalization. RedOne 2.0 proposes progressive, RL-prioritized post-training and reports gains at compact scales with less data.

1 Introduction

  • SNS combines heterogeneous task requirements, fast-changing norms, and multilingual, culturally diverse corpora, increasing distribution shift and the risk of brittle generalization.The passage notes risks including misreading community rules, policy over- or under-enforcement, and drifting as conventions change.Source: §1 · p. 1
  • SFT-heavy adaptation can improve in-distribution tasks at the expense of out-of-distribution generalization, especially for smaller models vulnerable to catastrophic forgetting.The introduction says common robustness mitigations can require very large data and compute budgets, while RL-centric training structure for SNS remains underexplored.Source: §1 · pp. 1–2
  • The three stages are Exploratory Learning to align and diagnose, targeted SFT on diagnosed weaknesses with general data, and Refinement Learning with SNS-centric RL signals.The final RL stage is intended to consolidate gains and smooth trade-offs across tasks.Source: §1 · p. 2
  • RedOne 2.0’s 4B variant surpasses the 7B counterpart by 2.41 on average, while using half RedOne’s data and achieving an 8.74 performance lift.The data comparison uses Qwen3-4B as the base.Source: §1 · p. 2
  • The authors position RedOne 2.0 as a competitive, cost-effective SNS-domain baseline with robustness under distribution shift.They describe empirical validation across general and SNS-specific abilities.Source: §1 · p. 2

2 Related Work

Related work spans SNS applications, general post-training, and domain adaptation. The cited approaches leave cost, domain specificity, and generalization challenges that RedOne 2.0 addresses with an RL-centric design.

2.1 LLM in Social Networking Services

  • SNS research covers many applications and unified approaches, but most approaches remain data-centric, relying on large annotated corpora and strong base models that raise costs.The passage contrasts this with RedOne 2.0’s stated stronger downstream performance using less data and smaller model scales.Source: §2.1 · p. 2

2.2 General LLM Post-training

  • General post-training combines supervised fine-tuning and preference optimization, but most approaches remain domain-agnostic and underuse specialized knowledge for vertical applications.The passage names InstructGPT, RRHF, DPO, GRPO, and DAPO as examples of post-training developments.Source: §2.2 · p. 2

2.3 Domain-specific LLM Post-training

  • Domain adaptation commonly combines continued pre-training, SFT, and preference optimization, yet heavy SFT can weaken generalization and later RL may only partly correct the drift.RedOne 2.0 is presented as emphasizing staged reinforcement and dynamic task sampling.Source: §2.3 · pp. 2–3

3 Methodology for RedOne 2.0

RedOne 2.0’s post-training proceeds through three dependent stages that progressively align the model to SNS while aiming to preserve general capability and stability.

3 Methodology for RedOne 2.0

  • Exploratory Learning initially aligns the base model to SNS and identifies SNS-specific tasks requiring targeted reinforcement.The stage surfaces deficiencies for the next stage to address.Source: §3 · p. 3
  • Targeted Fine-Tuning addresses deficiencies identified during exploration and creates exploration space for later optimization.Source: §3 · p. 3
  • Refinement Learning applies RL to SNS-domain data to stabilize and smooth model behavior across tasks.Source: §3 · p. 3

3.1 Dataset Definition

The dataset combines SNS-domain and general-domain data, covering a broad set of SNS capabilities and tasks before normalization into a shared question-answer format.

3.1 Dataset Definition

  • The SNS data spans more than 75 tasks, including information extraction, semantic matching, content understanding, user behavior modeling, dialogue, and translation.The general-domain portion draws on high-quality open-source datasets to support general capabilities.Source: §3.1 · p. 3
  • Collected data are normalized into a unified question-and-answer format.Source: §3.1 · p. 3
  • DSNS and DGEN denote the SNS-domain and general-domain dataset subsets, respectively.Source: §3.1 · p. 3

3.2 Exploratory Learning

Exploratory Learning broadly aligns the base model to SNS data while preserving general competence and surfacing difficult tasks. Task-specific rewards support DAPO-based training, whose gains and diagnosis guide targeted repair.

3.2 Exploratory Learning

  • Exploratory Learning immerses the base model in diverse SNS data to capture varied task distributions and interaction patterns while revealing remaining weaknesses.The aim is initial SNS alignment without committing early to narrowly scoped objectives.Source: §3.2 · p. 3

3.2.1 Data Construction.

  • About 750K SNS entries span 75 heterogeneous tasks, alongside 50K general examples with rationales to preserve reasoning and general competence.Balanced sampling preserves the visibility of long-tail behaviors.Source: §3.2.1 · p. 3

3.2.2 Reward Function.

  • Rewards are task-routed: exact match scores determinate answers, metrics evaluate open-ended tasks, sandbox execution assesses code, and pattern matching checks format.Pattern matching emphasizes adherence to specified formats rather than semantic content.Source: §3.2.2 · pp. 3–4

3.2.3 Domain Alignment.

  • The stage randomly mixes DSNS1 and DGEN1 into D1 for DAPO-based reinforcement learning.Its alignment objective jointly raises general competence and SNS-specific capability while surfacing difficult sub-tasks.Source: §3.2.3 · p. 4
  • The alignment stage yields broad, stable gains without premature specialization and produces a capability map that guides targeted repair.It also systematically surfaces sub-tasks that remain difficult because of the base model’s unfamiliarity.Source: §3.2.3 · p. 4

3.3 Targeted Fine-Tuning

Targeted Fine-Tuning repairs weak SNS tasks while aiming to preserve earlier gains. It uses hard SNS examples and a small soft-labeled general-data mixture, yielding more balanced preparation for RL refinement.

3.3.1 Data Preparation.

  • The 1.8M-example dataset contains 1.7M SNS instances and 100K general-domain instances, with SNS examples drawn from tasks identified as weak in the previous stage.The SNS data are stratified by capability and rare but impactful cases are upweighted; the general examples use judge-selected soft targets from eight stage-one completions.Source: §3.3.1 · p. 4
  • The stage applies plain SFT to hard SNS examples mixed with a small set of general-domain examples carrying soft labels.Its objective is to close gaps on underperforming SNS tasks while preserving gains from the previous stage.Source: §3.3.2 · p. 4

3.3.2 Targeted Learning.

  • Targeted Fine-Tuning yields consistent improvements on previously weak SNS tasks while maintaining first-stage gains on most capability types.The authors attribute a more balanced and robust model to SNS-prioritized repair combined with soft-label regularization from a small general set.Source: §3.3.2 · p. 4

3.4 Refinement Learning

Refinement Learning reapplies RL after targeted SFT, with training centered on SNS data. The authors report further improvements on both SNS-specific and general tasks, alongside better convergence and more robust domain adaptation.

3.4 Refinement Learning

  • Refinement Learning applies RL after the SFT-based stage, again centering training on SNS data.It uses approximately 400K examples from SNS and general sources, emphasizing difficult subsets, and increases the share of samples with rationale to 57.18%.Source: §3.4 · p. 4, §3.4.1 · p. 4

3.4.1 Further Refinement.

  • RL-based refinement yields further improvements on SNS-specific and general tasks, with better convergence and more robust domain adaptation than the previous stage.Table 1 compares results across General-Bench, SNS-Bench, and SNS-TransBench.Source: §3.4.1 · p. 5, Table 1 · p. 5

4 Experiments

Experiments assess RedOne 2.0 across general, SNS, and translation benchmarks, finding strong results at compact scale and gains across the model family. The section also describes the implementation and benchmark coverage.

4.1 Implementation Details

  • Exploratory Learning uses 500 steps and DAPO; Targeted Fine-Tuning runs for two epochs, and Refinement Learning mirrors the first stage’s configuration.The exploratory stage uses 5×10^-6 learning rate; Targeted Fine-Tuning uses cross-entropy loss with AdamW at 5×10^-6.Source: §4.1 · p. 5

4.2.1 Benchmarks.

  • The general evaluation covers six capability areas and CompassBench, while SNS evaluation uses SNS-Bench’s eight tasks and SNS-TransBench’s 2,858 English–Chinese cases.The selected general benchmarks cover knowledge, math, code, translation, instruction following, and hallucination detection; comparisons include proprietary, open-source, and SNS-specific models.Source: §4.2.1 · pp. 5–6, §4.2.2 · p. 6

4.3 Main Results

  • RedOne 2.0 4B scores 70.8 on General-Bench, the highest average reported, exceeding larger open models including Qwen3-8B and GLM-4-9B.The authors also report comparable or better results than some proprietary models or models above 100B parameters.Source: §4.3 · p. 6
  • RedOne 2.0 4B scores 67.57 on SNS-Bench, 0.69 above RedOne-7B, while the 4B and 30B-A3B variants average 47.67 and 49.54 on SNS-TransBench.Both translation variants rank in the top two overall averages; the 30B-A3B SNS-Bench score is 69.04.Source: §4.3 · p. 6
  • From 4B to 30B, average gains exceed 4.37 on General-Bench, 1.47 on SNS-Bench, and 1.87 on SNS-TransBench.Compared with RedOne-7B, RedOne 2.0 4B gains 6.97 on General-Bench and 0.69 on SNS-Bench.Source: §4.3 · p. 6

4.4 More Analysis

Further analyses test the staged pipeline across base-model scales, isolate stage contributions, and compare it with SFT-first and task-specific tuning. The reported results favor the staged and unified approach across multiple benchmarks.

4.4.1 Generalization Across Different Base-Model Scales.

  • The pipeline improves across base-model scales, and scaling up further amplifies overall performance; the authors attribute stronger gains partly to instruction-tuned backbones absorbing alignment signals.The 4B and 30B-A3B variants show superior gains over similar-scale models, particularly on SNS-related benchmarks.Source: §4.4.1 · pp. 6–7

4.4.2 Incremental Performance Over three-Stage Training.

  • Exploratory Learning scores 71.25% on General-Bench, 62.27% on SNS-Bench, and 43.35% on SNS-TransBench; refinement raises the average from 61.14% to 62.01%.Targeted Fine-Tuning raises SNS-Bench and SNS-TransBench scores to 65.67% and 47.72%, with a 1.21% General-Bench drop.Source: §4.4.2 · p. 7, Table 3 · p. 7

4.4.3 Superiority to the Naive SFT Followed RL Baseline.

  • RedOne 2.0 surpasses the naive SFT-followed-by-RL baseline by 1.00 on General-Bench, 4.54 on SNS-Bench, and 1.72 on SNS-TransBench.The authors report that SFT-first reduces general capability from 69.80 to 63.65, while subsequent RL only partly mitigates the seesaw.Source: §4.4.3 · p. 7
  • Against task-specific Qwen3-4B tuning, unified RedOne 2.0 4B outperforms on MRC by 9.00 and on CHLW by 11.87, while scoring 60.92 on QCorr and 47.67 on SNS-Trans.The task-specific models score 49.24 on QGen and 90.12 on Hash., respectively.Source: §4.4.4 · p. 7, Table 4 · p. 7

4.5 Online Application

RedOne 2.0 was deployed for personalized post-title rewriting on a large social platform, with online measures of advertiser value and title quality. Case studies show more engaging phrasing alongside a failure mode involving omitted critical details.

4.5 Online Application

  • The online deployment rewrites pre-published titles in real time for a platform with more than 3 million users; evaluation covers advertiser value and four human-rated quality dimensions.The test ran over several weeks and millions of posts.Source: §4.5 · p. 7
  • Advertiser Value increased by 0.43% in a statistically significant online test, alongside fewer vague titles and more practical, authentic, and interactive titles.Human evaluation found changes of −11.9% in vague titles and +7.1%, +12.9%, and +25.8% in practical, authentic, and interactive titles, respectively.Source: §4.5 · pp. 7–8

4.6 Case Study

  • Good cases use vivid or exclusive framing, but a bad case generalizes Osaka–Kyoto transport-card advice to Kansai travel and omits the key card reference.The authors note that engagement and stylistic appeal can come at the expense of informational precision, motivating stronger faithfulness constraints.Source: §4.6 · p. 8, Table 6 · p. 8, Table 7 · p. 8

5 Conclusion

RedOne 2.0 is an SNS-specific post-training framework for heterogeneous, dynamic, and culturally diverse tasks. Its RL-prioritized three-stage design and task-diverse training data are presented as a cost-effective, scalable baseline.

5 Conclusion

  • RedOne 2.0 uses Exploratory Learning to align and surface weaknesses, Targeted Fine-Tuning to repair deficiencies while retaining general competence, and RL Refinement to consolidate gains.Source: §5 · p. 8
  • The framework is supported by a task-diverse dataset spanning more than 75 SNS tasks and a high-quality general corpus.Source: §5 · p. 8
  • The conclusion presents RedOne 2.0 as a data-efficient, stable, and robust baseline that adapts even at compact scales without sacrificing safety or general usability.Source: §5 · p. 8

Related papers

Loading 2511.07070v1…