Source-linked AI summary
Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts
Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Xiyun Xu, Yang Song, Yiming Jia, Yuntao Wen, Yunzhi Xu, Zekai Wang, Zhenwei An, Zhicong Sun, Zongchao Chen
TL;DR
Nanbeige4.1-3B targets the open challenge of combining broad reasoning, coding, and agentic abilities in a 3B-scale model. It integrates preference modeling, complexity-aware code reinforcement learning, and long-horizon search supervision, and consistently outperforms comparable small models while remaining competitive with larger baselines.
Problem
Existing small language models often have fragmented capabilities, leaving unified 3B-scale generalist performance an open challenge.
Method
The model combines point-wise and pair-wise preference rewards, correctness- and efficiency-aware code RL, and turn- and trajectory-level supervision for long-horizon search.
Results
Nanbeige4.1-3B consistently outperforms comparable small models and remains competitive with larger baselines across diverse benchmarks and real-world challenges.
Takeaways & Limitations
The results show that a 3B model can combine broad competence with strong specialization across reasoning, coding, alignment, and tool-use tasks.
Abstract
from arXiv · showhide
We present Nanbeige4.1-3B, a unified generalist language model that simultaneously achieves strong agentic behavior, code generation, and general reasoning with only 3B parameters. To the best of our knowledge, it is the first open-source small language model (SLM) to achieve such versatility in a single model. To improve reasoning and preference alignment, we combine point-wise and pair-wise reward modeling, ensuring high-quality, human-aligned responses. For code generation, we design complexity-aware rewards in Reinforcement Learning, optimizing both correctness and efficiency. In deep search, we perform complex data synthesis and incorporate turn-level supervision during training. This enables stable long-horizon tool interactions, allowing Nanbeige4.1-3B to reliably execute up to 600 tool-call turns for complex problem-solving. Extensive experimental results show that Nanbeige4.1-3B significantly outperforms prior models of similar scale, such as Nanbeige4-3B-2511 and Qwen3-4B, even achieving superior performance compared to much larger models, such as Qwen3-30B-A3B. Our results demonstrate that small models can achieve both broad competence and strong specialization simultaneously, redefining the potential of 3B parameter models.
1 Introduction
Nanbeige4.1-3B addresses the challenge of building a unified 3B generalist by combining specialized training for reasoning, coding, and agentic behavior. It achieves broad, stable capabilities and outperforms comparable small models while supporting long-horizon search.
- Motivation: Existing small language models often specialize in reasoning, coding, or agency, leaving unified 3B-scale generalist capability an open challenge.Reasoning-focused models may struggle with deep search, while specialized code or agent models may lack broad reasoning and preference alignment.
- Approach: Nanbeige4.1-3B combines point-wise and pair-wise preference rewards, efficiency-aware code rewards, and turn- and trajectory-level supervision for long-horizon planning.The training recipe also uses careful SFT data mixing and multi-stage reinforcement learning to balance the domains.
- Results: Nanbeige4.1-3B consistently surpasses existing open-source SLMs across reasoning and coding tasks while exhibiting stable long-horizon agentic behavior.The model reliably sustains extended tool interactions on complex search-oriented tasks, unlike comparable small models that fail after a few turns.
- Results: Its deep-search capability is comparable to specialized search models at the tens-of-billions scale and approaches that of 100B+ general-purpose models.This comparison is reported specifically for the extended tool-interaction regime.
- Impact: The model is open-sourced to support research on efficient agent-capable language models and joint training under strict capacity constraints.The release includes the checkpoint and aims to contribute to understanding how reasoning, coding, and long-horizon behavior can be trained together.
2 Methods
The methods build Nanbeige4.1-3B through domain-specific optimization, data mixing, and staged reinforcement learning. Search data synthesis and staged code rewards target long-horizon agency, correctness, and efficiency while preserving balanced generalist performance.
- Overall Recipe: The training methodology separately optimizes general reasoning, long-horizon search, and code generation before integrating them through data mixing and multi-stage training.This integration is designed to retain domain-specific strengths under strict capacity constraints.
- SFT: Nanbeige4.1-3B uses enhanced SFT data distribution, length scaling, and quality improvements, including more code and challenging mathematics and general-domain problems.The context curriculum extends from 32k and 64k to 256k tokens, with the final stage emphasizing code, deep search, STEM, tool use, and general data.
- SFT: Nanbeige4.1-3B-SFT improves coding, mathematics, and alignment metrics, providing a stronger foundation for later reinforcement learning.The supplied table caption frames this as performance uplift from Nanbeige4-3B-SFT to Nanbeige4.1-3B-SFT.
- Point-wise RL: Point-wise RL reduces redundant reasoning and formatting errors, lowering LiveCodeBench-v6 overlong truncation from 5.27% to 0.38%.Arena-Hard V2 also rises from 60.2 to 66.6, while its hard-prompt subset rises from 46.1 to 54.1.
- Pair-wise RL: Pair-wise RL uses strong–weak response comparisons and full dialogue histories to sharpen preference alignment after point-wise RL.Multi-Challenge increases from 47.72 to 55.14, and Arena-Hard V2 increases from 66.6 to 73.8.
- Deep Search: Synthetic multi-hop QA and filtered long-range trajectories improve search performance across benchmarks, with xBench-DeepSearch-2505 increasing from 33.0 to 76.0.The dataset is derived from entity-relation graphs built upon Wikipedia and is open-sourced for further research.
- Code RL: Two-stage code RL first optimizes correctness and then activates time-complexity rewards only when all test cases pass.Across training, correctness rises sharply in Stage 1, while the gated time reward rises substantially in Stage 2.
- Overall Recipe: The unified training recipe maintains strong domain-specific performance while producing a balanced generalist model under strict capacity constraints.This conclusion connects the integrated objectives with the reported capability balance.
3 Experiment
The evaluation spans general reasoning, deep-search agentic tasks, and real-world coding challenges, comparing Nanbeige4.1-3B with similarly sized, specialized, and much larger models. Across these settings, it substantially outperforms small baselines, exceeds larger models on many benchmarks, and achieves strong deep-search and contest performance.
- Evaluation Setup: The evaluation covers code, mathematics, science, alignment, tool use, deep search, and out-of-distribution competitive programming tasks.General benchmarks measure core capabilities, deep-search tasks test iterative planning and tool interaction, and LeetCode contests provide a real-world stress test.
- General Task Evaluations: Nanbeige4.1-3B substantially outperforms Qwen3-4B-2507 and Nanbeige4-3B-2511 across all evaluated general domains.The comparisons cover code, math, science, alignment, and tool-use benchmarks.
- General Task Evaluations: With 3B parameters, Nanbeige4.1-3B surpasses Qwen3-30B-A3B-2507 and Qwen3-32B on most benchmarks, including coding, alignment, and tool-use tasks.It remains competitive with Qwen3-Next-80B-A3B, while that larger model retains advantages on selected mathematics and tool-use tasks.
- Deep-Search Evaluation: Nanbeige4.1-3B achieves state-of-the-art results on nearly all evaluated deep-search benchmarks, including GAIA (69.90), xBench-DeepSearch-05 (75.00), and SEAL-0 (41.44).These results place it on par with or above many larger research-agent and tool-equipped foundation models.
4 Conclusion
Nanbeige4.1-3B unifies reasoning, coding, and long-horizon search through specialized reward modeling and multi-level agentic supervision. It consistently outperforms comparable small models and remains competitive with larger baselines, while future work targets efficiency and stronger compact-model architectures.
- Conclusion: Nanbeige4.1-3B integrates point-wise and pair-wise preference rewards, efficiency-aware code rewards, and turn- and trajectory-level signals for agentic planning.These components target preference alignment, computational efficiency, and robust long-horizon behavior.
- Conclusion: Nanbeige4.1-3B consistently outperforms comparable small models and remains competitive with larger baselines across diverse benchmarks and real-world challenges.The conclusion covers reasoning, coding, and long-horizon search together.
- Future Work: Future work aims to improve inference efficiency with shorter outputs and fewer tool invocations while exploring architectural innovations for compact generalist models.The stated direction focuses on complex coding and research-agent scenarios.
B Leetcode Contest Examples
The appendix presents translated examples from LeetCode Weekly Contest 488 under virtual participation. Both the problem statements and model comments were originally in Chinese and are translated for clarity.
- Example Context: The examples come from LeetCode Weekly Contest 488 and use problems originally presented in Chinese.The generated code also contained Chinese comments.
- Example Context: English translations are provided for the problem descriptions and the model’s comments.The translations are included to make the examples clearer to readers.
B.1 Problem Q1: Count Dominant Indices
The dominant-index solution counts positions whose value exceeds the average of all elements to their right, excluding the rightmost position. It scans right to left, maintains a suffix sum, avoids floating-point arithmetic, and runs in O(n) time and O(1) space.
- Problem Definition: A dominant index i satisfies nums[i] > average(nums[i+1], ..., nums[n-1]), and the rightmost element is excluded.The task is to count all dominant indices.
- Optimization Strategy: The algorithm transforms nums[i] > sum / len into nums[i] * len > sum to avoid floating-point arithmetic.The implementation checks this strict inequality directly.
- Optimization Strategy: Scanning from right to left allows the algorithm to maintain the suffix sum and the number of elements to the right dynamically.The loop starts at index n - 2 and updates suffix_sum with nums[i + 1].
- Complexity and Boundary Conditions: The solution uses O(n) time and O(1) space, with a boundary condition that no valid index exists when the array length is at most one.The boundary check is implemented as if n <= 1.
B.2 Problem Q2: Apply Operations to Maximize Array Sum
The section presents stack-based linear-time solutions for adjacent-equal merging and dynamic programming for selecting exactly k ordered index pairs.
- Adjacent-equal merging: Adjacent equal elements are repeatedly replaced by their sum, and the final array is the stack content after no merges remain.
- Adjacent-equal merging: A stack simulation traverses the array, merging equal values with the stack top and accumulating through continuous merges.
- Adjacent-equal merging: O(n) time follows because each element is pushed and popped at most once, with O(n) maximum stack depth.
- Subarray cost constraint: For the subarray cost problem, cost equals (max(nums[l..r]) - min(nums[l..r])) × (r - l + 1), and the task counts subarrays with cost ≤ k.
- Subarray cost constraint: The model uses a sliding window with monotonic deques to maintain window maxima and minima efficiently.
C Time-Complexity Case Study
This case study compares functionally correct code before and after gated time rewards, focusing on whether reinforcement learning encourages more efficient algorithms.
- The study analyzes LiveCodeBench examples where both model outputs are correct but differ in algorithmic efficiency.
- The example problem maximizes the number of simultaneously formed kagamimochi pairs from sorted mochi sizes under a doubling constraint.
C.1.1 Complexity Analysis
The case study shows that replacing binary search and repeated feasibility scans with greedy two-pointer matching reduces the kagamimochi algorithm’s core cost.
- Kagamimochi matching: The before solution binary-searches K and checks each candidate by matching small mochi against large mochi under 2A[top] ≤ A[bottom].
- Kagamimochi matching: The after solution scans the smaller and larger halves with forward-moving pointers, advancing the smaller pointer only after valid matches.
- Complexity comparison: O(N) time results because each pointer advances at most N times and no sorting is needed, while space remains O(N).
- Complexity comparison: The after method improves the core algorithmic cost from O(N log N) to O(N) by eliminating binary search and using one-pass greedy matching.
C.2.1 Complexity Analysis
For detecting multiple subsequences, greedy leftmost and rightmost embeddings replace indexed occurrence searches, reducing per-test-case time from O(N + M log N) to O(N).
- Subsequence embeddings: The leftmost embedding scans A left-to-right, while the rightmost embedding scans A right-to-left to match B as early or late as possible.
- Before approach: The before approach builds a value-to-positions map and uses binary searches, requiring O(N + M log N) time and O(N) space.
- After approach: The after approach uses two greedy linear scans and achieves O(N + M) = O(N) time with O(M) space.
- Subsequence embeddings: At least two matching subsequences exist exactly when the leftmost and rightmost embeddings differ at some position.
C.3.1 Complexity Analysis
The heap-based method simulates adjacent swaps and can incur inversion-dependent costs, whereas the Fenwick-tree method aggregates required movements directly to achieve O(N log N) time.
- The heap method repeatedly resolves adjacent inversions by selecting the smallest-index inversion and rechecking nearby positions.
- Each heap operation costs O(log N), while the number of swaps equals the permutation's inversion count and can reach Θ(N^2).
- Repeated heap insertions and discards remain proportional to the number of swaps because each swap triggers constant-factor neighborhood updates.
- Worst-case heap-method complexity is O(inv(P) log N) = O(N^2 log N), with O(N) memory for the array and heap.
- The Fenwick-tree method processes input values once, counts smaller preceding elements, and aggregates movement costs using a closed-form arithmetic progression.
- O(N log N) time and O(N) space result from one logarithmic query and update per element, avoiding explicit adjacent-swap simulation.