Source-linked AI summary
PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learning
Yankai Wang, Yiding Sun, Qirui Wang, Pengbo Li, Chaoyi Lu, Dongxu Zhang
TL;DR
PointRFT addresses whether reinforcement learning can improve 3D point cloud fine-tuning, where existing evidence is limited and SFT can overfit scarce datasets. It replaces vanilla SFT with explicit accuracy and dispersion rewards, and consistently outperforms SFT, with the hybrid Pretraining-SFT-RFT paradigm achieving state-of-the-art base-to-new performance.
Problem
RL-based methods remain underexplored for 3D point cloud fine-tuning, while SFT is prone to overfitting on limited point cloud datasets.
Method
PointRFT replaces vanilla SFT with RL-based fine-tuning using explicit accuracy and dispersion rewards for point cloud perception.
Results
PointRFT consistently outperforms vanilla SFT across few-shot benchmarks, while Pretraining-SFT-RFT achieves gains of +10.51% over vanilla SFT baselines and state-of-the-art base-to-new performance.
Takeaways & Limitations
The hybrid Pretraining-SFT-RFT paradigm substantially unleashes point cloud foundation models’ representational capacity under data-scarce conditions.
Abstract
from arXiv · showhide
Understanding spatial dynamics and semantics in point cloud is fundamental for comprehensive 3D comprehension. While reinforcement learning algorithms such as Group Relative Policy Optimization (GRPO) have recently achieved remarkable breakthroughs in large language models by incentivizing reasoning capabilities through strategic reward design, their potential remains largely unexplored in the 3D perception domain. This naturally raises a pivotal question: Can RL-based methods effectively empower 3D point cloud fine-tuning? In this paper, we propose PointRFT, the first reinforcement fine-tuning paradigm tailored specifically for point cloud representation learning. We select three prevalent 3D foundation models and devise specialized accuracy reward and dispersion reward functions to stabilize training and mitigate distribution shifts. Through comprehensive few-shot classification experiments comparing distinct training paradigms, we demonstrate that PointRFT consistently outperforms vanilla supervised fine-tuning (SFT) across diverse benchmarks. Furthermore, when organically integrated into a hybrid Pretraining-SFT-RFT paradigm, the representational capacity of point cloud foundation models is substantially unleashed, achieving state-of-the-art performance particularly under data-scarce scenarios.
I. INTRODUCTION
PointRFT addresses limited cross-task and cross-domain generalization in point cloud fine-tuning by replacing vanilla SFT with explicit-reward reinforcement fine-tuning. Across few-shot classification experiments, it mitigates overfitting and forgetting, while the hybrid Pretraining-SFT-RFT paradigm achieves state-of-the-art base-to-new performance.
- Conventional Pretraining-SFT pipelines have constrained cross-task and cross-domain generalization and often require dedicated models for each dataset or task.
- PointRFT introduces reusable Pre-S, Pre-R, and Pre-S-R paradigms and benchmarks them across multiple few-shot classification tasks using three 3D foundation models.
- RL-based point cloud fine-tuning remains underexplored because objective physical ground truth conflicts with GRPO’s implicit exploratory feedback, while SFT overfits limited datasets.
- PointRFT replaces vanilla SFT with explicit, quantifiable accuracy and dispersion rewards to mitigate distribution shift during 3D fine-tuning.
- The hybrid Pretraining-SFT-RFT paradigm fully unleashes foundation-model capacity and attains state-of-the-art performance in the base-to-new scenario.
II. RELATED WORK
Prior point cloud representation learning relies mainly on contrastive or generative pretraining, followed by SFT that can mimic predefined answers rather than discover task-specific solutions. PointRFT shifts emphasis toward reward design tailored to 3D vision tasks.
- Point cloud representation pretraining is principally divided into contrastive methods, which align augmentations and separate samples, and generative methods, which reconstruct point clouds.
- After pretraining, SFT supports straightforward incremental training but is described as mimicking predefined answers rather than discovering what works best.
- PointRFT shifts training from SFT data scaling toward strategic reward functions tailored to specific 3D vision tasks.
B. RFT Paradigm in MLLM Perception
PointRFT adapts GRPO-style reinforcement fine-tuning to point cloud perception, using policy optimization and an epoch-wise old-model reference to stabilize updates. It supports combining SFT and RFT because both use the same labeled inputs.
- RFT Paradigm: PointRFT applies GRPO-style reinforcement fine-tuning to point cloud perception, extending RL-based optimization beyond image and video reasoning.The method is designed specifically for point cloud representation learning.
- RFT Paradigm: The method optimizes the point cloud model with reward-based policy gradients rather than SFT’s token-level loss.GRPO uses policy gradients computed from reward loss to encourage exploration of a larger solution space.
- Training Framework: PointRFT uses the pre-update base model as a detached reference policy and the updated model as the actively optimized policy.This reference-policy design is used to stabilize reinforcement fine-tuning.
- Training Framework: At each epoch, PointRFT updates the old model and incorporates it into the loss without gradient backflow.The pseudocode initializes an old model and copies the current policy into it before minibatch processing.
- Training Framework: SFT and RFT can be sequentially combined because they share labeled data and inputs, enabling an SFT-then-RFT training scheme.The framework treats the two paradigms as compatible rather than conflicting.
- Optimization: GRPO removes the critic model and estimates relative advantages, yielding higher computational efficiency.The objective uses group rewards and intra-group advantage estimation with clipping parameter ε.
B. Reward Design
PointRFT designs separate accuracy and dispersion rewards for point cloud representation learning. The reward construction targets correct predictions while preserving a sufficiently dispersed embedding distribution, with training weights adjusted across datasets.
- Reward Components: PointRFT decomposes its reward into accuracy and dispersion terms to support point cloud representation learning.The two components are summed into the final reward, with weights controlled by hyperparameters a and b.
- Accuracy Reward: The accuracy reward uses the softmax class-probability vector as a surrogate for varied responses unavailable in point cloud data.Point clouds provide data–label pairs but lack the chain-of-thought response groups used in language-model RL.
- Accuracy Reward: Correct predictions receive the accuracy reward, while per-sample rewards are clamped to preserve training stability.The reward is scaled so its expectation remains 1 under the stated formulation.
- Dispersion Reward: The dispersion reward addresses the requirement that embeddings remain globally dispersed in addition to forming cohesive same-class clusters.The paper motivates dispersion because cross-entropy alone leaves it unattended and can shrink features toward class centroids.
- Dispersion Reward: Removing the dispersion reward collapses PointRFT toward vanilla SFT, linking the dispersion component to the distinct reinforcement objective.The paper describes dispersion as a continuous, differentiable bridge from discrete labels to a learnable reward landscape.
- Reward Calibration: The reward weights a and b regulate training stability across datasets with varying label counts.The supplied few-shot evaluations cover ScanObjectNN, ModelNet40, and ShapeNetCore under multiple shot settings.
C. Training Paradigms
The paper frames SFT and RFT as complementary training regimes and introduces a meta-objective that switches among supervised, reinforcement, and hybrid optimization.
- Motivation: The 3D domain remains largely uncharted in comparisons of SFT and RFT for reshaping visual representations.
- Training paradigms: Φ ∈ {SFT, RL, Hybrid} switches the training regime, selecting supervised, reinforcement, or hybrid optimization.
A. Experiment Settings
The experiments use three pretrained point cloud foundation models and report computational cost across selected few-shot settings using GFLOPs and per-epoch training time.
- Models: PointRFT is evaluated on Point-MAE, HyperPoint, and ReCon, each pretrained on ShapeNet for 300 epochs.
- Computational cost: The computational comparison reports GFLOPs and training time in seconds for each epoch.
- Few-shot settings: The selected settings are 10-way 10-shot, 10-way 1-shot, and 5-way 1-shot.
B. Classification: ModelNet40
ModelNet40 evaluates few-shot 3D object recognition, and its comparison shows that PointRFT offers no advantage over vanilla SFT when support examples are ample.
- Dataset: ModelNet40 contains 40 CAD-based categories for few-shot 3D object recognition.
- Comparison results: PointRFT marginally underperforms vanilla SFT in ModelNet40 10-way 10-shot and 10-way 20-shot scenarios.
- Comparison results: Multimodal pre-training consistently improves accuracy over unimodal pre-training, with a wider advantage as support sets grow.
C. Classification: ShapeNetCore
ShapeNetCore broadens evaluation to 70 classes under sparse-support classification, where PointRFT outperforms competing paradigms and hybrid Pre-S-R adds substantial gains over vanilla SFT.
- Dataset: ShapeNetCore extends evaluation to 70 CAD classes and tests categorization with small fixed support sets.
- Comparison results: PointRFT surpasses all competing paradigms in every evaluated 10-way-1-shot and 10-way-5-shot regime.
- Comparison results: +10.51%, +13.30%, and +17.75% gains over vanilla SFT baselines are achieved with the Pre-S-R enhancement.
D. Classification: ScanObjectNN
On ScanObjectNN, PointRFT is evaluated under real-world distribution shifts and consistently outperforms SFT across few-shot splits, while hybrid Pre-S-R further improves performance. Ablations identify dispersion reward and moderate clipping as important training choices, with modest computational overhead offset by faster convergence and higher accuracy.
- Dataset: ScanObjectNN tests few-shot generalization on real-world scans affected by background clutter, partial occlusion, and unknown scale.The dataset contains 15 annotated object categories.
- Comparison Results: Pre-S-R further amplifies PointRFT's advantage under 5-way-5-shot evaluation.
- Hyperparameters: Disabling dispersion reward (b = 0) hurts accuracy, supporting its role as a soft regularizer.
- Hyperparameters: Large ε destabilizes optimization, while a moderate clip restrains effective parameter drift in low-data regimes.The passage attributes this instability to sparse RL updates and associated over-fitting.
- Computational Burden: PointRFT requires slightly more resources than SFT, but faster convergence and superior accuracy offset the additional computational burden.The computational footprint is measured by per-epoch training time and GFLOPs.