Source-linked AI summary
Partition-Invariant Tuning for 3D Scene Understanding
Hongqiang Lin, Tianle Wang, Shuiwang Li, Dongxu Zhang, Yiding Sun, Zihao Guo, Dongfu Yin
TL;DR
Existing PEFT methods largely focus on object-level point clouds, while large-scale scene serialization introduces partition variations that can entangle task adaptation with partition-specific perturbations. PointPiT combines scene-aware structural adaptation with gradient subspace optimization to address this mismatch. Across scene-level benchmarks, it achieves state-of-the-art performance among PEFT methods and competitive or better performance than full fine-tuning with less than 1% of the backbone's parameters.
Problem
Existing PEFT methods largely focus on object-level point clouds, while serialization-induced partition variations in large scenes can entangle task adaptation with partition-specific perturbations.
Source: §1 · p. 1Method
PointPiT combines SSA, which uses local geometric patterns and global scene context to mitigate representation shifts, with GSO, which favors stable update directions.
Source: §1 · p. 1, §2.1 · p. 2Results
PointPiT achieves state-of-the-art performance among PEFT methods and competitive or better performance than full fine-tuning with less than 1% of the backbone's parameters.
Source: §4 · p. 4Takeaways & Limitations
As an early exploration of scene-level PEFT, PointPiT is proposed as a strong baseline and a source of insights for future efficient 3D scene adaptation research.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Scene-level point cloud understanding remains challenging due to diverse geometries and spatial layouts. While pre-trained 3D point cloud foundation models (PFMs) offer strong transferability, full fine-tuning (FFT) incurs substantial computational and storage costs. Parameter-efficient fine-tuning (PEFT) provides a promising alternative, but existing PEFT methods largely focus on object-level point clouds and overlook serialization-induced partition variations in large-scale scenes. To address this issue, we propose PointPiT, a partition-invariant tuning framework for scene-level point clouds. Specifically, a Scene-aware Structural Adapter (SSA) integrates local geometric patterns with global scene context to mitigate partition-induced representation shifts. Moreover, Gradient Subspace Optimization (GSO) selects informative and partition-stable update directions, suppressing partition-dependent variations during optimization. Extensive experiments across multiple scene-level benchmarks demonstrate that PointPiT achieves competitive or even superior performance to full fine-tuning with less than 1% of backbone's parameters, while achieving consistent state-of-the-art performance among representative PEFT methods.
1. INTRODUCTION
Scene-level PEFT is underexplored, and serialization-induced partitions can divert limited adaptation capacity from task semantics. PointPiT addresses this with scene-aware representation adaptation and partition-stable gradient updates, achieving strong benchmark performance with far fewer trainable parameters.
1. INTRODUCTION
- Serialization-based grouping gives identical physical points different local contexts, risking the use of limited PEFT capacity on partition-specific artifacts rather than task-relevant semantics.The paper identifies the mismatch between serialization-induced partition variation and parameter-efficient adaptation as a challenge for large-scale scenes.Source: §1 · p. 1
- FFT can largely reconcile partition-induced discrepancies, linear probing inherits frozen-representation sensitivity, and LoRA lacks an explicit way to distinguish task-relevant variation from partition-specific perturbations.The paper says this entanglement limits effective exploitation of pretrained scene representations by existing PEFT methods.Source: §1 · p. 1
- PointPiT combines SSA, which uses global scene context to mitigate representation shifts, with GSO, which favors more stable update directions during optimization.The framework addresses partition sensitivity through forward representation adaptation and backward optimization.Source: §1 · p. 1
- With more than 99% fewer trainable parameters than full fine-tuning, PointPiT maintains comparable or better performance and achieves state-of-the-art performance among PEFT methods.The reported results cover multiple scene-level benchmarks.Source: §1 · p. 1
2. METHODOLOGY
PointPiT adapts scene-level point clouds with complementary forward representation adaptation and backward gradient optimization. SSA combines local geometry with global context, while GSO favors task-relevant, partition-stable update directions.
2.1. Overview
- PointPiT tokenizes each scene by serializing points, grouping the sequence, and projecting groups into feature tokens before processing them with hierarchical backbone blocks augmented by SSA.Source: §2.1 · p. 2
2.2. Scene-aware Structural Adapter
- SSA adapts pretrained geometric features and combines them with global scene context through attention-weighted aggregation and a learnable point-wise gate.Its structural pathway is intended to mitigate partition-induced representation shifts.Source: §2.2 · p. 2
2.3. Gradient Subspace Optimization
- GSO uses gradients from multiple scenes and valid partitions to seek update directions that balance informativeness with lower partition-dependent variation.The subspace dimension is capped at q.Source: §2.3 · p. 2
- GSO recovers the subspace from an eigenproblem of dimension at most BK rather than Dl, then projects gradients onto it without changing the trainable parameter set.The resulting updates reduce sensitivity to partition-dependent gradient variations.Source: §2.3 · pp. 2–3
3. EXPERIMENTS
Across diverse scene-level benchmarks, PointPiT outperforms representative PEFT methods and is competitive with or exceeds full fine-tuning, while using fewer than 1% of backbone parameters. Its ablations show complementary gains from SSA and GSO, alongside stable results across serialization configurations and moderate hyperparameter settings.
3.1. Experimental Setup
- Experiments use PTv3 with pretrained Sonata weights, frozen backbone parameters, and trainable SSA modules and task heads; evaluation reports segmentation metrics and trainable parameters.Benchmarks are ScanNet, ScanNet++, and S3DIS; the compared PEFT methods and FFT baseline share the same Sonata backbone without the decoder.Source: §3.1 · pp. 3–4
3.2. Main Results
- PointPiT consistently outperforms representative PEFT methods across diverse scene-level benchmarks and is competitive with or exceeds full fine-tuning on ScanNet Val.Qualitatively, it also produces more accurate and consistent predictions than FFT in complex scene regions.Source: §3.2 · p. 4, Figure 3 · p. 4
3.3. Ablation Study
- SSA and GSO contribute complementarily: GSO yields a modest improvement without SSA and further improves performance when paired with SSA.Table 3 reports the ablation of PointPiT components.Source: §3.3 · p. 4, Table 3 · p. 4
- PointPiT maintains consistent performance across test-time serialization configurations on identical scenes, while r = 64, q = 32, and λ = 0.5 achieve the best performance.Smaller r or q limits adaptation capacity or informative updates, while excessive values yield diminishing returns; overly strong partition regularization may suppress useful gradients.Source: §3.3 · p. 4, Table 4 · p. 4, Table 5 · p. 4
4. CONCLUSION
PointPiT addresses the mismatch between serialization-induced partition variation and parameter-efficient adaptation through scene-aware structural adaptation and gradient subspace optimization. It achieves state-of-the-art performance among PEFT methods and competitive or better performance than full fine-tuning with less than 1% of backbone parameters.
- PointPiT achieves consistent state-of-the-art performance among PEFT methods and competitive or even superior performance to full fine-tuning with less than 1% of backbone parameters.Source: §4 · p. 4
- The framework addresses the mismatch between serialization-induced partition variations and parameter-efficient adaptation using scene-aware structural adaptation and gradient subspace optimization.Source: §4 · p. 4
- The authors present PointPiT as an early exploration of scene-level PEFT and expect it to serve as a strong baseline and provide insights for future research.Source: §4 · p. 4