Source-linked AI summary
Triplet-Center Loss for Multi-View 3D Object Retrieval
Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai, Xiang Bai
TL;DR
Deep metric learning for 3D object retrieval has been relatively neglected compared with classification-focused approaches. The paper studies triplet and center losses, proposes triplet-center loss, and reports significant improvements over state-of-the-art retrieval methods across 3D and sketch-based shape retrieval tasks.
Problem
Deep learning research on 3D data has focused mainly on classification, while metric-learning approaches for 3D object retrieval have received limited attention.
Method
The paper proposes triplet-center loss, which learns class centers and jointly reduces intra-class variation while increasing inter-class separability within an MVCNN-based retrieval framework.
Results
The proposed losses significantly outperform state-of-the-art approaches on 3D object retrieval benchmarks and are effective on sketch-based 3D shape retrieval; on ModelNet40, TCL with softmax reaches 88.0% mAP, improving MVCNN by nearly 7%.
Takeaways & Limitations
Triplet-center loss produces more robust and discriminative embeddings that are appropriate for retrieval across both view-based 3D and cross-domain sketch-based tasks.
Takeaways & Limitations
Triplet loss is difficult to train because its triplets grow cubically with dataset size and its performance depends on time-consuming hard-triplet mining.
Abstract
from arXiv · showhide
Most existing 3D object recognition algorithms focus on leveraging the strong discriminative power of deep learning models with softmax loss for the classification of 3D data, while learning discriminative features with deep metric learning for 3D object retrieval is more or less neglected. In the paper, we study variants of deep metric learning losses for 3D object retrieval, which did not receive enough attention from this area. First , two kinds of representative losses, triplet loss and center loss, are introduced which could learn more discriminative features than traditional classification loss. Then, we propose a novel loss named triplet-center loss, which can further enhance the discriminative power of the features. The proposed triplet-center loss learns a center for each class and requires that the distances between samples and centers from the same class are closer than those from different classes. Extensive experimental results on two popular 3D object retrieval benchmarks and two widely-adopted sketch-based 3D shape retrieval benchmarks consistently demonstrate the effectiveness of our proposed loss, and significant improvements have been achieved compared with the state-of-the-arts.
1. Introduction
The paper addresses the limited use of deep metric learning for 3D object retrieval by introducing triplet-center loss within an MVCNN-based framework. TCL combines class-center attraction with inter-class separation, achieving improved retrieval performance.
- Deep learning methods for 3D data have primarily emphasized classification, while metric-learning approaches for large-scale 3D retrieval have received limited attention.
- The paper introduces triplet and center losses to learn distance-oriented features for 3D object retrieval.
- The retrieval framework uses MVCNN for view-based 3D representations, with TCL as supervision and optional softmax loss for performance boosting.
- The proposed losses, especially TCL, outperform state-of-the-art approaches on benchmarks including ModelNet40 and ShapeNet Core55.
- TCL learns one center per class, pulling samples toward their class center while pushing them away from centers of other classes.
2. Related work
Related work contrasts model-based and view-based 3D retrieval and notes that deep metric learning, although established in image retrieval, has been less used for 3D retrieval. The paper applies these losses and proposes TCL as a new alternative.
- 3D retrieval methods are broadly divided into model-based approaches that process 3D representations and view-based approaches that process rendered 2D projections.
- View-based methods: View-based methods assemble CNN-extracted features from one or multiple rendered views into compact shape descriptors for retrieval or classification.
- Deep metric learning: Deep metric learning losses such as triplet loss seek embeddings where same-category instances are closer than instances from different categories.
- Deep metric learning: The paper introduces triplet and center losses to 3D object retrieval, adapting metric-learning ideas previously used in image retrieval and re-identification.
- Triplet-center loss: TCL is proposed as a distinct loss whose reported improvements span two 3D shape-retrieval and two sketch-based retrieval benchmarks.
3. Proposed method
The proposed triplet-center loss (TCL) combines class-center modeling with triplet-style separation to learn compact, discriminative 3D shape embeddings efficiently. It can operate independently of softmax loss and avoids the costly triplet construction and hard-sample mining required by conventional triplet loss.
- Motivation: Softmax-trained features may separate classes without ensuring intra-class compactness, motivating metric-learning objectives for 3D shape retrieval.Softmax focuses on decision boundaries, whereas metric learning directly structures the embedding space.
- Triplet-center loss: TCL learns one center per class and requires each sample to be closer to its own center than to the nearest negative center by margin m.This explicitly combines intra-class compactness with inter-class separability using squared Euclidean distances.
- Joint supervision: TCL can supervise neural-network training independently of softmax loss, while combining TCL with softmax can produce more discriminative and robust embeddings.The trade-off between the two losses is controlled by the hyper-parameter λ; softmax can guide the randomly initialized, mini-batch-updated centers.
- Motivation: Triplet loss compares same-class and different-class samples, but its O(N^3) triplet count and hard-triplet mining make training time-consuming.The paper identifies triplet construction complexity and the difficulty of defining good hard triplets as practical limitations.
- Triplet-center loss: Unlike triplet loss, TCL forms N sample-center triplets instead of O(N^3) sample triplets, avoiding complex construction and hard-sample mining.Each TCL triplet contains a sample, its corresponding class center, and the nearest negative center.
- Comparison with center loss: Compared with center loss, TCL explicitly models inter-class separability in addition to reducing intra-class variation.Center loss requires joint softmax supervision for feasible learning, whereas TCL does not.
4. Experiments
The experiments evaluate TCL for within-domain and cross-domain 3D shape retrieval, comparing losses, retrieval methods, hyperparameters, and learned representations. TCL-based models achieve strong results across ModelNet40, ShapeNet Core55, and sketch-based benchmarks.
- Experimental scope: Experiments cover generic 3D shape retrieval, where queries and database examples are 3D models, and sketch-based retrieval, where queries are 2D sketches.The evaluated generic benchmarks are ModelNet40 and ShapeNet Core55; sketch-based evaluation uses SHREC’13 and SHREC’14.
- Parameter influence: TCL is robust to λ from 0.01 to 10, whereas its margin m is more sensitive; the default settings are m=5 and λ=0.01.With λ=0, softmax-only training achieves nearly 80.0% mAP, while adding TCL improves mAP by 7∼8%.
- Loss comparison: TCL and softmax loss with TCL perform best among the compared losses on ModelNet40, reaching 86.7% and 88.0% mAP respectively.Softmax loss with center loss reaches 83.5% mAP, while triplet loss is merely comparable to softmax loss.
- Generic retrieval results: 88.0% mAP is achieved by TCL+softmax loss on ModelNet40, improving nearly 7% over MVCNN and 3% over DLAN.The method also outperforms GIFT by nearly 6% in AUC and mAP respectively.
- Generic retrieval results: TCL+softmax loss outperforms MVCNN by nearly 10% in micro-averaged mAP and 12% in macro-averaged mAP on perturbed ShapeNet Core55.The results are reported as improvements on the challenging perturbed test set.
- Sketch-based retrieval results: 79.8% mAP is achieved on SHREC’13 by TCL+softmax loss without the sophisticated barycentric representations used by LWBR.LWBR obtains 75.2% mAP on SHREC’13, while the cited comparison identifies DCML as another strong method.
5. Conclusion
The paper proposes triplet-center loss for view-based 3D shape retrieval, combining intra-class compactness and inter-class separation. It reports effectiveness on both generic 3D retrieval and sketch-based cross-domain retrieval.
- Triplet-center loss combines triplet loss and center loss for view-based 3D shape retrieval.
- TCL minimizes intra-class variance while maximizing inter-class variance simultaneously.
- The resulting embeddings are more robust and discriminative, making them more appropriate for retrieval.
- TCL demonstrates effectiveness on generic 3D shape retrieval and sketch-based shape retrieval.