Source-linked AI summary
RouteGraph-Mona: Confusion-Aware Routing Fine-Tuning for Mineral Image Classification
Jierui Li, Zhiyuan Qi, Hao Zhu, Yufan Liu, Jixian Liu, Shaojie Jiang, Jianda Wang, Yaqi Liu, Xiaotong Li, Wei Wang
TL;DR
Mineral image classification is challenging because within-class appearances vary while different categories can look similar, and Mona’s static multi-scale aggregation cannot explicitly adapt routing to each image or category confusion. RouteGraph-Mona replaces that aggregation with sample-adaptive routing, then regularizes route signatures using class-wise anchors and confusion-weighted margins. Across three datasets and two visual backbones, it improves mean accuracy over Mona and remains competitive with representative baselines, while statistical support is limited by the small number of seeds.
Problem
Mineral image classification must handle substantial intra-class appearance variation and high inter-class visual similarity, while Mona’s static multi-scale aggregation limits sample-specific scale adaptation and explicit confusion modeling.
Method
RouteGraph-Mona replaces Mona’s static aggregation with sample-adaptive routing and regularizes compact route signatures using class-wise route anchors and a confusion-weighted route margin.
Results
RouteGraph-Mona improves mean accuracy over Mona across three mineral image datasets and two visual backbones while achieving competitive performance against representative fine-tuning and mineral classification baselines.
Takeaways & Limitations
The results support route-space regularization as a useful complement to feature adaptation for mineral image classification.
Takeaways & Limitations
The paired t-test did not reach statistical significance for accuracy (p = 0.151), so the findings are treated as a consistent trend rather than standalone significance evidence.
Abstract
from arXiv · showhide
Mineral image classification is important for geological exploration and resource development, but it remains challenging due to substantial intra-class variations in appearance and high inter-class visual similarity. Multi-cognitive Visual Adapter (Mona) is a vision-oriented parameter-efficient adapter that adapts pre-trained visual models by tuning only a few parameters. However, Mona statically aggregates responses from multiple scales, limiting its ability to accommodate sample-specific scale preferences and model confusion among visually similar mineral categories. To address this issue, we propose \textbf{RouteGraph-Mona}, a lightweight route-space regularization method built on Mona. Specifically, we replace Mona's static multi-scale aggregation with sample-adaptive routing. The resulting branch-selection behavior defines a compact routing space that captures each image's scale preferences. We then regularize the resulting routing signatures with class-wise route anchors and confusion-weighted margins. The route anchors encourage class-consistent routing patterns, while the margins promote greater separation between visually similar categories in the routing space. Experiments on three public mineral image datasets with two visual backbones show that RouteGraph-Mona consistently outperforms Mona in mean accuracy and remains competitive with representative fine-tuning methods and mineral image classification baselines.
2 Tsinghua University, Beijing, China
The paper’s keywords identify its focus on mineral image classification, parameter-efficient fine-tuning, adaptive routing, route-space regularization, and the Mona adapter.
- The paper centers on mineral image classification using parameter-efficient fine-tuning, adaptive routing, route-space regularization, and the Mona adapter.
1 Introduction
Mineral image classification supports geological applications but is difficult because mineral appearances vary within classes and overlap across classes. RouteGraph-Mona addresses these challenges by replacing Mona’s static multi-scale fusion with adaptive routing and regularizing compact route signatures using class anchors and confusion-aware margins.
- Motivation: Mineral image classification supports geological exploration, resource development, and resource evaluation.Traditional identification uses expert knowledge and specialized instruments, while image-based classification is easier to deploy.
- Motivation: Mineral images show substantial intra-class variation and high inter-class visual similarity, complicating classification.Samples within a category can differ in shape, color, and texture, while different categories may share local visual cues.
- Motivation: Mona’s static averaging may miss image-specific scale preferences and does not explicitly exploit category-level confusion.This limitation motivates adaptive routing and route-space regularization for mineral images.
- Proposed Method: RouteGraph-Mona replaces static multi-scale aggregation with sample-adaptive routing and summarizes routing weights into compact route signatures.The signatures represent branch-selection behavior and image-specific preferences over receptive-field scales rather than visual appearance details.
- Proposed Method: Class-wise route anchors and an online confusion graph regularize routing signatures to encourage category-consistent patterns and separate confusing categories.The confusion graph assigns larger route-margin penalties to visually confusing category pairs.
- Results: Experiments on three public datasets and two visual backbones show improved mean accuracy over Mona and competitive performance against representative baselines.The method’s reported evaluation scope includes fine-tuning and mineral image classification comparisons.
2 Related Work
Prior mineral image classification progressed from handcrafted descriptors and traditional classifiers to CNN- and Transformer-based learned representations. Related work also establishes pre-training and parameter-efficient fine-tuning as ways to adapt large visual models while reducing the cost of full fine-tuning.
- Mineral Image Classification: Early mineral image classifiers used handcrafted color, texture, and shape descriptors with traditional machine-learning classifiers.Deep learning later enabled representations to be learned directly from mineral images.
- Mineral Image Classification: CNN- and Transformer-based models have increasingly been used for mineral image recognition to learn discriminative visual representations.Recent work also explored DenseNet variants and YOLOv8-CLS.
- Parameter-Efficient Fine-Tuning: Pre-training and fine-tuning transfer visual knowledge from large-scale models to downstream recognition tasks.This paradigm addresses the difficulty of collecting and labeling large mineral image datasets.
- Parameter-Efficient Fine-Tuning: Parameter-efficient fine-tuning reduces computational and storage costs by updating lightweight modules instead of all backbone parameters.Examples include adapters, LoRA, visual prompt tuning, and other efficient adaptation modules.
- Parameter-Efficient Fine-Tuning: Mona extends vision-oriented parameter-efficient adaptation with filtering structures and scaled normalization for visual transferability.
3 Method
RouteGraph-Mona adapts pretrained visual backbones through sample-adaptive scale routing and compact route signatures, then regularizes routing with class-wise anchors and confusion-aware margins. The method updates lightweight task-specific components while retaining standard classification inference.
- Routing Signature Generation: RouteGraph-Mona replaces Mona’s static multi-scale averaging with sample-adaptive routing and summarizes branch-selection weights as compact route signatures.The signatures characterize image-specific preferences across Mona modules rather than visual feature embeddings.
- Parameter-efficient Fine-tuning Protocol: Training freezes the main backbone and updates Mona modules, scale routers, route anchors, and the classification head; inference requires neither labels nor graph updates.The adaptive scale router remains active and predictions use standard classification logits.
- Routing Signature Generation: Each Mona module uses three depthwise convolution branches with 3×3, 5×5, and 7×7 kernels, whose responses are fused by the router.The router uses global context and mean-pooled projected tokens to predict scale weights.
- Class-wise Route Anchors: Class-wise route anchors provide compact route prototypes that encourage samples to align with category-level routing tendencies.The main experiments use K = 1, giving one route prototype per mineral category.
- Online Confusion-guided RouteGraph Regularization: An online confusion graph estimates off-diagonal class-confusion tendencies from detached predictions and weights route-margin penalties toward difficult category pairs.Larger Guv values indicate stronger confusion from class u to class v, while the graph is updated with momentum.
- Training Objective and Inference: The final objective combines standard classification loss with anchor and graph regularization, controlled by λm.The anchor loss guides class-level routing tendencies, while the graph loss enlarges separation between confusing classes.
4 Experiments
The experiments evaluate RouteGraph-Mona on three cleaned mineral datasets using two pretrained visual backbones, standardized splits, metrics, and repeated runs. Comparisons include general fine-tuning approaches and representative mineral classification models.
- Compared methods: RouteGraph-Mona is compared with linear probing, full fine-tuning, adapter tuning, LoRA, Mona, and four mineral classification baselines.The mineral baselines include DenseNet-201, EfficientNet-B0, YOLOv8-cls, and SwinMin.
- Experimental protocol: Experiments use ViT-B/16 and Swin-B backbones with exact-hash grouping, shared splits, and seeds 0, 666, and 2026.The protocol prevents duplicate leakage and repeats all experiments across three seeds.
- Evaluation: Accuracy is the main metric, with each run evaluated using the checkpoint having the lowest validation loss.Fine-tuning baselines share the same splits, input resolution, augmentation, optimizer, early-stopping criterion, seeds, and checkpoint-selection rule.
- Implementation: RouteGraph-Mona uses one route anchor per class with λm = 0.03, mg = 0.05, and β = 0.95.Unless otherwise specified, experiments use 224 × 224 inputs, AdamW, weight decay 1 × 10−4, and early-stopping patience 15.
4.4 Main Comparison with Fine-Tuning Methods
RouteGraph-Mona improves mean accuracy over Mona across all three datasets and both backbones, while also improving class-balanced metrics in the main Swin-B MinetV2 setting. The results support a consistent trend, although the paired accuracy test is not statistically significant with three seeds.
- Overall comparison: RouteGraph-Mona improves mean accuracy over Mona across all three datasets under both ViT-B/16 and Swin-B.The gains are reported for every dataset-backbone combination.
- Overall comparison: 1.18, 0.78, and 2.40 percentage points are the ViT-B/16 gains over Mona on Minet, MinetV2, and MineralPhotos.Under Swin-B, the corresponding gains are 1.68, 1.42, and 2.59 percentage points.
- Baseline comparison: RouteGraph-Mona outperforms linear probing and provides stable gains over adapter tuning and LoRA while remaining competitive with full fine-tuning.The main pretrained backbone remains frozen for RouteGraph-Mona.
- Class-balanced evaluation: 1.42 percentage points is the Swin-B MinetV2 accuracy gain over Mona, alongside 1.64 and 1.59 point gains in macro-F1 and mA.RouteGraph-Mona reaches 79.64±0.98 accuracy, 79.00±0.84 macro-F1, and 78.97±1.12 mA.
- Statistical analysis: The paired t-test does not reach statistical significance for accuracy, with p = 0.151 across the small number of seeds.The authors treat the results as supporting evidence of a consistent trend rather than a standalone significance claim.
4.6 Accuracy–Efficiency Trade-off
RouteGraph-Mona achieves the best reported accuracy among the compared mineral classification baselines on all three datasets. Its trade-off profile combines low trainable-parameter ratios with additional training-time memory from routing and route-space regularization.
- Accuracy–efficiency trade-off: RouteGraph-Mona improves accuracy over Mona while retaining a low trainable-parameter ratio under both visual backbones.The method updates lightweight adaptation components, routers, route anchors, and the classification head without large-scale backbone updates.
- Accuracy–efficiency trade-off: Introducing routing and route-space regularization incurs additional training-time memory compared with Mona while preserving parameter-efficient updates.During inference, predictions use classification logits with the scale router active and no labels, graph updates, or auxiliary losses.
- Mineral classification baselines: RouteGraph-Mona achieves the best accuracy among the compared mineral classification baselines on Minet, MinetV2, and MineralPhotos.The comparison includes DenseNet-201, EfficientNet-B0, YOLOv8-cls, and SwinMin.
- Mineral classification baselines: 93.94%, 79.64%, and 81.09% are RouteGraph-Mona’s accuracies on Minet, MinetV2, and MineralPhotos, respectively.Compared with SwinMin, the gains are 2.53, 0.63, and 0.77 percentage points.
4.8 Ablation Study
The ablation study separates the contributions of adaptive routing, route anchors, and confusion-guided margins, then compares route-space regularization with generic feature-space constraints. Their combination performs best, while route-space application outperforms matched feature-space regularization.
- Component ablations: Adaptive routing provides small but consistent gains on Minet and MinetV2, establishing the basis for route-space modeling.Replacing static multi-scale aggregation with sample-adaptive routing is the first ablation improvement.
- Component ablations: The confusion-guided route margin increases MinetV2 accuracy from 78.38% to 78.77%, where inter-class visual ambiguity is stronger.Route anchors slightly improve MinetV2 accuracy and reduce Minet variability, while the margin supplies the larger MinetV2 gain.
- Combined model: 93.94% on Minet and 79.64% on MinetV2 are achieved when adaptive routing, route anchors, and the confusion-guided margin are combined.The components are described as complementary: routing defines the space, anchors provide category references, and the margin emphasizes confusing-category separation.
- Feature-space comparison: Generic feature-space constraints are inconsistent: Center Loss decreases accuracy on both datasets, while supervised contrastive learning improves MinetV2 but decreases Minet.These comparisons motivate evaluating the regularization space rather than assuming generic feature-level constraints transfer reliably.
- Feature-space comparison: Feature Anchor+Graph performs worse than RouteGraph-Mona on both datasets, isolating route-space regularization as the stronger matched design.The matched variant uses the same anchors, confusion-weighted margin, and online graph, but applies them to visual feature embeddings.
4.10 Effect of Confusion Graph Construction
The graph-construction study compares no-graph, uniform, random, and online confusion weighting on MinetV2 with Swin-B. Prediction-derived confusion weighting gives the best and most stable result among the evaluated strategies.
- Graph strategies: 78.46±1.71% accuracy in the no-graph setting provides the comparison point for graph-based weighting.The online confusion graph improves this setting by 1.18 percentage points.
- Graph strategies: Uniform weighting lowers accuracy to 78.30±2.86%, whereas random weighting reaches 78.69% but raises standard deviation to 2.61.Equal weighting lacks a stable benefit, while random weighting lacks semantic correspondence to actual category confusion.
- Graph strategies: 79.64±0.98% accuracy is achieved by the online confusion graph, improving the no-graph setting by 1.18 percentage points.It also has the lowest variance among the evaluated graph strategies.
- Graph strategies: The online confusion graph assigns stronger route-margin weighting to confusing category pairs using prediction-derived confusion.This strategy is presented as more effective and stable than fixed uniform or random weighting.
4.11 Hyperparameter Sensitivity
Hyperparameter sensitivity evaluates route anchors, regularization weight, and route margin on MinetV2 with Swin-B. The default settings are stable, while increasing anchor count or regularization strength does not consistently improve performance.
- Route anchors: A single route anchor per class already provides strong mean accuracy, while increasing K does not consistently improve performance and drops noticeably at K = 5.The study examines the number of route anchors K as one of three sensitivity factors.
- Regularization weight: λm = 0.03 achieves competitive performance, while the other evaluated regularization weights produce only moderate fluctuations.Increasing regularization strength does not consistently yield additional gains.
- Route margin: mg = 0.05 performs well, whereas increasing the route margin to 0.07 decreases performance.The route margin is the third sensitivity factor evaluated in Figure 4.
- Route-space analysis: Figure 5 reports confusion change and route-distance distributions to examine RouteGraph-Mona’s route-space geometry.Negative confusion-change values indicate reduced off-diagonal confusion errors.
- Route-space analysis: 15.3% lower total off-diagonal confusion error accompanies an increase in confusing inter-class route distance from 0.862 × 10^-2 to 1.206 × 10^-2.The separation gap rises from 0.076 × 10^-2 to 0.144 × 10^-2, becoming 1.90× as large as Mona’s gap.
5 Conclusion
RouteGraph-Mona replaces Mona’s static multi-scale aggregation with adaptive routing and regularizes routing signatures using class-wise anchors and an online confusion-guided margin. Across three datasets and two visual backbones, it improves mean accuracy over Mona and remains competitive with representative baselines.
- Conclusion: RouteGraph-Mona replaces static multi-scale aggregation with sample-adaptive routing and regularizes routing signatures using class-wise anchors and an online confusion-guided margin.The framework regularizes routing signatures formed from routing weights rather than visual feature embeddings directly.
- Conclusion: Across three mineral image datasets and two visual backbones, RouteGraph-Mona improves mean accuracy over Mona across the evaluated settings.It also achieves competitive performance compared with representative fine-tuning and mineral image classification baselines.
- Conclusion: One route anchor per class is empirically sufficient in the reported experiments.Future work may explore class-adaptive anchor allocation and more robust confusion-graph updating on larger, more diverse datasets.