Source-linked AI summary
Trusted Multi-View Classification with Dynamic Evidential Fusion
Zongbo Han, Changqing Zhang, Huazhu Fu, Joey Tianyi Zhou
TL;DR
Multi-view classifiers often prioritize accuracy but may produce unreliable predictions when views are noisy, corrupted, or out of distribution. TMC estimates uncertainty for each view and the overall decision, then fuses view evidence with variational Dirichlet modeling and Dempster-Shafer theory. Theoretical and empirical results validate its accuracy, robustness, and trustworthiness.
Problem
Existing multi-view classifiers can underestimate uncertainty, limiting reliable decision-making for noisy or corrupted views.
Method
TMC dynamically integrates views at the evidence level using variational Dirichlet distributions and Dempster-Shafer theory.
Results
Theoretical and empirical results validate TMC in classification accuracy, robustness, and trustworthiness.
Takeaways & Limitations
TMC identifies risky views, exploits informative views, and provides uncertainty for the final classification to support explainability.
Takeaways & Limitations
TMC’s late-fusion strategy limits interaction between views at the feature or representation level in scenarios requiring complementary information.
Abstract
from arXiv · showhide
Existing multi-view classification algorithms focus on promoting accuracy by exploiting different views, typically integrating them into common representations for follow-up tasks. Although effective, it is also crucial to ensure the reliability of both the multi-view integration and the final decision, especially for noisy, corrupted and out-of-distribution data. Dynamically assessing the trustworthiness of each view for different samples could provide reliable integration. This can be achieved through uncertainty estimation. With this in mind, we propose a novel multi-view classification algorithm, termed trusted multi-view classification (TMC), providing a new paradigm for multi-view learning by dynamically integrating different views at an evidence level. The proposed TMC can promote classification reliability by considering evidence from each view. Specifically, we introduce the variational Dirichlet to characterize the distribution of the class probabilities, parameterized with evidence from different views and integrated with the Dempster-Shafer theory. The unified learning framework induces accurate uncertainty and accordingly endows the model with both reliability and robustness against possible noise or corruption. Both theoretical and experimental results validate the effectiveness of the proposed model in accuracy, robustness and trustworthiness.
1 INTRODUCTION
Existing multi-view classifiers can be accurate but may underestimate uncertainty, making predictions unreliable when views are noisy or corrupted. TMC addresses this by dynamically integrating view evidence to support reliable, explainable decisions.
- Motivation: Existing multi-view methods often assume view quality is equal or stable across samples, despite view reliability varying by sample.Examples include medical diagnosis, where MR or PET may be more informative for different subjects, and image-text classification with noisy modalities.
- Motivation: Uncertainty estimation is needed to assess both the confidence of each view and the confidence of the final classification.This supports answers about how confident a decision is and why its confidence is high or low.
- Proposed Method: TMC integrates multi-view information at the evidence level using variational Dirichlet distributions and Dempster-Shafer theory.The framework is designed for sample-adaptive integration without additional computations or neural-network changes.
- Proposed Method: TMC explicitly learns subjective uncertainty, enabling reliable integration and decision explainability.The model provides uncertainty for individual views and the overall decision.
- Validation: Theoretical analysis and experiments on multiple-view datasets validate TMC in accuracy, robustness, and reliability.The authors attribute these benefits to uncertainty estimation and the multi-view integration strategy.
2 RELATED WORK
Related multi-view work learns shared or view-specific representations, while late-fusion methods combine decisions through averaging, voting, weighting, or learned models. Uncertainty-based approaches and Dempster-Shafer theory provide foundations for modeling and combining confidence or belief across sources.
- Multi-view Learning: CCA-based models seek common representations by maximizing correlation between views, while HM3L and AE2-Nets also model shared and view-specific information.Contrastive-learning methods have additionally been applied to multi-view representation learning.
- Late Fusion: Late fusion combines decisions through averaging, voting, weighting, or learnable models, with confidence-based fusion emerging more recently.The paper distinguishes TMC as an end-to-end fusion framework with theoretical guarantees.
- Uncertainty-Based Learning: Bayesian neural networks, MC-dropout, and ensemble methods estimate uncertainty through distributions or sampling around deterministic neural models.These approaches seek uncertainty while avoiding or addressing the limitations of standard deterministic networks.
- Dempster-Shafer Evidence Theory: Dempster-Shafer theory directly models epistemic uncertainty and combines beliefs from different sources using fusion operators.Dempster’s rule fuses shared belief and discounts conflicting belief through normalization factors.
3 TRUSTED MULTI-VIEW CLASSIFICATION
TMC represents each view with a variational Dirichlet distribution, converts it into belief and uncertainty masses, and fuses views using a reduced Dempster-Shafer rule. This evidence-level integration supports adaptive, theoretically analyzed classification, while ETMC adds a pseudo-view to recover representation-level interactions.
- Variational Dirichlet for Class Distribution: TMC models each view’s class-probability distribution with a variational Dirichlet distribution rather than relying directly on softmax outputs.The Dirichlet distribution is induced by a probabilistic encoder for each view and optimized through a variational lower-bound objective.
- Uncertainty and Evidence Theory: The Dirichlet parameters are converted into subjective opinions containing class-specific belief masses and an explicit overall uncertainty mass.Subjective logic represents uncertainty over the class frame, while evidence determines the assigned belief.
- DS-Combination Rule for Multi-View Classification: The reduced Dempster-Shafer combination rule fuses the opinions of multiple views by combining compatible evidence and normalizing conflicts.The reduction keeps the fusion tractable for K-class classification and retains uncertainty-based evidence fusion.
- Theoretical Analysis of the Combination Rule: Theoretical analysis states that adding an opinion can improve accuracy under reasonable conditions, reduce integrated uncertainty, and retain large uncertainty when both views are highly uncertain.The conclusions are presented through propositions on accuracy and uncertainty.
- Pseudo-View Enhanced TMC: TMC’s late-fusion strategy limits full cross-view interaction, so ETMC adds a pseudo-view to explore representation-level interactions while maintaining the original model’s trustworthiness.The pseudo-view is generated flexibly from the original views and is introduced to address scenarios requiring complementary feature information.
4 EXPERIMENTS
Experiments across vector-type, RGB-D, and image-text datasets evaluate TMC and ETMC for classification accuracy, robustness, uncertainty estimation, and computational cost. The methods consistently outperform comparison algorithms, remain effective under altered views or noise, and assign higher uncertainty to difficult or out-of-distribution samples.
- Vector-Type Data: ETMC outperforms single-view uncertainty-based methods on all datasets, improving accuracy by about 9.4% on PIE and 18.7% on Scene15 over the second-best models.The comparison uses the best-performing single view for existing uncertainty-based methods.
- RGB-D Scene Recognition: Higher-accuracy datasets generally produce lower in-distribution uncertainty, whereas out-of-distribution samples receive much higher uncertainty that usually increases with noise strength.These observations are reported for TMC and ETMC uncertainty distributions under Gaussian noise.
- Vector-Type Data: TMC consistently outperforms CCA-based algorithms across all evaluated vector-type datasets.The comparison reports mean and standard deviation over 30 runs.
- RGB-D Scene Recognition: ETMC achieves the best reported accuracy of 61.3% on NYUD Depth V2 and 72.5% on SUN RGB-D, while remaining robust when only one view is used.ETMC improves accuracy over TMC by 0.6% after fusion, although performance slightly degrades when only RGB is used.
- RGB-D Scene Recognition: Removing uncertain samples reduces the SUN RGB-D misclassification rate from 0.387 to 0.187.Subjective confusion matrices also show that the proportion of uncertain samples differs across classes.
5 CONCLUSION
The conclusion presents TMC as a trusted multi-view classification algorithm that fuses view uncertainty through Dempster-Shafer evidence theory. The authors report effectiveness in classification accuracy and uncertainty estimation.
- TMC produces trusted classification decisions by fusing uncertainty from multiple views using Dempster-Shafer evidence theory.The model dynamically identifies risky views, uses informative views, and estimates uncertainty for the final decision.
- The method dynamically identifies risky views and exploits informative views in the final decision.This view-level assessment supports explainability by indicating uncertainty in the current classification decision.
- Theoretical analysis and empirical results validate TMC's effectiveness in classification accuracy and uncertainty estimation.
6 APPENDIX
The appendix gives theoretical details for the variational decomposition and Dempster-Shafer fusion, and presents qualitative uncertainty analyses across several multi-view datasets. It also states the conditional-independence assumption used in one derivation and illustrates that uncertainty tends to track classification difficulty.
- Theoretical analysis: The appendix includes decomposition equations and proof details for propositions involving the variational Dirichlet formulation.It states the Dirichlet expectation E_Dir(µ|α)[log(µ_k)] = ψ(α_k) − ψ(S) and discusses KL divergence between Dirichlet distributions.
- Theoretical analysis: The derivation assumes pθm(y | µm, xm) = pθm(y | µm), meaning xm adds no information about y once µm is given.The passage characterizes this as a mild conditional-independence assumption.
- Qualitative experimental results: On SUN RGB-D and NYUD Depth V2, examples with the lowest uncertainty are correctly classified, whereas high-uncertainty examples include misclassified samples.The appendix attributes higher uncertainty in some cases to weak intuitive correlation with the ground-truth label.
- Qualitative experimental results: Uncertainty is relatively low for correctly classified samples and relatively high for misclassified samples on RGB-D scene recognition test data.This relationship is analyzed using uncertainty-density distributions.
- Qualitative experimental results: For Food101, the five highest-uncertainty examples are misclassified, while the five lowest-uncertainty examples are correctly classified.