PaperlayerView exact revision on arXiv

Source-linked AI summary

Trusted Multi-View Classification

Zongbo Han, Changqing Zhang, Huazhu Fu, Joey Tianyi Zhou

arXiv:2102.02051v1cs.LGcs.CV

TL;DR

Multi-view classifiers need to assess changing view quality because accurate predictions can still be unreliable, especially with poorly represented views. The paper proposes trusted multi-view classification, which combines view-derived evidence through Dirichlet distributions and Dempster-Shafer theory; experiments validate accuracy, reliability, and robustness, including out-of-distribution identification.

  • Problem

    Multi-view classifiers need sample-specific view-quality and uncertainty estimates because fixed view weighting can be unreliable when views are poorly represented.

  • Method

    Trusted multi-view classification dynamically fuses evidence from different views using Dirichlet distributions integrated with Dempster-Shafer theory.

  • Results

    Experiments validate the model’s effectiveness in classification accuracy, reliability, robustness, and out-of-distribution identification.

  • Takeaways & Limitations

    The framework provides uncertainty-aware and interpretable multi-view decisions by identifying risky views and exploiting informative ones.

  • Takeaways & Limitations

    Unlike some existing uncertainty methods, the model is contrasted with approaches requiring additional computations during inference.

Abstract

from arXiv · show

Multi-view classification (MVC) generally focuses on improving classification accuracy by using information from different views, typically integrating them into a unified comprehensive representation for downstream tasks. However, it is also crucial to dynamically assess the quality of a view for different samples in order to provide reliable uncertainty estimations, which indicate whether predictions can be trusted. To this end, we propose a novel multi-view classification method, termed trusted multi-view classification, which provides a new paradigm for multi-view learning by dynamically integrating different views at an evidence level. The algorithm jointly utilizes multiple views to promote both classification reliability and robustness by integrating evidence from each view. To achieve this, the Dirichlet distribution is used to model the distribution of the class probabilities, parameterized with evidence from different views and integrated with the Dempster-Shafer theory. The unified learning framework induces accurate uncertainty and accordingly endows the model with both reliability and robustness for out-of-distribution samples. Extensive experimental results validate the effectiveness of the proposed model in accuracy, reliability and robustness.

1 INTRODUCTION

The paper addresses unreliable multi-view predictions by dynamically assessing view quality and integrating views at the evidence level for trusted decisions. Its framework combines Dirichlet-based uncertainty with Dempster-Shafer theory and reports improved accuracy, reliability, and robustness.

  • Motivation: Complex deep multi-view models can produce unreliable predictions when views are poorly represented, such as abnormal sensor information.This limits their suitability for safety-critical applications.
  • Motivation: View quality can vary across samples, so models should adaptively assess each view and explain both the decision and its confidence.The paper highlights that different medical images may be informative for different subjects.
  • Method: Trusted multi-view classification integrates views at the evidence level using Dirichlet distributions and Dempster-Shafer theory instead of feature- or output-level fusion.Evidence from each view parameterizes class-probability distributions and supports uncertainty estimation.
  • Contributions: The model aims to produce effective, efficient, and interpretable sample-adaptive decisions without additional computations or neural-network changes.Interpretability comes from relating decisions to the uncertainty of individual views.
  • Results: Experiments validate superior accuracy, robustness, and reliability through the model’s uncertainty estimation and multi-view integration strategy.The reported evaluation emphasizes these three properties rather than a single metric.

2 RELATED WORK

Related work covers uncertainty estimation, multi-view representation learning, and evidence-theoretic fusion. The paper positions its approach alongside Bayesian methods, representation-based multi-view models, and Dempster-Shafer theory.

  • Uncertainty-based Learning: Bayesian uncertainty methods infer parameter posteriors using techniques such as Laplace approximation, MCMC, and variational inference, but can be costly relative to ordinary neural networks.MC-dropout is described as a more scalable alternative to Bayesian neural networks.
  • Multi-View Learning: CCA-based multi-view models learn common representations by maximizing correlations between views.These methods are presented as representative approaches to multi-view representation learning.
  • Dempster-Shafer Evidence Theory: The paper’s algorithm is illustrated as a pipeline from neural-network evidence through Dirichlet probabilities and uncertainty to Dempster-Shafer fusion across views.The figure distinguishes compatible evidence from mutually exclusive evidence during combination.
  • Multi-View Learning: Other multi-view methods model shared and view-specific information through explicit metrics or learned complete representations for classification.HM3L explicitly learns shared and view-specific metrics, whereas AE2-Nets implicitly learn a complete representation.
  • Dempster-Shafer Evidence Theory: Dempster-Shafer theory models epistemic uncertainty directly and combines beliefs from different sources by fusing shared evidence while normalizing conflicting beliefs.The paper uses this theory as the basis for combining multi-view opinions.

3 TRUSTED MULTI-VIEW CLASSIFICATION

The method represents each view with evidence-derived uncertainty, models class probabilities using Dirichlet distributions, and fuses views through Dempster-Shafer theory. It jointly trains view-specific evidence networks so predictions can reflect view quality and overall uncertainty.

  • Uncertainty and evidence: Evidence-based uncertainty estimation addresses softmax over-confidence and supports trusted multi-view decisions.Softmax confidence can remain high for erroneous predictions, whereas subjective logic adds an overall uncertainty mass.
  • Uncertainty and evidence: A Dirichlet distribution captures both the class-probability distribution and the uncertainty associated with the amount of observed evidence.More evidence for a category increases its assigned probability, while less total evidence increases overall uncertainty.
  • Uncertainty and evidence: Subjective logic maps each view’s evidence to Dirichlet parameters, class belief masses, and an overall uncertainty mass.The evidence vector induces Dirichlet concentration parameters and uncertainty for each view.
  • Multi-view evidence fusion: Dempster’s combination rule fuses independent view opinions into joint evidence and a joint Dirichlet distribution for final class probabilities and uncertainty.The fusion behavior depends on view uncertainty: two uncertain views yield low confidence, while a single confident view can determine the result.
  • Learning to form opinions: The model replaces softmax with a nonnegative evidence-producing activation and uses a multi-task loss to train view-specific opinions and their overall combination.The loss combines per-view and overall objectives, while a KL term encourages evidence for incorrect labels to shrink toward 0.

4 EXPERIMENTS

Experiments evaluate classification, uncertainty estimation, and robustness across six real-world datasets, including clean, noisy, and out-of-distribution settings. TMC outperforms competing uncertainty-based methods and remains effective when view-specific noise is introduced.

  • Experimental Setup: Experiments use six real-world datasets and compare TMC with single-view classifiers and uncertainty-based methods under multi-view noise.The compared methods include MCDO, deep ensemble, uncertainty-aware attention, and evidential deep learning.
  • Classification Performance: TMC improves accuracy over the second-best single-view uncertainty methods by about 7.6% on PIE and 14.8% on Scene15.The cited comparisons are against EDL/MCDO in terms of accuracy.
  • Robustness to Noise: When Gaussian noise is added to half of the views, all comparison methods lose accuracy, while TMC achieves impressive results across all datasets.Without noise, TMC achieves competitive results; its uncertainty-based fusion identifies view-specific noise.
  • Uncertainty Estimation: Out-of-distribution samples with Gaussian noise receive much higher uncertainty than in-distribution samples across all datasets.The experiments add noise with σ = 10 to 50% of test samples.
  • Uncertainty Estimation: Accuracy becomes more reliable as prediction uncertainty decreases, supporting trusted decisions through uncertainty thresholding.Figure 3 examines accuracy after applying uncertainty thresholds.

5 CONCLUSION

The conclusion presents TMC as a Dempster-Shafer-based approach for trusted multi-view decisions. It fuses view uncertainties, exploits informative views, identifies risky views, and provides uncertainty alongside classification.

  • Conclusion: TMC fuses uncertainty from multiple views to produce trusted classification decisions on multi-view data.The model is based on Dempster-Shafer evidence theory.
  • Conclusion: The model identifies risky views and exploits informative views in the final decision.This view-level assessment is part of the model’s decision-making process.
  • Conclusion: TMC provides interpretable uncertainty for the current decision while making the final classification.The conclusion also reports effectiveness in classification accuracy and out-of-distribution identification.

A.1 DEFINITION OF DIRICHLET DISTRIBUTION

The appendix defines the Dirichlet distribution through concentration parameters over a K-dimensional probability simplex. Its density uses the K-dimensional multinomial beta function for normalization.

  • Definition: The Dirichlet distribution is parameterized by K concentration parameters α = [α1, . . . , αK].These parameters define the distribution over class-probability assignments.
  • Definition: The probability density is defined over the K-dimensional unit simplex S_K.The simplex is introduced as the domain of the distribution.
  • Definition: The K-dimensional multinomial beta function B(α) appears as the distribution’s normalization function.The appendix identifies B(α) as the K-dimensional multinomial beta function.

A.2 ALGORITHM FOR TRUST MULTI-VIEW CLASSIFICATION

The algorithm trains evidential networks for each view, constructs view opinions, combines them into a joint opinion, and optimizes the resulting parameters. At test time, it calculates joint belief and uncertainty masses.

  • Training: Training initializes the neural-network parameters and repeatedly computes evidential outputs until convergence.The algorithm takes a multi-view dataset as input and updates the networks iteratively.
  • Training: Each view’s evidence e_v is converted into Dirichlet parameters α_v = e_v + 1 and an opinion M_v.The view opinion is obtained with Eq. 2.
  • Training: The algorithm obtains a joint opinion and joint Dirichlet parameters, then forms the overall loss from joint and view-specific parameters.The loss is optimized using gradient descent according to Eq. 11.
  • Testing: At test time, the trained networks calculate the joint belief mass and uncertainty mass.These quantities are the algorithm’s reported test-time outputs.

A.3 DETAILS OF THE DATASETS

The evaluation uses six real-world multi-view datasets, covering handwritten digits, bird image-text data, facial images, and other multi-view settings. These datasets represent multiple descriptors, modalities, and feature types.

  • Six real-world datasets are used to evaluate the trusted multi-view classification model.The datasets are introduced as the basis for the experimental evaluation.
  • Handwritten contains 2,000 samples from 10 digit classes with six descriptor types as views.
  • CUB contains 11,788 bird images paired with text descriptions across 200 bird categories, using GoogleNet and doc2vec features.
  • PIE contains 680 facial images from 68 subjects and uses three feature types as views.

A.4 COMPARISON WITH CCA-BASED ALGORITHMS

The method is compared with CCA-based representation-learning algorithms using repeated classification experiments. It consistently achieves better classification performance across the evaluated datasets, while differing from most baselines as a supervised classification model.

  • Thirty runs are used to report mean and standard deviation of classification accuracy for each method.
  • The baselines include CCA, DCCA, DCCAE, and BCCA, followed by a linear SVM classifier.
  • The proposed method consistently achieves better performance than the CCA-based algorithms on all datasets.
  • Unlike most CCA-based baselines, the proposed method is a classification model rather than an unsupervised representation-learning model.

A.5 EXPERIMENTAL ANALYSIS OF REMOVING VIEW

Removing views is evaluated on PIE and Scene15, while end-to-end experiments assess image-text and RGB-depth modalities. The reported results show improved performance with more information, strong Food101 accuracy, and uncertainty aligned with difficult or out-of-distribution samples.

  • Experimental analysis of removing view: Removing one view is tested on PIE and Scene15, both of which contain three views.
  • Experimental analysis of removing view: Using more view information tends to improve the model’s classification performance.
  • End-to-end experiment: End-to-end experiments use image-text data from UMPC-FOOD101 and RGB-depth data from NYUD.
  • End-to-end experiment: 91.3% accuracy is achieved on Food101, compared with 90.5% from the second performer.
  • Qualitative experimental results: The highest-uncertainty Food101 samples are difficult or out-of-distribution examples, whereas low-uncertainty samples are clearly classifiable and correctly classified.
Loading 2102.02051v1…