Source-linked AI summary
Reliable Conflictive Multi-View Learning
Cai Xu, Jiajun Si, Ziyu Guan, Wei Zhao, Yue Wu, Xiyue Gao
TL;DR
Conflictive multi-view data challenge methods that assume aligned views, while applications still need decisions and reliabilities for such instances. ECML learns view-specific evidence, forms opinions, and aggregates them conflict-aware; experiments on six datasets show improved accuracy, reliability, and robustness over baselines.
Problem
Real-world multi-view data can contain conflicting views, creating a need for decision results accompanied by reliabilities rather than merely eliminating conflictive instances.
Method
ECML learns view-specific evidence, constructs opinions with belief masses and decision reliabilities, and aggregates conflictive opinions with a conflictive opinion strategy.
Results
ECML outperforms baseline methods on accuracy, reliability, and robustness across 6 publicly available datasets.
Takeaways & Limitations
ECML provides decision results and associated reliabilities for conflictive multi-view data while modeling relations among common and view-specific reliabilities.
Takeaways & Limitations
Ground-truth uncertainties are usually unavailable, so the experiments manually construct conflictive instances and expect large prediction uncertainties.
Abstract
from arXiv · showhide
Multi-view learning aims to combine multiple features to achieve more comprehensive descriptions of data. Most previous works assume that multiple views are strictly aligned. However, real-world multi-view data may contain low-quality conflictive instances, which show conflictive information in different views. Previous methods for this problem mainly focus on eliminating the conflictive data instances by removing them or replacing conflictive views. Nevertheless, real-world applications usually require making decisions for conflictive instances rather than only eliminating them. To solve this, we point out a new Reliable Conflictive Multi-view Learning (RCML) problem, which requires the model to provide decision results and attached reliabilities for conflictive multi-view data. We develop an Evidential Conflictive Multi-view Learning (ECML) method for this problem. ECML first learns view-specific evidence, which could be termed as the amount of support to each category collected from data. Then, we can construct view-specific opinions consisting of decision results and reliability. In the multi-view fusion stage, we propose a conflictive opinion aggregation strategy and theoretically prove this strategy can exactly model the relation of multi-view common and view-specific reliabilities. Experiments performed on 6 datasets verify the effectiveness of ECML.
Introduction
Multi-view learning benefits from combining complementary views, but conflicting real-world views can undermine conventional methods. The paper therefore formulates reliable decision-making for conflictive instances and proposes ECML to produce decisions with reliabilities through conflict-aware opinion aggregation.
- Combining consistent and complementary views can provide more comprehensive data descriptions and support tasks including clustering, retrieval, and recommendation.
- Strict view alignment is often assumed, yet conflicting information across views can cause multi-view learning methods to degenerate or fail.The paper illustrates this with text and image views indicating different food categories.
- Existing approaches mainly eliminate conflictive instances by detecting outliers and removing them or constructing aligned replacements.
- Real-world applications require decisions for conflictive instances together with an assessment of whether those decisions are reliable.This motivates the Reliable Conflictive Multi-view Learning problem.
- ECML learns view-specific evidence, constructs opinions with belief masses and reliabilities, and aggregates conflicting opinions using a conflict-aware strategy.The method also uses average pooling as its fusion layer and theoretically relates final reliability to view-specific reliabilities.
- On 6 publicly available datasets, ECML outperforms baseline methods on accuracy, reliability, and robustness.
Related Work
Prior conflictive multi-view methods primarily remove or realign inconsistent instances, while uncertainty-aware deep learning estimates prediction uncertainty. ECML addresses the remaining need to make reliable decisions for conflictive instances.
- Conflictive Multi-View Learning: Conflictive multi-view learning mainly uses multi-view outlier detection or partial view alignment to handle inconsistent instances.
- Conflictive Multi-View Learning: Outlier-based methods identify abnormal cross-view behavior, whereas partially aligned methods establish alignment relationships for unaligned data.
- Conflictive Multi-View Learning: These approaches aim to eliminate conflictive instances, but applications may instead require making decisions for them.
- Uncertainty-aware Deep Learning: The paper positions reliable decisions for conflictive instances as distinct from methods that only eliminate conflicts or assign weights to uncertain views.
- Uncertainty-aware Deep Learning: Uncertainty-aware deep learning distinguishes aleatoric uncertainty from epistemic uncertainty and develops methods for estimating prediction uncertainty.
- Uncertainty-aware Deep Learning: Trusted Multi-View Classification and subsequent opinion aggregation methods extend evidential deep learning to multi-view learning.
The Method
ECML addresses conflictive multi-view data by learning view-specific evidence, converting it into opinions with uncertainty-based reliability, and aggregating those opinions. Its aggregation can increase uncertainty for conflicting views while distinguishing noisy and unaligned evidence.
- Conflictive multi-view data: Conflictive instances include noise views unrelated to ground-truth categories and unaligned views that indicate different categories from other views.The RCML setting includes both normal and conflictive instances, with manually constructed conflictive instances used to expect large prediction uncertainties.
- View-specific evidential learning: ECML uses view-specific evidential DNNs to learn support for each category and construct opinions containing belief masses and uncertainty-based decision reliability.The method models view-specific class probabilities with Dirichlet distributions parameterized by learned evidence.
- Opinion construction: View-specific opinions represent belief masses over categories, uncertainty as ambiguity or vacuity, and base-rate distributions over classes.The projected class probability is P_k = b_k + a_k u, while uncertainty is inversely related to the total available evidence.
- Conflictive fusion: ECML reduces the influence of noisy views and identifies unaligned views through highly conflicting opinions that may indicate unreliable perspectives.The method also minimizes conflict among opinions during training to capture multi-view common information and reduce conflicts caused by erroneous view-specific decisions.
- Training objective: The training objective adds KL divergence because the adapted evidential loss alone does not ensure lower evidence for incorrect labels.The method replaces softmax with a nonnegative evidence-producing activation and uses the KL term to regularize the resulting Dirichlet parameters.
- Conflictive fusion: Conflictive opinion aggregation combines view opinions so that the aggregated uncertainty decreases when the added opinion is less uncertain and increases when it is more uncertain.This property allows fusion to reflect whether an added view provides reliable or conflicting information.
Experiments
ECML is evaluated on six real-world multi-view datasets against feature-fusion and decision-fusion baselines, using normal and synthetically conflictive test sets. It outperforms the baselines on normal and conflictive data while estimating conflict and uncertainty in ways that track instance quality.
- Datasets: ECML is evaluated on six real-world multi-view datasets spanning handwritten numerals, birds, actions, scenes, images, and faces.The datasets use between two and six feature views and vary in size, category count, and modality.
- Compared methods: The comparison includes feature-fusion methods DCCAE, CPM-Nets, and DUA-Nets, plus decision-fusion methods TMC and TMDL-OA.The baselines cover classical, state-of-the-art, and uncertainty-aware multi-view approaches.
- Test construction: Conflictive test sets are created by adding Gaussian noise to partial test instances or misaligning information in a randomly selected view.Each method is run 10 times, with mean values and standard deviations reported.
- Performance comparison: ECML outperforms all baselines on normal test sets, including a 2.64% accuracy improvement over TMDL-OA on HMDB.The paper attributes this improvement to consistency loss and reports validation through ablation analysis.
- Performance comparison: On conflictive test sets, accuracy decreases for all methods, but ECML performs strongly across datasets through conflictive opinion aggregation.The reported comparison covers accuracy on conflictive test sets.
- Conflict and uncertainty analysis: ECML captures conflictive degrees between views and estimates greater uncertainty as injected noise intensity increases.On CUB, conflictive-instance uncertainty rises with σ from 0.1 to 10, supporting a relationship between estimated uncertainty and instance quality.
Conclusion
ECML addresses reliable decision-making for conflictive multi-view data by forming view-specific opinions, aggregating them through average pooling, and modeling common and view-specific reliabilities. Experiments on six real-world datasets confirmed its effectiveness.
- ECML forms view-specific opinions containing belief mass vectors and decision reliability for the RCML problem.
- ECML aggregates conflictive opinions with a simple average pooling layer.
- The aggregation strategy exactly models the relation between multi-view common and view-specific reliabilities.
- ECML minimizes conflict between opinions to guarantee consistency among different opinions.
- Six real-world datasets confirmed ECML's effectiveness.