Source-linked AI summary
Learnable Graph Convolutional Network and Feature Fusion for Multi-view Learning
Zhaoliang Chen, Lele Fu, Jie Yao, Wenzhong Guo, Claudia Plant, Shiping Wang
TL;DR
Multi-view learning lacks extensive methods that jointly learn discriminative node relationships with feature and graph fusion through GCNs. LGCN-FF combines a feature fusion network with a learnable GCN using adaptive graph refinement, and achieves superior performance against state-of-the-art graph-based algorithms in multi-view semi-supervised classification.
Problem
Limited work jointly optimizes feature fusion and graph fusion in GCN-based multi-view learning, while many methods use weighted sums of adjacency matrices or unrefined KNN graphs.
Method
LGCN-FF uses sparse autoencoders and a fully connected network for feature fusion, plus learnable adjacency weights, a DSA function, and multi-step optimization for graph fusion.
Results
LGCN-FF achieves superior performance compared with other state-of-the-art graph-based algorithms on multi-view semi-supervised classification tasks.
Takeaways & Limitations
The framework learns shared feature representations and more robust graph representations from heterogeneous views within one end-to-end neural network.
Takeaways & Limitations
The study focuses on undirected graphs and notes that real-world node relationships are more likely to be directed; it also relies largely on KNN-established graph information.
Abstract
from arXiv · showhide
In practical applications, multi-view data depicting objectives from assorted perspectives can facilitate the accuracy increase of learning algorithms. However, given multi-view data, there is limited work for learning discriminative node relationships and graph information simultaneously via graph convolutional network that has drawn the attention from considerable researchers in recent years. Most of existing methods only consider the weighted sum of adjacency matrices, yet a joint neural network of both feature and graph fusion is still under-explored. To cope with these issues, this paper proposes a joint deep learning framework called Learnable Graph Convolutional Network and Feature Fusion (LGCN-FF), consisting of two stages: feature fusion network and learnable graph convolutional network. The former aims to learn an underlying feature representation from heterogeneous views, while the latter explores a more discriminative graph fusion via learnable weights and a parametric activation function dubbed Differentiable Shrinkage Activation (DSA) function. The proposed LGCN-FF is validated to be superior to various state-of-the-art methods in multi-view semi-supervised classification.
1. Introduction
LGCN-FF addresses limited joint optimization of feature and graph fusion in multi-view semi-supervised classification by combining a feature fusion network with a learnable GCN. It uses adaptive graph refinement and multi-step training to learn representations from heterogeneous views.
- Motivation: Multi-view data combine heterogeneous descriptions of the same objects, motivating latent shared representations and graph-based propagation of supervision.The paper highlights frames, audio, and textual descriptions as an example of heterogeneous views.
- Limitations of Existing Methods: Existing approaches often use KNN-generated graphs without refinement, which can create undesired links and aggregate incongruent graph noise.The paper identifies inaccurate KNN edges and linear weighted adjacency fusion as limitations.
- Proposed Framework: LGCN-FF jointly trains feature fusion and learnable graph fusion networks within an end-to-end framework.Its multi-stage strategy optimizes corresponding parameter groups through separate steps.
- Feature Fusion: Sparse autoencoders and a fully connected network fuse heterogeneous features into an underlying representation.The sparse autoencoders process view-specific features before the fully connected network incorporates them.
- Graph Fusion: Learned adjacency weights and the Differentiable Shrinkage Activation function adaptively refine robust node relationships during graph fusion.The DSA function is presented as analogous to soft thresholding in sparse coding and low-rank approximation.
- Evaluation: LGCN-FF achieves superior performance to other state-of-the-art graph-based algorithms on multi-view semi-supervised classification tasks.The paper reports this outcome across its experimental evaluation without supplying a numerical value here.
2. Related Work
Related work spans spectral and first-order graph convolution, multi-view feature learning, and feature or graph fusion. The paper identifies a remaining need for frameworks that jointly conduct feature fusion and graph fusion.
- Graph Convolutional Networks: Spectral GCN applies graph convolution to signals using graph-Laplacian eigenstructure, while first-order approximations reduce computational cost for node classification.The layer operation uses graph structure and learnable layer-specific weights.
- Graph Convolutional Networks: GCN variants have been applied to diverse Euclidean and non-Euclidean learning tasks, including settings with limited supervised information.The reviewed examples include radial GCNs and self-supervised multi-stage GCN frameworks.
- Multi-View Learning: Multi-view learning methods exploit heterogeneous features and views for tasks such as discriminant representation learning, spectral clustering, and image fusion.The reviewed approaches include adaptive transition probabilities and nuclear-norm-based optimization.
- Feature and Graph Fusion: Feature and graph fusion methods combine complementary information across views using kernels, deep features, graph regularization, or mutual reinforcement.Examples include recurrent feature refinement, graph neural fusion, and unified graph recovery.
- Research Gap: Most reviewed methods focus on either feature fusion or graph fusion, motivating a co-training framework that performs both simultaneously.The paper positions simultaneous fusion as important because both forms influence GCN performance.
3. The Proposed Method
LGCN-FF jointly learns an intact multi-view feature representation and a more discriminative fused graph for semi-supervised classification. Its end-to-end framework combines view-specific feature fusion, learnable graph refinement, and multi-step optimization.
- Feature Fusion Network: LGCN-FF integrates multi-view features into an intact representation and merges multiple adjacency matrices into a unique graph.The shared representation H also serves as the node representation input to the learnable GCN.
- Feature Fusion Network: Sparse autoencoders map heterogeneous views with varying dimensions into same-dimensional sparse spaces before fully connected fusion.Each view uses a view-specific sparse autoencoder, and the fused representation is reconstructed across heterogeneous views.
- Learnable Graph Convolutional Network: The learnable GCN combines view-specific normalized adjacency matrices using automatically learned weights and semi-supervised classification loss.The initialized graphs may be generated with KNN, while the labeled sample set supplies the cross-entropy supervision.
- Learnable Graph Convolutional Network: The DSA function refines the weighted adjacency matrix with learnable edge-specific coefficients and thresholds, reducing noise and promoting graph sparsity.Only relationships whose coefficients exceed corresponding learned thresholds are activated, while symmetry is maintained for undirected graphs.
- Training Strategy: Training follows an ADM-inspired four-step optimization over sparse autoencoders, fully connected layers, the shared input H, and learnable GCN parameters.Each step uses its own loss or shared reconstruction loss while holding uncorrelated variables fixed during back propagation.
4. Experimental Analyses
Experiments evaluate LGCN-FF on seven multi-view datasets against classical, adaptive, regression-based, self-representation, and graph-based methods. Results indicate strong classification performance, robustness with limited labels, refined graph structure, useful sparsity, and convergent collaborative training.
- Datasets: Seven publicly available multi-view datasets span image, text, handwritten-digit, article, and multilingual document data with varied views and feature dimensions.The evaluation includes ALOI, BBCnews, BBCsports, MNIST, Wikipedia, MSRC-v1, and Reuters.
- Compared methods: Compared methods include KNN, AMGL, MVAR, MLAN, AWDR, HLR-M2VS, ERL-MVSC, and three GCN-based approaches.The GCN-based methods are GCN fusion, SSGCN fusion, and Co-GCN.
- Classification results: LGCN-FF reaches remarkable classification performance across all test datasets, with larger improvements over GCN-based methods on BBCnews, BBCsports, MSRC-v1, and Reuters.Classification accuracy is measured with 10% randomly labeled data, averaged over five runs with standard deviations.
- Classification results: With 5% or 10% labeled samples, LGCN-FF performs satisfactorily across datasets, while other algorithms generally require more supervision for comparable accuracy.The improvement is reported as more significant on BBCnews, BBCsports, and MSRC-v1; MNIST remains competitive at 5% labels and improves with more labels.
- Refined adjacency matrices: The DSA-refined adjacency matrices are sparser and more robust than direct weighted sums because some entries diminish or disappear, making critical node relationships more pronounced.Figure 4 visualizes partial matrices, with darker colors indicating higher values and red boxes marking diminished or disappearing connections.
- Impact of β: Accuracy fluctuates only slightly with β, but declines marginally at β=0, indicating that suitable sparse-autoencoder regularization benefits hidden-representation learning.β controls the sparsity penalty degree of the sparse autoencoders.
- Convergence analysis: Lsa generally declines within 10 iterations, Lfc decreases after roughly 100 iterations in most cases, and Llgcn reaches its lowest value within 500 iterations on all datasets.The reported loss behavior supports early termination when Llgcn stops decreasing for several iterations.
5. Conclusion
The paper concludes that LGCN-FF jointly learns feature and graph fusion for multi-view semi-supervised classification, with experiments validating its superiority. It identifies directed relationships and KNN-derived graph construction as directions for further graph-fusion research.
- 5. Conclusion: LGCN-FF jointly learns feature and graph fusion through an end-to-end neural network framework for multi-view learning.Its feature fusion network combines multiple sparse autoencoders with a fully connected network, while the learnable GCN integrates topology graphs.
- 5. Conclusion: The learnable GCN adaptively integrates topology graphs, while the DSA function learns a more robust shared adjacency matrix.The framework jointly optimizes feature and graph representations with a multi-step strategy.
- 5. Conclusion: Experimental results validated LGCN-FF's superiority on multi-view semi-supervised classification tasks.
- 5. Conclusion: Future work should address directed node relationships and graph fusion when topology is established primarily through KNN.The paper notes that most existing GCN-based methods focus on undirected graphs and that many real-world datasets lack natural topology networks.