Source-linked AI summary
Data Fusion by Matrix Factorization
Marinka Žitnik, Blaž Zupan
TL;DR
Heterogeneous data sources can provide complementary evidence for predicting relations, but existing fusion strategies may discard data structure or require nontrivial transformations. The paper develops penalized matrix tri-factorization that jointly factorizes relation matrices while preserving block structure and source-specific patterns. Its data-fusion algorithm achieves higher accuracy than factorization-based early integration in the reported gene-function experiment and is presented as flexible, accurate, and fast.
Problem
Heterogeneous data-fusion tasks need methods that combine diverse sources while retaining their structure and requiring little preprocessing.
Method
The paper simultaneously factorizes relation matrices with constrained matrix tri-factorization, sharing factors across sources while modeling source- and object-type-specific structure.
Results
0.826 F1 score versus 0.576 for factorization-based early integration on the 1,000 most GO-annotated genes.
Takeaways & Limitations
The approach can model matrix-expressible data, including feature representations, ontologies, associations, and networks, and is positioned for tasks such as link prediction and clustering.
Takeaways & Limitations
Early integration neglects modular data structure, while the target relation is assumed to be a partially observed [0, 1]-matrix.
Abstract
from arXiv · showhide
For most problems in science and engineering we can obtain data sets that describe the observed system from various perspectives and record the behavior of its individual components. Heterogeneous data sets can be collectively mined by data fusion. Fusion can focus on a specific target relation and exploit directly associated data together with contextual data and data about system's constraints. In the paper we describe a data fusion approach with penalized matrix tri-factorization (DFMF) that simultaneously factorizes data matrices to reveal hidden associations. The approach can directly consider any data that can be expressed in a matrix, including those from feature-based representations, ontologies, associations and networks. We demonstrate the utility of DFMF for gene function prediction task with eleven different data sources and for prediction of pharmacologic actions by fusing six data sources. Our data fusion algorithm compares favorably to alternative data integration approaches and achieves higher accuracy than can be obtained from any single data source alone.
1. Introduction
Heterogeneous, partially related data sources can reveal patterns that direct data alone may miss, motivating data-fusion methods that preserve source structure while combining evidence. The paper introduces constrained matrix factorization for intermediate fusion with minimal data transformation.
- Motivation: Heterogeneous data sources may contain complementary information that reveals patterns hidden when directly related data are considered alone.The motivating example combines clinical, demographic, environmental, social, and entertainment data.
- Fusion strategies: Early integration converts all sources into one feature table but neglects their modular structure.Its unified representation can model cross-source relationships, but loses source modularity.
- Fusion strategies: Late integration builds a separate model per source and fuses predictions, requiring independently crafted transformations into target relations.Such transformations may be nontrivial for sources such as movie preferences and social relationships.
- Fusion strategies: Intermediate integration retains each source’s structure and merges sources through a joint predictive model.It is often preferred for predictive accuracy, although each model type requires a new inference algorithm.
- Paper contribution: The paper develops constrained matrix factorization for intermediate fusion with minimal preprocessing of attribute-based data, ontologies, associations, and networks.The method is evaluated against multiple kernel learning and random forests.
2. Background and Related Work
Prior matrix-factorization and kernel-based methods provide routes to intermediate data integration, but heterogeneous representations often require transformations whose kernel choices affect performance. The paper extends matrix-factorization approaches to heterogeneous data sets and objects of different types.
- Matrix factorization: Matrix factorization approximates a data matrix with low-rank factors found by optimizing reconstruction discrepancy.Two-factor decomposition writes R ≈ WH with k much smaller than the matrix dimensions.
- Matrix factorization: Penalized matrix tri-factorization incorporates must-link and cannot-link constraints to model relations between objects of the same type.Such constraints can encode information from ontologies and social networks.
- Prior factorization approaches: Joint matrix-factorization methods decompose multiple matrices into shared basis and source-specific coefficient factors, with some variants adding network regularization.These approaches generally describe objects of the same type.
- Prior factorization approaches: The paper extends prior approaches by simultaneously handling heterogeneous data sets and objects of different types.This extension preserves structured, source-specific factorization while broadening the kinds of relations that can be fused.
- Kernel-based integration: Kernel-based intermediate integration combines multiple normalized kernel matrices using learned coefficients for shared objects.Multiple kernel learning extends single-kernel SVMs to classification, regression, and clustering.
- Kernel-based integration: MKL represents heterogeneous object types and structures through kernels, making performance dependent on the chosen transformations and kernels.Strings, vectors, and graphs are converted into kernel matrices before integration.
3. Data Fusion Algorithm
The algorithm fuses heterogeneous relations among multiple object types by simultaneously factorizing relation matrices while incorporating same-type constraints. Shared object-specific factors preserve system structure, support target-relation prediction, and integrate incomplete collections of data sources.
- Data representation: DFMF represents available relations between object-type pairs as sparse matrices and same-type information as constraint matrices, including when some pairwise relations are missing.Constraint examples include social networks and drug interactions; cannot-link entries are positive for dissimilar objects, while similar-object entries are negative.
- Joint factorization: The method simultaneously factorizes all relation matrices while constraining them with same-type relations, preserving block structure and sharing factors across related sources.Source-specific factors capture relations between object types, while object-type-specific factors are reused across every relation involving that type.
- Target prediction: Target relations are predicted by reconstructing unobserved entries of a partially observed [0, 1]-valued matrix in the context of all other data sources.Stopping monitors convergence for the target matrix; in the experiments, ε was set to 10^-5 and assessed every fifth iteration.
- Joint factorization: The block relation matrix R is tri-factorized into G and S, with each relation reconstructed as GiSijGⱼᵀ.Factors Gi are specific to objects of type Ei and shared across relations involving that type; Sij defines the relation between object types Ei and Ej.
- Constrained optimization: The objective balances approximation of input data with penalties or rewards induced by cannot-link and must-link constraints, and supports multiple constraint matrices per object type.The optimization uses the Frobenius norm and trace; iterative updates of Gi and Sij converge to a local minimum.
- Parameter estimation: Model ranks are selected by varying candidate ranks and evaluating explained variance, residual sum of squares, and cophenetic correlation through internal cross-validation.Ranks are chosen where cophenetic correlation begins to fall, explained variance is high, and the RSS curve shows an inflection point.
4. Experiments
The experiments fuse eleven heterogeneous data sources for gene function prediction in Dictyostelium discoideum, using relation and constraint matrices around the gene–ontology target. Models are evaluated by cross-validation and compared with multiple kernel learning and random-forest early integration.
- Data: Eleven data sources were fused to predict 148 gene ontology annotation terms in Dictyostelium discoideum.The study used cross-validated accuracy for the 148 classes.
- Baselines: The study compared the fusion algorithm with multiple kernel learning and random-forest early integration.Random forests used concatenated 9,362-dimensional gene profiles, while kernel-based fusion used source-specific kernels.
- Data: The fusion configuration represents genes, ontology terms, experimental conditions, publications, MeSH descriptors, and KEGG pathways as connected object types.Relations include gene annotations, expression measurements, publication links, pathways, and ontology-associated evidence.
- Data: The target gene–ontology relation was balanced with an equal number of unsupported non-associations and constrained using gene-interaction and term-similarity scores.Additional constraints used STRING interaction scores and shortest-path-based ontology similarity.
- Evaluation: Model quality was estimated with tenfold cross-validation, withholding all data for test-set genes during training and averaging F1 scores across runs.F1 was defined as the harmonic mean of precision and recall.
5. Results and Discussion
The factorization approach performs competitively or better than alternative fusion methods, benefits from additional data and constraints, and is substantially faster than MKL. Retaining structured data blocks is important for predictive performance.
- Predictive performance: Matrix factorization was at least comparable to MKL and substantially more accurate than early integration by random forests on slim GO terms.All three approaches improved when more genes and data were included.
- Predictive performance: Factorization produced consistently higher F1 scores than MKL and random forests for nine selected GO terms.With one or two exceptions, F1 scores were very high and relevant to candidate-gene prioritization.
- Computational performance: 18 minutes versus 77 minutes: factorization completed one whole-genome cross-validation iteration faster than MKL.The comparison used a standard desktop computer.
- Sensitivity to data sources: Adding data sources increased prediction accuracy across tested combinations.The experiment added R13, Θ1, or both to the target source R12.
- Sensitivity to constraints: Adding object-type-specific constraints improved the factorization model’s predictive performance.Constraint sparsity was varied by holding out randomly selected protein-protein interactions.
- Initialization: The informed initialization algorithms surpassed random initialization, with random Acol performing best among them.This experiment used 1,000 highly GO-annotated D. discoideum genes and selected factorization ranks.
- Early integration: 0.826 versus 0.576: structured data fusion achieved a higher cross-validated F1 score than factorization-based early integration.Early integration lost source-specific and system structure by treating the data as one matrix.
6. Conclusion
The paper proposes a flexible data fusion algorithm that requires minimal preprocessing and models collections of matrix-expressible data sets. Its principal advantages are reported as accuracy, time response, and applicability across heterogeneous data types.
- Conclusion: The proposed algorithm requires minimal, if any, preprocessing and provides accuracy and time-response advantages over kernel-based methods.These features are identified as the algorithm’s principal advantages.
- Conclusion: The approach can model any collection of data sets that can each be expressed as a matrix.The paper identifies gene function prediction as one example and anticipates applications including association mining and link prediction.