PaperlayerView exact revision on arXiv

Source-linked AI summary

Bayesian Consensus Clustering

Eric F. Lock, David B. Dunson

arXiv:1302.7280v1stat.MLcs.LG

TL;DR

Clustering heterogeneous data sources requires methods that preserve source-specific structure while integrating shared information. The paper proposes a Bayesian consensus model that estimates source-specific and overall clusterings together, and reports greater robustness than joint clustering and greater power than separate clustering.

  • Problem

    Separate clustering may lack power and miss inter-source associations, while joint clustering may overlook features specific to individual data sources.

  • Method

    BCC simultaneously estimates an overall clustering and source-specific clusterings whose adherence strengths are learned from the data.

  • Results

    BCC learns the level of cluster agreement and is more robust than joint clustering and more powerful than clustering each data source separately.

  • Takeaways & Limitations

    BCC provides a flexible consensus-clustering framework that models uncertainty and borrows information across heterogeneous data sources.

  • Takeaways & Limitations

    BCC assumes a simple dependence between data sources centered on an overall clustering, which may not make sense when no overall clustering is sought.

Abstract

from arXiv · show

The task of clustering a set of objects based on multiple sources of data arises in several modern applications. We propose an integrative statistical model that permits a separate clustering of the objects for each data source. These separate clusterings adhere loosely to an overall consensus clustering, and hence they are not independent. We describe a computationally scalable Bayesian framework for simultaneous estimation of both the consensus clustering and the source-specific clusterings. We demonstrate that this flexible approach is more robust than joint clustering of all data sources, and is more powerful than clustering each data source separately. This work is motivated by the integrated analysis of heterogeneous biomedical data, and we present an application to subtype identification of breast cancer tumor samples using publicly available data from The Cancer Genome Atlas. Software is available at http://people.duke.edu/~el113/software.html.

1 Motivation

Multi-source biomedical data combine heterogeneous measurements for common samples, creating a need for clustering methods that preserve source-specific structure while integrating shared information.

  • Multi-source data comprise heterogeneous datasets describing a common set of objects, with each dataset representing a distinct measurement mode or domain.
  • TCGA collects multiple genomic data types for common breast cancer tumor samples, including RNA expression, DNA methylation, microRNA expression, and proteomic data.
  • Integrating disparate genomic sources provides a more comprehensive understanding of cancer genetics and molecular biology.
  • Separate analyses may lack power and miss inter-source associations, whereas joint analysis may overlook features specific to individual data sources.
  • The paper seeks an integrative clustering method that is computationally scalable and robust to the unique features of each data source.

2 Related Work

Existing approaches either cluster sources separately or combine them into one joint clustering, while integrative methods model dependence across sources to balance shared and source-specific structure.

  • Multi-source clustering commonly uses separate clustering with possible post hoc integration or a single clustering combining all data sources.
  • Separate clustering can be followed by consensus clustering, which seeks an overall partition agreeing most with the data-specific clusterings.
  • Joint clustering exploits shared structure but can fail to recognize features specific to each data source.
  • MDI clusters each data source separately while simultaneously modeling dependence between the clusterings, allowing information sharing across sources.
  • The paper’s approach differs from MDI by modeling adherence to an overall clustering rather than dependence between each pair of data sources.
  • A finite Dirichlet mixture model represents each object through a mixture component, with Dirichlet-distributed cluster probabilities and Bayesian posterior estimation using Gibbs sampling.

3 Integrative model

The integrative model assigns each data source its own clustering while linking those clusterings through a shared overall clustering. It accommodates disparate data structures and allows source-specific adherence to the consensus to vary.

  • Integrative model: The model supports disparate data structures by assigning each source its own probability model and parameters.Examples include Gaussian random-field spectral densities for images and multivariate probability mass functions for categorical vectors.
  • Integrative model: Each data source has a separate clustering, while an overall clustering links the source-specific assignments.Source-specific clusterings are dependent on the overall clustering, which unifies them across sources.
  • Integrative model: αm controls source m’s adherence to the overall clustering and is interpreted as the probability that Lmn = Cn.When αm = 1, the source-specific clustering equals the overall clustering; the αm values are estimated from the data.
  • Integrative model: The number of represented source-specific clusters can exceed the number represented in the overall clustering.An object assigned outside the overall clusters in a source indicates that its data do not conform well to any overall pattern.
  • Marginal forms: For M = 2, the model and MDI’s pairwise-dependence model are functionally equivalent under a parameter substitution, but not when M > 2.The distinction reflects overall-clustering adherence in this model versus general pairwise dependence in MDI.

4 Computation

The computation uses Gibbs sampling to estimate the consensus and source-specific clusterings jointly, while learning source adherence and selecting a practical cluster count. The approach is designed to quantify uncertainty and borrow information across data sources, but high-dimensional structured data can induce too many clusters.

  • Computation: Gibbs sampling approximates the posterior distribution of the overall clustering, source-specific clusterings, and model parameters.The framework does not require specific forms for the source likelihoods and uses conjugate priors when possible.
  • Computation: O(MNK) operations complete each MCMC iteration when the overall clustering is incorporated into the algorithm.Including the consensus clustering can improve computational efficiency dramatically, especially as the number of sources grows.
  • Computation: Uncertainty is modeled for both the source-specific clusterings and the consensus clustering, with all parameters estimated statistically.Posterior samples can be aggregated to obtain point estimates for the clusterings.
  • Computation: The source-specific and consensus clusterings are estimated simultaneously, allowing information to be borrowed across sources for more accurate assignments.This differs from two-stage procedures that first estimate separate clusterings and then integrate them.
  • Computation: Each source’s adherence strength is learned from the data, either separately or under a shared α1 = . . . = αM assumption.The shared-adherence option is favored when M = 2 for identifiability reasons.
  • Choice of K: High-dimensional structured data can lead the model to select a large number of clusters, even with very small Dirichlet prior concentration parameters.The authors therefore use a heuristic that selects K by maximizing mean adjusted adherence; in practice it selects a small number of clusters revealing shared structure.

5 Simulation

The simulations assess whether BCC estimates source adherence accurately and whether its clustering accuracy adapts to different levels of agreement across data sources.

  • Accuracy of α estimation: BCC estimated α accurately, with 95% credible intervals containing the true value in 91 of 100 simulations.The estimated values were generally close to the true α without substantial bias.
  • Clustering accuracy: The simulations compare separate, joint, dependent, and Bayesian consensus clustering using average incorrect cluster assignments.Relative error is evaluated across 100 simulated datasets with weak signal distinguishing the clusters.
  • Clustering accuracy: Joint clustering performs well when α ≈ 1, whereas separate clustering is superior when α ≈ 0.5.These methods are strongest at the respective extremes of perfect agreement and little relationship between sources.
  • Clustering accuracy: BCC is competitive with separate clustering near α ≈ 0.5 and with joint clustering near α ≈ 1 for both two- and three-source simulations.BCC and dependent clustering learn the level of agreement, providing a bridge between the two extremes.
  • Clustering accuracy: Dependent clustering performs worse in the three-source simulation because its pairwise-dependence model lacks an overall clustering.The resulting model has less power to learn the underlying structure when M > 2.

6 Application to Genomic Data

BCC is applied to four heterogeneous genomic data sources measured on the same 348 breast cancer tumor samples. The resulting overall and source-specific clusterings show shared structure alongside source-specific distinctions.

  • Data: The TCGA application analyzes 348 common tumor samples across RNA expression, DNA methylation, miRNA expression, and protein-array data.The sources contain 645 genes, 574 probes, 423 miRNAs, and 171 proteins, respectively.
  • Data: The four data sources use different platforms and represent different biological components while measuring the same samples.This shared sample structure motivates an integrative analysis.
  • Overall clustering: BCC selects K = 3 clusters, whose overall partition has a significant but weak association with TCGA’s four comprehensive subtypes.The weak association suggests that the two partitions may not be driven by the same biological signal.
  • Visualization: Source-specific clusters are more clearly distinguished than overall clusters, while overall clusters remain represented to some degree in every data source.The PCA views therefore show both data-source-specific structure and shared overall structure.

7 Conclusions and discussion

The paper presents BCC as a flexible, scalable Bayesian model that estimates an overall clustering together with source-specific clusterings. Its main scope boundary is the assumption that an overall clustering and a simple dependence structure are appropriate.

  • Conclusions: BCC jointly models an overall clustering and a clustering specific to each data source.The framework is presented as consensus clustering that models uncertainty and borrows information across sources.
  • Limitations: BCC may be inappropriate when an overall clustering is not sought or does not make sense as an assumption.In such settings, a more general dependence model such as MDI or a context-specific model may be more appropriate.
  • Extensions: BCC’s normal-data implementation can be extended to clustering models with sparse features or more general covariance structures.The paper notes that these extensions are straightforward but does not develop them in the presented implementation.
  • Computation: The MCMC procedure alternates updates for source-specific clusterings, cluster parameters, adherence parameters, the overall clustering, and overall cluster probabilities.The implementation uses normal-gamma mixture components with diagonal covariance matrices for the data-source distributions.

B Cluster size illustration

The cluster-size illustration shows how source-specific inclusion probabilities change with adherence to an overall clustering. Lower adherence produces more uniform source-specific cluster probabilities.

  • Cluster-size illustration: Figure 4 compares overall cluster inclusion probabilities with source-specific probabilities at αm = 0.95, 0.75, and 0.10.The illustration uses K = 10 overall clusters with skewed overall inclusion probabilities.
  • Cluster-size illustration: When αm is close to 1, source-specific inclusion probabilities closely resemble the overall clustering probabilities.As αm approaches 1/K, the source-specific probabilities become more uniform.
  • Cluster-size illustration: Clusters with zero probability in the overall clustering can receive positive probability in a source-specific clustering.Source-specific clusterings generally represent more clusters than the overall clustering.

C Equivalence of BCC and MDI

For two data sources with matched marginal cluster probabilities, BCC and MDI are functionally equivalent after parameter substitution. This equivalence does not extend to more than two data sources.

  • C Equivalence of BCC and MDI: For M = 2, BCC and MDI are compared under conditions that allow equivalence through parameter substitution.The comparison assumes equal adherence in BCC and equal marginal cluster probabilities in MDI.
  • C Equivalence of BCC and MDI: Under BCC, equal and unequal source-specific cluster assignments receive probabilities determined by α and the overall-cluster probabilities.The displayed joint distribution distinguishes k1 = k2 from k1 ≠ k2.
  • C Equivalence of BCC and MDI: Under MDI, pairwise clustering dependence is parameterized through φ and the marginal cluster probabilities.The MDI joint distribution is presented for matched marginal probabilities across the two sources.
  • C Equivalence of BCC and MDI: For M > 2, BCC and MDI have no such equivalence, regardless of restrictions on Π and Φ.MDI models pairwise dependence between clusterings rather than adherence to an overall clustering.

D Prior comparison for α

The prior study examines how choices for α affect BCC estimates using simulated two-source data and alternative truncated beta priors. Estimates are inaccurate under very precise priors but appear robust under moderately precise priors.

  • D Prior comparison for α: The simulation generates two 200-observation datasets from two clusters, with source labels agreeing with the overall clustering with probability α.Cluster-specific observations are drawn from Normal(1.5, 1) or Normal(−1.5, 1) distributions.
  • D Prior comparison for α: BCC is estimated assuming equal adherence, α = α1 = α2, under truncated beta priors with varied a and b.The study also describes MCMC-based computational procedures for the clustering comparisons.
  • D Prior comparison for α: Uniform priors with a = b = 1 produce relatively unbiased α estimates in the simulation.The manuscript illustrates this behavior in Figure 1.
  • D Prior comparison for α: Very precise priors with large a and b strongly influence α estimates and make them inaccurate.Figure 5 displays estimates and prior densities for alternative choices of a and b.
  • D Prior comparison for α: α estimates appear robust for moderately precise priors.This conclusion accompanies the comparison of scatterplots across alternative truncated beta priors.

F.1 Choice of K

The TCGA application selects the number of clusters by maximizing adjusted adherence to an overall clustering. K = 3 gives the highest adjusted adherence, while source-specific clusterings show differing adherence levels and significant but not exceptionally strong associations with TCGA subtypes.

  • F.1 Choice of K: The application evaluates K = 2, . . . , 10 using 10,000 MCMC iterations per candidate, with the first 2,000 iterations used as burn-in.The model estimates source-specific adherence parameters separately for GE, ME, miRNA, and RPPA data.
  • F.1 Choice of K: K = 3 maximizes mean adjusted adherence at ¯α*_K = 0.57, with comparable adherence for K = 4 and K = 6.The estimate is averaged over MCMC iterations and accompanied by a 95% credible interval.
  • F.2 MCMC mixing: For K = 3, estimated adherence is α = 0.91 for GE, α = 0.69 for ME, α = 0.56 for miRNA, and α = 0.70 for RPPA.The MCMC draws converge within approximately the first 1,000 iterations to an approximately stationary distribution.
  • F.2 MCMC mixing: The overall cluster inclusion probabilities for K = 3 average ˆπ1 = 0.24, ˆπ2 = 0.28, and ˆπ3 = 0.48.These probabilities also quickly converge to a stationary distribution over the MCMC draws.
  • F.3 Comparison with TCGA subtypes: All BCC-versus-TCGA subtype associations are significant with p-value < 0.01, but the associations are not exceptionally strong.The comparisons cover overall and source-specific subtype partitions for GE, ME, miRNA, and RPPA.
  • F.3 Comparison with TCGA subtypes: Heatmaps show apparent cluster effects in many variables across the GE, ME, miRNA, and RPPA datasets.Samples are grouped by their relevant source-specific cluster.
Loading 1302.7280v1…