PaperlayerView exact revision on arXiv

Source-linked AI summary

Nonparametric Bayes Modeling of Populations of Networks

Daniele Durante, David B. Dunson, Joshua T. Vogelstein

arXiv:1406.7851v3stat.ME

TL;DR

Replicated network data call for flexible inference on their population distribution, a target that models for single networks do not address directly. The paper uses a Bayesian nonparametric mixture of low-rank latent-space representations to model that distribution. In simulations and human connectome data, the approach accurately infers expected network structures and substantially improves on state-of-the-art alternatives.

  • Problem

    Replicated network data are increasingly available, creating a need for a flexible representation of their population distribution.

    Source: §1 · pp. 1, 3

  • Method

    A Bayesian nonparametric mixture of low-rank factorizations with latent-space representations models the population distribution of network-valued data.

    Source: §1 · pp. 2–3

  • Results

    In human connectome data and simulations, the approach accurately infers expected network structures and substantially improves state-of-the-art alternatives.

    Source: §7 · p. 25

  • Takeaways & Limitations

    The approach models the probabilistic generative mechanism of a network-valued random variable.

    Source: §7 · p. 25

    Without modification, the computational algorithms fail to scale to massive networks at the high spatial resolutions provided by modern imaging modalities.

    Source: §7 · p. 25

Abstract

from arXiv · show

Replicated network data are increasingly available in many research fields. In connectomic applications, inter-connections among brain regions are collected for each patient under study, motivating statistical models which can flexibly characterize the probabilistic generative mechanism underlying these network-valued data. Available models for a single network are not designed specifically for inference on the entire probability mass function of a network-valued random variable and therefore lack flexibility in characterizing the distribution of relevant topological structures. We propose a flexible Bayesian nonparametric approach for modeling the population distribution of network-valued data. The joint distribution of the edges is defined via a mixture model which reduces dimensionality and efficiently incorporates network information within each mixture component by leveraging latent space representations. The formulation leads to an efficient Gibbs sampler and provides simple and coherent strategies for inference and goodness-of-fit assessments. We provide theoretical results on the flexibility of our model and illustrate improved performance --- compared to state-of-the-art models --- in simulations and application to human brain networks.

1. INTRODUCTION

Flexible procedures for inference on the population distribution of replicated network data are lacking. The paper proposes a dimension-reducing Bayesian nonparametric model for that distribution and for studying network properties across replicates.

1. INTRODUCTION

  • The model targets the unknown population distribution of replicated networks, rather than analyzing each network separately or averaging them.The networks are treated as realizations from a common network-valued random variable.Source: §1 · p. 2
  • A mixture of low-rank factorizations with latent-space representations reduces dimensionality while incorporating network information within components.The formulation places priors on factorization terms to induce a provably flexible prior for the network probability mass function.Source: §1 · pp. 2–3
  • The model supports inference on expected network summary measures and on their distributions across replicates.The paper frames this as inference from the posterior distribution of the network probability mass function.Source: §1 · p. 4
  • The authors report that the approach improves performance over competitors in simulations and apply it to human brain networks.The introduction identifies the simulation study and brain-network application as later evaluations.Source: §1 · p. 4

2. NONPARAMETRIC MODEL

The model uses a mixture of latent-space components to address the dimensionality of network distributions while retaining flexibility over network-valued probability mass functions. It supports inference on expected network summaries and their distributions across replicates, with Monte Carlo inference and an analytically available expected network.

2.1 Notation and motivation

  • The full network-distribution pmf requires intractably many parameters, motivating dimension reduction that exploits shared network structure without imposing overly restrictive assumptions.Even for V = 68 brain regions, the passage reports 2^{68·67/2} − 1 free parameters.Source: §2.1 · pp. 4–5

2.2 Model formulation and properties

  • The mixture of low-rank factorizations can represent any network pmf while reducing dimensionality from 2^{V(V−1)/2}−1 to H{1+R(V+1)}+V(V−1)/2−1 parameters.Source: §2.2 · p. 6

2.3 Identifiability and inference

  • The factorization is not identifiable: shared similarity and component deviations can be shifted against each other, and infinitely many positive semidefinite matrices can yield the same deviation.The passage notes this over-completeness as a limitation of the parameterization.Source: §2.3 · p. 7
  • For each network summary measure, the model supports inference on both its expectation and its full distribution across replicates.Both are functionals of the population distribution of networks.Source: §2.3 · p. 7
  • Following Ghosh and Dunson (2009), the model imposes no identifiability constraints on component edge-probability vectors because they are unnecessary for identifying the network distribution.Source: §2.3 · p. 7
  • The formulation facilitates Monte Carlo integration for inference on expected summary measures and their distributions.Source: §2.3 · p. 7
  • To generate a network, the model draws a mixture component using its mixing probabilities, then samples edges as conditionally independent Bernoulli variables with that component’s edge probabilities.The component edge-probability vector is factorized according to (2).Source: §2.3 · pp. 7–8
  • The expected network is the weighted sum of the component edge-probability vectors, with mixture probabilities as weights.This gives an averaged network architecture analytically.Source: §2.3 · p. 8

3. PRIOR SPECIFICATION AND PROPERTIES

The prior specification gives the mixture model sufficient conditions for broad support over network probability mass functions and strong posterior consistency as the number of observed networks grows. Shrinkage priors favor simpler mixture and low-rank representations, while the asymptotic results concern fixed node count.

3. PRIOR SPECIFICATION AND PROPERTIES

  • Sufficient prior conditions yield full support for the induced prior on representable network probability mass functions.The conditions include adequate mixture capacity and support for the component-specific edge probabilities and mixture weights.Source: §3 · pp. 8–9
  • The theoretical results address consistent estimation of the population distribution as n grows with V fixed, not a growing-node regime.The paper identifies theory for V increasing with n as an interesting extension.Source: §3 · p. 9
  • With fixed node count, full prior support gives strong posterior consistency for the true network probability mass function as the number of networks grows.The posterior assigns probability one to every L1 neighborhood of the truth as n → ∞.Source: §3 · p. 9
  • A double-shrinkage prior favors automatic adaptation by shrinking away redundant mixture components and toward low-rank structure within components.The paper reports empirical evidence that extra components are efficiently deleted in this model.Source: §3 · pp. 9–10
  • The shared similarity vector and component-specific deviations structure edge log-odds so components share variability while retaining component-specific variation.When the deviations shrink toward zero, correlations for the same edge across components approach one, collapsing toward a single low-rank representation.Source: §3 · p. 11

4. POSTERIOR COMPUTATION

Posterior computation alternates mixture allocation with updates to shared and component-specific parameters using Pólya–gamma augmentation. Posterior predictive checks compare observed network measures with replicated distributions, which also estimate distributions of those measures.

4. POSTERIOR COMPUTATION

  • The Gibbs sampler allocates networks to mixture components, then updates shared and component-specific parameters using Bayesian logistic regression with Pólya–gamma augmentation.Networks assigned to a component are conditionally iid given its edge-probability vector, allowing the update to use aggregated networks.Source: §4 · p. 11
  • Posterior samples of mixture weights and component edge probabilities generate network replicates whose summary measures provide posterior estimates of expected network measures.The procedure simulates networks for each MCMC draw, computes the measures, and averages them.Source: §4 · p. 13

4.1 Goodness-of-fit via posterior predictive checks

  • Posterior predictive checks compare observed network measures with their replicated distributions; observed values in the tails indicate inadequate model flexibility.Replicates are drawn from the posterior predictive distribution of the network pmf.Source: §4.1 · pp. 13–14
  • The posterior predictive distribution of each network summary measure also estimates its density under the modeled population distribution.These estimates provide information about how topological properties vary across replicates.Source: §4.1 · p. 14

5. SIMULATION STUDY

The simulation study evaluates whether the mixture model represents populations with diverse network structures, examining predictive and inferential behavior and posterior concentration across network sizes in an application-mimicking setting.

5. SIMULATION STUDY

  • High a1 and a2 settings can shrink component variability toward zero, forcing edge-probability vectors to coincide and collapsing the mixture to one low-rank representation.The authors caution against settings that induce priors on λ(h) strongly concentrated near zero.Source: §5 · p. 15
  • ≈1,200 effective samples out of 4,000 indicate good mixing, while posterior computation takes 43 minutes in a naive R implementation.The authors note significant room to improve computation time.Source: §5 · p. 15
  • Posterior predictive checks show improved flexibility and uncertainty quantification across density, homophily, transitivity, centrality, and degree-distribution measures.The checks also support an adequate representation of the mechanism underlying the simulated networks.Source: Figure 4 · p. 16, §5 · p. 16
  • Compared with hierarchical latent-distance and bilinear models, our approach preserves multimodal summaries and yields posterior expectations centered near truth.The competing models rule out multimodal patterns and concentrate mass around averaged structures.Source: §5 · pp. 16–17, Figure 5 · p. 17
  • Posterior inference for the expected network structure is also accurate.Figure 6 shows posterior summaries for the expected structure alongside its true expectation.Source: §5 · p. 17, Figure 6 · p. 18
  • Increasing sample size facilitates posterior concentration for all expected network summary measures when network size is fixed at V = 20.The study compares n = 40 and n = 100, consistent with the stated strong posterior consistency results.Source: §5 · p. 18
  • The second scenario evaluated posterior concentration as network size increased under a generative setup mimicking the brain-network application.It considered V = 68 and V = 34, with four mixture components and two-dimensional latent spaces.Source: §5 · p. 18
  • Posterior concentration was good at both network sizes and increased with V, as reported in Table 1.The table reports posterior concentration for varying n and V settings.Source: §5 · p. 19, Table 1 · p. 19
  • The authors suggest concentration improves with V because the latent dimension remains fixed, allowing cross-edge information borrowing to improve efficiency in modeling mixture-component latent coordinates.Source: §5 · p. 19

6. APPLICATION TO HUMAN BRAIN NETWORKS

The KKI-42 brain-network application evaluates how the proposed model captures observed network measures and supports inference on expected structure and topology. It examines connectivity-based node locations, communities, and high-degree hubs, while noting limitations of simpler alternatives for higher-order structures.

Posterior Predictive for Network Measures

  • The model incorporates false-positive and false-negative measurement errors, but the analysis cannot separate error-related variability from true between-subject network variation without fiber ground truth.Source: §6 · pp. 19–20
  • Moderate changes in a1 and a2 do not substantially affect posterior inference on expected network measures, and similar robust results hold for posterior predictive distributions of network summary measures in model checking and the inference focus.The comparison uses the initial setting a1 = 2.5 and a2 = 3.5.Source: §6 · p. 21
  • The method substantially improves characterization of brain-network distributions, capturing multimodal patterns and uncertainty that competitors’ overly concentrated predictions miss.The competitors’ predictions cluster around the averaged structure and rule out many observed network measures.Source: Figure 7 · p. 21, §6 · pp. 21–22
  • Posterior inference indicates high hemispheric homophily and small-world topology, with short average path length and high transitivity.The reported expected measures include network density, homophily, transitivity, centrality, and degree distribution.Source: Figure 8 · p. 22, §6 · p. 22
  • Compared with equal-edge Erdős–Rényi networks, the connectomes have similar expected average path length but lower expected transitivity, confirming small-worldness.Source: §6 · p. 23

6. APPLICATION TO HUMAN BRAIN NETWORKS

  • Figure 10 shows connectivity-inferred node locations aligning with real brain coordinates and revealing hemisphere communities and lobe-related subcommunities.The visualization uses the posterior mean of the edge-probability vector.Source: §6 · p. 24, Figure 10 · p. 24
  • Corpus callosum, isthmus, precuneus, and superior temporal regions have high expected degree, marking them as central anatomical regions.The passage reports these as examples among other high-degree regions.Source: §6 · p. 24
  • Relative edge frequencies and hierarchical latent-distance or bilinear models yield similar insights and offer parsimonious alternatives for collapsed architectures or networks concentrated around averages.These alternatives are suited to settings focused on collapsed architectures or probabilistic mechanisms concentrated around averaged structures.Source: §6 · pp. 24–25
  • However, these models perform poorly when inference targets complex probability mass functions with higher-order topological and dependence structures.The passage refers to Section 5 and Figure 7 for this limitation.Source: §6 · p. 25

7. DISCUSSION

The paper proposes a Bayesian nonparametric model for the population generative mechanism of network-valued data. It reports accurate expected-structure inference and improved flexibility, while identifying very large networks as a scaling challenge.

7. DISCUSSION

  • The approach models the probabilistic generative mechanism of network-valued data using a Bayesian nonparametric formulation.The discussion presents this as a flexible but tractable and efficient procedure.Source: §7 · p. 25
  • Simulations and human connectome data show accurate inference of expected network structures and substantial improvement over state-of-the-art alternatives in adapting to the generative mechanism.The model is described as flexible, tractable, and efficient for network-valued data, with procedures for inference on topological properties and network summary measures.Source: §7 · p. 25
  • The discussion identifies supervised modeling with predictor variables as an ongoing direction.It gives phenotypes as an example of variables that could be incorporated in supervised extensions.Source: §7 · p. 25
  • Allowing mixture probabilities or latent coordinates to depend on phenotypes extends the model to supervised settings and may improve higher-order functional modeling.The passage states this can reduce concerns arising from model misspecification.Source: §7 · p. 25
  • Without modification, the computational algorithms fail to scale to very large node sets; sparsity-aware models and more efficient sampling are proposed directions.The passages identify high spatial resolution from modern imaging as motivating the scaling challenge.Source: §7 · p. 25
  • The mixture representation supports potential clustering of networks through posterior inference on component indicators, although accurate clustering is beyond this paper's scope.The discussion suggests adapting existing mixture-model clustering methodologies.Source: §7 · p. 25

APPENDIX

The appendix supplies proof details for the model's probability representation and prior-support arguments. These include a representation of component log-odds and conditions under which induced priors assign positive probability to neighborhoods.

APPENDIX

  • The appendix treats network configurations as cells in a contingency table, with the lth variable indicating presence or absence of an edge.The proof invokes independent Bernoulli variables over the remaining node pairs.Source: APPENDIX · p. 26
  • The proof represents component-specific log-odds as D(h) = L(X(h)Λ(h)X(h)^T), with X(h) and a nonnegative diagonal Λ(h) chosen for each component.The passage notes that such a representation can be constructed for any D(h).Source: APPENDIX · p. 26
  • The support argument requires full support for Πν on the probability simplex and positive prior probability in neighborhoods of the component edge-probability vectors.The proof cites conditions in Theorem 3.1 for these support properties.Source: APPENDIX · p. 27
  • The proof transfers neighborhood support from log-odds vectors to edge-probability vectors through the element-wise logistic mapping.It relies on the mapping being one-to-one and continuous.Source: APPENDIX · p. 28
  • For the stated inverse-gamma conditionals, the proof establishes positive prior probability in neighborhoods of positive λ vectors.It uses λ1 ∼ Inv-Ga(a1, 1) and λr | λr−1 ∼ Inv-Ga(a2, λr−1), with positive parameters.Source: APPENDIX · p. 28

Related papers

Loading 1406.7851v3…