PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint Modelling of Multiple Network Views

Isabella Gollini, Thomas Brendan Murphy

arXiv:1301.3759v3stat.MEstat.AP

TL;DR

Latent space models have an intractable posterior, and multiple network views on the same nodes can require a joint summary. The paper proposes variational inference and a latent space joint model with shared latent positions; it demonstrates the approach on longitudinal friendship and protein interaction networks.

  • Problem

    Latent space model posteriors are analytically intractable, while multiple network views on the same nodes call for a model that summarizes their information jointly.

    Source: §1 · p. 2

  • Method

    The paper uses variational inference for latent space models and introduces a joint model in which one latent variable explains connectivity across multiple views.

    Source: §1 · p. 2, §3 · p. 8

  • Results

    The joint model is demonstrated on friendship networks for 50 girls at three time points and on Saccharomyces cerevisiae genetic and physical protein interactions.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The model summarizes longitudinal measurements and multiple relations among the same nodes in a shared latent space.

    Source: §6 · p. 23

    The conclusion identifies extensions for cluster formation and explicit modeling of sequential network structure as future work.

    Source: §6 · p. 24

Abstract

from arXiv · show

Latent space models (LSM) for network data were introduced by Hoff et al. (2002) under the basic assumption that each node of the network has an unknown position in a D-dimensional Euclidean latent space: generally the smaller the distance between two nodes in the latent space, the greater their probability of being connected. In this paper we propose a variational Bayes approach to estimate the intractable posterior of the LSM. In many cases, different network views on the same set of nodes are available. It can therefore be useful to build a model able to jointly summarise the information given by all the network views. For this purpose, we introduce the latent space joint model (LSJM) that merges the information given by multiple network views assuming that the probability of a node being connected with other nodes in each network view is explained by a unique latent variable. This model is demonstrated on the analysis of two datasets: the excerpt of 50 girls from `Teenage Friends and Lifestyle Study' data at three time points and the Saccharomyces cerevisiae genetic and physical protein-protein interactions.

1 Introduction

The paper develops variational inference for latent space models and introduces a joint model that summarizes multiple network views using shared latent positions. It demonstrates the joint model on longitudinal friendship data and protein interaction networks.

1 Introduction

  • Network modeling includes exponential random graph models, stochastic blockmodels, and latent space models, among other approaches.The passage also notes recent reviews of network modeling approaches.Source: §1 · p. 2
  • Variational inference estimates the analytically intractable LSM posterior and is described as less computationally intensive than MCMC, allowing large networks to be handled.The comparison is with the MCMC procedure proposed by Hoff et al. (2002).Source: §1 · p. 2
  • The latent space joint model uses a unique latent variable to explain connectivity across multiple network views and estimates it through iterative EM updates.Independent LSM fits initialize the joint posterior approximation, which is then used to update the view-specific estimates until convergence.Source: §1 · p. 2
  • The paper demonstrates the joint model on 50 girls’ friendship networks at three time points and on Saccharomyces cerevisiae genetic and physical protein interactions.The implementation is available in the lvm4net R package.Source: §1 · p. 3
  • The paper proceeds from LSM and variational inference to the joint model, missing-link data, dataset analyses, cross-validation, and discussion.The analyses use two example datasets and consider performance with missing edges.Source: §1 · p. 3

2 Latent Space Model

The distance model represents nodes in a Euclidean latent space, where proximity corresponds to greater tie probability. Its variational EM procedure approximates the intractable posterior by optimizing a lower bound.

2 Latent Space Model

  • The distance model treats networks as reciprocal: a link from one node to another implies a high probability of a link in the reverse direction.This makes it particularly suitable for undirected networks or directed networks with strong reciprocity.Source: §2 · pp. 3–4
  • The model represents the adjacency matrix using latent positions and uses squared Euclidean distance, which clarifies visualization and requires fewer estimation approximations.The paper notes that an empirical comparison with Euclidean distance is provided in supplementary material.Source: §2 · p. 4

2.1 Variational Inference Approach

  • For d_ij > 1, the probability curves for squared Euclidean distance decrease faster than those for Euclidean distance.Figure 1 compares link probability as a function of the distance metrics.Source: §2.1 · p. 5, Figure 1 · p. 5
  • Variational inference introduces a variational posterior and maximizes a lower bound on the log marginal likelihood to approximate the intractable posterior.This corresponds to minimizing the KL divergence between the variational posterior and the true posterior.Source: §2.1 · pp. 4–6
  • Jensen’s inequality approximates the expected log-likelihood; the squared-distance model needs this approximation, whereas the Euclidean-distance model requires three first-order Taylor expansions.The paper says the alternative Jaakkola–Jordan bound would require further approximations and be more difficult to compute.Source: §2.1 · pp. 6–7
  • Variational EM alternates latent-position posterior estimates with parameter updates, minimizing posterior KL divergence in O(N^2) time and converging in a few iterations.The estimation calculations are described as simple.Source: §2.1 · p. 7

3 Latent Space Joint Model

The LSJM uses a shared latent variable to combine information from multiple network views on the same nodes. Variational EM estimates both joint and view-specific latent positions, allowing each view to be interpreted in the context of the others.

3 Latent Space Joint Model

  • A continuous latent variable with prior zi ∼ N(0, σ2ID) summarizes information from K network views on the same N nodes.The network data use the views together to identify a node's position in a D-dimensional latent space.Source: §3 · pp. 8–9
  • The joint model is specified through the likelihood of all K network views conditional on the latent positions and view-specific parameters.The dyad yijk is 1 when nodes i and j are linked in view k, and 0 otherwise.Source: §3 · p. 9
  • The LSJM estimates overall latent positions from all views and positions conditional on each individual view.Fitting the joint model provides estimates of both posterior distributions.Source: §3 · p. 10
  • View-specific latent-position estimates are updated from the joint estimates, so disconnected nodes or subgraphs can be located using their relations in other views.This avoids the usual tendency of classical LSM to push unconnected nodes apart to maximize likelihood.Source: §3 · p. 10
  • The variational EM procedure estimates each view's latent-position posterior separately, then merges those estimates to obtain the posterior given all views.The passage describes an E-step for the joint posterior and an M-step for model parameters.Source: §3 · pp. 10–11

4 Missing Link Data

The LSJM addresses missing-link prediction by combining information from multiple network views. The paper describes posterior-mean link probabilities, a thresholding rule, and cross-validation procedures for missing links and nodes.

4 Missing Link Data

  • Missing links are assigned probabilities from the posterior means of the model parameters and latent positions.The probability uses the estimated overall positions and the view-specific parameter.Source: §4 · p. 12
  • A link is classified as present when its predicted probability exceeds a view-specific threshold set to the median probability among observed links in that view.Source: §4 · p. 12
  • For missing-link evaluation, 10-fold cross-validation splits possible dyads within each view and compares separate LSM fits with the LSJM.Source: §4 · p. 12
  • For a node with no observed links in one view, the LSJM predicts its links using the other views; the evaluation holds out nodes in 10 folds.The paper does not use a single LSM for this setting because the latent-position estimate would rely only on the prior.Source: §4 · p. 12

5 Applications

The applications fit separate latent space models and the joint model to three yearly friendship networks and to genetic and physical protein-interaction networks. The LSJM’s link-prediction results vary by dataset and network view, while the applications also examine missing nodes and estimated network structure.

5.1 Computational Aspects

  • The analysis used a two-dimensional latent space, ten random starts, and rotation matching by singular value decomposition during the first ten EM iterations.The EM stopping tolerance was 10^-2; fit assessment used ROC curves, AUC, and boxplots of estimated probabilities for true positive and true negative links.Source: §5.1 · p. 13

5.2 Excerpt of 50 girls from ‘Teenage Friends and Lifestyle Study’

  • The girls’ dataset contains three yearly friendship networks for an excerpt of 50 girls present at all measurement points, with 113, 116, and 122 links.The waves span 1995–1997.Source: §5.2 · p. 14

5.2.1 LSM

  • Separate LSM fits to the three girls’ networks had similar posterior α estimates, which the authors interpret as indicating little change in network density over time.The reported q(α) means were −0.63, −0.66, and −0.48; the in-sample ROC analysis suggested the fits were good.Source: §5.2.1 · p. 14, Figure 5 · p. 15

5.2.2 LSJM

  • The girls’ LSJM represents each node with overall and network-specific positions, with arrows showing position variation across the three time points.For missing-node cross-validation, withholding 10% of nodes produced a 9% misclassification rate in each network; estimated degree distributions were described as quite similar to the true ones.Source: §5.2.2 · pp. 16, 18, Figure 6 · p. 16, Figure 9 · p. 18
  • AUCs were 97%, 96%, and 99% for LSJM versus 89%, 97%, and 98% for single LSM across the three girls’ networks.The authors also report misclassification rates of 4% for each LSJM network, versus 4%, 5%, and 5% for single LSM.Source: Figure 8 · p. 17, §5.2.2 · p. 18

5.3 Saccharomyces Cerevisiae Protein-Protein Interactions

  • The PPI data comprise two undirected networks among 67 Saccharomyces cerevisiae proteins: 294 genetic-interaction links and 190 physical-interaction links.The respective densities were 0.066 and 0.043.Source: §5.3 · p. 19

5.3.1 LSM

  • In separate PPI LSM fits, the genetic-interaction network was denser than the physical-interaction network, as indicated by the posterior α estimates.The reported q(α) distributions were N(−0.332, 0.003) and N(−1.001, 0.005), respectively.Source: §5.3.1 · pp. 19–20, Figure 11 · p. 20

5.3.2 LSJM

  • 94% was the LSJM AUC for both PPI networks; single LSM AUCs were 66% for genetic interactions and 97% for physical interactions.The LSJM missing-node misclassification rates were 24% for genetic interactions and 20% for physical interactions.Source: §5.3.2 · pp. 21–22, Figure 15 · p. 22

6 Conclusions

The paper presents LSJM for summarizing multiple network views in one latent space and uses variational inference to make estimation practical for larger networks. Its examples illustrate view-specific positions and prediction; clustering and explicit sequential modeling are proposed as future extensions.

6 Conclusions

  • Variational inference allows LSJM to be applied to networks of thousands of nodes.The model summarizes different relations on the same nodes in a shared latent space, with view-specific positions updated from the overall positions.Source: §6 · p. 23
  • The examples show how LSJM can support interpretation of node positions across longitudinal measurements and multiple protein-interaction relations.The paper also states that the resulting information allows effective visualization and prediction.Source: §6 · p. 23
  • Future extensions may add cluster formation through Gaussian-mixture latent positions or explicitly model sequential structure.Source: §6 · p. 24

Supplementary Material for “Joint Modelling of Multiple Network Views”

The supplied supplementary-material passages identify the authors but provide no substantive section content.

Supplementary Material for “Joint Modelling of Multiple Network Views”

non-squared Euclidean model

The supplementary comparison examines squared and non-squared Euclidean latent-space models, then compares fitting and prediction results on a connected genetic PPI component and datasets from the main paper.

non-squared Euclidean model

  • The comparison evaluates visualization and prediction for Euclidean versus squared Euclidean distance models.Euclidean distance is defined as d_ij = |z_i − z_j|, while squared Euclidean distance is d_ij = |z_i − z_j|^2.Source: non-squared Euclidean model · p. 28

1.1 Connected component of the genetic PPI network

  • On the 57-node, 294-edge genetic PPI component, lvm4net is much faster than latentnet; latentnet and lvm4net yield similar positions and fit well, while VBLPCM performs more poorly.The comparison uses CPU time and visual fit assessments.Source: §1.1 · p. 29, Table 2 · p. 29, Figure 1 · p. 29

1.2 Datasets used in the main paper

  • For the main-paper datasets, latentnet and lvm4net both perform well in AUC; VBLPCM is excluded because it only works with connected components.The supplementary tables report AUC and fitting times.Source: §1.2 · p. 31, Table 3 · p. 31, Table 4 · p. 31

Data

The dataset comprises genetic and physical protein-protein interaction networks among 67 Saccharomyces cerevisiae proteins, downloaded from BioGRID.

Data

  • The dataset contains two undirected networks of genetic and physical protein-protein interactions among 67 Saccharomyces cerevisiae proteins.The proteins are listed in Figure 4, and the data were downloaded from BioGRID.Source: Data · p. 32, Figure 4 · p. 32

3 Estimation procedures for the LSM parameters

The LSM estimation procedures use Taylor-series expansions in both the E-step and the M-step.

3.1 E-Step LSM

  • The E-step uses a second-order Taylor-series expansion involving the gradient G and Hessian H of f evaluated at ˜zi = ˜zi.Source: §3.1 · p. 33
  • The E-step also uses a first-order Taylor-series expansion, with Jacobian J evaluated at ˜Σ = ˜Σ0.Source: §3.1 · p. 34

3.2 M-Step LSM

Loading 1301.3759v3…