PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalized Determinantal Point Processes for Attraction and Repulsion with Distinct Interaction Kernels

Hideaki Ishibashi, Takahiro Kawashima, Hideitsu Hino

arXiv:2610.12894v1stat.MLcs.LGstat.CO

TL;DR

DPPs model repulsion, but many applications also involve attraction, and NDPP attraction kernels are difficult to interpret and specify. This paper proposes DRPPs, which use separate symmetric kernels and an auxiliary-variable representation for inference and learning. Its analyses and experiments find that DRPPs represent transitive attraction with fewer parameters, while non-transitive attraction requires compensation through the repulsion kernel.

  • Problem

    Standard DPPs cannot express attraction, while NDPP kernels that capture both effects are difficult to interpret and specify.

    Source: §1 · p. 1

  • Method

    DRPPs model repulsion and attraction separately with two symmetric positive-definite kernels and use Gaussian auxiliary variables to obtain a mixture-of-DPPs representation.

    Source: §1 · p. 1, §3.1 · p. 3, §6 · p. 8

  • Results

    For transitive attraction, DRPPs match or outperform comparison models with fewer parameters, while non-transitive attraction requires compensation through the repulsion kernel.

    Source: §1 · p. 2, §5.2 · p. 7, §6 · p. 8

  • Takeaways & Limitations

    DRPPs are parameter-efficient for transitive and clustered attraction, whereas their structural bias constrains flexibility relative to broader models such as SimplePGC.

    Source: §6 · p. 8

    The structural inductive bias that enables parameter efficiency also limits flexibility relative to broader models such as SimplePGC.

    Source: §6 · p. 8

Abstract

from arXiv · show

Determinantal point processes (DPPs) model subset selection probability with repulsion, but many applications also exhibit attraction arising from pairwise relations or shared latent structures. Nonsymmetric DPPs (NDPPs) capture both effects, but their kernels are difficult to interpret and specify. We propose the determinant-ratio point process (DRPP), which models repulsion and attraction via a ratio of the determinants of two symmetric positive-definite kernels. This formulation preserves the feature-based interpretation of standard DPPs while allowing the two effects to be specified separately. Furthermore, we introduce an auxiliary-variable representation that enables efficient MCMC sampling and yields tractable expectation formulas for key statistics including the normalizing constant, which together facilitate maximum-likelihood learning. We theoretically characterize the interaction structures well suited to DRPPs: for clustered attraction with $C$ latent groups, DRPPs exactly realize the target pairwise interactions with an attraction rank $C$, whereas NDPPs can require a rank that grows with the number of items given the same repulsion component. Conversely, nontransitive pair-specific attraction may require compensation through the DRPP repulsion kernel. Experiments support these predictions, showing that DRPPs fit transitive attraction with substantially fewer parameters than NDPPs.

1 INTRODUCTION

Standard DPPs model repulsion, while attraction is common in applications but difficult to specify in NDPPs. The paper introduces DRPPs to model the two effects separately and characterizes the interaction structures they represent efficiently.

1 INTRODUCTION

  • Standard DPPs cannot express attraction, while NDPPs can but are difficult to specify because their skew-symmetric attraction component lacks a Gram-matrix interpretation.NDPP kernels are therefore typically learned from data.Source: §1 · p. 1
  • DRPPs model repulsion and attraction separately through two feature-based kernels, allowing both effects to be specified from prior knowledge.Items similar under the repulsion kernel are less likely to be selected together; similarity under the attraction kernel makes co-selection more likely.Source: §1 · p. 1
  • An auxiliary-variable representation makes DRPP sampling and estimation practical despite the absence of a closed-form normalizing constant.Gaussian latent variables yield a mixture of DPPs; the resulting samples support estimates of key statistics and the gradient needed for maximum-likelihood learning.Source: §1 · pp. 1–2
  • For clustered attraction under uniform repulsion, DRPPs realize target interactions with attraction rank C, while NDPPs may require rank growing with the ground-set size.For non-transitive pair-specific attraction, DRPPs require compensation through the repulsion kernel.Source: §1 · p. 2
  • Experiments find comparable held-out likelihood to more flexible models with fewer parameters on synthetic data, and better performance than NDPPs with fewer parameters on real data.The reported parameter-efficiency advantage is for transitive attraction.Source: §1 · p. 2

2 RELATED WORK

Related models extend DPPs to capture attraction, but differ in interpretability and in how they represent the two interaction types. Some alternatives control attraction and repulsion globally or induce positive dependence indirectly rather than parameterizing attraction separately.

2 RELATED WORK

  • Standard DPPs use feature-vector Gram matrices, whose determinant assigns lower probability to subsets of similar items, and thus model repulsion rather than positive correlations.The determinant corresponds to the squared volume spanned by the selected feature vectors (Kulesza and Taskar, 2012).Source: §2 · p. 2
  • NDPPs capture attraction and repulsion and retain tractable inference, but their skew-symmetric component lacks a natural Gram interpretation, complicating prior specification.Poinas (2026) established well-definedness conditions and couplings with intra-mark repulsion and inter-mark attraction; controlling attraction range remains open.Source: §2 · p. 2
  • Immanantal, α-determinantal, and discrete kernel point processes broaden determinant-based models, but govern attraction and repulsion globally and cannot express mixed pairwise interactions.DKPPs use symmetric kernels and can be specified from item features (Kawashima and Hino, 2025).Source: §2 · p. 2
  • The permanental-determinantal process separates attraction and repulsion into factors, but computing its permanent is #P-hard and both factors share one kernel.The determinantal clustering process models within-cluster attraction but does not include repulsion.Source: §2 · pp. 2–3
  • Mixtures of low-rank DPPs and DetPGCs can induce positive dependence, but do not parameterize attraction as a distinct interaction mechanism.DetPGCs allow arbitrary within-group distributions while retaining DPP-based repulsion across groups (Zhang et al., 2021).Source: §2 · p. 3

3 DETERMINANT-RATIO POINT PROCESS (DRPP)

DRPPs define subset weights using separate symmetric positive-definite kernels for repulsion and attraction. A Gaussian latent-variable representation connects them to DPPs, supporting sampling, statistic estimation, likelihood learning, and fixed-cardinality variants.

3.1 Definition and Standard Form of DRPP

  • DRPP subset weights use the determinant of a repulsion kernel divided by the determinant of an attraction kernel, with both kernels symmetric positive definite.The repulsion determinant penalizes similar items, while the inverse attraction determinant increases their joint probability; setting S = I recovers a standard DPP.Source: §3.1 · p. 3

3.2 DRPP as Infinite Mixture of DPPs

  • Marginalizing Gaussian auxiliary variables represents a DRPP as a continuous mixture of DPPs, enabling expectations of set statistics to be averaged over the latent distribution.The representation provides tractable expressions for statistics such as expected cardinality and marginal probabilities.Source: §3.2 · pp. 3–4

3.3 Sampling

  • The latent representation supports rejection-free samplers that update the full subset in one Monte Carlo step.The methods include a block Gibbs sampler and a two-step sampler using elliptical slice sampling for the latent variables.Source: §3.3 · pp. 4–5

3.4 Learning DRPPs

  • The log-normalizer gradient can be expressed as an expectation over latent variables, allowing its estimation for maximum-likelihood training.The learned parameters typically include item qualities and the low-rank repulsion and attraction factors.Source: §3.4 · p. 5

3.5 DRPP with Fixed Cardinality

  • For fixed subset size k, k-DRPPs use conditional k-DPP updates, and their normalizer and gradient formulas can be adapted from the variable-size case.The gradient of the relevant elementary symmetric polynomial has no tractable closed form and is evaluated by automatic differentiation.Source: §3.5 · p. 5

4 COMPARISON OF DRPP AND NDPP

The section compares DRPP and NDPP through their pairwise interaction structures. DRPPs realize clustered attraction with rank equal to the number of clusters, while arbitrary pair-specific attraction can be represented by NDPPs and may require repulsion compensation in DRPPs.

Interaction measure

  • The section measures pairwise interaction Γij using unnormalized weights for the empty set, each singleton, and the pair, with the normalizing constants canceling.For DRPPs, Γij is a ratio of repulsive and attractive terms and is independent of item qualities Q.Source: §4 · p. 6

Clustered attraction

  • For fixed cluster count C, DRPPs realize the target interactions at rank C, whereas the skew-symmetric rank required by an NDPP grows linearly with N under the same repulsion component.The clustered attraction is transitive: if i attracts j and j attracts k, then i also attracts k.Source: §4 · p. 6

Pair-specific attraction

  • For any attraction graph and edge strengths, an NDPP with L = I + A exactly represents the target pairwise interactions.DRPP attraction is formed from low-dimensional latent-vector inner products, making it suited to attraction arising from shared factors.Source: §4 · pp. 6–7
  • A DRPP can express non-transitive attraction only when its repulsion kernel provides sufficient cancellation; with K = I, patterns with γ > 2 cannot be realized.For two attractive edges of strength γ > 2 with the third pair neutral, the required cancellation is explicit in the stated condition δ < 1 − 2/γ.Source: §4 · p. 6

Comparison

  • DRPPs suit shared-factor attraction, while unconstrained-rank NDPPs can specify pair-specific attraction entries individually.The paper evaluates the two models separately because real-world data may exhibit both structures.Source: §4 · pp. 6–7

5 EXPERIMENTAL RESULTS

The experiments compare validation NLL against parameter count across synthetic interaction types and three real datasets. DRPPs are parameter-efficient for transitive attraction, while their advantage diminishes for non-transitive attraction and varies across real datasets.

5.1 Experimental Setup

  • The study evaluates validation NLL against trainable-array parameter count for DRPP, NDPP, DPP, SimplePGC, and attraction-only variants; real-data DRPP NLL is approximated using ESS.Synthetic NLL is computed exactly.Source: §5.1 · p. 7
  • Synthetic datasets use N = 60 items with cluster, continuous, and skew pairwise interactions; cluster and continuous attraction are transitive, while skew attraction is non-transitive.Subsets have fixed size k = 3; the real datasets are Amazon Baby Registries, KDD Cup 2000, and Microsoft COCO.Source: §5.1 · p. 7

5.2 Parameter Efficiency

  • On Amazon, DRPPs match or beat NDPPs with an order of magnitude fewer parameters; on KDD and COCO they beat NDPPs with fewer, while SimplePGC achieves lower NLL.On KDD and COCO, SimplePGC achieves lower NLL even with a comparable number of parameters.Source: §5.2 · p. 7, Figure 1 · p. 8
  • On transitive cluster and continuous data, DRPPs match SimplePGC with several to about ten times fewer parameters and beat NDPPs; on skew data, the best NLLs of DRPPs, NDPPs, and DPPs are similar.For the cluster type, NDPPs do not match DRPP NLL across the explored hyperparameter settings.Source: §5.2 · p. 7, Figure 1 · p. 8
  • On cluster data, DRPP validation NLL is minimized at attraction rank rS = 6, matching the six clusters, while NDPP NLL saturates and remains higher beyond rA = 30.The DRPP rank result agrees with the theoretical rank-C construction for clustered attraction.Source: §5.2 · p. 7, Figure 2 · p. 8

6 CONCLUSION

The conclusion presents DRPP as a ratio of feature-based symmetric positive-definite kernels for modeling repulsion and attraction. It highlights efficient latent-variable inference and learning, while noting that DRPP’s structural bias limits flexibility and leaves mode finding without formal approximation guarantees.

6 CONCLUSION

  • DRPP models repulsion and attraction through a ratio of determinants of feature-based symmetric positive-definite kernels.The conclusion also connects the formulation to auxiliary Gaussian variables and an infinite mixture of DPPs.Source: §6 · p. 8
  • Auxiliary Gaussian variables enable efficient MCMC sampling, tractable estimation of key set statistics, and maximum-likelihood learning.The conclusion describes DRPP as an infinite mixture of DPPs.Source: §6 · p. 8
  • DRPP represents transitive and clustered attraction with high parameter efficiency relative to NDPPs, whereas non-transitive attraction requires cancellation through the repulsion kernel.Source: §6 · p. 8
  • DRPP’s structural bias constrains flexibility relative to SimplePGC; mode finding is unaddressed, and available difference-of-submodular optimization lacks formal approximation guarantees.The conclusion also identifies reducing variance in the ESS-based estimator and extending theoretical analysis as future directions.Source: §6 · p. 8

Generalized Determinantal Point Processes for Attraction and Repulsion with Distinct Interaction Kernels: Supplementary Materials

The supplementary proof expresses a DRPP pairwise interaction through its repulsion and attraction kernels. It shows that the two kernels contribute separately to that interaction.

Generalized Determinantal Point Processes for Attraction and Repulsion with Distinct Interaction Kernels: Supplementary Materials

  • Pairwise interaction decreases as items become more similar under the repulsion kernel and increases as they become more similar under the attraction kernel.Source: A PROOFS OF THEOREMS · p. 11
  • In standard form, the repulsion-kernel quality changes item-selection likelihood but does not affect pairwise interactions.The passage relates this observation to Anzai and Hino (2026).Source: A PROOFS OF THEOREMS · p. 11
  • The NDPP proof derives singleton and pair subset weights from the symmetric and skew-symmetric parts of its kernel.Source: A PROOFS OF THEOREMS · pp. 11–12

A.1 Proof of Theorem 2

The proof constructs clustered interactions using separate repulsion and attraction kernels. Its construction uses attraction rank equal to the number of clusters and a block-diagonal skew-symmetric matrix for the NDPP comparison.

A.1 Proof of Theorem 2

  • The construction sets repulsion-kernel similarity to κ for both within-cluster and between-cluster pairs, while attraction-kernel similarity is σ within clusters and zero between them.Source: §A.1 · p. 12
  • The DRPP construction uses attraction rank C to realize the target within-cluster interactions exactly.In standard form, the attraction kernel has rank rS = C.Source: §A.1 · p. 12
  • The NDPP comparison yields a block-diagonal skew-symmetric matrix whose cluster block has rank 2⌊mc/2⌋.The proof states the rank formula for each cluster block Ac.Source: §A.1 · pp. 12–13

A.2 Proof of Theorem 3

The supplied proof passage states that the NDPP pairwise interaction equals the target on edges and one on non-edges. The learning material describes likelihood and gradient calculations for variable- and fixed-size observations, plus held-out likelihood estimation without the normalizing constant.

A.2 Proof of Theorem 3

  • The proof states that the NDPP pairwise interaction equals γij on edges and 1 on non-edges.Source: §A.2 · p. 13

B.1 Learning Algorithm

  • For variable- or fixed-size observations, the likelihood’s data term is computed exactly from determinants of subset-sized matrices.The normalizing-constant gradient is expressed as an expectation of ∇θ log wθ(z) under P(z) or Pk(z).Source: §B.1 · p. 14

B DETAILS OF LEARNING DRPPS

  • The variable-size and fixed-size learning cases use different normalizers: det(I + ˜L(z)) and ek(˜L(z)), respectively.Source: §B.1 · p. 14

B.2 Hyperparameter Selection via Held-out Likelihood

  • The held-out likelihood estimator avoids the normalizing constant and uses subset-sized determinants and wθ(z(t)).Source: §B.2 · p. 15

B.3 Efficient Evaluation

The evaluation uses diagonal structure for fast computation and low-rank structure to avoid costly dense decompositions. Newton’s identities avoid eigendecomposition for fixed-size statistics, but cancellation errors grow with k.

B.3 Efficient Evaluation

  • When R_K = I, diagonal structure allows det(I + L̃) in O(N) time and e_k(L̃) in O(Nk); dense evaluation naively requires O(N^3).For r_K > 0, the method exploits low-rank structure rather than a dense decomposition.Source: §B.3 · p. 15
  • Writing L̃(z) as a diagonal matrix plus WW⊤ lets the matrix determinant lemma reduce evaluation to a diagonal determinant and an r_K × r_K matrix.This avoids summing over subsets and the O(N^3) cost of eigendecomposition.Source: §B.3 · p. 15
  • For fixed-size statistics, Newton’s identities use power sums p_j = tr(L̃^j) to avoid eigendecomposition, but cancellation errors grow as k increases.The power sums are the traces of powers of L̃.Source: §B.3 · p. 15

C.1 Synthetic Datasets

The synthetic settings test transitive cluster and continuous attraction, plus a non-transitive skew structure. DRPP represents the transitive interactions exactly, while NDPP represents skew exactly.

C.1 Synthetic Datasets

  • Synthetic samples come from a pairwise log-linear model with fixed subset size k = 3, enabling exact normalizer computation and validation NLL evaluation.The study generated 10,000 training and 10,000 validation samples for each synthetic dataset.Source: §C.1 · p. 16
  • DRPP represents cluster attraction exactly with r_S = C and continuous attraction with r_S = 2; the continuous construction uses a quarter circle to keep attraction monotonic with angular distance.For the cluster construction, items in the same group share one feature direction and items in different groups are orthogonal.Source: §C.1 · p. 16
  • The skew interaction is represented exactly by an NDPP’s skew-symmetric component with r_A = 1.The construction uses A = BC⊤ − CB⊤.Source: §C.1 · pp. 16–17

C.2 Real Datasets

The real-data setup filters subsets, defines train/validation/test evaluation, and measures excess transitivity. Hyperparameter sweeps vary model complexity, with baselines reimplemented in JAX.

C.2 Real Datasets

  • Subsets with fewer than two items are removed from Amazon, KDD, and COCO; they account for more than 90% of raw KDD data, but none of Amazon’s.The stated rationale is that these subsets contain no information about item interactions and would otherwise dominate likelihood.Source: §C.2 · p. 17
  • KDD uses distributed splits, while Amazon and COCO use fixed-seed 70/15/15 splits; test sets are excluded from fitting and NLL evaluation.For ESS-based DRPP likelihood evaluation, the first 1,000 validation subsets are used.Source: §C.2 · p. 17
  • Excess transitivity is measured as graph clustering beyond the average of 100 degree-preserving random graphs; positive pairwise log odds define graph edges.The log odds use a Haldane–Anscombe correction, and the random graphs are generated by double-edge swaps.Source: §C.2 · p. 17

C.3 Hyperparameter Setting

  • The hyperparameter study varies ranks for DRPP, NDPP, and DPP, and maximum group size K and component count C for SimplePGC.Other hyperparameters are fixed, and SimplePGC uses the epoch with the best validation NLL.Source: §C.3 · pp. 17–18

C.4 Implementation of Baselines

  • The baselines are reimplemented in JAX following their original models and objectives, with a validation-set change to SimplePGC early stopping.The implementations are based on the official SimplePGC and NDPP implementations.Source: §C.4 · p. 18
Loading 2610.12894v1…