Nonparametric Hypothesis Testing of High-dimensional Clustering With Application to Single-cell RNA Data

1 Introduction

• Clustering labels guide downstream biological interpretation, but conventional cluster-number criteria cannot formally test subgroup existence, leaving single-population stochastic variation as a competing explanation.The motivating question is whether an observed partition reflects genuine biological heterogeneity or can be explained by one population.Source: §1 · p. 2
• SigClust shows elevated Type-I error and spuriously detects multiple clusters on unimodal generalized-normal data with nonstandard shape parameters (Figure 1).The passage identifies β = 2 as standard normal; Figure 1 reports median p-values over 10 repetitions using 10,000 samples.Source: §1 · p. 3, Figure 1 · p. 4
• SigClust-LCP combines a log-concave single-cluster null with Monte Carlo inference to test high-dimensional clustering beyond Gaussian mixtures.The method estimates the data distribution via log-concave projection and derives the null cluster index distribution for a p-value.Source: §1 · pp. 3, 5
• Prior LCP methods become infeasible around d > 6, while available high-dimensional restricted estimators require independence or symmetry assumptions that may not hold.The introduction contrasts polynomial dimension scaling for local estimators with exponential scaling for the standard MLE and notes grid-integration costs can grow sharply with dimension.Source: §1 · p. 4
• A constrained score-matching LCP estimator enables moderate-to-high-dimensional use without shape or independence assumptions.It targets the gap between general score matching and LCP estimation, with score matching minimizing Fisher divergence and allowing neural-network approximation of the probability score.Source: §1 · pp. 4–5

Sections 2–3

• Section 2 introduces SigClust-LCP’s methodology and computational framework, followed by theoretical properties in Section 3.Source: §1 · p. 5

• The motivating problem is deciding whether putative cellular subpopulations reflect genuine heterogeneity or a single unimodal population.Source: §1 · p. 5

Section 4

• Section 4 compares SigClust-LCP with existing methods in simulations designed to mimic distributional features of modern biomedical data.Source: §1 · p. 5

Section 5

• Section 5 demonstrates the method’s practical utility through analysis of a high-dimensional single-cell RNA sequencing dataset.Source: §1 · p. 5

Section 6

• Section 6 discusses methodological implications and directions for future research.Source: §1 · p. 5

2.1 Clustering Significance for Gaussian Mixtures

• Original SigClust tests a Gaussian null against a Gaussian mixture using the 2-means cluster index, but its Gaussian reference can overestimate the index and cause false rejections for near-uniform unimodal data.The cluster index is the ratio of within-cluster to overall variation; SigClust estimates its null distribution by Monte Carlo simulation from a fitted Gaussian.Source: §2.1 · pp. 5–6

2.3 Nonparametric Clustering Significance

• SigClust-LCP tests a log-concave single-cluster null using the 2-means cluster index and Monte Carlo samples from an estimated log-concave density.It compares the observed cluster index with its empirical null distribution to obtain a p-value.Source: §2.3 · p. 8
• To make log-concave estimation tractable, SigClust-LCP assumes independent unit-variance log-concave components and uses covariance eigendecomposition to pre-whiten the data.The stated model writes X = SA, with independent log-concave components in S and observed covariance ATA.Source: §2.3 · pp. 8–9
• For scalability, the method estimates an ICA representation, fits log-concave densities component-wise, and performs testing in truncated PCA space, which preserves pairwise distances and clustering structure.The PCA truncation also reduces the computational cost of MLE-based log-concave projection.Source: §2.3 · p. 9

2.4 Score Matching Estimation for Log-concave Projection

• SigClust-LCP uses score matching to estimate the log-concave log-likelihood gradient rather than the likelihood directly, avoiding ICA’s independence assumption and its loss of higher-order correlations.The score-matching objective is defined using Fisher divergence between the true data density and a log-concave density.Source: §2.4 · p. 9
• Denoising score matching replaces the inaccessible data score with a known conditional score, such as Gaussian noise, and is equivalent to standard score matching up to a constant.The procedure adds noise to observed samples and estimates the score by approximating the added noise.Source: §2.4 · p. 10
• To ensure the estimated score comes from a log-concave distribution, the method constrains a neural network to be concave using weight-sign conditions and convex activation functions.The unconstrained network can estimate complex densities but does not guarantee log-concavity.Source: §2.4 · p. 11
• The testing algorithm uses the fitted score in unadjusted Langevin sampling to simulate the null distribution of the clustering index and derive its p-value.It compares the observed two-means clustering index with indices from simulated batches.Source: §2.4 · pp. 11–12

3.3 Property for PCA-based Dimension Reduction

• For very high-dimensional data, the paper considers PCA-based dimension reduction and analyzes SigClust-LCP p-values after PCA.PCA or MDS is recommended alongside SigClust to reduce dimension in practice.Source: §3 · p. 12, §3.3 · p. 16

3.1 Asymptotic Properties for Score Matching LCP

• Under regularity conditions, score-matching LCP is consistent in Fisher divergence and estimates log-concave densities with polynomial sample complexity n = O(d^β), β > 1.The results apply when the number of neural-network parameters grows linearly or polynomially with the data dimension.Source: §3.1 · p. 13

3.2 Properties of SigClust-LCP

• Under unimodal non-Gaussian nulls, original SigClust can be miscalibrated: its p-value tends to 1 when RFX < 2/π, and multivariate results extend this analysis.The one-dimensional results also identify anti-conservative behavior when RFX is above 2/π.Source: §3.2 · p. 14
• For log-concave distributions, the result indicates that statistical size can be controlled under a condition, though the cited passage does not specify it.Source: §3.2 · p. 14
• With estimated references, p-value consistency depends on Wasserstein-2 estimation rate and the m/n regime; full-data testing can fail at large n, motivating consistent subset tests aggregated for power.The theorem gives rate conditions for uniform consistency, while the proposed remedy tests data subsets and aggregates their p-values.Source: §3.2 · p. 15
• All SigClust methods, including original Gaussian SigClust, are powerful for mixtures beyond independent Gaussian mixtures when moment and covariance conditions are mild.The stated scope includes non-Gaussian distributions with higher-order correlations.Source: §3.2 · p. 15
• For the stated two-cluster setting with known log-concave noise, Algorithm 1's empirical p-value converges to 0 as dimension grows with fixed sample size.The theorem assumes independent samples from two clusters and places conditions on the noise distribution's eigenvalues and trace.Source: §3.2 · p. 15

3.3 Property for PCA-based Dimension Reduction

• For mixing independent sub-exponential noise, the leading principal component can still recover the clustering structure when the signal-to-noise ratio is sufficient.The passage notes that log-concave noise may be sub-exponential rather than sub-Gaussian.Source: §3.3 · p. 16
• Under Lemma 3.7’s assumptions, the leading principal component recovers cluster assignments with sup-norm error o_P(1/√n).The result applies to two clusters with independent log-concave noise and a sufficiently strong clustering signal-to-noise ratio.Source: §3.3 · p. 16
• With q = 1 under Lemma 3.7’s settings and assumptions, SigClust-LCP’s empirical p-value converges to 0 in probability.The argument contrasts an observed cluster index converging to 0 with a null cluster index bounded away from 0.Source: §3.3 · p. 17

4.1 Generalized Normal Distributions

• SigClust-LCP, RIFT, and MRIFT controlled Type-I error at the nominal 0.05 level across generalized-normal settings; SigClust-LCP also retained error control under non-independent correlations, unlike RIFT and MRIFT.Other methods failed to control size across the three dimensional settings.Source: §4.1 · p. 18, Figure 2 · p. 19
• As dimension increased, SigClust-LCP(Score) gained power over SigClust-LCP(ICA) and outperformed RIFT and MRIFT; at d = 50, SigClust-LCP power was comparable to theirs.All methods became more likely to reject the null as cluster difference increased.Source: §4.1 · p. 18, Figure 2 · p. 19

4.2 Beta Distributions

• Overall, SigClust-LCP consistently outperforms other methods in controlling Type-I error and exhibits comparable power; Figure 3 reports simulation results for beta distributions.The caption states that a = 0 denotes the null hypothesis of a single cluster.Source: §4.2 · p. 19, Figure 3 · p. 20

4.3 Normal Distributions

• Under standard-normal simulations, SigClust-LCP and SigClust-MDS preserved Type-I error, while SigClust-Hard, SigClust-Soft, and k-means inference became increasingly anti-conservative as dimension increased.The reported setting used n = 1000, independent variables, and a unidirectional signal; covariance-estimation inaccuracies were given as the reason for inflation.Source: §4.3 · pp. 19–20, Figure 4 · p. 21
• In the standard-normal simulations, SigClust-LCP required slightly stronger signal than SigClust-MDS to reject the null; SigClust-LCP(Score) had comparable power at d = 50 and 100 and outperformed RIFT and M-RIFT across all settings.Source: §4.3 · p. 20, Figure 4 · p. 21

4.4 Count Distributions

• For negative-binomial and multinomial counts, SigClust-LCP controlled Type-I error under both distributions; CHOIR was anti-conservative under both, while scSHC and Recall were anti-conservative under multinomial counts.These simulations used n = 1000 and d = 1000.Source: §4.4 · p. 21, Table 1 · p. 22
• For count distributions, SigClust-LCP had power comparable to scSHC and Recall, though it was slightly conservative in some scenarios.Source: §4.4 · p. 21, Table 1 · p. 22

4.5 Sensitivity Analysis

• The sensitivity analysis considers multiple SigClust-LCP hyperparameters, including the truncation dimension.Source: §4.5 · p. 22
• SigClust-LCP preserves nominal Type-I error across truncation dimensions 2, 5, 10, and 30, and standard neural-network architectures generally yield similar performance.Source: §4.5 · p. 22
• Across sample sizes and correlation structures, SigClust-LCP consistently outperforms other methods while exhibiting desirable statistical properties.Source: §4.5 · p. 22
• Score-matching LCP estimates a 5-dimensional density within one minute, compared with more than 20 minutes for LogConcDEAD, and has a computational advantage over ICA-based LCP in high dimensions.Source: §4.5 · p. 22
• Clustering-significance testing with SigClust-LCP completes in 1–5 minutes across a large variety of sample sizes and data dimensions.Source: §4.5 · p. 22

Analysis approach

• The analysis applies a continuous log-concave reference distribution to count-valued expression data after log-normalization and dimension reduction.The authors note that discreteness does not itself violate log-concavity, using the Poisson distribution as an example.Source: §5 · p. 23

Within-annotation comparisons

• Several methods that fail to control Type-I error infer spurious subclusters; Gaussian-based methods classify most annotated states as further clusterable, possibly reflecting non-Gaussian expression distributions.scSHC suggests extra subpopulations in battery and tentacle cells, while CHOIR appears to overcluster the endodermal lineage.Source: §5 · pp. 23–24

Cross-lineage comparisons

• Recall rejects 7 of 20 mixed-lineage groups, while scSHC and CHOIR also show spurious clustering under the single-cell-state null.Recall's limited rejections may mean it overlooks genuine biological signals in practice.Source: §5 · p. 24

Within-annotation comparisons

• SigClust-LCP, RIFT, MRIFT, and Recall support most original manual annotations, while body-column populations may contain multiple subpopulations and other annotated states fit one community.Seven of eight methods reject the single-cluster null for both body-column cell states, supporting possible additional positional subpopulations.Source: §5 · p. 23, Table 2 · p. 24

Cross-lineage comparisons

• Among methods preserving Type-I error, SigClust-LCP was one of the most powerful, rejecting all tested alternatives involving cells from multiple lineages.The displayed Table 3 rows cover pairs with at least one method not significant at 0.05.Source: §5 · p. 24, Table 3 · p. 25
• For 20 mixed-lineage pairs, SigClust-LCP, MRIFT, and CHOIR reject all nulls, while RIFT and scSHC fail to reject two and four, respectively.SigClust-Hard, SigClust-Soft, and k-means inference do not preserve Type-I error, so their alternative-hypothesis p-values are not directly comparable.Source: §5 · p. 24, Table 3 · p. 25

Discussion

• The framework uses log-concave projection to construct a clustering null beyond the restrictive Gaussian assumptions common in existing methods.It accommodates a broad class of unimodal distributions.Source: §6 · p. 25
• A score-matching algorithm adapted to log-concave distributions supports null-distribution estimation in moderate to high dimensions and broadens the applicability of LCP solvers.The approach is motivated by clustering-based definitions of putative cell types or states that are carried into downstream analyses.Source: §6 · p. 25

Future directions

• The authors identify multi-class clustering, including hierarchical and tree-structured settings, as a natural framework extension.Source: §6 · p. 25

Limitations

• Because score-based LCP requires a score function and is naturally suited to continuous distributions, explicitly accommodating discrete data remains future work despite robustness to gene-expression counts.Source: §6 · p. 25