PaperlayerView exact revision on arXiv

Source-linked AI summary

Semi-Supervised Learning under Spatially Biased Sampling

Bright Wiredu Nuakoh, Francky Fouedjio, Stephen Bradshaw, Yaw Kwaafo Awuah-Mensah, Wei Hong Tan, Emet Arya, Ebenezer Afrifa-Yamoah

arXiv:2609.07982v1cs.LG

TL;DR

SSL assumes labelled and unlabelled data share a distribution, an assumption spatially biased sampling can violate. The paper varies mismatch separately from spatial dependence and non-stationarity, finding abrupt degradation and testing diagnostics across synthetic and real datasets.

  • Problem

    SSL assumes labelled and unlabelled samples share a distribution, but spatially biased label collection can make their covariate marginals diverge.

    Source: §1 · pp. 1–2

  • Method

    The paper varies spatial mismatch, autocorrelation, and non-stationarity independently in a synthetic framework and evaluates SSL methods, divergence diagnostics, and real spatial datasets.

    Source: §1 · p. 2, §3.2 · p. 5, §3.1 · p. 5

  • Results

    Across synthetic and real settings, SSL shows threshold-like degradation under mismatch, with a synthetic breakpoint around 0.71–0.77; non-stationarity also degrades performance independently.

    Source: §5 · pp. 20–21, §6 · p. 21

  • Takeaways & Limitations

    Global divergence metrics track performance loss more reliably than a naïve local statistic, while a kernel-weighted correction improves the localized correlation.

    Source: §6 · p. 21

    Real-dataset directions depend on the fixed split or ordering, and six severity levels cannot distinguish a true breakpoint from a smooth nonlinear curve.

    Source: §4.7 · p. 17, §4.10 · p. 19

Abstract

from arXiv · show

Standard semi-supervised learning (SSL) typically relies on labelled and unlabelled data sharing a common marginal distribution. This assumption is often violated by biased spatial sampling mechanism, when labels are collected under spatially biased or preferential site selection. We treat this marginal mismatch, spatial autocorrelation, and spatial non-stationarity as three distinct mechanisms, varied independently via a labelled-sampling concentration parameter, a spatial length scale, and a non-stationarity strength parameter, and ask how mismatch degrades SSL, whether the cluster and manifold assumptions survive it, and how the resulting failure can be diagnosed. Using a controlled synthetic framework alongside PovertyMap-WILDS, California housing, socio-economic and US air quality monitoring datasets, we systematically vary the degree of mismatch while accounting for spatial autocorrelation and non-stationarity. Through a series of analyses including a segmented-regression changepoint, we show that in the synthetic generator, SSL performance does not degrade gradually but instead exhibits a threshold-like breakdown between approximately 0.71 and 0.77 once distribution mismatch becomes sufficiently severe. We further demonstrate that spatial non-stationarity contributes to performance loss independently of marginal mismatch and that models become increasingly overconfident outside the regions where labels are available. To support practical deployment, we evaluate several distribution-divergence measures as indicators of reliability and introduce a kernel-weighted local divergence metric that provides a more stable estimate of spatial mismatch than a na\"ive localised approach. These findings provide empirical evidence and diagnostic tools for better documenting the risk of incorporating unlabelled spatial data into semi-supervised learning workflows.

1 Introduction

The paper distinguishes marginal mismatch from spatial autocorrelation and non-stationarity, then tests how these mechanisms affect SSL. It contributes evidence on performance breakdown, divergence-based reliability indicators, and hypotheses about method sensitivity and mitigation.

1 Introduction

  • SSL relies on labelled and unlabelled samples sharing a distribution, but spatially biased coverage can violate that assumption.Spatial autocorrelation and non-stationarity do not by themselves imply marginal mismatch.Source: §1 · pp. 1–2
  • Spatial autocorrelation, non-stationarity, and marginal mismatch are distinct mechanisms, and only biased sampling directly makes labelled and unlabelled covariate marginals diverge.Autocorrelation concerns spatial similarity, while non-stationarity concerns geographic variation in the covariate–outcome relationship.Source: §1 · p. 2
  • The paper frames mismatch and non-stationarity as distinct dataset shifts, introduces a kernel-weighted local divergence estimator, and finds a threshold-like breakdown around α ≈0.71–0.77 in five of six method/metric combinations.The sixth combination, label propagation’s accuracy breakpoint, is much less precisely located.Source: §1 · p. 2
  • The study asks whether increasing marginal mismatch degrades SSL, whether cluster- and manifold-based methods are more sensitive to spatial heterogeneity, and whether spatially explicit or alignment-aware training mitigates losses.These questions are expressed as four hypotheses, including whether spatially explicit SSL outperforms standard approaches under mismatch.Source: §1 · pp. 2–3

2 Related Work

Prior work identifies SSL’s structural assumptions and the risks of out-of-distribution unlabelled data, while leaving geographic mismatch and its interaction with spatial structure less established. The paper situates its study among domain adaptation, spatial learning methods, and applications where labels are geographically concentrated.

2 Related Work

  • Earlier SSL work showed that out-of-distribution observations in the unlabelled pool reduce performance, but did not test geographic sampling mismatch.The cited study induced shift by withholding classes; safe SSL asks whether unlabelled data can be used without underperforming the supervised learner.Source: §2 · p. 3
  • Spatial autocorrelation challenges independence-based guarantees, while non-stationarity changes the covariate–label relationship; their separate effects and interaction with SSL remain underexplored.The literature also reports that ignoring autocorrelation can inflate apparent performance, although appropriate validation design remains debated.Source: §2 · p. 3
  • Domain adaptation studies covariate shift and divergence measures, but global metrics can hide local mismatch while naïve local estimates become unreliable with small samples.PovertyMap-WILDS includes geographic shift but treats it as a fixed split rather than a varied quantity and does not compare SSL families.Source: §2 · p. 3
  • Geographically weighted models and geostatistical pseudo-labelling offer spatially explicit alternatives, but prior geostatistical comparisons lacked a controlled mismatch sweep and comparisons across graph- and consistency-based SSL.Fouedjio and Talebi (2022) reported better performance than self-training on continuous geochemical targets.Source: §2 · p. 4
  • Housing transactions, sparse socio-economic surveys, and non-randomly sited air-quality monitors create settings where spatially biased labels make extrapolation consequential.These applications motivate combining limited labels with dense unlabelled covariates.Source: §2 · p. 4

3 Problem Formulation and Methods

The methods formalize marginal consistency separately from spatial dependence and non-stationarity, then vary these mechanisms in synthetic and real spatial settings while measuring global and local mismatch.

3.1 Notation and probabilistic formulation

  • SSL assumes matched labelled and unlabelled covariate marginals plus a cluster- or manifold-style conditional structure; spatial dependence and non-stationarity alone do not violate marginal matching.Source: §3.1 · pp. 4–5

3.2 Synthetic data-generating process

  • The synthetic generator varies autocorrelation range, non-stationarity strength, and labelled-site concentration independently while holding the spatial field and response law fixed.Source: §3.2 · pp. 5–6

3.3 Measuring distribution mismatch

  • Global mismatch is measured with one-directional KDE KL, standardized-feature W1, and RBF-kernel MMD; KL may understate mismatch when PU extends beyond PS support.Source: §3.3 · p. 6
  • Kernel-weighted local MMD replaces hard grid cells with continuous spatial weights and effective sample sizes, avoiding all-or-none support and grid-estimate noise under severe mismatch.The grid approach can become noise-dominated as local overlap collapses.Source: §3.3 · pp. 6–7

3.4 Semi-supervised learning methods

  • The main comparison centers on supervised-only, self-training, and label propagation, representing cluster and manifold assumptions; seven additional method families are reported in the supplement.Source: §3.4 · p. 7

3.5 Distribution-aware framework

  • Distribution-aware self-training reweights labelled points toward PU; geographic weighting and a distance-adaptive threshold extend it in the three-component framework.Source: §3.5 · p. 7

3.6 Data sources

  • Four real datasets cover satellite poverty, housing, county socio-economic outcomes, and PM2.5; labelled regions are restricted while training regions supply unlabelled/in-region tests and held-out regions supply OOD tests.Source: §3.6 · pp. 7–9, Table 1 · p. 9

3.7 Experimental protocol and changepoint analysis

  • The synthetic sweep spans six α levels, three autocorrelation scales, and two non-stationarity levels (324 runs), with a 2,160-run rerun and seed-bootstrap breakpoints.Performance is assessed using accuracy, F1, expected calibration error, and the spatial generalisation gap.Source: §3.7 · pp. 9–10

4 Results

Increasing marginal mismatch produces a threshold-like decline in out-of-region accuracy, with label propagation degrading more gradually than the other baselines. The comparison also finds that SSL methods continue to outperform matched supervised-only controls across the mismatch sweep.

4.1 Effect of marginal distribution mismatch

  • Both SSL methods outperform supervised-only on identical data draws at every mismatch level, with confidence intervals excluding zero.Thus, the observed performance decline does not show that unlabelled data are harmful relative to not using them.Source: §4.1 · p. 11
  • Accuracy remains relatively stable through α ≤0.6, then declines sharply; the spatial generalisation gap widens to 0.05–0.07 at α = 1.The widening gap indicates increasingly poor generalisation beyond labelled regions.Source: Figure 4 · p. 11, §4.1 · p. 11, Figure 5 · p. 12
  • Out-of-region accuracy drops 4.2 pp for label propagation, 5.6 pp for self-training, and 6.2 pp for supervised-only; label propagation degrades significantly less than both baselines.Paired seed-cluster bootstrap comparisons find label propagation’s drop reliably smaller than each baseline’s.Source: §4.1 · p. 11, Table 2 · p. 12, §4.2 · p. 12

4.2 Cluster and manifold sensitivity

  • Synthetic results provide only partial support for greater cluster- or manifold-assumption sensitivity: self-training’s 5.6 pp decline is not distinguishable from supervised-only’s 6.2 pp decline.Label propagation’s 4.2 pp decline is reliably smaller than both baselines’.Source: §4.2 · p. 12

4.3 Spatial non-stationarity in isolation

Spatial non-stationarity reduces accuracy independently of marginal mismatch, while divergence metrics are associated with out-of-region performance. Global measures track performance more strongly than localised estimates, and the kernel-weighted local estimator is not supported as a within-severity, per-run reliability signal.

4.3 Spatial non-stationarity in isolation

  • At α = 0, increasing non-stationarity from 0 to 1.5 reduces accuracy by about 3.3 percentage points; supervised-only accuracy falls from 0.883 to 0.850.Across tested α values, the non-stationarity effect remains about 2.8–3.4 points below the stationary case, indicating largely independent contributions from non-stationarity and marginal mismatch.Source: §4.3 · pp. 12–13

4.4 Divergence metrics as proxies for performance loss

  • Global KL divergence, maximum mean discrepancy, and Wasserstein distance correlate with out-of-region accuracy at r = −0.52, −0.48, and −0.45, respectively.All three 95% confidence intervals exclude zero; the proposed kernel-weighted local estimator improves on the fixed-grid estimator from r = −0.28 to r = −0.40, but remains weaker than global metrics.Source: §4.4 · p. 13, Figure 6 · p. 14, Table 3 · p. 14
  • After controlling for α, length-scale, and non-stationarity, global metrics retain moderate correlations with accuracy, while the kernel-weighted local estimator falls to r = −0.20.Within individual α levels, its accuracy correlation is not distinguishable from zero; it mainly indicates severity regime rather than which run will underperform.Source: §4.4 · p. 13

4.5 Effect of mismatch on calibration and accuracy

Severe mismatch affects both predictive accuracy and confidence calibration outside labelled regions. On PovertyMap-WILDS, the threshold-like decline recurs as labelled coverage narrows, but the apparent differences among methods are not statistically conclusive.

4.5 Effect of mismatch on calibration and accuracy

  • Out-of-region expected calibration error roughly doubles from α = 0 to α = 1, while in-region calibration error remains stable or improves slightly.Self-training’s out-of-region calibration error increases by 0.062 points, compared with 0.041 for supervised-only and 0.038 for label propagation.Source: §4.5 · p. 14, Table 4 · p. 15

4.6 Benchmark validation on PovertyMap-WILDS

  • PovertyMap-WILDS out-of-region accuracy drops 14.6 percentage points for self-training, 12.1 for label propagation, and 10.2 for supervised-only, but paired method differences are not statistically distinguishable.The paired 95% confidence intervals include zero for all three comparisons.Source: Table 5 · p. 15, §4.6 · p. 15
  • On PovertyMap-WILDS, all methods remain comparatively stable while several training countries remain available, then decline when labelled data are restricted to one to three countries.This reproduces the threshold shape observed in the synthetic experiments.Source: §4.6 · p. 15, Figure 7 · p. 16
  • Unlike the synthetic experiments, supervised-only has the best point estimate on PovertyMap-WILDS, but its difference from label propagation is uncertain because their confidence intervals overlap.The result should not be treated as a clear reversal in relative robustness.Source: §4.6 · p. 15

4.7 Validation on real datasets

Across three real-world datasets, mismatch effects varied: housing and air quality declined sharply only at severe mismatch, while socio-economic results depended on the method. Alternative spatial splits sometimes changed the reported direction.

4.7 Validation on real datasets

  • 12.2–13.8 percentage-point losses affected three of four housing methods, while all air-quality methods lost 11.0–15.4 percentage points at maximum mismatch.Label propagation’s housing decline was smaller at 4.3 percentage points but still significant; all air-quality confidence intervals excluded zero.Source: §4.7 · p. 16
  • Self-training and re-weighted self-training gained 5.0 and 5.2 percentage points on socio-economic data, while supervised-only and label propagation showed no reliable change.A plain logistic regression changed by only +0.3 points when trained on anchor-region labels; anchor-to-held-out feature–label correlation similarity was 0.974.Source: §4.7 · pp. 16–17, Table 6 · p. 17, Table 7 · p. 17
  • The socio-economic increase was specific to iterative pseudo-labelling methods, not a general property of restricting labelled data to one region.A same-size random-region control reached 0.664 accuracy versus 0.888 for the anchor region at α = 1; the authors flag the anchor-specific increase as unexplained.Source: §4.7 · p. 17
  • Reported real-dataset directions were not stable across alternative spatial splits or PovertyMap-WILDS country orderings.Housing’s decline shrank below one point under one alternative split, while other directions flipped under at least one alternative split or ordering.Source: §4.7 · p. 17

4.8 The distribution-aware framework

Re-weighting modestly reduced mismatch-related performance loss, but adding geographic-proximity weighting and adaptive pseudo-labelling did not help further. Real-dataset patterns remained mixed across datasets and methods.

4.8 The distribution-aware framework

  • Across five datasets, average performance drop decreased from 6.6 percentage points for standard self-training to 5.6 percentage points with re-weighting.On PovertyMap-WILDS, the reduction in performance loss was 2.7 percentage points in the twenty-seed experiments.Source: §4.8 · p. 17
  • Housing and air quality showed late-onset accuracy declines with mismatch, whereas socio-economic accuracy increased for self-training and re-weighted self-training.The figure describes out-of-region accuracy across mismatch severity by method for the three real-world datasets.Source: Figure 8 · p. 18
  • The complete framework’s average performance drop was 11.5 percentage points, and its additional proximity-weighting and adaptive pseudo-labelling components did not improve performance further.A domain-adversarial variant outperformed the original framework on four of five datasets.Source: §4.8 · pp. 17–18
  • The mitigation analysis was a secondary investigation, with the complete framework and further results reported in supplementary sections.The full framework and ablation analyses are in Supplementary Material Sections S1.5 and S2.1.Source: §4.8 · p. 18

4.9 Spatial awareness across method families

The expanded evaluation across ten SSL methods offered only partial support for spatially explicit methods’ robustness. Performance varied by method family and dataset size.

4.9 Spatial awareness across method families

  • The spatial graph neural network’s average endpoint drop was 4.9 percentage points versus 6.9 for supervised-only, but its drop was more than twice as large on air quality.The authors connect the air-quality exception to the small dataset’s potentially inadequate k-nearest-neighbour graph for reliable message passing.Source: §4.9 · pp. 18–19
  • Fouedjio and Talebi’s geostatistical method showed no clear robustness advantage in the twenty-seed experiments, though it remained strongest on the smallest dataset.Modern consistency-based methods, including FixMatch and Mean Teacher, had smaller average declines than self-training under increasing mismatch.Source: §4.9 · p. 19
  • The geographically weighted network was the least robust method across the evaluated settings.The broader comparative study and method-specific analyses are presented in Supplementary Material Section S2.2.Source: §4.9 · p. 19

4.10 Sensitivity analysis and location of the breakdown

Segmented regression on the twenty-seed synthetic experiments estimated threshold-like breakdowns, with uncertainty varying by method. The authors caution that the sparse severity grid cannot validate a true changepoint.

4.10 Sensitivity analysis and location of the breakdown

  • Five of six breakpoint estimates had relatively narrow confidence intervals, but label propagation’s accuracy breakpoint interval was [0.44, 0.78].The broad interval accords with label propagation’s more gradual decline.Source: §4.10 · p. 19
  • 0.71–0.77 was the estimated breakpoint range across methods and metrics in the synthetic experiments.The sharpest decline began slightly later than visual inspection suggested; Figure 9 displays fitted regressions and breakpoint locations.Source: Table 9 · p. 19, §4.10 · p. 19, Figure 9 · p. 20
  • Six severity levels and five fitted parameters make the segmented model nearly interpolating, so the analysis cannot distinguish a true break from a smooth nonlinear curve.The authors call for denser α sampling, especially between 0.6 and 1.0, and describe the estimates as specific to the synthetic generator rather than a validated changepoint.Source: §4.10 · pp. 19–20

5 Discussion

Marginal mismatch degrades SSL non-uniformly across methods and sharply beyond a synthetic-generator threshold, while spatial non-stationarity independently degrades performance. Divergence measures can help assess reliability, but mismatch diagnostics and covariate-shift corrections do not address every failure mode.

5 Discussion

  • Between roughly 0.71 and 0.77 on the synthetic mismatch axis, SSL shows a threshold-like breakdown whose location has not been calibrated for real labelling campaigns.The threshold extends the matched-versus-out-of-distribution framing of Oliver et al. (2018) to a continuously increasing mismatch severity.Source: §5 · p. 20
  • In synthetic experiments, label propagation’s smaller accuracy drop is distinguishable from self-training and supervised-only, but FixMatch is more robust than conventional self-training.On PovertyMap-WILDS, accuracy rankings are not statistically distinguishable; calibration instead shows self-training becoming overconfident outside labelled regions while label propagation appears comparatively robust.Source: §5 · p. 20
  • Spatial non-stationarity degrades performance independently of marginal mismatch, distinguishing changes in covariate–outcome relationships from changes in covariate distributions.The two mechanisms have comparable effects in isolation, so covariate-shift correction alone is unlikely to eliminate degradation.Source: §5 · pp. 20–21
  • Global divergence measures tracked performance degradation, while a kernel-weighted local estimator produced more stable estimates than the naïve fixed-grid approach under sparse sampling.The diagnostics and estimated breakpoint can help assess geographic coverage, but re-weighting can only partially mitigate failure when non-stationarity also matters.Source: §5 · p. 21

6 Conclusion

Biased sampling, rather than spatial data itself, is associated with a sharp SSL breakdown, while non-stationarity can degrade performance independently of mismatch. The threshold and divergence diagnostics are useful as relative reliability indicators, but the breakpoint is not calibrated for deployment.

6 Conclusion

  • SSL performance collapses between mismatch severities 0.71 and 0.77 in the synthetic generator; affected models can become confidently wrong outside labelled regions.The threshold breakdown and self-training’s greater point-estimate fragility replicate on PovertyMap-WILDS; threshold degradation also recurs across three real datasets, two consistently.Source: §6 · p. 21
  • Spatial non-stationarity degrades performance through a mechanism separate from mismatch, including when labelled and unlabelled marginals are identical.The conclusion distinguishes these mechanisms from spatial dependence, which is not itself the culprit.Source: §6 · p. 21
  • Global divergence metrics tracked loss more reliably than a naïve localized statistic, and kernel weighting raised localized correlation from −0.28 to −0.40.The diagnostics can serve as relative reliability indicators across candidate labelled samples for the same deployment target.Source: §6 · pp. 21–22
  • The synthetic breakpoint is not calibrated to an absolute deployment-ready threshold, and correcting concept shift rather than only covariate shift remains unresolved.Density-ratio re-weighting helps only marginally even where covariate shift dominates, while the fuller framework can worsen results.Source: §6 · pp. 21–22

Related papers

Loading 2609.07982v1…