PaperlayerView exact revision on arXiv

Source-linked AI summary

Puffin: Probabilistic Learning of Spatial Detail From Coarse Observations

Chaitanya Jobanputra, Sebastian Vollmer, Gerrit Großmann

arXiv:2610.11914v1cs.LG

TL;DR

Censuses and registers often provide coarse totals rather than fine-scale socioeconomic measures, despite the importance of local detail. PUFFIN learns probabilistic subregion estimates from satellite embeddings and coarse totals, conditioning predictions on each total. Across evaluations, its accuracy varies with scale and signal; calibrated intervals require a fine-label-derived calibration factor.

  • Problem

    Socioeconomic outcomes are often reported only as coarse regional totals, leaving fine-scale maps unavailable for applications that need local detail.

    Source: §1 · p. 1

  • Method

    PUFFIN learns subregion outcome distributions from satellite embeddings using regional totals alone, then conditions the prior on each observed total to produce exact aggregate-consistent estimates.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Within-region R2 ranges from 0.12 to 0.84, with learned splits above bicubic interpolation across the reported scales; performance is weaker where fine-scale signal is absent or misaligned with embeddings.

    Source: §5 · p. 9, §7 · p. 11

  • Takeaways & Limitations

    Point estimates need no fine-resolution labels and apply where regional totals and globally available satellite embeddings exist, while success depends on fine variation being visible in the embeddings.

    Source: §1 · p. 2, §5 · p. 9, §7 · p. 11

    Calibrated intervals are not label-free: their global factor κ is estimated from fine-scale validation errors, and transferring it to data without fine labels does not work.

    Source: §5 · p. 10

Abstract

from arXiv · show

High-resolution socioeconomic variables are important for applications such as urban planning, public health, disaster response, and resource allocation. In practice, however, these variables are often observed only at a coarse spatial resolution. We introduce Puffin, a probabilistic framework for statistical disaggregation that raises the resolution of coarse totals using high-resolution satellite embeddings as covariates. Instead of predicting a single value for each fine-resolution subregion, Puffin learns a probability distribution and is trained through an aggregation-aware likelihood. At inference, Puffin conditions these predictions on the observed regional total and splits it among the subregions. The resulting fine-scale estimates are consistent with the observed aggregate and come with calibrated uncertainty, without requiring fine-resolution labels for training. We evaluate Puffin on German and US census, employment, and election data across population, jobs, and other count variables, and study when statistical disaggregation succeeds or fails across regions, countries, and targets.

1 INTRODUCTION

The paper targets fine-scale mapping of socioeconomic outcomes when censuses and registers provide only coarse totals, using globally available satellite embeddings. PUFFIN learns subregion distributions from totals alone, conditions them on each observed total, and evaluates where its disaggregation succeeds or fails.

1 INTRODUCTION

  • PUFFIN predicts a prior distribution for each subregion from satellite embeddings, then conditions it on the observed regional total to produce a posterior that splits the total exactly.Figure 1 depicts the posterior means; training uses regional totals without fine-resolution labels.Source: Figure 1 · p. 1, §1 · p. 2
  • Censuses and registers report one total per tract or county because finer surveys are expensive and such statistics protect privacy, while satellite imagery provides broad-coverage socioeconomic signals.The introduction describes imagery at 10 to 100 m resolution and cites prior work for its socioeconomic signal.Source: §1 · p. 1
  • Existing gridded population products spread regional totals across curated auxiliary layers such as night lights, roads, or building footprints, and publish a single value per cell.The layers are laborious to assemble and differ from country to country.Source: §1 · p. 2
  • PUFFIN frames disaggregation as probabilistic constraint satisfaction, learning from regional totals alone without fine-resolution labels and returning predictive distributions rather than bare point values.Its posterior is consistent with each aggregate, and two-stage calibration is reported to give intervals with nominal marginal coverage across resolutions, countries, and targets.Source: §1 · p. 2
  • Held-out-region evaluations span Berlin and Germany, irregular US units, multiple resolutions, population and jobs, and identify settings where linear baselines match or PUFFIN does not help.The cited overview names precinct turnout and rare events among the cases where PUFFIN does not help.Source: §1 · p. 2

2 NOTATION AND PROBLEM SETTING

The setting links unobserved fine-scale outcomes to observed regional totals through sum aggregation. Subregion covariates and the totals provide the information for inferring fine-scale values with uncertainty.

2 NOTATION AND PROBLEM SETTING

  • Each coarse region contains M same-sized subregions in the simplified notation, though equal size and a fixed subregion count are not required in practice.Regions are observation units such as census tracts or counties; subregions are target units such as grid cells or US census blocks.Source: §2 · p. 2
  • The fine-scale outcome for each subregion is non-negative, while only a single total per coarse region is observed.The subregion outcome may, for example, be the number of residents in a grid cell or census block.Source: §2 · pp. 2–3
  • Each subregion has contextual covariates represented by an embedding, and fine- and coarse-scale outcomes are linked by sum aggregation.The embeddings may be 64-dimensional AlphaEarth features; the formulation does not depend on that particular choice.Source: §2 · p. 3
  • Fine outcomes are never observed individually, making the problem weakly supervised: each regional total is shared across its subregions.Source: §2 · p. 3
  • The goal is to infer fine-scale outcomes that agree with the regional aggregates while representing uncertainty in those inferred values.Source: §2 · p. 3

3 OUR METHOD: PUFFIN

PUFFIN predicts independent subregion outcome distributions from contextual embeddings and fits them using observed regional totals. It conditions the learned prior on each total, then implements area-aware, non-negative estimates and calibrated intervals.

3 OUR METHOD: PUFFIN

  • PUFFIN learns a prior over subregion outcomes from contextual covariates and conditions it on the observed regional total, yielding a closed-form posterior that conserves the total exactly.The model is trained through the aggregation constraint rather than fine-resolution labels.Source: §3 · p. 3, §3.1 · p. 3

3.1 OVERALL IDEA

  • The prior assumes independent subregions, and the Gaussian posterior adjusts predicted means upward or downward according to the total residual, with larger-variance subregions adjusted more.The prior is fitted by maximizing the likelihood assigned to observed regional aggregates, not through the posterior.Source: §3.1 · pp. 3–4

3.2 IMPLEMENTATION

  • For unequal-sized subregions, the network predicts mean and variance per unit area and uses area to convert both into counts.The area term cancels on a uniform grid; varying numbers of subregions per region require no change beyond using the region-specific count.Source: §3.2 · p. 4
  • Scaling variance by subregion area improves US-block within-region R2 by about 0.10 compared with dropping the area factor, while allowing larger blocks to absorb more of a regional correction.US census block areas span about four orders of magnitude between the 1st and 99th percentiles.Source: §3.2 · p. 4
  • Negative MAP estimates are clipped to zero and the remaining estimates rescaled to preserve each total, with an even split used if all estimates vanish.The resulting PUFFIN estimate is the reported point estimate; posterior variances supply the intervals.Source: §3.2 · p. 4
  • Posterior variances are rescaled in two stages to form intervals: coarse residuals set tile-specific factors, while a single validation-set factor κ uses fine-scale errors.Only κ uses fine labels, on validation regions disjoint from those evaluated.Source: §3.2 · p. 4
  • A single-hidden-layer network with 32 hidden units and about 2,000 parameters predicts subregion means and variances; training minimizes the aggregate likelihood loss.Totals are standardized by average training density, and whole regions are minibatched when country-scale configurations do not fit in memory.Source: §3.1 · pp. 3–4, §3.2 · p. 4

4 EXPERIMENTS

The experiments test whether satellite embeddings can recover subregion structure from aggregate totals using spatially held-out census, employment, and election data. PUFFIN performs well for population and jobs, but not when turnout does not follow the embedding or events are rare.

4 EXPERIMENTS

  • Five-fold spatial cross-validation scores within-region R2, which measures how well each method splits totals; shared baselines also reproduce totals and use no fine labels.The baselines are region average, bicubic interpolation, and Goodman-style ecological regression.Source: §4 · p. 5

4.1 EXPERIMENT 1: GERMAN CENSUS, CITY TO COUNTRY

  • Across Berlin’s 18 configurations, PUFFIN scores 0.12–0.68, while Goodman-style regression leads overall, 0.448 versus 0.427, winning 12 configurations.Goodman wins all six 10-km configurations, where Berlin has only 73 training regions; PUFFIN’s fine-scale error starts rising after about 100 epochs (Figure 4). Figure 2 illustrates recovered within-region population structure.Source: §4.1 · p. 6, Figure 2 · p. 6
  • In Germany, PUFFIN beats Goodman-style regression in 16 of 18 configurations, with mean within-region R2 of 0.561 versus 0.544.The margins are at most 0.04; the prior alone scores 0.483, below Goodman-style regression.Source: §4.1 · p. 6, Table 1 · p. 7

4.2 EXPERIMENT 2: US CENSUS, POPULATION AND JOBS

  • Intervals achieve 0.87–0.93 marginal coverage at the nominal 90% level across configurations, countries, and targets, but this is a weak test because most finest-scale subregions are empty.The calibration factor κ is a quantile of validation errors.Source: §4.2 · p. 7
  • On US census blocks, PUFFIN scores +0.377 for population and +0.209 for jobs; bicubic interpolation scores −0.41 and +0.06, and Goodman-style regression −1.04 and +0.05, respectively.Both baselines perform worse than the region average on population; PUFFIN remains ahead on inhabited blocks and subregion-R2 comparisons.Source: §4.2 · p. 7, Table 2 · p. 9

4.3 EXPERIMENT 3: WHERE PUFFIN DOES NOT HELP

  • For precinct turnout, every method adds error: bicubic interpolation scores −0.10, Goodman-style regression −0.31, and PUFFIN −0.43, all below the region average.Turnout does not follow the embedding, and precinct area actively misleads because area varies much more than turnout within counties.Source: §4.3 · pp. 8–9
  • For gun violence, only 3% of census blocks record an incident, and PUFFIN stays close to the region average at +0.01.The intervals retain nominal coverage in this negative case, so the reported failure is in the point estimate rather than mis-scaled intervals.Source: §4.3 · p. 9

5 DISCUSSION AND LIMITATIONS

PUFFIN’s within-region performance suggests satellite embeddings capture fine-scale variation, but its uncertainty intervals and structural assumptions constrain where its outputs transfer. The authors also note missing comparisons that limit the evidence for variance weighting.

5 DISCUSSION AND LIMITATIONS

  • Within-region R2 ranges from 0.12 to 0.84, above bicubic interpolation everywhere and far above it at all but the finest scale; gains point to embeddings rather than spatial smoothness.A linear density with the same embeddings keeps up in a single city and pulls ahead where training regions are few.Source: §5 · p. 9
  • Calibrated intervals do not transfer to a new resolution or unit type without a small labeled sample to set the global factor κ.The per-tile calibration transfers, but the global factor—estimated from fine-scale validation errors—does not; point estimates remain fine-label-free.Source: §5 · p. 10
  • The evidence for variance weighting rests on the prior comparison and area ablation because the same network with squared loss and a proportional split, and published gridded products, were not compared.Source: §5 · p. 10
  • Structural limits include independent subregions, one averaged embedding per unit, area-scaled variance that understates uncertainty in small dense units, and one variable at a time.The Gaussian prior permits negative counts, while intervals are symmetric and may extend below zero; lower endpoints should be read as zero.Source: §5 · p. 10

6 RELATED WORK

Prior disaggregation approaches conserve totals through curated auxiliary layers, fixed splits, covariance assumptions, or aggregate regression. PUFFIN instead learns a probabilistic split from satellite embeddings and coarse totals, linking work on aggregate learning, ecological inference, embeddings, and conformal intervals.

6 RELATED WORK

  • WorldPop, GHS-POP (Schiavina et al., 2023), and LandScan (Lebakula et al., 2025) use curated layers for dasymetric mapping, while Tobler’s interpolation applies a fixed mass-preserving rule.POMELO uses open geodata and a deep network; related work maps population from Sentinel imagery.Source: §6 · p. 10
  • Area-to-point kriging offers coherent downscaling with uncertainty, but assumes covariance structure, whereas PUFFIN learns splits from covariates.Other exact-conservation methods include climate downscaling and time-series regression disaggregation.Source: §6 · p. 10
  • CLAM (Großmann et al., 2026) is closest in spirit: coarse totals train fine-field recovery for causal local effects, unlike exact-conservation disaggregation of observed counts with calibrated uncertainty on real census and satellite data.Source: §6 · p. 10
  • Unlike supervised satellite regression that predicts at or within reporting units, PUFFIN disaggregates below them using aggregate-only supervision; its interval calibration adapts to new areas with coarse residuals, while fine labels set one factor per setting.The paper relates its training signal to learning from aggregate observations and learning from label proportions.Source: §6 · p. 10

7 CONCLUSIONS AND FUTURE WORK

PUFFIN learns subregion distributions from coarse totals and conditions them to produce non-negative estimates that preserve aggregates and carry calibrated uncertainty. Its point estimates apply across the studied settings without fine-resolution reference grids, while calibrated intervals require a small labeled sample; the authors identify extensions and evaluation gaps.

7 CONCLUSIONS AND FUTURE WORK

  • PUFFIN’s posterior conditions a learned prior on observed totals, producing non-negative estimates that reproduce each total exactly and carry calibrated uncertainty across resolutions, two countries, and several variables.Where fine-scale signal is nearly absent or does not follow the embedding, estimates fall back to or below the regional average.Source: §7 · p. 11
  • Point estimates need only published totals and globally available embeddings, but calibrated interval widths still require a small labeled sample.Source: §7 · p. 11
  • The authors propose head-to-head tests against deployed gridded products on held-out regions, broader country coverage, cell-level embeddings, count-dependent variance, and jointly conditioned correlated variables.Source: §7 · p. 11
  • Generative AI aided writing and literature retrieval, but did not generate the research idea, method, or experimental results; authors reviewed and edited assisted text.Source: AI USE STATEMENT · p. 11
  • The authors report that all data sources are public and that code producing every reported number is linked in the abstract.Source: REPRODUCIBILITY STATEMENT · p. 11
  • Fine-scale estimates of sensitive variables can be mistaken for observations or misused to single out small areas; the authors intend the method for aggregate planning and research, not individual inference.The paper notes that Experiment 3 shows such estimates can be unreliable.Source: ETHICS STATEMENT · p. 11

A EXPERIMENTAL DETAILS

The experiments test calibration transfer across settings and isolate the contribution of the area term, while documenting data construction and evaluation choices. They also expose limits of transferred calibration and the treatment of unobserved zero-count cells.

German Grids

  • German grids use 18 settings from 200 m to 10 km, with subregions at least 100 m wide and AlphaEarth embeddings averaged over each subregion.The study uses 2022 embeddings for Germany, 2020 for US census data, and 2024 for precincts.Source: A EXPERIMENTAL DETAILS · p. 14

Empty Subregions; US Units

  • German cells absent from the census file are labeled zero, while cells outside Germany are dropped; US census files already include zero-count blocks.The US data cover the contiguous states and DC; embedding-mosaic gaps leave block population about 1.3% below the published total, and Alaska and Hawaii are absent.Source: A EXPERIMENTAL DETAILS · pp. 14–15

Calibration Details

  • Calibration uses 10 km tiles; when a tile holds one region, γ2 rests on a single residual, making that estimate noisy.Stage 1 uses observed totals for every region, while Stage 2 fits κ on validation subregions; interval widths are not out of fold, and symmetric lower bounds can fall below zero at low counts.Source: A EXPERIMENTAL DETAILS · p. 15

Area Term Ablation

  • Removing the area factor lowered within-region R2 from +0.365 to +0.260 on population and from +0.208 to +0.112 on jobs, while the mean still scaled with area.The controlled ablation attributes the gap to the variance term alone.Source: A EXPERIMENTAL DETAILS · p. 15

Transferring κ

  • Transferred κ produced coverage from 0.32 to 1.00, while self-calibration reached 0.895 to 0.900 at nominal 0.90; residual scales differed, and country effects could not be separated.The transfer test varied resolution, unit type, and country together.Source: A EXPERIMENTAL DETAILS · p. 15

Optimization; Baselines

  • Training standardizes embeddings using training regions, selects checkpoints by validation aggregate likelihood, and uses five fixed spatial folds across methods and seeds.Baselines interpolate or regress regional density, then apply it to subregions and rescale to regional totals.Source: A EXPERIMENTAL DETAILS · pp. 14–15

B ADDITIONAL RESULTS

Additional analyses compare subregion-level performance with within-region results, report calibration coverage, and show learning curves and a city-scale jobs example. Method rankings are largely stable, but some comparisons change for inhabited-only subsets and US jobs.

Additional Results

  • Interval coverage is 0.897 to 0.902 across Germany configurations and 0.871 to 0.926 across Berlin configurations.Tables 7 and 8 repeat within-region comparisons on inhabited cells; Table 9 reports the US inhabited-block comparison.Source: B ADDITIONAL RESULTS · p. 15, Table 7 · p. 19, Table 8 · p. 19
  • Figure 6 plots training loss and fine-scale test error over training for Berlin at 1 km, Germany at 4 km, and Germany at 10 km.The figure presents learning curves for these remaining settings.Source: Figure 6 · p. 16
  • Figure 7 contrasts region averages, PUFFIN estimates, and ground truth for New York City jobs, showing concentrated jobs in small commercial blocks that PUFFIN recovers from embeddings and tract totals.The region average cannot express that block-level concentration.Source: Figure 7 · p. 16
  • Method ordering is largely unchanged on subregion-R2, but bicubic interpolation and Goodman-style regression swap on US jobs, and Goodman wins fewer Berlin and Germany inhabited-only configurations.Goodman wins 11 rather than 12 Berlin configurations and 4 rather than 2 Germany configurations when only inhabited subregions are scored.Source: B ADDITIONAL RESULTS · p. 16, Table 5 · p. 18, Table 6 · p. 18
Loading 2610.11914v1…