Learning a Size-Weight Frontier for Synthetic-Augmented Inference

Chengpiao Huang; Kaizheng Wang

Source-linked AI summary

Learning a Size-Weight Frontier for Synthetic-Augmented Inference

Chengpiao Huang, Kaizheng Wang

arXiv:2608.28576v1Aug 28, 2026stat.MEcs.AIcs.LGstat.ML

TL;DR

Synthetic data may reduce variance when real data are scarce but can bias inference and undermine coverage if treated as real. The paper learns a size–weight frontier from historical tasks and provides simultaneous finite-sample coverage for configurations on or below it, while experiments achieve nominal coverage and narrower intervals.

Problem
Synthetic data can reduce estimation variance but may introduce bias and unreliable coverage when naively combined with scarce real data.
Method
The framework learns a size–weight frontier from historical tasks to calibrate how many weighted synthetic observations can augment real data.
Results
Coverage remains above the nominal level 1 −α = 0.9 across both LLMs, proxy constructions, real sample sizes, and evaluated weights.
Takeaways & Limitations
The learned frontier supports downstream configuration selection while maintaining simultaneous coverage for configurations on or below it.
Takeaways & Limitations
The coverage guarantee is marginal over the task population and does not imply target coverage for every individual task.

Abstract

from arXiv · show

Synthetic data can improve statistical inference when real data are scarce, but naively treating synthetic samples as real data can introduce bias and lead to unreliable inference. We develop a general framework for synthetic-augmented inference across a population of related tasks. It characterizes synthetic augmentation by the number of synthetic observations and their weight. Central to our framework is a size-weight frontier that specifies, for each weight, the largest synthetic sample size for which all smaller sizes attain the target task-marginal coverage. We estimate this frontier from historical tasks, and establish a finite-sample coverage guarantee simultaneously for all size-weight configurations on or below the estimated frontier. In experiments using large language model responses to augment opinion survey data, our procedure achieves target coverage and substantially narrows confidence intervals.

1 Introduction

The paper frames synthetic augmentation as a coverage problem: synthetic data can reduce variance but may bias confidence sets when treated as real. It introduces a learned size–weight frontier that calibrates valid augmentation across related tasks.

Synthetic samples can narrow confidence intervals yet fail to maintain valid coverage when their distribution differs systematically from the human population.
The framework assigns each synthetic observation weight λ, while k controls the number of synthetic observations added to n real responses.
The central question is how many synthetic observations can be used at each weight while maintaining valid confidence-set coverage.
Historical tasks with additional real data provide reference confidence sets for evaluating and calibrating size–weight configurations on a target task.
For each candidate weight, the size–weight frontier gives the largest valid synthetic sample size, with all smaller sizes also covered.
Uniform coverage below the learned frontier permits selecting a configuration after calibration, such as by minimizing average confidence-set volume.

2 Problem Setup

The framework studies confidence-set construction when scarce real data are augmented with potentially misspecified synthetic observations. It calibrates synthetic sample size and weight through a size–weight frontier whose configurations preserve task-marginal coverage.

Motivating example: Synthetic data can reduce estimation variance, but distributional mismatch may produce narrower confidence intervals that fail valid coverage.The motivating survey example uses LLM-generated responses to augment a small human sample.
Calibration data: Historical tasks with additional real data provide reference confidence sets for calibrating which synthetic size–weight configurations attain target coverage.This setup supports cold-start inference for new users, items, or survey questions with limited real data.
General framework: An augmentation configuration (λ, k) assigns weight one to each real observation and nonnegative weight λ to each of k synthetic observations.The framework allows a target task to have n real observations and up to K synthetic observations.
Coverage criterion: Coverage is evaluated marginally over the task population, rather than requiring the target coverage level for every individual task.The paper explicitly notes that marginal coverage is weaker than task-conditional coverage.
Confidence-set constructions: The framework accommodates standard confidence-set constructions, including CLT and empirical Bernstein intervals, weighted hybrid-data intervals, and bootstrap sets for general parameters.When λ = 0, the example intervals reduce to their real-data-only counterparts.
Size–weight frontier: For each candidate weight, the oracle frontier is the largest synthetic size such that every smaller size achieves at least 1 −α coverage, with the frontier nonincreasing across ordered weights.Every configuration on or below the frontier therefore satisfies the target coverage condition.

3 Learning the Size–Weight Frontier

The paper learns valid synthetic size–weight configurations from historical tasks using coverage proxies, then constructs a frontier with simultaneous coverage guarantees for all configurations on or below it.

Calibration: Historical tasks provide calibration data for learning which synthetic size–weight configurations achieve the target coverage for a future task.The framework assumes historical tasks from the same task population with additional real observations.
Coverage proxies: Reference confidence sets proxy the unknown event that a candidate confidence set contains the task parameter.The full-sample proxy uses all available real observations, while the split-sample proxy uses held-out observations and conditional independence.
Frontier learning: The algorithm estimates the frontier recursively, retaining each weight–size pair only when empirical proxy coverage exceeds 1−(1−γ)α for every smaller size.The recursion makes the estimated frontier non-increasing across ordered weights, and k = 0 is always feasible.
Proxy trade-offs: The split-sample proxy reserves data for its reference set and requires task-conditional validity, whereas the full-sample proxy requires only marginal coverage validity.The full-sample proxy uses all available real observations; the split-sample proxy can reuse its reference set when the target level α changes.
Guarantee: Theorem 3.1 guarantees, with probability at least 1−δ over calibration data, target coverage for every configuration on or below the learned frontier, up to an error of order J−1/2.The logarithmic factor depends on K ∧ M; once M ≥ K, further grid refinement does not increase the statistical penalty.
Post-calibration selection: Uniform coverage permits selecting a configuration after learning the frontier using the same calibration data without an additional coverage penalty.One supported selection rule minimizes average confidence-set volume across calibration tasks.

4 Numerical Experiments

The experiments evaluate learned synthetic-augmentation frontiers on opinion-survey data, measuring coverage validity, confidence-interval efficiency, and effective synthetic sample size. Across GPT-4o and GPT-5 mini, the learned frontier maintains nominal coverage while admitting useful synthetic information.

Experiment Setup: The evaluation uses WorldValuesBench survey questions, two LLM simulators, and at most K = 200 synthetic responses per question.The dataset contains 235 questions from 64 countries, with population means used as ground truth; the LLMs are GPT-4o and GPT-5 mini.
Experiment Procedure: Historical questions are split into calibration and test sets, and Algorithm 1 learns frontiers for n ∈{10, 30, 50} using full-sample or split-sample proxies.The calibration-to-test ratio is 3:2, with 50 independent partitions and response draws; synthetic weights use a grid from 0 to 1 and k ranges to 200.
Learned versus Oracle Frontiers: For small weights around 0.2 or below, learned and oracle frontiers are capped at K = 200 because coverage remains valid even with all simulated samples.In this regime, the simulation budget rather than the coverage constraint limits the frontier.
Learned versus Oracle Frontiers: For larger weights, the learned frontier decays as λ approaches 1 and remains below but close to the oracle across sample sizes and proxy constructions.The split-sample proxy produces a frontier marginally closer to the oracle than the full-sample proxy.
Coverage Validity: Coverage remains above the nominal level 1 −α = 0.9 across both LLMs, both proxies, all real sample sizes, and every evaluated λ at the learned frontier.The empirical results are consistent with the uniform coverage guarantee.
Effective Synthetic Sample Size: The effective synthetic sample size λ · bkn(λ) grows as Kλ at small weights, stabilizes for λ ≥0.25, and generally increases with n.GPT-4o shows greater alignment with the real population than GPT-5 mini, while the split-sample proxy extracts more synthetic information from both LLMs.

5 Discussion

The discussion frames the learned size–weight frontier as a coverage-preserving limit on synthetic augmentation and examines effective synthetic sample size at that frontier.

The framework calibrates both the number of augmented synthetic samples and their weight for confidence-set construction from scarce real data.
The estimated frontier provides a simultaneous coverage bound for every size–weight configuration on or below it.This permits selecting a final configuration using a downstream objective based on calibration data.

A.1 Proof of Lemma 3.1

The proof begins by fixing a configuration and relating proxy quantities for confidence-set coverage to the actual coverage probability.

The argument fixes a configuration (λ, k) and suppresses it from the notation I_j,n(λ, k).
For the full-sample proxy, the proof relates the proxy to the actual coverage probability p_n.
For the split proxy, the proof uses conditional independence between I^split_j and I_j,n given f_j.

A.2 Proof of Theorem 3.1

The theorem proof controls frontier overestimation on selected weights, extends the bound across the full grid using monotonicity, and derives coverage guarantees for all configurations below the frontier.

The proof defines α′ = (1 −γ)α + η to calibrate the relevant coverage threshold.
The selected weights have distinct reference sizes, so their number satisfies |Ω| ≤ K.
Claim A.1 establishes that the expected proxy coverage at each selected frontier threshold is below 1 −α′.
Claim A.2 extends the inequality from selected weights to the entire grid using the construction of Ω and monotonicity of b̄_k.
A union bound controls the probability that the learned frontier exceeds the reference frontier across the selected weights.
Hoeffding’s inequality bounds deviations of the empirical proxy coverage because it averages J independent Bernoulli variables.
On the resulting event, the full-sample proxy has coverage at least 1 −α −η, while the split proxy has coverage at least 1 −α −(1 −γ)^−1η.

A.3 Proof of Corollary 3.1

The corollary proof introduces a random variable and invokes the uniform coverage theorem with a logarithmic calibration term.

The proof defines a random variable before applying the uniform coverage theorem.
Theorem 3.1 yields the stated result with probability at least 1 −δ over the calibration data.
The proof uses the bound 2 log(K ∧M) / J and completes the argument by noting 2 log(K ∧M) ≥ 2 log 2 > 1.