Source-linked AI summary

Exact Minimax One-Bit Unbiased Compression: Heavy-Tail Necessity and Finite-Randomness Approximation

Tao Jiang, Minbo Gao, Shaowei Cai

arXiv:2609.27860v1cs.LG

TL;DR

The paper studies one-bit unbiased compression and the challenge of exact-B-bit Gaussian-gradient communication without bounded-range gradients. It characterizes optimal codes and finite-randomness approximations, establishing the exact fixed-source infimum and showing that exact Gaussian minimax attainment forces an order-three moment transition.

  • Problem

    The work asks how to achieve exact-B-bit Gaussian-gradient communication without bounded-range gradients and what optimal one-bit codes require.

    Source: §1 · pp. 1–2
  • Method

    It proves an unrestricted-code converse and equality characterization, constructs a distribution-centered random-threshold code, and analyzes finite-randomness approximations.

    Source: §1 · p. 2
  • Results

    The fixed-source infimum is m^2 + J(P)^2, while exact Gaussian minimax attainment forces the order-three absolute-moment transition.

    Source: §1 · p. 2
  • Takeaways & Limitations

    On Kim’s continuous quadratic hard family, coordinate allocation yields a fixed-bit Gaussian-gradient scheme with lower-bound-matching parameter dependence.

    Source: §8 · p. 9

    The optimal constant for joint B-bit vector coding and the joint bias–variance–randomness–storage tradeoff remain open directions.

    Source: §7 · p. 9

Abstract

from arXiv · show

A pointwise-unbiased one-bit compressor reconstructs every real input in expectation while transmitting one bit. For a scalar source $P$ with CDF $F$, mean $m$, and $\mathcal J(P)=\int_{\mathbb R}\sqrt{F(r)(1-F(r))}\,dr$, we prove that the infimum of the source-averaged reconstruction second moment over all public-coin one-bit codes unbiased on $\mathbb R$ is $m^2+\mathcal J(P)^2$. For regular full-support sources, a distribution-centered random-threshold code attains this value; a converse over arbitrary randomized binary encoders and an equality analysis characterize every attaining code up to null sets, bit relabeling, and public-seed refinement. For the Gaussian location family $\mathcal N(μ,σ^2)$ with $|μ|\le cσ$, the equal prior on the endpoint means is least favorable and the minimax value is $σ^2Λ_c^2$. Exact Gaussian minimax optimality forces a critical heavy tail: at the endpoint means, absolute moments are finite exactly for $p<3$, and $\Pr(W>t)=Θ(t^{-3}/\sqrt{\log t})$. A Cauchy-mixture robustification inflates the second moment by at most $1/(1-η)$ while making every positive-order absolute moment finite. Finite-support public randomness with finite decoder means cannot achieve exact unbiasedness on $\mathbb R$, but a bounded-output approximation using exactly $R$ shared random bits has explicit bias and second-moment bounds converging to the minimax constant. Finally, coordinate allocation communicates exactly $B$ bits per Gaussian-gradient query. On Kim's continuous quadratic hard family, the expected optimization guarantee matches the lower bound in its dependence on $(σ,d,B,\varepsilon)$, and a finite-variance high-probability bound incurs only a logarithmic confidence factor.

1 INTRODUCTION

The paper studies exact one-bit unbiased reconstruction for unbounded Gaussian observations and develops finite-randomness implementations for fixed-bit Gaussian-gradient optimization. It establishes scalar optimality and characterizes the costs and limits of exact and approximate implementations.

1 INTRODUCTION

  • The fixed-source infimum is m^2 + J(P)^2, attained for regular full-support sources by a distribution-centered random-threshold code.A sharp rearrangement inequality gives the converse over arbitrary randomized binary encoders, and equality characterizes every attaining code up to null sets, bit relabeling, and public-seed refinement.Source: §1 · p. 2
  • For X ∼ N(µ, σ²) with |µ| ≤ cσ, the equal prior on endpoint means is least favorable, and its resulting code has second-moment risk increasing with |µ|, yielding an exact minimax theorem.Source: §1 · p. 2
  • Exact Gaussian minimax optimality forces every attaining code to have an order-three absolute-moment transition.The paper describes this transition as unavoidable by essential uniqueness.Source: §1 · p. 2
  • A Cauchy-mixture robustification bounds worst-case second-moment inflation by 1/(1 − η) and makes every positive-order absolute moment finite.Finite-support public randomness with finite decoder means cannot yield exact full-line unbiasedness; an R-bit threshold grid instead gives bounded outputs with explicit bias and second-moment bounds.Source: §1 · p. 2
  • Allocating s = min{B, d} scalar codes across coordinates gives a Gaussian-gradient scheme using exactly B bits per query and a lower-bound-matching dependence on (σ, d, B, ε).On Kim’s continuous quadratic hard family, coordinatewise median-of-means aggregation also provides a high-probability bound using only finite variance.Source: §1 · p. 2
  • The exact minimax constants apply to scalar one-bit coding; the optimal constant for arbitrary joint B-bit vector encoders remains open.The scope also notes a strongly-convex extension under localization of the gradient mean.Source: §1 · p. 2

2 RELATED WORK AND THE UNIFORM-SOURCE PRECEDENT

The section situates exact one-bit source-dependent second-moment optimization alongside uniform-source coding, multi-sample mean estimation, and gradient-compression work. It presents pointwise-unbiased reconstruction as a complementary per-sample primitive and connects finite-variance robust aggregation to high-probability optimization.

Uniform-source precedent

  • For uniform X in [0,1], the threshold density is (8/π)√(r(1 − r)) and the optimum average variance is π²/64 − 1/12.Corollary 2 recovers these formulas exactly; the related calculations restrict unbiasedness to [0,1].Source: §2 · p. 3

Related-work comparison

  • The work extends uniform-source average-variance calculations to exact full-line source and Gaussian minimax second moments, with an all-code converse, equality characterization, tail necessity, finite-randomness tradeoff, and optimization consequences.The closest prior results include [0,1] uniform-source average variance, [0,1] worst-case MSE, and full-line scalar components whose objective is distortion after rotation rather than exact source-dependent second moment.Source: §2 · p. 3, Table 1 · p. 3

One-bit mean estimation

  • One-bit Gaussian and log-concave mean-estimation studies optimize multi-sample estimation, often localizing an unknown mean, whereas this work studies pointwise-unbiased reconstruction and its exact source-averaged second moment.The cited literature includes Kipnis and Duchi (2017; 2022), Cai and Wei (2024), Lau and Scarlett (2026), Miao (2026), and Hu and Zhong (2026).Source: §2 · p. 3

Gradient compression and robust aggregation

  • A coordinatewise median-of-means argument shows that the exact code’s infinite third absolute moment does not preclude high-probability optimization under finite variance.The section places this result alongside standard communication-efficient methods including random sparsification, rotations, and adaptive quantization.Source: §2 · p. 3

3 PROBLEM FORMULATION AND FIXED-SOURCE OPTIMUM

The section establishes the exact minimum source-averaged reconstruction second moment for pointwise-unbiased one-bit codes and shows that a distribution-centered threshold code attains it under full-support conditions. A matching converse applies to arbitrary randomized binary encoders, and equality conditions characterize the attaining code for regular full-support sources.

Fixed-source optimum

  • The infimum source-averaged second moment is m^2 + J(P)^2 for any Borel source with finite second moment and finite J(P).Here m is the source mean; the infimum is over codes pointwise unbiased on R.Source: §3 · p. 4

Attainment

  • For nondegenerate sources, the same value is attainable on the convex hull of the support, and full-line attainment holds when the CDF has full support.When the support is bounded or one-sided, full-line codes approach the value arbitrarily closely by adding an arbitrarily small full-support threshold component.Source: §3 · p. 4

Uniform-source special case

  • For Unif[0,1], the interval-unbiased special case recovers Brady (2021)'s two reconstruction-level formulas.Source: §3 · p. 4

Equality characterization

  • For continuous strictly increasing full-support CDFs, every code attaining m^2 + J(P)^2 is the threshold code up to null sets, bit relabeling, and public-seed refinement.Equality conditions force deterministic threshold encoders for almost every public state.Source: §3 · p. 5

4 GAUSSIAN MINIMAX OPTIMALITY AND THE TAIL PRICE

For the Gaussian location family, the equal prior on the endpoint means is least favorable, and the exact minimax second-moment value is σ^2Λ_c^2. Exact minimax optimality requires a critical heavy tail, while Cauchy-mixture robustification gives finite positive-order moments at bounded second-moment cost.

Gaussian minimax value

  • For full-line pointwise-unbiased public-coin one-bit codes, the exact Gaussian location minimax result states that worst-case second-moment risk is attained at µ = ±cσ, and Bayes and worst-case bounds coincide.Source: §4 · p. 5
  • A distribution-centered random-threshold code attains the value by drawing Z ∼ ρ_c and sending M = 1{X/σ ≥ Z}.Its decoder uses the centering 1 − F_c(Z).Source: §4 · p. 5
  • Every attaining minimax code is equivalent, up to null sets, bit relabeling, and public-seed refinement, to the optimal code for the endpoint mixture.The worst-case risk is attained at the two endpoint means.Source: §4 · p. 5

Tail price

  • At endpoint means, the minimax code has finite second moment but an infinite third absolute moment; every attaining Gaussian minimax code shares the same failure of third and higher absolute moments.Source: §4 · pp. 5–6
  • Mixing the optimal threshold density ρc with Cauchy density r as ρc,η(z) = (1 − η)ρc(z) + ηr(z), 0 < η < 1, yields a perturbed code that is pointwise unbiased on R.The mixture imposes a polynomial lower envelope on the threshold density.Source: §4 · p. 6

Scale misspecification

  • Scale misspecification preserves pointwise unbiasedness but need not preserve moment finiteness; the Cauchy-mixture code retains all positive-order absolute moments under every finite mismatch.Source: §4 · p. 6

5 FINITE PUBLIC RANDOMNESS AND BOUNDED OUTPUTS

Finite public randomness with finite decoder means cannot preserve exact pointwise unbiasedness over all real inputs, but a bounded-output approximation retains controlled bias and approaches the Gaussian minimax second-moment constant. Its guarantees trade shared-randomness and threshold-grid size against approximation error.

Finite-support impossibility

  • No one-bit code with finite-support public randomness and finite conditional decoder means can be unbiased for every real input.Each conditional expectation lies in the convex hull of finitely many decoder means and is therefore bounded.Source: §5 · p. 6

Finite-randomness approximation

  • A uniform R-bit seed selects among thresholds with probabilities π_i, while the encoder sends the one-bit threshold comparison; the code stores K threshold/decoder entries and counts.The theorem specifies N = 2^R > K.Source: §5 · p. 6
  • The second-moment inflation is at most 1 + η when N ≥ K(1 + η)/η, and S_K,L converges to Λ_c as K then L grow.The code uses exactly R shared random bits; rounding stored decoder levels to precision τσ adds at most τσ to bias and root second moment. The finite-grid diagnostic may fall below one because the implementation is only approximately unbiased, with the theorem controlling its bias.Source: §5 · pp. 6–7
  • R = O(log K + log(1/η)) shared random bits suffice for a bounded-output code approaching the minimax constant.For target approximation level ζ, the explicit selection sets L − c = Θ(log(1/ζ)), chooses even K ≳ max{L, D_c, L L_2}/ζ, and sets R ≥ ⌈log2(K(1 + η)/η)⌉.Source: §5 · p. 7

6 FIXED-BIT GAUSSIAN-GRADIENT COMPRESSION AND OPTIMIZATION

The section gives fixed-bit Gaussian-gradient compression and optimization guarantees, including exact unbiasedness, matching expected-rate performance on Kim’s hard family, and implementations using finite randomness. It also uses finite variance to obtain a high-probability guarantee despite the exact endpoint code’s infinite third absolute moment.

6.1 FIXED-BIT VECTOR COMPRESSION

  • Each query communicates exactly B bits, and the vector decoder is unbiased for every deterministic g in R^d.The construction samples a uniform subset of size min{B, d}; when B > d, it pads the message with deterministic bits.Source: §6.1 · p. 8
  • Robustifying each scalar component inflates the second-moment bound by at most 1/(1 −η) and makes every positive-order coordinatewise absolute moment finite.Source: §6.1 · p. 8

6.2 EXACT-CODE GUARANTEES ON KIM’S CONTINUOUS QUADRATIC HARD FAMILY

  • On Kim’s continuous quadratic hard family (Kim, 2026), the estimator matches the lower bound in its dependence on (σ, d, B, ε).It queries at zero, averages decoded gradients, negates the average, and projects onto the cube.Source: §6.2 · p. 8
  • Finite variance suffices for a high-probability quadratic guarantee using a balanced coordinate schedule and coordinatewise median-of-means, with exactly B bits per query.The stated condition is T min{B, d} ≥ Cd log(2d/δ0), and the guarantee holds with probability at least 1 −δ0.Source: §6.2 · p. 8

6.3 END-TO-END FINITE-RANDOMNESS IMPLEMENTATION

  • The finite-randomness implementation uses sR shared random bits per query and bounded decoder outputs; its guarantee separates sampling variance from squared finite-approximation bias.It uses a deterministic balanced coordinate schedule, where s = min{B, d}; if d does not divide Ts, fewer than d/gcd(d, s) additional rounds make it divide Ts.Source: §6.3 · p. 8

7 DISCUSSION

The discussion separates query-complexity behavior, moment requirements for exact optimality, and limitations of exact unbiasedness, then identifies relaxations and open directions.

7 DISCUSSION

  • Unbounded Gaussian observations preserve the expected query-complexity rate on the localized quadratic family, while exact scalar second-moment optimality entails an endpoint order-three absolute-moment transition.The transition concerns absolute moments at the endpoint means.Source: §7 · p. 9
  • Exact pointwise unbiasedness on R is incompatible with finite public support and finite decoder means, as well as uniformly bounded decoder outputs.A small second-moment slack restores all positive-order absolute moments; finite shared randomness and bounded outputs instead yield controlled approximation bias.Source: §7 · p. 9
  • Open directions include optimal joint B-bit vector coding constants, the joint bias–variance–randomness–storage tradeoff, and localization for general strongly-convex objectives.Source: §7 · p. 9

8 CONCLUSION

The paper characterizes optimal unbiased one-bit compression and its Gaussian minimax behavior, then quantifies tradeoffs from robustification and finite randomness. It also gives a fixed-bit Gaussian-gradient scheme whose parameter dependence matches a lower bound on Kim’s continuous quadratic hard family.

8 CONCLUSION

  • The work characterizes the source-dependent optimum for public-coin, pointwise-unbiased one-bit compression, identifies a least-favorable Gaussian prior, and finds an order-three absolute-moment transition at the Gaussian minimax optimum.Attaining the Gaussian minimax optimum forces this transition.Source: §8 · p. 9
  • Cauchy-mixture and finite-randomness constructions quantify complementary tradeoffs among second moment, higher moments, bias, and shared-randomness complexity.Source: §8 · p. 9
  • Coordinate allocation yields a fixed-bit Gaussian-gradient scheme with lower-bound-matching parameter dependence on Kim’s continuous quadratic hard family.Source: §8 · p. 9

REPRODUCIBILITY STATEMENT

The supplementary code archive provides scripts, tests, and instructions to regenerate reported numerical outputs; numerical scripts are diagnostic only, and no numerical computation enters any theorem proof.

REPRODUCIBILITY STATEMENT

  • The archive includes arbitrary-precision evaluations of Λc, Figure 1 panel-generation scripts, finite-grid and probability-rounding checks, unit tests, and exact communication accounting; reproduce all.sh regenerates numerical outputs and runs the test suite.Code and exact reproduction instructions are available at github.com/jiangxioabai/onebit unbiased compression; the archive reproduces Table 2, Figure 1, and finite-grid calculations used in Theorem 8.Source: REPRODUCIBILITY STATEMENT · p. 9

AI USE STATEMENT · A CODE-CLASS REDUCTIONS AND DISTRIBUTIONAL PRELIMINARIES

The authors describe their use of generative AI and their verification of the manuscript. The preliminaries reduce randomized decoding and establish distributional tools and inequalities used in the proofs.

AI USE STATEMENT

  • Generative AI assisted with proof-strategy exploration, algebra checks, literature triage, debugging, and prose editing; the authors independently verified claims, citations, and numerical outputs.The authors state they are responsible for the final manuscript and provide complete proofs and reproduction scripts.Source: AI USE STATEMENT · p. 9

A CODE-CLASS REDUCTIONS AND DISTRIBUTIONAL PRELIMINARIES

  • Replacing a randomized decoder by its conditional mean preserves pointwise unbiasedness and weakly decreases source-averaged second moment.After this reduction, each fixed-seed decoder has two output values, while encoder private randomness is represented by its probability of sending one message.Source: A CODE-CLASS REDUCTIONS AND… · p. 10
  • The randomized distributional transform is uniform on [0, 1], and the generalized witness is defined by conditioning γ of that transform on X.For continuous strictly increasing F, the witness reduces to γ(F(x)); bounded odd truncations are also defined.Source: A CODE-CLASS REDUCTIONS AND… · pp. 10–11
  • The randomized bathtub inequality bounds the correlation for measurable message probabilities, and the same upper bound applies to clipped witnesses.The passage states the bound in terms of α, the mean message probability, but its displayed expression is incomplete.Source: A CODE-CLASS REDUCTIONS AND… · p. 11
  • For continuous strictly increasing F and 0 < α < 1, equality in the bathtub inequality requires a message probability that is an upper or lower quantile-interval indicator almost surely.The equality characterization is up to a null set.Source: A CODE-CLASS REDUCTIONS AND… · p. 11
  • When EX^2 and J(P) are finite, the preliminaries establish a dual correlation identity using quantile coupling and clipped-quantile limits.The proof couples X to its generalized quantile through the distributional transform and justifies the limiting step by monotone approximation and tail integrability.Source: A CODE-CLASS REDUCTIONS AND… · pp. 11–12

B PROOF OF THE FIXED-SOURCE CHARACTERIZATION … L RESOURCE ACCOUNTING AND SCOPE SUMMARY

Together, these parts characterize fixed-source equality and limits on exact full-line unbiasedness, then analyze Gaussian endpoint risks, reconstruction tails, finite-randomness, and vector compression. They connect these results to optimization guarantees, including matching dependence for expected quadratic bounds and high-probability and localized strongly-convex extensions, while specifying fixed-bit accounting and distinguishing numerical illustration from proof.

B.2 ACHIEVABILITY ON THE CONVEX HULL OF THE SUPPORT

  • When the source support’s convex hull is all of R, the distribution-centered threshold code is unbiased on R; otherwise, mixing its threshold density with an everywhere-positive density restores full-line unbiasedness.For a degenerate source, the construction also has full-line second-moment risk exactly m^2.Source: §B.2 · pp. 12–13

C EQUALITY CASES AND ESSENTIAL UNIQUENESS

  • For continuous, strictly increasing full-support sources, every attaining code is equivalent to the distribution-centered threshold code, up to bit relabeling and public-seed refinement.The characterization assumes a finite second moment and 0 < J(P) < ∞.Source: C EQUALITY CASES AND ESSENTIAL… · pp. 13–14

D.2 SECOND-MOMENT MONOTONICITY OF THE MIXTURE-CENTERED CODE

  • The symmetric mixture-centered code has nondecreasing risk in |a| and equal endpoint risks, yielding the Gaussian minimax result and essential uniqueness after restoring scale.The endpoint risks average to the fixed-source risk under the endpoint mixture; equality in the fixed-source theorem identifies the attaining code.Source: §D.2 · p. 15

E.1 MOMENT PHASE TRANSITION; E.2 ENDPOINT SURVIVAL ASYMPTOTICS

  • For Gaussian endpoints, absolute reconstruction moments are finite for p < 3, while the positive-tail p = 3 moment is finite only when a < c; the negative tail follows by reflection.Essential uniqueness transfers the endpoint survival-tail obstruction to every attaining minimax code.Source: §E.1 · p. 15, §E.2 · p. 16

E.3 CAUCHY-MIXTURE ROBUSTIFICATION; E.4 SENSITIVITY TO SCALE MISSPECIFICATION

  • The symmetric Cauchy-mixture design keeps worst-case risk at the endpoints and gives every positive-order absolute moment finite, including under scale misspecification.Pointwise unbiasedness holds for any positive design scale, although the attaining code’s second moment can become infinite; the Cauchy-mixture alternative has finite moments for finite parameters.Source: §E.3 · p. 16, §E.4 · p. 17

F.1 WHY EXACT FULL-LINE UNBIASEDNESS NEEDS INFINITELY MANY PUBLIC-SEED STATES

  • A finite public-seed set cannot give exact full-line unbiasedness, and no uniformly bounded decoder can do so regardless of seed cardinality.With finitely many seed states, conditional mean reconstructions lie in the bounded convex hull of finitely many decoder values.Source: §F.1 · p. 17

F.2 PROBABILITY ROUNDING; F.3 BIAS, SECOND MOMENT, AND OUTPUT MAGNITUDE; F.4 CONVERGENCE AND STORED PRECISION

  • With R shared bits, rounded threshold probabilities yield a midpoint-staircase approximation to clipping, and the finite-code constant converges explicitly to Λ_c.The finite-state construction maps the uniform seed to thresholds using integer probability counts.Source: §F.2 · p. 17, §F.3 · p. 18, §F.4 · p. 19

G VECTOR COMPRESSION AND FIXED-BIT ACCOUNTING

  • For a deterministic vector, the construction samples s = min{B, d} coordinates and gives an unbiased reconstruction for each coordinate.Selected coordinates use independent scalar reconstructions, while unselected coordinates return zero.Source: G VECTOR COMPRESSION AND FIXED-BIT… · p. 19
  • Adaptive optimization requires fresh thresholds and subsets after each iterate is selected, using independent public-seed blocks.Each iterate must depend only on previous seed blocks and the transcript; exactly one bit is sent per selected coordinate, with deterministic padding when B > d.Source: G VECTOR COMPRESSION AND FIXED-BIT… · p. 19

H EXPECTED QUADRATIC OPTIMIZATION GUARANTEE

H.1 FINITE-RANDOMNESS IMPLEMENTATION

  • The finite-randomness implementation uses a deterministic cyclic coordinate schedule and an independent R-bit shared seed for each active coordinate.When B > d, remaining message positions are deterministic padding; divisibility adjustments require fewer than d / gcd(d, s) additional rounds, and stored-level rounding follows Theorem 8's perturbation bounds.Source: §H.1 · p. 20

I HIGH-PROBABILITY QUADRATIC ESTIMATION FROM FINITE VARIANCE

  • An elementary coordinatewise median-of-means proof applies Lemma 18 to every coordinate: for k the smallest odd integer at least 8 log(2d/δ0), if Ts ≥ 4dk, the stated guarantee holds with probability at least 1−δ0.Source: I HIGH-PROBABILITY QUADRATIC… · pp. 20–21

J LOCALIZED STRONGLY-CONVEX EXTENSION

  • Projected SGD achieves expected squared distance O(1/t) under localized Gaussian gradient means, strong convexity, and smoothness.The localization controls the Gaussian mean, not individual stochastic-gradient realizations; removing it also requires locating an arbitrarily translated optimum.Source: J LOCALIZED STRONGLY-CONVEX EXTENSION · pp. 21–22

K NUMERICAL EVALUATION AND FIGURE GENERATION

  • Numerical calculations use arbitrary-precision quadrature to illustrate the theory, but are not used in any proof.The figure-generation script evaluates risk integrals, a parametric survival curve, and finite-grid constants.Source: K NUMERICAL EVALUATION AND FIGURE… · p. 22

L RESOURCE ACCOUNTING AND SCOPE SUMMARY

  • Table 4 summarizes communication resources, implementation guarantees, and scope.Source: Table 4 · p. 23
Loading 2609.27860v1…