PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Nonparametric Regression with Heterogeneous Clients

Xiaoran Jiang, Anand N. Vidyashankar

arXiv:2610.32178v1stat.MLcs.LG

TL;DR

The paper studies when borrowing a federated predictor improves an individual client’s prediction despite heterogeneous populations and estimation uncertainty. It develops a summary-based Nadaraya–Watson framework with paired gain analysis and inference that retain local–server dependence. The analysis distinguishes gain from persistent target correction and variance reduction, and treats robustness and nonregular boundaries separately.

  • Problem

    Federated aggregation under client heterogeneity leaves the client-specific question of whether borrowing improves out-of-sample prediction relative to a local predictor.

    Source: §1 · p. 1

  • Method

    The paper combines client kernel summaries into a federated Nadaraya–Watson estimator and analyzes paired gain on independent audits, retaining dependence between local and shared fits.

    Source: §1 · pp. 1–2

  • Results

    Positive gain can arise from correcting persistent target mismatch or reducing fitting variability, while client gains remain dependent through the shared server estimator.

    Source: §1 · p. 1, §9 · p. 35

  • Takeaways & Limitations

    A good federated predictor does not imply positive transfer for every client; borrowing’s benefit depends on the client’s evaluation target and amount borrowed.

    Source: §9 · p. 35

    When the first-order gain derivative vanishes, the linear approximation does not capture the leading fitting fluctuation, requiring separate quadratic boundary analysis.

    Source: §5.2 · p. 14

Abstract

from arXiv · show

Whether a federated predictor benefits an individual client depends on both population heterogeneity and estimation uncertainty. We study this question for Nadaraya--Watson regression in a triangular array with potentially different fitting and evaluation distributions. Each client transmits local kernel summaries, allowing the server to construct a covariate-density-weighted federated estimator without centralizing individual observations. A client combines its fitted local predictor with the federated predictor through a prescribed weight, and gain is the paired reduction in squared prediction loss on an independent audit observation. Exact identities characterize both the gain of the realized fitted pair and its repeated-fitting mean. We also establish pointwise inference for the federated estimator. A two-stage analysis of client gains separates fitting variability from audit variability, retains the covariance created when a client's observations enter both its local estimator and the common server estimator, and yields fixed-block simultaneous inference across clients. Robustness is examined under two perturbation models. A Kullback--Leibler neighborhood provides a variance-sensitive lower certificate in a regular positive-gain-variance regime, while an $L_2$ neighborhood yields an exact least-favorable regression direction and robust value. Exact-model, vanishing-variance, and exact-zero-direction boundaries are kept separate because their scalar inputs are nonregular. The framework thus distinguishes decisions about a realized fitted pair from claims about the learning procedure under repeated fitting.

1 Introduction

The paper asks when federated prediction improves a particular client’s out-of-sample prediction, given heterogeneous populations and estimation uncertainty. It develops client-specific gain identities and inference while retaining dependence between local and server fits.

1 Introduction

  • Federated kernel summaries produce a covariate-density-weighted combination of client regression functions, rather than a simple average.The paper asks whether access to this predictor improves a client’s out-of-sample prediction over its local predictor.Source: §1 · p. 1
  • Persistent population gain requires displacement between the local procedure target and the evaluation regression plus a favorable federated correction; variance reduction can also yield finite-sample gain.Conditional-regression shift is one example of target displacement under the stated shrinking-bandwidth assumptions.Source: §1 · p. 1
  • The paper defines gain as the paired reduction in squared prediction loss and estimates it using an independent client audit sample.This avoids estimating several risks separately.Source: §1 · p. 2
  • The paper’s focus differs from prior work by studying client-specific paired gain under moving heterogeneous fitting and evaluation laws, with covariance retained when local data also enter the server fit.The passage situates this focus alongside personalized, agnostic, decentralized, and predictive-error inference literature.Source: §1 · p. 2
  • The four contributions include federated-estimator inference, realized-pair and repeated-fitting gain inference, dependence-aware simultaneous client inference, and KL and L2 robustness analyses.The robustness analyses treat vanishing-variance and exact-zero-direction boundaries separately.Source: §1 · p. 2
  • Targets may move across stages, and the primary fitting theorem is nondegenerate first order; exact-model and other boundary regimes are treated separately.The primary theorem does not determine the sign of gain.Source: §1 · p. 2

2 Federated regression and prediction protocol

The framework aggregates client-level kernel summaries into a federated predictor and compares a prescribed local-federated mixture with the local fit. Independent evaluation samples assess predictive gain separately from fitting.

2.1 Triangular array of client systems

  • The system is a stage-indexed sequence in which client counts, laws, and sample sizes may vary, with fitting samples independent across clients.The number of clients may grow with the stage.Source: §2.1 · p. 3

2.2 Federated target and estimator

  • Weighted client numerator and denominator summaries define the federated population target and allow its estimator to be built without transmitting individual observations.Only the local kernel summaries are transmitted; denominator flooring and clipping have the same pointwise limit as the raw ratio under stated conditions.Source: §2.2 · pp. 3–4

2.3 Fitting and evaluation samples

  • Each client’s local estimator and the shared server predictor use fitting data, so they are generally dependent because the client contributes to the server fit.Independent second-stage samples are used to evaluate gain.Source: §2.3 · p. 4
  • The evaluation law may represent the current client population or a future one, while ordinary foldwise reuse for fitting and evaluation is not covered.Such reuse has a different fold-averaged estimand and dependence structure.Source: §2.3 · p. 4
  • The prescribed weight balances local and federated information, but borrowing can introduce bias when the client regression differs from the federated target, so improved prediction is not guaranteed.The personalized predictor is compared with the local fit under the client’s evaluation law.Source: §2.3 · p. 5

3 Pointwise inference for the federated estimator

The paper establishes pointwise inference for the federated Nadaraya–Watson estimator in a growing heterogeneous client system. Under stated regularity and undersmoothing conditions, its error is characterized around the federated population target.

3 Pointwise inference for the federated estimator

  • The federated estimator has pointwise consistency and asymptotic normality under conditions accommodating a growing client population and changing client distributions.The theorem is stated for a fixed query point and the federated population target.Source: §3 · pp. 5, 8
  • Consistency and a nondegenerate stochastic scale require the first three assumptions, while centered normal approximation additionally requires higher-order moments and undersmoothing.Under the first three assumptions, pointwise variance is comparable to the aggregate scale q_t.Source: §3 · p. 7
  • The server can estimate pointwise variance without client-level observations using three additional scalar summaries per client at each query point.The variance estimator is constructed from the additional second-moment summaries.Source: §3 · p. 7

4 Predictive gain and positivity

The section distinguishes client-specific predictive gain from federated-estimator accuracy, separating realized fitted-pair gains from repeated-fitting marginal gains. It identifies persistent target displacement and variance reduction as distinct routes to positive gain.

4 Predictive gain and positivity

  • The section evaluates borrowing by comparing the personalized predictor with the local predictor under the client’s evaluation law.Improved prediction is not guaranteed.Source: §4 · p. 8

4.1 Paired gain, fitted gain, and marginal gain

  • The exact two-stage decomposition distinguishes gain for the realized fitted pair from marginal gain under repeated fitting.These correspond to different inferential questions: the fitted pair versus the learning procedure.Source: §4.1 · p. 8

4.2 Population procedure targets

  • Persistent positive population gain requires a persistent displacement between the local procedure target and the client’s evaluation regression.Finite-sample marginal gain may still arise through variance reduction when the local procedure is correctly targeted.Source: §4.2 · p. 9

4.3 Expected fitted gain and bias–variance conditions

  • Federation can improve expected fitted gain without persistent target correction when its variance reduction exceeds the added squared-bias cost.The stated sufficient condition is lower federated than local fitting variance.Source: §4.3 · pp. 10–11
  • The empirical gain curve provides a direct way to assess predictive improvement using independent evaluation data.The section identifies target correction and reduced fitting variability as two mechanisms for improvement.Source: §4.3 · p. 11

5 Inference for predictive gain

The section develops gain inference that separates audit-sample uncertainty from fitting-sample uncertainty, then extends it to fixed client blocks where the shared server fit induces dependence. The results distinguish inference for a realized fitted pair from inference for repeated fitting and treat boundary regimes separately.

5.1 Conditional evaluation inference

  • Conditional evaluation inference concerns the realized fitted predictors and should not be read as a confidence bound for the repeated-fitting gain Γ_j,t.The audit observations are independent of the fitting data, and the conditional lower bound targets the realized pair.Source: §5.1 · p. 12

5.2 One-client procedure-level theorem

  • Procedure-level inference combines fitting and evaluation uncertainty, accounting for dependence between local and federated predictors that share client observations.The independent audit stage contributes a separate variance component; together they provide an uncertainty scale for the marginal gain.Source: §5.2 · p. 14
  • When the first-order gain derivative vanishes, the quadratic term matters and a separate quadratic-martingale boundary analysis is required.The main procedure-level theorem applies to the nondegenerate first-order regime.Source: §5.2 · p. 14

5.3 A worked feasible regime

  • In the worked example, auditing the same fitted procedures under the original client law gives negative transfer.The example verifies the regular conditions for a two-client model with different fitting regressions and an audit regression between them.Source: §5.3 · p. 15

5.4 Client-indexed gain process

  • For every fixed client block, the joint limit retains fitting dependence from the common server estimator, while independent audit streams contribute diagonal covariance.Consistent covariance estimation and compatible finite-block limits are available under additional conditions.Source: §5.4 · pp. 15–16
  • The sufficient condition for a growing all-client event does not require client independence, but requires accumulated approximation errors and Gaussian lower-tail probabilities to vanish.The fixed-block theorem does not assert a central limit theorem uniform over all clients.Source: §5.4 · p. 16

6 KL-robust predictive-gain certificates

This section studies predictive-gain robustness to perturbations of the joint evaluation distribution within a KL neighborhood. It derives an exact robust value, a variance-sensitive lower certificate, and inference for that certificate under regular positive-variance conditions.

6.1 Exact robust value and variance-sensitive certificate

  • The KL analysis characterizes the fitted worst-case expected gain and provides a variance-sensitive lower certificate whose positivity suffices for positivity of the exact robust value.The certificate is developed under a conditional sub-gamma lower-tail assumption.Source: §6.1 · p. 17

6.2 Certificate inference and client process

  • The regular certificate inference regime requires positive gain variance; when variance vanishes, the square-root variance term is nonsmooth and the regular theorem does not follow.The fitting-covariance calculations also require additional cross-fitting information.Source: §6.2 · pp. 18–19
  • For fixed client blocks, certificate inference gives Gaussian orthant limits; a positive conditional lower confidence bound, rather than empirical-certificate positivity alone, supports robust benefit for the realized pair.The all-client result requires a sufficient summability condition.Source: §6.2 · p. 18
  • A critical radius characterizes how much distributional perturbation certificate positivity tolerates, but the regular theorem supplies no universal sample-size radius.A normalized limit is stated only when the specified model verifies the required relative limits for the mean, variance, and tail scale.Source: §6.2 · pp. 18–19

7 L2-robust regression-function perturbations

This section considers robustness to perturbations of the evaluation regression function while holding the evaluation covariate distribution fixed. It derives an exact least-favorable direction and robust value, then develops inference for regular and boundary regimes.

7.1 Exact least-favorable direction

  • The L2 analysis derives an exact least-favorable regression perturbation acting against the borrowing direction and an exact fitted robust value.When the local and personalized predictors coincide almost surely, the gain is zero.Source: §7.1 · p. 19

7.2 Inference and client process

  • For persistent nonzero mean borrowing directions and bounded radii, fixed-block robust-gain inference has a Gaussian process limit under the stated conditions.Studentization and grid implementation require additional aggregate cross-fitting and grid-norm conditions.Source: §7.2 · p. 20
  • When the mean borrowing direction is exactly zero, the oracle-norm robust value has a studentized Gaussian limit under separate martingale and audit conditions.Local-to-zero directions form a distinct transition regime, and vanishing directions require separate boundary analysis.Source: §7.2 · p. 20

8 Numerical Experiments

The numerical experiments assess two-stage gain inference, bias correction and variance rescue, exact-boundary behavior, client-indexed joint inference, and KL- and L2-robust certification. They report certificate rates and coverage across KL radii, and examine persistent and vanishing-direction regimes under L2 robustness.

8.1 Experiment 0: Two-client verification of two-stage gain inference

  • Fitting-variance ratios were 0.938–0.980 with shared local–server covariance retained, versus 0.626, 0.740, and 1.354 when omitted across the negative-, near-zero-, and positive-gain regimes.Realized-fit intervals had coverage essentially at the nominal 95% level; marginal intervals including fitting and evaluation uncertainty were close to 95%, while evaluation-only intervals substantially under-covered.Source: §8.1 · p. 22, Table 1 · p. 22

8.2 Experiment 1: Bias correction and variance rescue

  • Across the grid, the variance contribution stayed near 7.9 × 10^-3, while bias contribution varied; at (bL, bF) = (0.25, 0.25), variance rescue dominated as bias correction nearly vanished.The transition arose mainly because increasing bF reduced the bias-correction component, while the variance contribution changed little; positive gain could persist without material bias correction.Source: §8.2 · p. 24, Table 2 · p. 24, Figure 1 · p. 25
  • In Experiment 1, fitting-variance ratios ranged from 0.917 to 1.037 and two-stage 95% coverage ranged from 0.926 to 0.950, versus 0.521 to 0.814 for evaluation-only intervals.Fitting randomness was essential for procedure-level inference even though gains were evaluated on an independent audit sample.Source: §8.2 · pp. 24–25, Table 2 · p. 24, Figure 1 · p. 25

8.3 Experiment 2: Exact-model quadratic boundary

  • Quadratic-boundary fitting-variance ratios improved from 0.558 to 0.639 to 0.769 across increasing sample regimes, while persistent-linear ratios remained 0.103, 0.100, and 0.094.At the exact-model boundary the first-order derivative vanishes, so the centered quadratic term governs fitting fluctuation; the persistent-linear approximation captures only about one tenth of the Monte Carlo fitting variance.Source: §8.3 · p. 26, Figure 3 · p. 27
  • Quadratic-plus-evaluation intervals reached 95% coverages of 0.868, 0.935, and 0.952 as fitting samples increased, versus 0.650, 0.755, and 0.814 for persistent-linear intervals.Evaluation-only coverage was 0.507, 0.667, and 0.782, respectively.Source: §8.3 · p. 26, Figure 3 · p. 27

8.4 Experiment 3: Client-indexed joint inference

  • For the five-client block, marginal 95% coverages ranged from 0.942 to 0.961 and total-variance ratios from 0.942 to 0.995 despite substantial client heterogeneity.The common federated estimator induced off-diagonal correlations of 0.342–0.440; the estimated correlation matrix had mean absolute error 0.0205 and maximum absolute error 0.0397.Source: §8.4 · pp. 28–29, Table 3 · p. 28, Figure 4 · p. 29
  • For the local-margin joint event, empirical probability 0.315 closely matched Gaussian approximations 0.317 and 0.327, whereas independence gave 0.158.For the all-positive event, the approximation was less accurate: 0.889 versus empirical probability 0.921.Source: §8.4 · p. 29, Table 4 · p. 30

8.5 Experiment 4: KL-robust gain and certificate inference

  • In the regular regime, the variance-sensitive KL certificate lower-bounds the exact fitted KL-robust gain under the sub-gamma condition.The experiment evaluates KL-robust gain, certificate positivity, and confidence-bound coverage; the supplied passages give no numerical outcomes.Source: §8.5 · p. 30
  • At cρ = 0, 0.25, and 2, the KL certificate means are 0.0606, 0.0454, and 0.0171 versus exact robust gains of 0.0606, 0.0457, and 0.0196.The certificate never exceeded the exact empirical dual in 1,000 replications; the mean gap rose from 0 to 2.47 × 10−3, remaining conservative and close.Source: §8.5 · p. 31, Figure 6 · p. 31
  • The certificate was positive in every replication for cρ ≤ 1 and in 98.6% at cρ = 2; one-sided 95% lower-bound coverage ranged from 0.933 to 0.937.The coverage calculation combines fitting and evaluation uncertainty for the procedure-level marginal certificate.Source: §8.5 · p. 31, Figure 6 · p. 31, Table 5 · p. 32

8.6 Experiment 5: L2 robustness in persistent and vanishing-direction regimes

  • At n = 200, the fitted L2-robust value fell from 0.0631 at τ = 0 to −0.0129 at the largest radius, while audit estimates differed by about 6.0 × 10−4 on average.The robust value decreases monotonically with radius, consistent with the exact linear penalty in τ.Source: §8.6 · p. 33, Figure 7 · p. 33
  • Mean L2 error of the estimated least-favorable regression function fell from 0.1680 at n = 100 to 0.0699 at n = 800.The estimated curve moves toward the population least-favorable oracle as fitting sample size increases.Source: §8.6 · p. 33, Table 6 · p. 33, Figure 7 · p. 33
  • At cτ = 1, full-to-nominal fitting-variance ratios were approximately 0.231, 0.211, and 0.184 across sample-size regimes, as direction-term covariance caused substantial cancellation.The ratios decrease substantially for positive radii, so the stage-specific coefficient κB materially changes fitting variance and cannot be ignored.Source: §8.6 · pp. 34–35, Figure 8 · p. 34
  • At (nT, nA) = (120, 480), combined fitting/evaluation 95% coverage was 0.929, 0.937, 0.941, and 0.931 for cτ = 0, 0.5, 1, and 2.At cτ = 1, the mean square-root remainder fell from 6.80 × 10−3 to 1.43 × 10−3 across the reported sample-size regimes; standardization improved with sample size for moderate radii, but cτ = 2 remained distorted.Source: Figure 8 · p. 34, §8.6 · p. 35

9 Discussion

The pointwise inference proofs establish denominator stability, control the federated target and variance integrand, and bound the order of the leading variance.

A Pointwise Federated-Estimator Inference: Proofs

  • The proofs formulate inference at a fixed query point while allowing a growing, heterogeneous client population and stage-varying system features.The fitting samples are independent across clients; client laws, sample sizes, weights, kernels, and bandwidths may vary by stage.Source: A Pointwise Federated-Estimator… · p. 36

A.2 Denominator stability and leading variance

  • The federated denominator is consistent, and the leading variance is bounded above and below by constant multiples of its aggregate scale.The uniform target and variance-integrand controls support the variance expansion; no clientwise lower bound on density at the query point is needed.Source: §A.2 · pp. 36, 38, 41
  • The client-level variance expansion and smoothing-bias bounds provide the ingredients for pointwise consistency and a central limit theorem.The proof treats local regions around the query point separately from the tails when controlling smoothing bias.Source: §A.2 · pp. 39–40

A.3 Pointwise consistency and central limit theorem

The proofs establish pointwise consistency and an oracle central limit theorem for the federated estimator, with consistency results for its distributed variance estimator. The gain setup also separates fitting and audit stages and retains the possibility that a selected client’s observations enter both fits.

A.3 Pointwise consistency and central limit theorem

  • The oracle pointwise central limit theorem builds on a Lyapunov condition for the triangular array of client observations.The section also states pointwise consistency and smoothing-bias bounds under the listed assumptions.Source: §A.3 · pp. 42, 43, 46, 48

A.4 Consistency of the distributed variance estimator

  • The distributed variance estimator is ratio-consistent, supporting a studentized pointwise limit.The proof separately establishes concentration of the oracle second moment and negligibility of plug-in centering.Source: §A.4 · pp. 49, 51, 52, 54

B.2 Primitive setup and leverage notation

  • The gain-inference setup allows the selected client’s observations to enter both fits, and specializes its theorem to one-dimensional covariates with a common bandwidth.Observation-specific leverage coefficients record contributions to the local and federated empirical averages.Source: §B.1 · p. 55, §B.2 · p. 56

B.1 Triangular-array setup and gain targets

  • The gain setup distinguishes the realized fitted-predictor gain from its fitting-sample mean and gives an exact two-stage decomposition.The selected client’s evaluation law and the prescribed local–federated borrowing weight define the prediction protocol.Source: §B.1 · p. 55

B.3 Fitting-sample linearization

The fitting-sample analysis linearizes gain and establishes its asymptotic normality while retaining covariance from shared observations. It also gives a fitting-variance consistency result under an aggregate cross-fitting condition.

B.3 Fitting-sample linearization

  • Integrated ratio linearization and second-moment bounds control the fitted local and federated ratios used in the gain expansion.The centered quadratic gain is stated to be negligible under the listed assumptions.Source: §B.3 · pp. 57–59
  • The fitting-sample gain has a central limit theorem based on its linearized influence contributions.When selected-client observations enter both local and server fits, their covariance is retained in the expansion.Source: §B.3 · pp. 57, 59

B.4 A consistent estimator of the fitting variance

  • The fitting-variance estimator is consistent under the theorem’s assumptions, including aggregate cross-fitting stability.Finite-bandwidth proxy equivalence and plug-in bounds contribute to the result.Source: §B.4 · pp. 60–61
  • Feasible nominal studentization requires an aggregate cross-fitting condition and is not implied by basic kernel summaries and bandwidth conditions alone.A model-specific nuisance-rate lemma would be needed to establish that condition from those primitives.Source: §B.4 · p. 62

B.5 Conditional evaluation inference

This section develops conditional evaluation and predictive-gain inference for a realized fitted pair, distinguishing conditional coverage from inference under repeated fitting and treating both nondegenerate and exact-model boundary regimes. It also gives fixed-block inference at the exact-model quadratic boundary and interprets moving-stage gain inference, including its variance dependence and scope limits.

B.5 Conditional evaluation inference

  • With positive finite conditional variance and finite third absolute moment, the standardized evaluation error has a conditional Berry–Esseen bound and its random conditional law converges to N(0, 1).The result is conditional on the fitting realization.Source: §B.5 · pp. 62–63
  • The conditional studentized audit theorem gives a lower confidence bound whose coverage is conditional on the realized fitted predictors and targets Gfit_t.Source: §B.5 · p. 63
  • Evaluation-variance stabilization and sample-variance consistency justify estimating the conditional evaluation variance as the number of audit observations grows.The variance ratio stabilizes under the stated assumptions, and the sample variance is consistent when Neval → ∞.Source: §B.5 · pp. 63–64

B.6 Standalone theorem

  • In the nondegenerate first-order regime, predictive-gain inference combines fitting and evaluation errors with asymptotically independent standard-normal limits; the total studentized error is asymptotically standard normal.The total-error result does not require the variance fraction to converge.Source: §B.6 · pp. 64–65
  • When Γ_t/s_t converges to a finite value, the corollary characterizes the probability of positive marginal gain; that probability tends to one or zero when the ratio diverges positively or negatively.Near the zero-gain boundary, replacing the moving target Γ_t with its limit requires more than ordinary convergence.Source: §B.6 · p. 65

B.7 Exact-model quadratic boundary

  • At the exact-model boundary, a high-level Hall–de Jong condition supports a quadratic fitting CLT, and combined gain inference has asymptotically independent fitting and evaluation terms.The high-level condition is not derived from the basic kernel conditions.Source: §B.7 · pp. 67–69
  • At the exact-model quadratic boundary, compatible fixed-block limits yield convergence in R^N with the product topology and fixed-block orthant limits; the moving all-client event still needs coordinatewise summability.The conclusion also holds with consistent coordinatewise scales.Source: §B.7 · p. 71
  • The block estimators are theoretical cross-fitted estimators using observation-pair kernels, not recoverable from basic server summaries without added client computation or communication.Source: §B.7 · p. 71

B.8 Interpretation

  • The inference targets the deterministic moving-stage marginal gain Γ_t without requiring convergence; replacing it by Γ in a standardized margin requires (Γ_t − Γ)/s_t → 0.Source: §B.8 · p. 71
  • Fitting variance preserves dependence between the selected client’s local and common server fits, while evaluation variance is conditional on fitted models and uses an independent second-stage sample.Conditional convergence proves asymptotic independence of the normalized fitting and evaluation fluctuations; it is not assumed.Source: §B.8 · p. 71

C A reusable two-stage client-array assembly theorem

The reusable assembly combines finite-block fitting and conditional evaluation limits to infer client gains, while preserving dependence from the shared server fit. The fitting and evaluation Gaussian limits are independent, and the studentized limit follows under covariance approximation.

C A reusable two-stage client-array assembly theorem

  • Fixed-block fitting and evaluation limits assemble into independent Gaussian limits, with studentization valid when the covariance estimator approximates the limiting covariance.The common assembly theorem applies after the relevant scalar fitting expansions are established.Source: C A reusable two-stage client-array… · pp. 71–72

D.1 Client-indexed setup

  • Client gain coordinates need not be independent: observations can enter both a client's local fit and the shared server fit, which enters every client coordinate.The decomposition is orthogonality, not unconditional independence.Source: §D.1 · pp. 72–73

D.2 Finite-dimensional fitting limit

  • Each gain's fitting influence includes both the local-fit contribution and the common-server contribution, and the finite-block fitting CLT applies under the stated Lyapunov and remainder conditions.The primitive one-client Nadaraya–Watson assumptions imply the finite-block conditions in the nondegenerate first-order regime.Source: §D.2 · pp. 73–74

D.3 Finite-dimensional conditional evaluation limit

  • Conditional evaluation limits combine the client-specific evaluation streams for each fixed block, with coordinates degenerating at zero when evaluation variability is negligible relative to total variability.Evaluation streams are independent across clients and independent of the fitting data.Source: §D.1 · p. 73, §D.3 · p. 75

D.4 Finite-dimensional gain theorem

The analysis develops finite-block and client-indexed inference for gains, then applies the two-stage framework to KL-robust fitted certificates. Its conclusions preserve shared-server dependence and distinguish finite-block results from growing all-client claims.

D.4 Finite-dimensional gain theorem

  • The finite-dimensional gain theorem combines fitting and evaluation Gaussian limits, which are independent, while the resulting client correlation matrix may be singular.The gains share dependence through the fitted server estimator.Source: §D.4 · p. 76, §D.8 · p. 80

D.5 Consistent finite-block covariance estimation

  • Finite-block covariance approximation supports studentization of the gain vector under the stated covariance-estimation conditions.The proof uses plug-in fitting covariance and evaluation covariance estimates.Source: §D.5 · pp. 77–78

D.6 The client-indexed gain process

  • With compatible covariance limits, standardized gains converge to a coherent Gaussian sequence; consistent studentizing scales preserve the coordinatewise conclusion.This is a product-topology result for fixed collections of clients.Source: §D.6 · p. 78, §D.8 · p. 81

D.7 Positive-gain probabilities

  • For a fixed client block, positive-gain probabilities converge to a Gaussian orthant probability determined by the limiting correlation matrix.For local margins, replacing the mean by the observed gain does not consistently estimate the standardized margin.Source: §D.7 · p. 79

D.8 Interpretation and scope

  • The fitting covariance is generally non-diagonal because clients share the server estimator, whereas independent fresh evaluation streams yield diagonal evaluation covariance.Product-topology convergence alone does not establish positivity for a moving all-client event; the paper gives summability as a sufficient route, and the exact-model boundary requires separate input.Source: §D.8 · pp. 80–81, §D.7 · p. 80

E.1 Clientwise fitted certificate

  • The fitted certificate distinguishes the realized predictors' gain from the evaluation-law variance of one paired gain.The certificate is defined using client-specific robustness parameters.Source: §E.1 · p. 81

E.1.1 Why the certificate is robustly sufficient

  • The variance-sensitive KL certificate requires positive fitted gain and variance for a positive certified radius; a radius of order qt is justified only when both mean and variance have order qt.The exact fitted KL dual is stated under finite absolute first moment and an exponential-integrability condition; vanishing gain variance is a nonsmooth boundary for the Gaussian delta-method inference.Source: §E.1.1 · pp. 82–83

E.2 Empirical certificate and two-stage decomposition

  • The empirical certificate has an exact two-stage decomposition, and the nonlinear square-root evaluation term is not conditionally centered, although its first-order influence representation is..Source: §E.2 · p. 84

E.3 One-client conditional evaluation theorem

This section develops one-client and finite-block inference for KL certificates and L2-robust values, covering fitting and evaluation variability, covariance and variance estimation, and cross-fitted influences. It also characterizes limits on certificate positivity and gives an exact least-favorable regression result for the fitted L2-robust value, under stated regularity conditions.

E.3 One-client conditional evaluation theorem

  • Under the regular conditional certificate regime, the theorem and studentized corollary provide conditional inference; a positive lower confidence bound supports the realized-pair implication, unlike empirical certificate positivity alone.The lower-bound interpretation concerns the realized fitted pair.Source: §E.3 · pp. 84, 86

E.4 One-client fitting expansion of the gain mean and variance

  • The joint fitting expansion characterizes the gain mean and variance, and under Assumption 6.2 yields a one-client fitting CLT for the KL certificate.The expansion permits a singular bivariate covariance when the certificate contrast remains nondegenerate.Source: §E.4 · pp. 88–90

E.5 Feasible certificate influences under cross-fitting

  • Cross-fitted certificate influences are supported by finite-bandwidth proxy equivalence, aggregate plug-in and cross-covariance bounds, conditional on nuisance stability; no universal primitive nuisance rate is claimed.The required nuisance-stability condition is high-level, with verification depending on model-specific smoothness, tails, and empirical-process complexity.Source: §E.5 · pp. 91–93

E.6 Finite-dimensional fitting limit

  • The finite-dimensional fitting limit is set up using the certificate influences and the one-client remainder from the KL-certificate fitting corollary.The supplied passage gives this construction but not the limit theorem’s conclusion.Source: §E.6 · p. 93

E.6 Finite-dimensional fitting limit; E.7 Finite-dimensional conditional evaluation limit; E.8 Finite-dimensional certificate theorem

  • Fitting and conditional evaluation limits yield a finite-block certificate CLT with independent Gaussian fitting and evaluation components.The fitting representation preserves dependence from a client’s observations entering both its local estimator and the common server estimator; the evaluation limit allows coordinates to degenerate at zero after total standardization.Source: §E.6 · p. 93, §E.7 · p. 95, §E.8 · p. 96

E.9 Consistent finite-block covariance estimation

  • The finite-block covariance estimator consistently approximates the fitting and evaluation covariance, supporting studentization when the certificate CLT holds.The evaluation covariance is diagonal; the fitting covariance need not be.Source: §E.9 · pp. 97–98

E.10 The client-indexed certificate process

  • Under compatible covariance limits, the client-indexed KL-certificate process converges to a centered Gaussian sequence, also with consistent studentizing scales under the stated conditions.Source: §E.10 · p. 98

E.11 Robust-positive certificate probabilities

  • For a fixed client block, the limiting probability that every empirical variance-sensitive KL certificate is positive follows from the finite-block CLT.Certificate positivity is pathwise sufficient for positivity of the exact fitted robust gain, but the repeated-fitting mean of the certificate does not guarantee positivity for every fitted sample.Source: §E.11 · p. 99

E.12 Interpretation and scope

  • The process result concerns the variance-sensitive lower certificate, not the exact KL dual value, and applies in a regular positive-gain-variance regime.At zero variance or an exact-model boundary, the certificate’s leading fluctuation may be a nonsmooth norm or absolute-value functional; a separate one-client boundary theorem is needed.Source: §E.12 · p. 100

F.1 Fitted L2-robust value

  • For every fitted local and personalized predictor pair, the L2-robust value has an exact least-favorable regression result; when the predictors coincide in L2, the robust value is zero.Source: §F.1 · p. 101

F.2 Persistent-direction fitting expansion

  • The persistent-direction fitting expansion introduces mean fitted curves and a deterministic borrowing direction.Source: §F.2 · p. 102
  • Under Assumptions 7.1, F.1, and F.2, the one-client L2-robust value satisfies a fitting CLT.The fitting expansion allows the same observation to influence both the local and server estimators, retaining their dependence.Source: §F.2 · pp. 103–104

F.3 Consistent fitting-variance estimation

  • Under Assumption F.3, the proposed fitting-variance estimator is ratio-consistent, but the result relies on high-level plug-in and norm-approximation conditions.The section does not assert a primitive theorem connecting these conditions to grid spacing, density-weight estimation, client sample sizes, and communication cost.Source: §F.3 · p. 105

F.4 Evaluation-sample contribution

  • With the fitted borrowing norm held fixed, the fresh evaluation sample contributes an exact fluctuation and a conditional normal approximation when the standardized third-moment bound vanishes in probability.Evaluation samples are independent across clients and of the fitting data; under the stated condition, random conditional laws converge in probability mode to N(0, 1).Source: §F.4 · pp. 105–106

F.5 Same-audit-sample estimation of the borrowing norm

This section develops robust-gain inference when borrowing norms or directions are estimated on the audit sample, treating exact-zero mean borrowing directions separately and clarifying the scope of the inference. It extends the analysis to grid-based and multi-client settings and uses a bounded two-client example to show how changing the evaluation regression can reverse transfer while the fitting model family stays fixed.

F.5 Same-audit-sample estimation of the borrowing norm

  • With the borrowing norm estimated from the same audit sample, the conditional gain limit is normal and its variance estimate is ratio consistent.The variance accounts for both norm estimation and its covariance with paired gain; regularity requires a norm floor, bounded standardized moments, and positive audit variance.Source: §F.5 · p. 107

F.6 One-client combined theorem

  • The combined L2-robust gain has asymptotically independent fitting and evaluation limits, enabling a studentized robust-positive probability result.The fitting and audit variance contributions combine for the overall limit.Source: §F.6 · p. 108

F.7 Grid-based norm and least-favorable function

  • A feasible grid-based robust value follows when norm error is negligible on the total standard-error scale; ordinary norm consistency alone is insufficient.The least-favorable function is also shown consistent.Source: §F.7 · p. 109

F.8 Finite-dimensional client process

  • For any fixed client block, robust gains converge to a Gaussian process with dependent fitting coordinates and independent evaluation coordinates; covariance and correlation estimators are entrywise consistent.The fitting dependence retains common-server effects, while the normalized evaluation covariance is diagonal.Source: §F.8 · pp. 110–111

F.9 Sequence-space limit and positivity

  • Compatible fixed-block limits yield a centered Gaussian sequence, with fixed-block robust positivity and all-client positivity under a summability condition.The all-client result states that every empirical L2-robust value exceeds its threshold with probability tending to one.Source: §F.9 · pp. 111–112

F.10 Exact-zero mean borrowing direction

  • At an exact-zero mean borrowing direction, a separate fitting CLT handles the nondifferentiable norm through quadratic expansions; the exact-model boundary is variance-driven.The exact-model result requires its stated Hall–de Jong conditions, and the nominal gain’s linear influence vanishes there.Source: §F.10 · pp. 113–114
  • At that boundary, the actual borrowing-direction size governs the L2 scale: cancellation can reduce it, and leverage bounds alone do not justify directional noncancellation.Relating the scale to r_j,t/h_t requires a separate directional noncancellation condition.Source: §F.10 · pp. 114–115
  • Under the fixed-block conditions, exact-zero-direction gains admit studentized Gaussian inference and robust-positive orthant limits; estimated fitting variance is consistent.The combined theorem also transfers inference to a grid under its stated conditions, and thresholds are exceeded with probability tending to one under the additional threshold condition.Source: §F.10 · pp. 115–118

F.11 Scope

  • The fitted robust value is positive exactly when gain remains positive throughout the L2 ball; same-audit inference must include norm-estimation influence, while the d_j,t → 0 regime is outside these results.Conditional audit inference targets the realized fitted robust value, whereas fitting-plus-audit inference targets the repeated-fitting mean; deterministic least-favorable directions are claimed only in the persistent-direction regime.Source: §F.11 · p. 118

G A fully worked regular regime

  • A bounded two-client example checks that the assumptions for regular pointwise, nominal-gain, KL-certificate, and L2 procedures can hold together.The example is intended to verify the general theorems’ requirements, not replace them.Source: G A fully worked regular regime · p. 118

G.1 Model and exact gain geometry

  • Positive and negative transfer occur within one supported model family when only the evaluation regression changes.For the unchanged client law, the paired audit gain has the opposite sign to the procedure-target comparison.Source: §G.1 · p. 119

G.2 Pointwise and fitting rates

  • On the interior audit region, pointwise consistency, the oracle central limit theorem, and distributed variance estimation apply; the scalar gain has positive gain and audit variance.An independent audit sample of size N_E,n ≍ n satisfies the stated conditional Berry–Esseen, variance-consistency, and studentization requirements.Source: §G.2 · pp. 119–120

G.3 Feasible nuisance and variance estimation

  • A bounded independent pilot and fitting folds separate from evaluated influence observations yield nuisance bounds, fitting-variance ratio consistency, and the regular KL nuisance condition.The stronger fourth-order leverage comparison is checkable in this model but does not follow from ordinary no-dominance.Source: §G.3 · p. 120

G.4 Regular KL and L2 robustness

The section specifies KL and L2 robustness results in a regular positive-variance regime, then describes communication and confidence bounds that distinguish realized-fit from repeated-fitting claims.

G.4 Regular KL and L2 robustness

  • Bounded responses and clipped predictions give a deterministic conditional range bound, yielding a sub-gamma assumption with c_n = BW/3.The text attributes the conditional mgf inequality to Bernstein’s inequality.Source: §G.4 · p. 121
  • The gain variance converges to a positive constant, placing the model in the regular KL regime with admissible radii determined by fitted critical-radius formulas, not a sample-size radius.The formulas are given in Proposition E.3.Source: §G.4 · p. 121
  • The deterministic borrowing direction converges to (1 − α)b, so the L2 persistent-direction theorem applies to the limiting fitted robust value.Source: §G.4 · p. 121
  • When borrowing norm is estimated from audit covariates, audit variance uses the full influence in (627); an external grid is allowed only under Assumption F.4.Source: §G.4 · p. 121

G.5 Communication and confidence bounds

  • For fitting-gain variance, clients compute local observation-level influences and combine local and server contributions before squaring, retaining shared-observation covariance.Audit responses stay local; client 1 returns paired-gain summaries and, when needed, scalar KL or L2 influence summaries.Source: §G.5 · p. 121
  • The two lower confidence bounds target different objects: the realized fitted predictor pair and the learning procedure under repeated fitting.Source: §G.5 · p. 121

Related papers

Loading 2610.32178v1…