PaperlayerView exact revision on arXiv

Source-linked AI summary

Confounding-Valid Conformal Inference for Counterfactual KPIs in Wireless Networks

Abdessamed Qchohi, Jessica Moysen Cortes, Matteo Zecchin

arXiv:2609.05073v1cs.LGcs.NIeess.SPstat.ML

TL;DR

When telemetry omits information used by the controller, hidden confounding can invalidate coverage guarantees for counterfactual methods relying on observational telemetry alone. CV-CCI addresses this problem while maintaining prescribed coverage across hidden-confounding levels and producing informative prediction sets.

  • Problem

    Telemetry that omits information used by the controller creates hidden confounding that can invalidate coverage guarantees for counterfactual methods relying on observational telemetry alone.

  • Method

    CV-CCI combines abundant, potentially confounded telemetry with a methodology that provides finite-sample coverage guarantees up to a user-specified tolerance.

  • Results

    Across different levels of hidden confounding, CV-CCI maintained the prescribed coverage while producing informative prediction sets.

  • Takeaways & Limitations

    Methods relying primarily on observational telemetry increasingly violated coverage as confounding increased, while alternative valid methods produced wider or more variable prediction sets.

Abstract

from arXiv · show

Conformal counterfactual inference enables network operators to use logged telemetry to reliably answer 'what-if' questions about network operation. These answers typically take the form of prediction sets that contain, with a user-defined probability, the key performance indicators (KPIs) that would have been observed under alternative control actions. A key challenge is that logged telemetry may omit variables used by the controller, resulting in hidden confounding and invalidating the statistical guarantees of counterfactual analysis. In principle, this issue can be addressed using randomized telemetry, collected by assigning control actions independently of the network state. However, because such randomization may disrupt normal operation, randomized telemetry is typically scarce, causing counterfactual analysis based solely on it to produce uninformative prediction sets. To address these challenges, we propose Confounding-Valid Counterfactual Conformal Inference (CV-CCI), which combines abundant, potentially confounded observational telemetry with limited randomized data through the General Synthetic-Powered Inference (GESPI) principle. CV-CCI leverages observational data to improve efficiency while using randomized data to retain finite-sample coverage guarantees under arbitrary hidden confounding. Experiments on two representative radio access network (RAN) control tasks show that CV-CCI remains valid under hidden confounding while producing more efficient prediction sets than state-of-the-art confounding-valid baselines.

I. INTRODUCTION

Counterfactual KPI inference in RANs must handle hidden confounding when logged telemetry omits information used by the controller. CV-CCI combines abundant observational telemetry with scarce randomized telemetry to retain coverage while producing more informative prediction sets.

  • Motivation: Logged telemetry may omit controller inputs that affect both action selection and KPIs, creating hidden confounding and invalid observational coverage guarantees.Examples include fine-grained measurements, internal states, and signals from other controllers that are absent from aggregated, delayed, or filtered logs.
  • Motivation: Randomized telemetry restores valid counterfactual coverage by assigning actions independently of observed and hidden network information.Such interventions can be costly, suboptimal, or operationally incompatible, so they are typically scarce.
  • Motivation: Prediction sets calibrated only on observational telemetry may fail coverage, whereas sets calibrated only on randomized telemetry are valid but wide and uninformative.The scheduling example contrasts these failure modes for residual backlog under alternative policies.
  • Contribution: CV-CCI combines observational and randomized telemetry, using the former for efficiency and the latter as a validity guardrail against arbitrary hidden confounding.The method is motivated by the tension between abundant but potentially invalid observational data and scarce but valid randomized data.
  • Related work: The paper positions CV-CCI within conformal counterfactual inference and GESPI-based combinations of observational and randomized data.Related methods include weighted conformal prediction, sensitivity analysis, and density-ratio approaches whose high-dimensional estimation errors may affect coverage and efficiency.

C. Contributions

The paper formulates counterfactual KPI inference under partially observed controller information and introduces CV-CCI to combine observational and randomized telemetry. It establishes finite-sample coverage under arbitrary hidden confounding and evaluates the method in representative RAN-control tasks.

  • Problem: The problem setting concerns counterfactual KPIs when controller actions use information only partially available in logged telemetry.The logged state may omit contextual information used for action selection, producing hidden confounding.
  • Method: CV-CCI combines abundant, potentially confounded observational telemetry with limited randomized telemetry through a conformal counterfactual KPI methodology.The method specializes the GESPI principle to wireless counterfactual prediction.
  • Guarantees: CV-CCI provides finite-sample marginal coverage under arbitrary hidden confounding up to a user-specified tolerance without observing or modeling the hidden confounder.Randomized telemetry supplies the target-distribution sample while observational data contribute additional information.
  • Evaluation: Across MAC-layer scheduling and handover settings, CV-CCI maintains reliable coverage as hidden confounding increases and produces more informative, less variable sets than randomized-only methods.The evaluation compares the proposed method with methods calibrated exclusively on randomized telemetry.

1) Action-assignment regimes:

The paper distinguishes observational operation, where a controller may choose actions using the full context, from randomized assignment, where actions are independent of context. Telemetry records the logged state, action, KPI, and regime while omitting hidden context, with observational data assumed abundant relative to randomized data.

  • Action-assignment regimes: Under the observational regime, the deployed controller selects actions according to an arbitrary policy that may depend on the full decision context.The action-selection probability is represented by pobs(a | X).
  • Action-assignment regimes: Under the randomized regime, actions are assigned independently of the decision context according to a known policy prnd(·).Positivity requires every action to have nonzero selection probability.
  • Action-assignment regimes: The two regimes share the distributions of contexts and potential KPIs and differ only in their action-assignment mechanisms.This requires regime assignment to be independent of context and potential KPIs.
  • Telemetry logging: Each logged decision contains the logged state, selected action, resulting KPI, and assignment regime, while the controller’s hidden context is not recorded.The full context is decomposed into logged components ˜X and unobserved components U.
  • Telemetry logging: The observational and randomized datasets contain nobs and nrnd independent samples, respectively, with the practically relevant regime nobs ≫ nrnd.Randomized experimentation is limited by operational costs, safety constraints, or service-level requirements.

B. Problem Definition

The paper formulates reliable counterfactual KPI estimation as constructing prediction sets for KPIs under alternative actions, with user-defined marginal coverage. Hidden confounding prevents observational telemetry alone from generally identifying the target potential-outcome distribution, while randomized telemetry restores the needed independence but may yield uninformative sets.

  • Counterfactual KPI estimation asks which KPI would have been observed under a target action given logged network context.
  • For miscoverage level α, reliability requires the prediction set Γa(˜X, D) to contain the target potential KPI Y(a) with probability at least 1 −α.
  • Hidden confounding arises because an unobserved variable U can affect both selected action A and potential outcomes, so conditioning only on logged state ˜X need not ensure ignorability.
  • Consequently, observational samples assigned to action a may not represent the target potential-KPI distribution and may not support nontrivial distribution-free prediction sets.
  • Randomized telemetry makes action independent of the full decision context, enabling valid counterfactual analysis, but returning the entire KPI space would be covered yet uninformative.
  • Prediction-set quality is measured by average size, with smaller inefficiency indicating more informative restrictions on plausible potential KPI values.
  • Prediction sets for individual potential KPIs can induce valid prediction sets for action effects through the Minkowski difference, enabling uncertain comparisons between candidate actions.

B. General Synthetic-Powered Inference

GESPI combines reliable data with auxiliary data through a guardrail that preserves validity while allowing efficiency gains. For prediction sets, its aggregation limits shrinkage relative to reliable-only inference and controls degradation by a user-specified tolerance.

  • GESPI combines abundant, potentially biased auxiliary data with limited reliable data to improve efficiency while limiting worst-case degradation.
  • The framework wraps a base inference procedure that controls target risk at level α on data sampled from the target distribution.
  • GESPI aggregates reliable-data, auxiliary-powered, and guardrail outputs, with the guardrail using reliable data at error level α+ϵ.
  • The auxiliary-powered set can reduce reliable-data set size only within the degradation allowed by ϵ.
  • For prediction sets, the outer intersection prevents the GESPI set from exceeding the reliable-only α-level set, while the guardrail union preserves the reliable-only α+ϵ set.
  • Under standard loss monotonicity assumptions, GESPI provides a deterministic guarantee regardless of the auxiliary-data distribution Q.
  • The paper specializes GESPI by combining abundant potentially confounded observational telemetry with a smaller amount of randomized telemetry for counterfactual prediction sets.

IV. CONFOUNDING-VALID COUNTERFACTUAL CONFORMAL INFERENCE

CV-CCI applies GESPI to combine randomized telemetry, which supplies validity under hidden confounding, with observational telemetry, which can improve efficiency. Its finite-sample guarantee limits marginal-coverage deterioration to the user-specified tolerance even under arbitrary hidden confounding and propensity-score errors.

  • CV-CCI is a conformal framework for counterfactual estimation of wireless KPIs under hidden confounding.
  • The method uses randomized telemetry as reliable data and observational telemetry as auxiliary data within a GESPI-wrapped CCKE construction.
  • CV-CCI constructs randomized-only and pooled WCP prediction sets, using weighted calibration scores for observational samples and constant randomized weights.
  • The pooled set may be more efficient when observational data are informative, but it alone lacks marginal coverage because its observational component may remain confounded.
  • CV-CCI applies GESPI aggregation so randomized data act as a validity guardrail while observational data can improve efficiency.
  • For α + ϵ < 1, Proposition 1 establishes finite-sample coverage for every logged context under the stated sampling assumptions.
  • The guarantee holds up to tolerance ϵ under arbitrary hidden confounding and arbitrary errors in estimated observational propensity scores.

V. EXPERIMENTS

The experiments evaluate CV-CCI against state-of-the-art counterfactual conformal inference baselines on two representative radio access network control tasks.

  • The evaluation compares CV-CCI with state-of-the-art counterfactual conformal inference baselines.
  • The two tasks are medium access control-layer scheduling and handover in radio access networks.

A. Baselines

The baselines differ in how they use observational and randomized telemetry to address confounding, validity, and efficiency. The resource-allocation setup hides the PRB budget, which can influence both scheduling actions and residual-backlog KPIs.

  • wSCP-DR: wSCP-DR estimates randomized-to-observational density ratios and uses them to reweight action-specific telemetry for hidden-confounding correction.
  • wSCP-DR: wSCP-DR Inexact directly uses regressed interval endpoints, improving computational efficiency but lacking finite-sample coverage guarantees.
  • wSCP-DR: wSCP-DR Exact adds split-conformal calibration to restore finite-sample validity, but reserves scarce randomized data and may widen intervals.
  • Guardrail: Guardrail trains quantile regressors on observational telemetry and calibrates with randomized samples at error level α + ϵ.
  • Guardrail: Guardrail satisfies marginal coverage 1 −α −ϵ under hidden confounding and isolates CV-CCI’s auxiliary observational-data benefit.
  • Resource-allocation setup: The resource-allocation experiment hides the instantaneous PRB budget U = NPRB, which affects scheduling policy and the residual UE-backlog KPI.

2) Results:

Experiments vary hidden-confounding strength and tolerance using scarce randomized telemetry to compare coverage, efficiency, and failure rates. CV-CCI and Guardrail remain coverage-valid, while CV-CCI generally provides the strongest balance of robustness and efficiency.

  • RR outcome: At λ = 0, CCKE, wSCP-DR Exact, CV-CCI, and Guardrail achieve coverage close to or above nominal 1 −α = 0.9, unlike wSCP-DR Inexact.
  • RR outcome: As λ increases from 0 to 1, CCKE coverage decreases from approximately 0.91 to 0.78, while wSCP-DR Inexact decreases from approximately 0.79 to 0.67.
  • RR outcome: CV-CCI, Guardrail, and wSCP-DR Exact maintain coverage above nominal across λ, but wSCP-DR Exact produces wider sets because data splitting reduces calibration efficiency.
  • RR outcome: CV-CCI and Guardrail achieve the most favorable balance between robustness to hidden confounding and prediction-set efficiency.
  • PFCA outcome: CV-CCI’s empirical failure probabilities are 30% at λ = 0 and 26% at λ = 1, versus 47% and 41% for Guardrail.
  • Tolerance analysis: CV-CCI stays near nominal coverage 0.90 across all ϵ values, while Guardrail tracks 1 −α −ϵ and becomes more variable as ϵ increases.
  • Tolerance analysis: Incorporating observational telemetry reduces variability from the limited randomized calibration sample while maintaining coverage near nominal across ϵ.

C. Handover

The handover task models counterfactual throughput when a neighboring base station’s unobserved load affects both handover acceptance and outcomes. Randomized handover decisions provide telemetry independent of observed and hidden contexts.

  • Handover setup: A UE may remain connected to BS1 or hand over to BS2, producing a throughput KPI for each alternative action.
  • Hidden confounding: Telemetry records RSS context, action, and throughput but omits BS2’s instantaneous load, which can influence both handover decisions and throughput.
  • Decision process: BS1 proposes handover using recent RSS measurements, after which BS2 accepts or rejects based on its instantaneous load.
  • Hidden confounding: The hidden confounder is U = ℓ2, and λ controls its influence: λ = 0 yields independence, whereas larger λ produces stronger hidden confounding.
  • Randomized telemetry: Randomized telemetry assigns the handover decision independently of observed and hidden contexts.

2) Results:

Across handover and no-handover experiments, CV-CCI achieved the most favorable balance between empirical coverage and prediction-set efficiency. It maintained coverage near or above nominal levels under increasing hidden confounding while reducing variability and interval width relative to valid randomized-data baselines.

  • Handover: CV-CCI consistently achieved the best trade-off between statistical validity and prediction-set efficiency in the handover experiment.Its performance matched Guardrail's robustness while producing substantially narrower prediction intervals.
  • Handover: At λ = 8, CCKE and wSCP-DR Inexact reached empirical coverage of 0.75 and 0.71, respectively, despite producing comparatively narrow prediction intervals.Both methods violated the coverage requirement for every λ > 0.
  • Handover: CV-CCI, Guardrail, and wSCP-DR Exact maintained empirical coverage above the nominal level for all tested confounding strengths.CV-CCI and Guardrail stayed close to nominal coverage, whereas wSCP-DR Exact showed noticeable overcoverage.
  • Failure probability: Under the strongest confounding, Guardrail failed the prescribed coverage in 40% of folds, compared with 13% for CV-CCI.Without hidden confounding, the corresponding failure rates were 18% for Guardrail and 9% for CV-CCI.
  • Tolerance sensitivity: As ϵ increased, Guardrail's coverage declined toward 1 −α −ϵ and became less stable, whereas CV-CCI remained close to nominal coverage and was considerably less sensitive.The observational component reduced variability and kept coverage closer to nominal across tolerance choices.

VI. CONCLUSIONS

The conclusions identify hidden confounding in incomplete network telemetry as a threat to counterfactual coverage guarantees and present CV-CCI as a remedy. Across MAC-layer scheduling and handover experiments, CV-CCI maintained prescribed coverage while producing more informative prediction sets than alternatives relying primarily on observational or limited randomized telemetry.

  • Conclusions: Missing controller-used variables in telemetry create hidden confounding that can invalidate coverage guarantees for observational counterfactual methods.This motivates combining observational and randomized telemetry rather than relying on observational data alone.
  • Conclusions: CV-CCI combines abundant potentially confounded observational telemetry with limited randomized data to construct counterfactual KPI prediction sets with finite-sample coverage guarantees.The guarantee holds up to a user-specified tolerance.
  • Conclusions: Across MAC-layer scheduling and handover experiments, CV-CCI maintained the prescribed coverage under different levels of hidden confounding.The experiments demonstrated the methodology's practical effectiveness in both control scenarios.
  • Conclusions: Methods relying primarily on observational telemetry increasingly violated coverage as confounding strengthened, while valid alternatives produced wider or more variable prediction sets.The wider intervals or greater variability were associated with limited randomized sample sizes.
  • Future work: Future work targets sequential and online inference, interference-aware controllers, and adaptive intervention strategies that minimize network disruption.These extensions address evolving conditions, interacting controllers, and more informative data collection.
Loading 2609.05073v1…