Policy Optimization and Statistical Inference for Online Contextual Matrix Games

• Online contextual matrix games address settings where strategic agents’ payoff matrices evolve with contextual information, unlike contextual bandits or online matrix games alone.The framework treats contexts as exogenous factors affecting the current payoff matrix and studies two-player zero-sum games with noisy bandit feedback.Source: §1 · pp. 3–4
• OnGameLearn balances exploration and exploitation across player actions and contexts while estimating Nash equilibria through regularized learning and minimax optimization.The algorithm incorporates UCB, Thompson Sampling, and ϵ-Greedy exploration to support valid statistical inference and dynamically update both players’ strategies.Source: §1 · pp. 4–5
• The authors formalize policy value for stochastic matrix-game policies and propose a doubly robust estimator for game value under Nash equilibrium.The estimator is designed for strategy evaluation in the presence of contextual and strategic uncertainty.Source: §1 · p. 4
• The framework provides statistical inference for payoff-matrix parameters under adaptively collected, non-i.i.d. data, unifying policy optimization and inference.The resulting procedures quantify uncertainty while learning strategically dependent policies online.Source: §1 · p. 4
• The paper establishes statistical guarantees for payoff-matrix parameter estimates and estimated Nash equilibria despite the minimax structure of equilibrium estimation.This directly characterizes equilibrium estimation rather than relying only on regret-based convergence arguments.Source: §1 · p. 5
• The estimated Nash equilibrium is shown to be consistent, with parameter estimators additionally attaining asymptotic normality despite dependence induced by adaptive data.Source: §1 · p. 5
• The paper develops a T-consistent doubly robust value estimator for stochastic policies in matrix games, both with and without contexts.Source: §1 · p. 5
• The proposed learning procedures achieve sublinear regret bounds.Source: §1 · p. 5
• T-consistency holds under a mild margin condition and accommodates both full-support and locally stable sparse Nash equilibria.Source: §1 · p. 5

2.1 Online Zero-sum Matrix Games

• Online zero-sum matrix games involve repeated action choices with unknown payoffs, bandit feedback, and goals of estimating the matrix, designing optimal strategies, and evaluating game value.Players observe only the opponent’s action and reward for the selected action pair; outcomes follow a sub-Gaussian noise model.Source: §2.1 · p. 7
• Nash equilibrium is defined by the matrix-game saddle-point inequalities, while policy value is x⊤Ay and optimal policy value is attained under strategic equilibrium.A strongly convex–strongly concave formulation yields a unique saddle point.Source: §2.1 · p. 7

2.2 Online Contextual Zero-sum Matrix Games

• Online contextual matrix games reveal zt ∈ R^d and model the payoff matrix as M(zt), allowing context-dependent mixed strategies and Nash-equilibrium policy values.The framework primarily uses linear contextual payoffs with sub-Gaussian noise, but permits flexible parameterizations when payoff and equilibrium estimation remain consistent.Source: §2.2 · p. 8

3 Proposed Method: OnGameLearn

• OnGameLearn combines payoff-matrix estimation, regularized Nash-equilibrium optimization, UCB, TS, or ϵ-Greedy exploration, and doubly robust policy-value estimation.Its iterative components aggregate historical observations, optimize strategies, and combine model predictions with observed rewards.Source: §3 · p. 9, §3.1 · p. 11

3.1 Method for Online Matrix Games

• In online matrix games, OnGameLearn estimates unobserved payoff matrices from bandit feedback, regularizes the objective for a unique saddle point, and derives exploratory behavior policies.UCB and TS clip exploratory Nash strategies, whereas ϵ-Greedy follows estimated equilibrium strategies with probability 1 − ϵ_t and explores uniformly otherwise.Source: §3.1 · pp. 9–10

3.2 Method for Online Contextual Matrix Games

• In contextual games, OnGameLearn estimates action-pair parameters by ordinary least squares, forms context-dependent payoff matrices, computes regularized equilibria, and preserves positive action-pair selection probabilities.Contextual UCB, TS, and ϵ-Greedy strategies support estimation and inference before doubly robust value estimation is updated.Source: §3.2 · pp. 11–12

4.1 Results for Online Matrix Games

• For online matrix games, OnGameLearn yields tail-bound and consistency guarantees for payoff-matrix estimation, and estimated Nash equilibria converge to the true equilibrium set.Under a unique-equilibrium assumption, behavior strategies converge to limiting strategies; the asymptotic variance depends on the limiting exploration rate and true Nash equilibrium.Source: §4.1 · pp. 13–15
• The theory establishes asymptotic normality and T^-1/2 convergence of the payoff-matrix estimator, saddle-value convergence, T-consistency of doubly robust value estimation, and inference for optimal game value.Source: §4.1 · pp. 15–16
• For the UCB strategy, under ct = O(√log t), Theorem 8 gives a regret bound without Assumptions 4.1 and 4.2, allowing multiple Nash equilibria and zero equilibrium probabilities, while ϵt ≍ t^-1/4 balances leading terms.Source: §4.1 · pp. 16–17

4.2 Results for Online Contextual Matrix Games

• For online contextual matrix games, the theory covers parameter-estimator tail bounds, Nash-equilibrium and behavior-strategy convergence, asymptotic normality, saddle-value and doubly robust value-estimator rates, and regret bounds.Complete proofs are provided for these contextual-game guarantees.Source: §4.2 · p. 17
• OnGameLearn admits tail bounds for parameter estimates, yielding consistency of the estimated contextual payoff matrix.The tail-bound result applies under Assumption 4.3 with non-increasing ϵ_t; Corollary 10 then establishes payoff-matrix consistency.Source: §4.2 · pp. 17–18
• The estimated mixed strategies converge in ℓ2-norm to the optimal strategy sets, while contextual behavior strategies converge to their optimal counterparts under uniqueness and vanishing exploration conditions.The contextual behavior-strategy result requires Assumption 4.4, ϵ_t → ϵ_∞, and c_t/t → 0.Source: §4.2 · p. 18
• The parameter estimators are asymptotically normal, with asymptotic variance depending on the optimal strategies through the expectation integrand.The asymptotic normality follows from the Martingale Central Limit Theorem together with parameter-estimator convergence.Source: §4.2 · pp. 18–19
• Under Assumption 4.5 (Margin), when the conditions in Corollary 14 hold and ηt = O(t−1/2), the doubly robust value estimator is T-consistent.Source: §4.2 · pp. 19–20
• With ϵ_t ≍ t^-1/4 and η_t = O(t^-1/2), OnGameLearn achieves sublinear cumulative contextual Nash-gap regret under all three exploration strategies.For UCB, the result additionally assumes c_t = O(√log t); contextual payoff-matrix estimation error is controlled uniformly over realized contexts.Source: §4.2 · p. 20

5.1 Simulation Studies for Online Matrix Games

• In matrix games, OnGameLearn’s mixed strategies converge steadily and remain stable after exploration, while baseline methods converge more slowly and fluctuate more, especially Tsallis-INF under violated bounded-payoff assumptions.Both ϵ-Greedy and UCB variants nearly overlap in the equilibrium-convergence plots; EXP3 and Tsallis-INF are regret-minimization baselines and do not provide payoff-parameter inference.Source: §5.1 · pp. 21–22
• Under ϵ-Greedy exploration, payoff estimates achieve SE/MCSD convergence to 1, bias approaching zero, and nominal 95% coverage for all Aij parameters.These metrics evaluate proper variance estimation, unbiasedness, and Wald-type confidence-interval coverage; EXP3 and Tsallis-INF do not provide statistical inference for payoff parameters.Source: Figure 3 · p. 22, §5.1 · p. 22
• The matrix-game doubly robust estimator converges to the optimal policy value V*A, while baseline estimates fluctuate more, stabilize more slowly, or retain finite-sample bias.The log-log estimation-error plot has slopes aligned with the theoretical 1/√T convergence rate stated in Theorem 7.Source: §5.1 · pp. 22–23

Hotel pricing games

• The application models Hotel 2 and Hotel 4 as competing row and column players with binary above- or below-average pricing actions and three contextual features.The features are advance purchase days, length of stay, and party size, with an intercept included; profits use nightly rates, estimated costs, room counts, and stay lengths.Source: §6 · p. 25

Results

• Across inference metrics, OnGameLearn approaches nominal standard-error, bias, and coverage levels as time steps increase.Figure 7 evaluates the average SE-to-MCSD ratio, average bias, and coverage probability against nominal levels of 1, 0, and 95%.Source: §6 · p. 26, Figure 7 · p. 26
• T^-1/2 convergence is retained by the doubly robust value estimator after imposing a small lower bound on estimated Nash equilibrium probabilities.The lower bound stabilizes inverse-probability weights when Assumption 4.5 is difficult to verify directly.Source: §6 · p. 26, Figure 8 · p. 27
• $29.4 is the approximate expected zero-sum game value, implying Hotel 2 loses and Hotel 4 gains about $29.4 per transaction under Nash equilibrium.The result suggests customers are more likely to choose Hotel 4 when both hotels adopt Nash equilibrium strategies.Source: §6 · pp. 26–27

7 Conclusion

• OnGameLearn integrates policy learning and evaluation for online contextual matrix games, with statistical inference guarantees, a T-consistent doubly robust value estimator, and demonstrated effectiveness in numerical experiments and a real-world application.The framework covers both multi-armed and contextual matrix-game settings.Source: §7 · p. 27

A Behavior Policy

• The behavior policy combines estimated Nash-equilibrium strategies with exploration algorithms to avoid becoming trapped in a specific action pair, and extends these schemes to context-dependent games.The contextual construction ensures each action pair is selected with probability at least t/(mk).Source: A Behavior Policy · pp. 34, 36

B Discussion

• The discussion identifies bootstrap-based two-sided Wald-type confidence intervals for optimal value and extensions from two players to general multi-agent scenarios as future work.The proposed multi-agent extension retains Nash equilibrium as the strategy characterization and requires corresponding statistical inference techniques.Source: §7 · p. 27, B Discussion · p. 36

B.1 Convergence Rate of Estimated Nash Equilibrium

• The estimated regularized Nash equilibrium is consistent and has an explicit convergence rate, with the contextual-game extension requiring the additional realized error-bound condition in Assumption D.2.The convergence-rate theorem concerns equilibria computed from estimated payoff matrices and extends the same argument to contextual games under that condition.Source: §B.1 · pp. 36–37

B.2 Computation of the Regularized Nash Equilibrium

• Regularized Nash equilibria are computed by solving saddle-point problems over probability simplices with gradient descent–ascent and projection, while regularization ensures existence and uniqueness.Compact convex feasible sets and continuity guarantee solutions for fixed estimated payoff matrices.Source: §B.2 · p. 37
• In the non-contextual setting, each gradient descent–ascent iteration costs O(mk), giving total optimizer-computation cost O(Nmk) for N inner iterations.The dominant operation is matrix–vector multiplication with the estimated payoff matrix.Source: §B.2 · p. 38
• In the contextual setting, constructing the estimated payoff matrix costs O(mkd), followed by O(Nmk) for solving the regularized minimax problem for each observed context.The construction evaluates mk inner products between d-dimensional vectors.Source: §B.2 · p. 38
• Computational burden can be reduced by using fewer inner iterations and warm-starting from the previous optimizer because estimated payoff matrices typically change gradually over time.Warm starts can reduce the number of iterations needed for practical convergence.Source: §B.2 · p. 38

B.3 Extension to Model Misspecification

• The misspecified setting allows the working linear payoff model to differ from the true conditional mean payoff while retaining the prior exploration and regularity conditions.The true payoff is represented as r_t = φ_i_t,j_t(z_t) + e_t.Source: §B.3 · pp. 38–39
• Inverse-probability-weighted least-squares estimation yields contextual Nash policies converging to the Nash equilibrium policies of the pseudo-payoff game.Behavior policies converge to clipped or randomized versions of the limiting policies, depending on the exploration strategy.Source: §B.3 · pp. 38–39
• The payoff estimator admits an asymptotic normal approximation under model misspecification.This extends the inferential results beyond the correctly specified working model.Source: §B.3 · p. 39
• The value estimator is consistent for the pseudo-true contextual game induced by Nash equilibrium of M†(z), not the game induced by the true conditional mean functions φ_ij(z).Formal statements and proofs are deferred to Section F, with corresponding simulations in Section C.3.Source: §B.3 · p. 39

C Additional Experiments Results

• The additional experiments supplement Section 5 with Tsallis-INF hyperparameter studies and UCB-based statistical inference in non-contextual online matrix games.These results are reported in Section C.1.Source: C Additional Experiments Results · p. 39
• Additional experiments also evaluate contextual matrix games under UCB exploration, model misspecification, and larger action spaces with m, k > 2.These settings correspond to Sections C.2, C.3, and C.4, respectively.Source: C Additional Experiments Results · p. 39

C.1 Additional Results for Online Matrix Games

• The Tsallis-INF sensitivity analysis evaluates several learning-rate coefficients because Gaussian-noise payoffs are not uniformly bounded and importance-weighted updates can be sensitive to large noisy observations.The schedule is ηt = η0/√t + 1, and the original Tsallis-INF paper suggests η0 = 2.0 for the standard bounded-loss setting.Source: §C.1 · p. 39
• Figures C.1 and C.2 assess estimated Nash-equilibrium convergence and T-consistency of the doubly robust value estimator under alternative η0 settings.The reported settings include η0 = 2.0 in Figure C.1 and η0 = 4.0 in Figure C.2.Source: Figure C.1 · p. 40, Figure C.2 · p. 40, §C.1 · p. 40
• The online-game UCB experiment provides statistical inference results for OnGameLearn as an additional exploration-strategy evaluation.It complements the ϵ-greedy results reported in the main text.Source: §C.1 · p. 40
• OnGameLearn performs well under UCB across all three evaluation metrics in online matrix games, supporting inference beyond ϵ-greedy exploration and agreeing with Theorem 5.These findings indicate that the inference procedure is not limited to the original exploration strategy.Source: §C.1 · p. 40

C.2 Additional Results for Contextual Matrix Games

• The contextual-game UCB experiment evaluates statistical inference for OnGameLearn, including consistency and asymptotic normality of the payoff estimator.This extends the contextual analysis beyond the original ϵ-greedy exploration strategy.Source: §C.2 · p. 40, Figure C.3 · p. 41
• OnGameLearn continues to perform well under UCB across all evaluation metrics in contextual matrix games, supporting applicability beyond ϵ-greedy exploration and consistency with Theorem 13.Figure C.3 reports consistency and asymptotic normality of the payoff estimator under UCB exploration.Source: §C.2 · p. 41, Figure C.3 · p. 41

C.3 Simulation Results under Model Misspecification

• OnGameLearn converges to the target Nash equilibrium and accurately estimates the least-false game value under model misspecification.These findings are consistent with Theorems 20 and 22.Source: §C.3 · p. 42, Figure C.5 · p. 42
• The misspecified-model simulations provide empirical evidence for asymptotic normality of the WLS estimator for least-false parameters.This supports Theorem 21.Source: §C.3 · p. 42, Figure C.6 · p. 43

C.4.1 Online Matrix Games

• OnGameLearn provides consistent Nash-equilibrium estimates for 4 × 4 games under both ϵ-greedy and UCB, whereas EXP3 and Tsallis-INF are unstable.The two OnGameLearn curves nearly overlap across all equilibrium coordinates, while the baselines do not yield stable or consistent estimates.Source: §C.4.1 · p. 44, Figure C.8 · p. 44
• In 4 × 4 online matrix games, the payoff-matrix estimator exhibits asymptotic normality under both ϵ-greedy and UCB exploration, verifying Theorem 5.The 16 matrix-element curves approach the nominal reference line across all three evaluation metrics.Source: §C.4.1 · p. 44, Figure C.9 · p. 45, Figure C.10 · p. 46

C.4.2 Contextual Matrix Games

• OnGameLearn significantly outperforms baselines in larger contextual games, although OnGameLearn-UCB converges slightly more slowly.In the 4-action contextual setting, the method continues to perform well under contextual variation.Source: §C.4.2 · p. 45

D.1 Tail Bound of Payoff Matrix Estimator (Theorem 1)

• Theorem 1’s proof derives payoff-estimator concentration by treating centered action-pair indicators as a bounded martingale-difference sequence and combining two bounds.Conditional payoff noise is assumed mean-zero sub-Gaussian for the selected action pair.Source: §D.1 · pp. 46–48

D.2 Convergence of the Estimated Nash Equilibrium (Theorem 3)

• Theorem 3’s proof uses strong convexity-concavity, continuity, convex strategy spaces, and semicontinuity conditions to establish convergence of the estimated Nash equilibrium.The regularized game has a unique equilibrium represented by the optimization maps X*(A, η) and Y*(A, η).Source: §D.2 · pp. 48–49

D.3 Convergence of the Behavior Strategies (Theorem 4)

• The behavior-strategy proofs analyze exploration probabilities for ϵ-greedy, UCB, and TS, with ϵ-greedy yielding limiting behavior policies and UCB using a consistent payoff estimate.The UCB upper bound includes an exploration-control constant c_t.Source: §D.3 · p. 50, §D.3.1 · p. 51, §D.3.2 · p. 51

D.3.2 Proof of UCB

• The exploration bonus vanishes in probability as t →∞, so the optimistic payoff matrix approaches the estimated payoff matrix.Source: §D.3.2 · p. 51
• Non-expansive Euclidean projection and jointly continuous clipping preserve convergence when the clipping parameter satisfies ϵ < 1/d.Source: §D.3.2 · p. 51
• Clipped behavior strategies converge to the projections of the true Nash equilibrium strategies onto the corresponding truncated simplices.This follows from the asymptotically vanishing exploration bonus together with the continuity of projection and clipping mappings.Source: §D.3.2 · pp. 51–52

D.3.3 Proof of TS

• Thompson-sampled payoff matrices converge entrywise in probability to the true matrix, and their induced equilibrium strategies converge to projections of the true strategies onto truncated simplices.The proof combines vanishing posterior sampling error with estimator consistency and arguments analogous to earlier equilibrium-convergence results.Source: §D.3.3 · p. 53

D.4 Asymptotic Normality of Payoff Matrix Estimator (Theorem 5)

• The payoff-matrix estimator’s asymptotic normality is proved by applying a martingale central limit theorem after verifying conditional Lindeberg and variance-limit conditions.The variance estimator’s consistency is then established using expansions, a weak law of large numbers, and continuous-mapping arguments.Source: §D.4 · pp. 53–56

D.5 Convergence Rate of of Saddle Value (Corollary 6)

• The saddle-value convergence proof treats the payoff matrix and regularization parameter as a joint parameter and bounds derivatives using compact simplices and bounded regularization terms.It defines the saddle value through the regularized payoff function and applies a mean-value argument between estimated and true parameters.Source: §D.5 · pp. 56–57

T-consistency of Value Estimator (Theorem 7)

• Under the stated margin conditions, the doubly robust value estimator has uniformly bounded importance ratios and achieves the proven square-root convergence rate across the three exploration strategies.The proof uses almost-sure convergence of payoff and equilibrium estimates, with local support stability setting importance ratios to zero outside the true equilibrium support.Source: T-consistency of Value Estimator… · pp. 57–60

D.7 Regret Bound (Theorem 8)

• The regret-bound proof begins by defining the time-t payoff matrix used to compute regularized equilibrium strategies and the corresponding strategies.The supplied passage covers only this setup, not the resulting regret rate.Source: §D.7 · p. 62
• The analysis establishes GapA (ext, eyt) ≤ 4LAϵt + 2 ∥A −Bt∥max + ηt(log m + log k), as stated in equation (D.5).Source: §D.7 · p. 64

D.8 Convergence Rate of Estimated Nash Equilibrium (Theorem 17)

• Estimated equilibria converge by first bounding payoff-estimation error, then converting Nash gap to equilibrium distance through Hoffman’s error bound.In contextual games, the argument requires a uniform realized error-bound condition that excludes nearly degenerate payoff matrices; under it, C(z_t) = O_p(1).Source: §D.8 · pp. 68–69

E.1 Tail Bound of Parameter Estimator (Theorem 9)

• The parameter-estimator proof controls the minimum eigenvalue of the contextual design matrix before deriving a high-probability estimation-error bound for each action pair.The argument uses the exploration sequence and Assumption 4.3 to establish the required design-matrix control.Source: §E.1 · pp. 69–71

E.2 Consistency of Payoff Matrix Estimator (Corollary 10)

• The payoff-matrix estimator is consistent: for every fixed h > 0, its estimation error probability converges to zero.The proof bounds the concentration terms using logarithmic growth in t and the exploration schedule.Source: §E.2 · p. 72

E.3 Convergence of the Estimated Nash Equilibrium (Theorem 11)

• Context-dependent payoff-matrix consistency and upper semi-continuity of the equilibrium mappings establish convergence of the estimated Nash equilibrium.The contextual matrix M(z_t) is replaced by its estimator, and Lemma 5 supplies the required equilibrium-mapping regularity.Source: §E.3 · pp. 72–73

E.4 Convergence of the Contextual Behavior Policy (Theorem 12)

• The contextual behavior-policy proof analyzes limiting exploration probabilities separately for ϵ-Greedy, UCB, and Thompson sampling.These three cases determine the limiting behavior strategies under the different exploration mechanisms.Source: §E.4 · p. 73

E.4.2 Proof of UCB

• Under UCB, the clipped contextual behavior policy converges to the projection of the true contextual equilibrium-induced joint action distribution.The proof uses contextual upper confidence bounds and the estimated-equilibrium convergence result.Source: §E.4.2 · pp. 74–75

E.4.3 Proof of TS

• Under Thompson sampling, vanishing posterior-sampling error implies that the clipped behavior policy converges to the projection of the true contextual equilibrium-induced joint action distribution.The proof represents the posterior draw through a Gaussian perturbation and shows that its sampling error vanishes in probability.Source: §E.4.3 · p. 75

E.5 Asymptotic Normality of Online Estimator (Theorem 13)

• Theorem 13 proves asymptotic normality of the online estimator by verifying a conditional Lindeberg condition and identifying the limiting conditional variance.The proof applies the Martingale Central Limit Theorem and combines the components with Slutsky’s Theorem.Source: §E.5 · pp. 77–78
• The variance estimator is shown to be consistent by decomposing its terms and applying the Weak Law of Large Numbers and continuous mapping.The final remainder term converges in probability to zero by Lemma 7.Source: §E.5 · pp. 78–79

E.6 Convergence Rate of Saddle Value (Corollary 14)

• Corollary 14 derives the convergence rate of the contextual saddle value by applying the saddle-value argument to estimated contextual payoff matrices and their asymptotic estimator distribution.The contextual substitutions use M(z_t) and the estimated matrix, together with the regularization parameter η_t.Source: §E.6 · pp. 79–80

T-consistency of Contextual Saddle Value Estimator (Theorem 15)

• Theorem 15 establishes square-root-rate convergence for the contextual saddle value estimator under either condition of the formal uniform-margin Assumption E.1.The assumption covers uniform separation involving contextual equilibrium quantities and regularized contextual-game equilibria.Source: T-consistency of Contextual Saddle… · pp. 80–82
• The consistency proof treats ϵ-Greedy, UCB, and TS separately, using eventual exploration, contextual payoff control, continuity of equilibrium mappings, and vanishing regularization.For UCB and TS, the contextual payoff matrices satisfy the required bounds through the estimated behavior-strategy parameters.Source: T-consistency of Contextual Saddle… · pp. 81–82
• The estimator’s main decomposition has J1 = Op(T^-1/2), combining the saddle-value rate with a Central Limit Theorem for i.i.d. contextual equilibrium values.The J1 term is bounded using Corollary 14 and η_t = Op(t^-1/2).Source: T-consistency of Contextual Saddle… · p. 83
• The martingale component satisfies J2 = Op(T^-1) = op(T^-1/2) because importance ratios are eventually uniformly bounded and conditional noise variances remain bounded.Thus J2 is asymptotically smaller than the J1 contribution in the consistency decomposition.Source: T-consistency of Contextual Saddle… · p. 83

F Analysis of Contextual Matrix games under Model Misspecification

• The contextual WLS estimator satisfies a tail bound and is consistent under bounded linear approximation error and the stated regularity conditions.The consistency result follows when the sample size and error tolerance satisfy the conditions of the tail-bound theorem.Source: F Analysis of Contextual Matrix games… · pp. 84, 86, 92
• Estimated contextual behavior strategies converge when each least-false contextual game has a unique Nash equilibrium.The limiting behavior strategies depend on the exploration strategy.Source: F Analysis of Contextual Matrix games… · p. 86
• Under the strategy-convergence conditions, the contextual WLS estimator is asymptotically normal with a consistently estimable asymptotic variance.The proof combines martingale central limit arguments, convergence of the weighted design matrix, and consistency of the variance estimator.Source: F Analysis of Contextual Matrix games… · pp. 87, 91, 92
• The contextual value estimator is consistent under model misspecification when the conditions for contextual WLS asymptotic normality hold.The pseudo value uses pseudo-optimal strategies but evaluates payoffs under the true contextual payoff matrix.Source: F Analysis of Contextual Matrix games… · pp. 92–94
• Policy consistency transfers to value consistency, but a sqrt(T) rate requires an additional orthogonality condition or sufficiently fast policy convergence.Because pseudo-target strategies need not be saddle-point optimal for the true payoff matrix, plug-in value error includes first-order policy-estimation terms.Source: F Analysis of Contextual Matrix games… · p. 95

Adaptive concentration

• The auxiliary lemmas provide tail bounds for adaptive Bernoulli sums, positive-semidefinite matrix sums, and self-normalized action-pair estimates.The scalar bound assumes a predictable mean lower bound, the matrix bound controls minimum eigenvalues under bounded PSD increments, and the self-normalized result applies conditional sub-Gaussian observations.Source: G Auxiliary Results · pp. 96, 98, 99
• Under bounded contextual covariates and conditional sub-Gaussian, mean-zero noise, a self-normalized contextual martingale bound controls adaptive parameter estimation.The result is derived by exponential-moment arguments, Gaussian averaging, determinant control, and Markov’s inequality.Source: G Auxiliary Results · pp. 101, 103

Equilibrium continuity

• If the game parameters converge and the limiting game has a unique saddle point, every corresponding sequence of saddle strategies converges to that limiting equilibrium.The continuity result uses the parameterized payoff f(x, y, A, η), the saddle-set correspondence S(A, η), and continuity-attainment conditions for both players.Source: G Auxiliary Results · pp. 103–104

Saddle-value sensitivity

• For continuous games on compact strategy sets with continuously differentiable parameter dependence, the saddle-value difference is bounded between gradient evaluations at cross-indexed optimal strategies.The bound applies the mean value theorem to the minimax and maximin representations and extends prior absolute-continuity results to saddle values.Source: G Auxiliary Results · p. 104

Concentration of contextual IPW terms

• Lemma 9 derives concentration bounds for contextual IPW terms under conditional sub-Gaussian reward noise and action-pair sampling probabilities.The proof uses martingale differences, conditional variance control, Freedman’s inequality, moment-generating-function bounds, and Markov’s inequality.Source: G Auxiliary Results · pp. 106–107

Properties of product norm

• Lemma 10 shows that the product-space expression combining Frobenius and Euclidean components is a valid norm.Positive definiteness, absolute homogeneity, and the triangle inequality follow from the corresponding properties of the component norms.Source: G Auxiliary Results · pp. 108–109