Source-linked AI summary
SafeAdapt-Poker: Counterfactual Risk Calibration for Budgeted Adaptation under Opponent Shift
Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Rui Chen, Daren Zha, Jun Xiao
TL;DR
Opponent histories can mislead under behavioral shifts, leaving the problem of adapting while preserving a strong anchor and bounded deviation. SafeAdapt-Poker separates local action downside from the future value of budget consumption and allocates among support-preserving deviations. On exact Leduc, it reports a +0.0718 weak-pool gain with a −0.0126 worst-opponent delta, alongside improved delayed-opportunity capture over myopic allocation.
Problem
The paper asks how an agent can spend bounded anchor-relative deviation using only pre-decision information when opponent evidence may be sparse, deceptive, or non-stationary.
Source: §1 · p. 1Method
SafeAdapt estimates local action downside and future budget opportunity cost separately, then combines them with a pathwise KL ledger and pre-interaction screening of frozen controllers.
Source: §1 · p. 1Results
On exact Leduc, the full controller reports +0.0718 weak-pool gain with a −0.0126 worst-opponent delta and captures 0.7841 delayed opportunities versus 0.6318 for myopic allocation.
Source: §1 · p. 1Takeaways & Limitations
The anchor-deviation certificate is distinct from exploitability and worst-opponent value, which the paper measures separately.
Source: §5 · p. 8Calibration bounds rely on the declared calibration-session distribution, and systematic out-of-distribution error can cause premature spending or excessive budget retention.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Opponent histories can mislead after behavioral shifts. SafeAdapt-Poker formulates budgeted adaptation under opponent shift as a dual-counterfactual sequential-control problem using only information available before each decision. At each decision the controller asks two distinct questions: what local downside a deviation from the frozen anchor would induce under matched legal continuations, and what future adaptation value would be lost by consuming part of the remaining deviation budget now. The first question is answered by a monotone strategic-risk model over deviation intensity; the second by a paired budget-state continuation model that prices current budget consumption. The controller selects among support-preserving candidate deviations by trading immediate adaptation value against these two quantities, a pathwise cumulative-KL ledger bounds total anchor-relative deviation, and a finite family of frozen controllers is screened on complete paired calibration sessions before held-out interaction. A finite-candidate perturbation bound shows that the allocation objective degrades continuously with continuation-value estimation error. Evaluation covers exact-game exploitation and safety, four opponent shifts, local-risk calibration, delayed-opportunity allocation, direct opportunity-value estimation, value-error robustness, factorial mechanism attribution, budget sensitivity, abstract-HUNL cross-play, and player/time-held-out public-history behavior. On exact Leduc the full controller attains a matched weak-pool gain of +0.0718 with a worst-opponent delta of -0.0126, raises delayed-opportunity capture from 0.6318 under myopic allocation to 0.7841 under the same session budget, and reaches +42.7 mbb/hand in abstract-HUNL cross-play; the retained public-history screen reports 0.5820 action accuracy and 0.9277 log loss versus 0.5039 and 0.9778 for a public prior.
1 Introduction
SafeAdapt-Poker addresses how to adapt from public opponent history while preserving an anchor when evidence is sparse, deceptive, or shifting. It separates the local downside of changing an action from the future value of the budget spent, then constrains and screens adaptation before held-out interaction.
1 Introduction
- Opponent histories can support exploitation, but sparse, deceptive, or non-stationary evidence makes preserving a strong baseline important.The design issue is session-level control: local adaptation may accumulate risk or rely on stale history after a shift.Source: §1 · p. 1
- A pathwise ledger bounds cumulative anchor-relative deviation, while a finite family of frozen controllers is screened on paired sessions before held-out interaction.The protocol links decision-level risk calibration, trajectory-level accounting, and deployment-level controller selection.Source: §1 · pp. 1–2
- The dual-counterfactual formulation separates local strategic downside under a matched anchor continuation from future value lost by consuming adaptation capacity.A monotone risk model and paired budget-state continuation model estimate the two quantities separately.Source: §1 · p. 1
- +0.0718 weak-pool gain with a −0.0126 worst-opponent delta, while delayed-opportunity capture reaches 0.7841 versus 0.6318 for myopic allocation.These exact-Leduc results are reported for the full controller and the same session budget, respectively.Source: §1 · p. 1
2 Related Work
Related work spans equilibrium-oriented poker agents, safe exploitation and opponent modeling, and methods for uncertainty, constraints, and sequential allocation. SafeAdapt combines these threads around a public-only session budget and separates local intervention risk from the future value of retained capacity.
Poker agents and anchor policies
- CFR and neural variants underpin equilibrium-oriented poker agents, while systems such as DeepStack, Libratus, ReBeL, and Pluribus also use search, abstraction, opponent-pool design, and engineering.The paper keeps anchor strength separate from the adaptation module and from evaluation oracles.Source: §2 · p. 2
Closest safe exploitation and online modeling
- Restricted Nash Response, robust counter-strategies, opponent modeling, Safe Opponent Exploitation, SES, and OX-Search study balancing exploitation with robustness or adaptation safety.StratFormer is a learned sequence baseline on Leduc; SafeAdapt instead adds a public-only session budget, shift detector, and separate local-risk and retained-budget estimates.Source: §2 · p. 2
Uncertainty, constraints, and sequential allocation
- SafeAdapt connects KL-ledger constraints and finite-resource allocation to sequential decision making, while distinguishing current-action downside from the future value of remaining budget.Related foundations include calibration, uncertainty, constrained MDPs, trust regions, safe policy improvement, and counterfactual rollout methods.Source: §2 · p. 3
Distillation and selective adaptation
- Distillation compresses a teacher policy, whereas SafeAdapt measures and gates anchor-relative deviation; selective prediction motivates a closed gate that excludes history from the current action.Source: §2 · p. 3
3 Method
SafeAdapt-Poker uses public information available before each decision to adapt a frozen anchor under a finite cumulative-KL budget. Its method combines history support, uncertainty and shift signals with risk-aware candidate selection, budget valuation, and leakage-controlled evaluation.
3.1 Cross-Hand Interaction and Public Information; 3.2 Anchor-Relative Objectives and Splits
- The online state uses the hero information state, pre-decision public history, remaining KL budget, and remaining horizon; private cards, hidden shift labels, and future outcomes are excluded.Opponent and sequence IDs are disjoint across calibration and evaluation, and time groups are also split when available.Source: §3.1 · p. 3, §3.2 · p. 3
3.2 Anchor-Relative Objectives and Splits; 3.3 SafeAdapt-Poker
- Anchor-relative evaluation compares expected value against a frozen reference, while the public-history screen reports behavior prediction separately from matched game-study values.The full method encodes public sequences; its inputs include public betting and card information, observed actions, hand boundaries, and sequence position.Source: §3.2 · p. 3, §3.3.2 · p. 4
3.3.1 Count-Based History Baseline
- The count-based history baseline uses a small public-state bucket and calibrated history counts, mixing them with the anchor only within a local KL radius and count gate.The retained PHH adapter uses λ=1 for the history branch and has no latent opponent-type variable.Source: §3.3.1 · pp. 3–4
3.3.2 History Encoder, Uncertainty, and Shift
- Independent history heads estimate epistemic uncertainty, while fast and slow summaries support a shift signal that can reduce history weight and the next allowed deviation.A count-gated residual baseline provides a comparison that isolates the learned uncertainty and shift signals.Source: §3.3.2 · p. 4
3.3.3 History-Conditioned Residual and Training Objective
- The residual is parameterized relative to anchor logits and trained to maximize weak-pool value while controlling mean, tail, and worst-opponent degradation.Thresholds, anchor checkpoints, and opponent pools are selected on training or validation opponents.Source: §3.3.3 · p. 4
3.3.4 Dual-Counterfactual Risk and Opportunity Control
- The controller estimates local downside under a matched anchor continuation and budget opportunity cost from paired future continuations, then selects feasible candidates by gain minus that cost.Feasibility includes risk, budget, support, and shift conditions; Myopic SafeAdapt removes the opportunity-cost term.Source: §3.3.4 · p. 4
- The finite-candidate allocation bound limits selection-score loss to 2εg + 4εV when gain and continuation-value errors are uniformly bounded.The opportunity-cost estimate error is bounded by 2εV, and the objective degrades continuously with model error.Source: §3.3.4 · pp. 4–5
3.3.5 Pathwise KL Ledger and Heuristic Projection; 3.3.6 Trajectory-Level Anchor-Deviation Analysis; 3.3.7 Leakage-Free Execution Contract
- A pathwise KL ledger debits each chosen deviation without refill and returns to the anchor when budget is exhausted or shift crosses its threshold; trajectory guarantees require shared kernels and a bounded return.The leakage-free execution contract freezes calibration before held-out sequences and joins outcome or value oracles only after traces are immutable.Source: §3.3.5 · p. 5, §3.3.6 · p. 5, §3.3.7 · p. 5
4 Experiments
Across matched experiments, SafeAdapt combines stronger exact-Leduc gain with limited worst-opponent loss, and is evaluated on calibration, opponent shifts, budget allocation, large-game cross-play, and public histories.
4.1 Experimental Setup and Research Questions
- Full SafeAdapt reaches +0.0718 weak-pool gain and −0.0126 worst-opponent degradation, with the lowest adaptive-method exploitability and recovery time.The comparison uses a fixed anchor, opponent pool, seat protocol, solver configuration, and statistical unit.Source: §4.1 · p. 5, Table 1 · p. 6
4.2 Counterfactual Risk Calibration
- The monotone risk curve reaches AUROC 0.8819 and AUPRC 0.7886, while reducing ECE to 0.0318 and accepted downside to 0.0119.Against the unconstrained classifier, AUROC and AUPRC are 0.8147 and 0.7118, ECE is 0.0594, and accepted downside is 0.0201.Source: §4.2 · p. 5, Table 2 · p. 6
4.3 Opponent Shift and Recovery
- Under bait-and-switch, Full SafeAdapt has 0.0578 regret and recovers in 11.9 hands, versus 0.0963 and 22.5 for OX-Search.Across the four shifts, its detection AUROC ranges from 0.856 to 0.902 and maximum drawdown from 0.0237 to 0.0409.Source: Figure 2 · p. 6, §4.3 · p. 6
4.5 Delayed-Opportunity Budget Allocation
- With the same session budget, Full SafeAdapt captures 0.7841 of high-value decisions versus 0.6318 for myopic allocation, while spending less budget early.It spends 0.2153 before the high-value window, compared with 0.3281 for myopic allocation, and its gain and worst delta are +0.0698 and −0.0131.Source: §4.5 · pp. 6–7, Table 3 · p. 7
4.4 Safety–Exploitation Frontier and Ablations
- Increasing the session budget raises weak-pool gain from +0.0258 to +0.0754 while worst-opponent degradation grows from −0.0047 to −0.0196; removing the ledger yields −0.0915.The very small budget projects 78.4% of decisions, compared with 24.2% under the large budget.Source: §4.4 · p. 6, Figure 3 · p. 7
4.6 Factorial Mechanism Attribution
- Risk supervision alone improves safety and recovery, opportunity value alone raises delayed capture, and their combination is best across all reported columns.The combined controller reaches +0.0718 gain, −0.0126 worst delta, 8.9 recovery hands, and 0.7841 delayed capture.Source: Table 4 · p. 7, §4.6 · p. 7
4.7 Budget-Opportunity Estimation Quality; 4.8 Opportunity-Value Robustness
- Paired estimation reduces opportunity-cost MAE from 0.0335 to 0.0182 and allocation regret from 0.0152 to 0.0068; large-noise capture remains 0.6627 versus 0.6318 for myopic allocation.Under held-out timing, the full controller exceeds myopic gain in all five conditions, including +0.0228 on the unseen bursty schedule.Source: §4.5 · p. 6, §4.7 · p. 7, §4.8 · p. 7
4.9 Large-Game Cross-Play; 4.10 Real PHH History Generalization
- Under matched compute and duplicate deals, Abstract-HUNL cross-play reaches +42.7 ± 3.3 mbb/hand; across complete-hand clusters, safe history improves player-set accuracy by +0.0716 and time-group accuracy by +0.0579 (Table 5; Table 21).Across complete-hand clusters, safe history also lowers log loss, Brier score, and ECE, raises shift AUROC from 0.6412 to 0.8127, and halves the false-adaptation rate (Table 21).Source: Table 5 · p. 8, §4.9 · p. 8, §4.10 · p. 8
5 Limitations, Ethics, and Conclusion
SafeAdapt-Poker controls adaptation with a pathwise deviation ledger and selects frozen controllers through closed-loop calibration. The conclusion connects these mechanisms to several evaluation settings while distinguishing session value from snapshot exploitability.
5 Limitations, Ethics, and Conclusion
- The complete-entry audit is limited to recorded stakes, sites, human play, and time groups, while tiny Leduc and abstract HUNL cover declared finite-game and action-space settings.Current actions, private cards, outcomes, styles, and value maps remain outside the online trace; the release excludes raw hands and identifying metadata.Source: §5 · p. 8
- Systematic out-of-distribution continuation-value error can still cause premature spending or excessive budget retention, despite the perturbation bound.The bound applies under uniform continuation-value error; the finite-family calibration proposition is a selection-time statement under the declared calibration-session distribution.Source: §5 · p. 8
- The conclusion links dual counterfactuals, a pathwise ledger, and closed-loop calibration to calibration, shift recovery, opportunity allocation, cross-play, and public-history behavior.Exploitability and worst-opponent value are measured separately from the anchor-deviation certificate.Source: §5 · p. 8
A Closest-Work Comparison
- SafeAdapt-Poker combines public-only sequence input, explicit shift protection, and a cumulative session-level deviation ledger, properties absent together from the compared methods.The comparison uses the same anchor-relative estimands across methods.Source: A Closest-Work Comparison · p. 11
B.1 Safety Conditions
- Online state is restricted to pre-decision information, while calibration, threshold selection, and anchor selection precede held-out opponent evaluation.The protocol separates action agreement, analytic utility, and strategic EV, and uses clustered intervals and controls before poker-strength claims.Source: §B.1 · p. 11
B.2 Information Boundary
- The offline oracle is joined only after the policy trace and replay digest are frozen.The table identifies the policy-facing and post-hoc fields enforced by this information boundary.Source: §B.2 · p. 11
C Strategic Evaluation Definitions
- Snapshot exploitability measures the induced single-hand policy conditional on a frozen controller context, whereas repeated-session value uses complete budget-state interaction.Snapshot NashConv sums unilateral best-response improvements, and exploitability is that quantity divided by two.Source: C Strategic Evaluation Definitions · p. 11
C.1 Proof of the Finite-Candidate Perturbation Bound
- The finite-candidate perturbation result bounds per-decision objective loss by 2ε_g + 4ε_V over the estimated feasible set.Risk-feasibility errors enter through that set and are evaluated separately in the calibration study.Source: §C.1 · p. 11
C.2 Reproducibility Fields
The reproducibility fields record configuration, data splits, oracle integrity, and costs. Additional protocols specify the statistical units and table scopes for budget, shift, and closest-work comparisons.
C.2 Reproducibility Fields
- The release records configuration, selection and calibration digests, counterfactual-labelled states, candidate intensities, label-generation cost, and controller-family grids while excluding identifying metadata and host details.The stability audit is separate from the primary PHH table, and full calibration fields are in Appendix D.Source: §C.2 · p. 11
- The eight selected PHH entries use disjoint calibration and evaluation hands, player-set groups, and time groups, with train-only calibration and frozen traces checked before oracle joins.Replay checks cover policy traces and exact rollouts where applicable.Source: §C.2 · p. 11
- The frozen configuration records seeds, model and optimizer settings, KL cap, session budgets, temperature, gate, and separate GPU, latency, throughput, memory, and monetary-cost fields.The ledger measures cumulative KL deviation.Source: §C.2 · p. 11
D Additional Experimental Protocols
- Additional protocols fix each table’s protocol and statistical unit before reporting values, and the session-budget sweep holds anchor, opponent pool, and seeds constant across budget levels.Projection activity is the fraction of decisions whose raw distribution violates the active allowance.Source: D Additional Experimental Protocols · p. 11, §D.1 · p. 11, Table 8 · p. 12
C.2 Reproducibility Fields
- The closest-work table compares online updating, explicit shift protection, and session-level cumulative deviation budgets; the information-boundary table places oracle joins after trace and digest freezing.A dash indicates an absent property.Source: Table 6 · p. 12, Table 7 · p. 12
D.2 Opponent-Shift Recovery Values
- The shift-recovery table reports post-shift regret, maximum drawdown, recovery hands, and shift-detection AUROC, retaining right-censored sessions in the failure ledger.Recovery requires K consecutive hands with positive rolling anchor-relative value after the first post-switch hand.Source: §D.2 · p. 12
D.3 Component Ablation Values
The component-ablation table reports independently retrained rows under the matched Leduc protocol and attributes the two counterfactual mechanisms separately.
D.3 Component Ablation Values
- Table 10 reports independently retrained component-ablation rows under matched Leduc conditions, with risk and opportunity-value mechanisms attributed separately in Table 4.The passage provides protocol and attribution scope, but no ablation values.Source: §D.3 · p. 12
D.4 Counterfactual Risk Calibration
The section evaluates held-out risk calibration across deviation intensities and shift schedules. The monotone model improves calibration and accepted downside across intensity strata, while the unconstrained classifier’s ordering violations increase with intensity.
D.4 Counterfactual Risk Calibration
- Held-out calibration freezes candidate deviation intensities before joining the continuation oracle and evaluates metrics on opponent–sequence–seed blocks.The target labels downside when η_tj L_t < −ε_loc.Source: §D.4 · p. 12
- The monotone model lowers ECE by 45–49% and accepted downside by 40–43% in every intensity stratum, while unconstrained order violations rise from 0.1148 to 0.1834.Figure 4 shows ECE, accepted downside, and within-state order-violation rates; the monotone model’s violation rate is zero by construction.Source: §D.4 · p. 12, Figure 4 · p. 12
- Risk calibration is also stratified by the predeclared shift schedule, with the stationary row matching the main counterfactual-risk comparison.Table 12 reports the shift-stratified results.Source: §D.4 · p. 12
D.5 Delayed-Opportunity Budget Allocation
The delayed-opportunity studies test whether an allocator retains budget for later high-value decisions, then examine sensitivity to opportunity-value noise and the closed-loop controller-selection audit. Additional configuration details specify the abstract-HUNL evaluation setup.
D.5 Delayed-Opportunity Budget Allocation
- Early KL measures budget spent before the high-value window, and capture measures high-value decisions at which sufficient budget remains; Figure 5 also marks the offline oracle as an upper-bound diagnostic.Methods share the opponent schedule, chance outcomes, candidate menu, and total KL budget; the oracle is added after online traces are frozen.Source: §D.5 · p. 12, Figure 5 · p. 13
- The full controller’s gain margin is +0.0299 for late opportunities and +0.0228 for unseen bursty timing, while the early-opportunity margin is +0.0037.The bursty schedule was excluded from training; Figure 6 compares session gains under held-out timing conditions.Source: §D.5 · pp. 13–14, Figure 6 · p. 14
D.6 Opportunity-Value Estimation and Robustness
- Larger opportunity-value perturbations increase early spending and reduce capture, weak-pool gain, and worst-opponent value.Figure 7 plots these four quantities under the matched delayed-opportunity protocol.Source: §D.6 · p. 14, Figure 7 · p. 14
D.7 Closed-Loop Controller Calibration Audit
- The calibration audit evaluates a predeclared frozen controller family and reports calibration downside, group-wise upper confidence bounds, acceptance, held-out downside, and threshold violations.From 16 to 128 candidates, calibration worst UCB rises from 0.0468 to 0.0538 and the accepted fraction falls from 13/16 to 87/128; weak EV rises from +0.0684 to +0.0722, with zero evaluation violations.Source: §D.7 · p. 15, Table 18 · p. 15
D.8 Abstract-HUNL Configuration and Cost Ledger
- The abstract-HUNL configuration fixes its abstractions, CFR anchor, held-out opponent pool, matched compute, and rollout counts before evaluation.The pool contains 12 opponents from four families, with 1,024 duplicate deals and seat swaps; Table 19 tracks supervision and deployment costs.Source: §D.8 · p. 15
D.9 Controller Family and Reproducibility Fields
The reproducibility fields specify a frozen candidate grid and record the supervision and deployment costs of the controller, alongside an evidence ledger for retained diagnostics.
D.9 Controller Family and Reproducibility Fields
- The release records labelled-state counts, candidate intensities, rollout or exact-tree calls, label-generation and training time, and the independent validation split used to select ε_loc and γ.Table 19 defines label and training times as offline compute hours, latency as mean per-decision wall-clock time, and memory as peak online resident memory.Source: §D.9 · p. 15, Table 19 · p. 15
- The evidence ledger binds each retained diagnostic row to a frozen configuration, source and selected-entry digests, replay record, and calibration record.Measured diagnostic rows are kept separate from matched strategic studies.Source: Table 20 · p. 15
- The base controller grid contains 36 frozen candidates formed from four risk thresholds, three budgets, and three penalty weights.Calibration uses threshold ρ = 0.08 and family-wise confidence 1 − δ = 0.95, with ties broken by lowest_risk before held-out evaluation.Source: §D.9 · p. 15
D.10 Completed Experiments and Public-History Fixture
The completed experiments report public-history behavior metrics and a separate held-out tiny-Leduc diagnostic. The evaluation protocol specifies the fixture, replay checks, and the references used for the paired trade-off.
D.10 Completed Experiments and Public-History Fixture
- Table 21 reports complete-entry public-history metrics, with accuracy treated as auxiliary and paired intervals computed on complete-hand clusters.Table 22 reports behavior agreement on an eight-entry fixture; current actions and raise buckets are post-hoc historical labels.Source: §D.10 · p. 15, Table 21 · p. 16, Table 22 · p. 16
- The entry-split stability run reproduces paired intervals, with safe-history bootstrap action accuracy of 0.5211 on the player-set axis and 0.5762 on the time-group axis.The run resamples complete hands over four player-set and two time-group evaluation groups; replay passes.Source: §D.10 · p. 16
E.1 Evaluation Protocol
- The held-out run uses three train and three held-out opponent IDs across five declared styles, with 96 train and 96 held-out rows per seed.Seeds are 13, 17, and 23; opponent identity, style, private card, and analytic weak and strong values are stored only in the oracle artifact.Source: §E.1 · p. 16
- Figure 8 displays paired weak-gain and worst-strong trade-offs with descriptive intervals, while exact scoring enumerates the tiny-Leduc tree under a fail-closed 4,096-node limit.Trace, exact-rollout, and combined replay checks pass for this run.Source: §E.1 · p. 16
E.2 Diagnostic Results
The retained finite-game diagnostic reports adaptive-policy utility criteria and describes the trade-off between weak-proxy gain and strong-proxy degradation on a fixed fixture.
E.2 Diagnostic Results
- Each adaptive policy has a positive held-out weak-gain lower interval, and the safe-gate worst-opponent lower interval remains above the predeclared −ε boundary of −0.05.Intervals describe the fixed fixture; weak gain is paired with the no-memory policy and strong delta is relative to the fixed uniform anchor.Source: Table 23 · p. 16, §E.2 · p. 16
- The stats policy has the largest weak-proxy gain and the largest strong-proxy degradation among adaptive rows, while the gated policy shows the intended descriptive trade-off.Figure 8 shows retained weak-proxy gain and worst strong-proxy delta against a zero-reference anchor.Source: §E.2 · p. 16, Figure 8 · p. 16
- Exact utilities are negative on the fixed contribution states, so their ordering is reported as an analytic diagnostic rather than a general performance comparison.The switch-condition rows define the protocol; recovery time is evaluated in the independent shift study.Source: §E.2 · p. 16