Source-linked AI summary
Nous: Learning and Certifying Memory Decisions Before Source Calibration
Pranav Singh
arXiv:2610.00094v2cs.LGcs.AI
TL;DR
Memory systems must decide whether to revise state predictions from reports whose reliability may be unknown, while existing source-estimation analyses do not settle when policies can be learned or improvements certified from the same records. The paper develops policy-learning and certification methods that avoid source identification, and reports task-specific sample requirements alongside a MultiWOZ evaluation in which balanced batches retain part of the candidate's gain and receive more positive certificates under weak audits.
Problem
Existing source-estimation analyses do not determine when a memory policy can be learned or its improvement certified from the same observation law.
Source: §1 · pp. 1–2Method
The paper compares learning, source calibration, and revision certification, then uses compatible-channel bounds, witness regions, and prediction-count-preserving batches to certify policy gains.
Source: §1 · p. 2Results
Decision learning and powered certification require quadratically fewer records than fixed-precision source estimation on the specified HMM family, while experiments report increased acceptance under weak audits and an accuracy gain sacrificed by balancing.
Source: §1 · p. 2, §10 · p. 16Takeaways & Limitations
The results establish task-specific information requirements and provide a policy-level certification framework for memory revision in Nous.
Source: §10 · p. 16The studies do not validate natural reporting channels, real source lineage, open-ended attributes, or online batch scheduling, and their audits are simulated.
Source: §9 · p. 15
Abstract
from arXiv · showhide
Agent memory systems update state decisions from reports whose reliability may be unknown. Existing analyses of source estimation do not determine when a policy can be learned or its improvement certified without identifying the reporting channel. We study these three tasks using the same observed records. For a specified hidden Markov family with continuous source uncertainty, decision learning and powered certification have quadratic sample complexity, whereas fixed-precision source estimation has quartic complexity. We characterize a sharp identified interval for policy gain under an unknown shared-background channel and derive finite-sample certificates under bounded history dependence and conditional copying. Independently trained witness regions support general history spaces, and disagreement-conditioned auditing improves power for sparse revisions. For dependent histories, prediction-count-preserving batches cancel the unknown reporting background and admit conditional certificates. A MultiWOZ 2.4 evaluation uses text-processing policies on 1,000 human-written test dialogues with simulated audits. Balanced batches retain 2.26 percentage points of the full candidate's 7.34 percentage-point mean gain and obtain more positive certificates under weak audits. These results establish task-specific information requirements and provide an auditable policy-revision framework for Nous.
1 Introduction
Nous addresses when memory policies can be learned and their gains certified without identifying the reporting source. It distinguishes decision learning, source calibration, and revision certification, then compares their information requirements and describes how policy-level certificates fit into memory updates.
1 Introduction
- Existing source-estimation analyses do not establish when a memory policy can be learned or its improvement certified from the same observation law.The paper distinguishes decision learning, source calibration, and revision certification as separate statistical targets.Source: §1 · pp. 1–2
2 Memory Decisions and the Evidence Contract
2.1 Nous Integration
- Nous keeps the native Bayesian updater and persistence format unchanged, while an adapter applies or declines proposals according to their revision receipts.The adapter rejects stale states and mismatched policy identifiers; policy updates are recorded separately from observational updates.Source: §2.1 · p. 3
2.2 Information Boundaries
- Policy, witness, and batch selection precede fresh certification reports; population receipts use iid deployment records, while batch receipts condition on a preselected set.The batch setting permits dependent histories under a conditional report-channel contract.Source: §2 · p. 2, §2.2 · p. 3
3.1 Shared Observation Mechanism
- The shared-observation construction uses the same future report for learning and auditing, without an additional pinned-audit sensor.Theorem 1 gives O(ℓ^-2) records per split to learn and certify the specified candidate against the incumbent.Source: §3.1 · p. 4
3 Learning and Certification before Source Calibration
- On a continuous-source HMM family, decision learning requires Θ(ℓ^-2) records, while uniform source estimation to error below 1/16 requires Θ(ℓ^-4).The tasks use the same full records, and powered certification of a fixed proposal has matching quadratic requirements.Source: §1 · p. 2, §3.1 · p. 4
3.2 Sufficient Conditions
- For broader observation spaces, order-ℓ^r decision and audit margins with KL orders ℓ^2r and ℓ^2q yield Θ(ℓ^-2r) learning and powered certification, versus an Ω(ℓ^-2q) source lower bound.Theorem 2 separately gives Θ(ℓ^-2) current-state decision learning and an Ω(ℓ^-4) lower bound for estimating source informativeness on its stated class.Source: §3.2 · p. 5
4 Policy-Gain Identification
This section characterizes policy-gain uncertainty when reports pass through an unknown shared-background channel. It gives a sharp interval for attainable clean gains and explains when the gain's sign remains unresolved.
4 Policy-Gain Identification
- The categorical channel preserves the state with probability ρ and otherwise draws from an unknown shared background distribution.This channel family contains every positively oriented binary class-conditional channel.Source: §4 · p. 5
- Prediction-mass changes add observable bias, so the sufficient rule ∆Y > max_k m_k cannot accept an entirely one-direction change.For binary labels, the rule is ∆Y > |E[g − b]|.Source: §4 · p. 5
- For finite positive-mass histories and nonconstant observable reports, attainable clean gains form an interval over the compatible channel family.Theorem 3 characterizes the interval without requiring histories containing only one clean class.Source: §4 · p. 6
- A nonpositive lower endpoint leaves the gain's sign unresolved because an observationally identical compatible world has nonpositive gain.More records from that observation law cannot resolve this ambiguity; an interval crossing zero leaves the sign unresolved.Source: §4 · p. 6, Figure 2 · p. 6
5 Finite-Sample Certification and Repeated Releases
The section develops finite-sample certificates for fixed policies and extends them to repeated releases, bounded channel misspecification, and sparse policy changes. These certificates bound the risk of accepting a non-improving revision under their stated calibration conditions.
5 Finite-Sample Certification and Repeated Releases
- History-only witness regions can upper-bound unknown background mass from regional report frequencies, without estimating every conditional report probability.For finite history alphabets, independent training selects minimum-frequency cells, and increasing training and calibration counts recovers the sharp condition when its margin is positive.Source: §5 · pp. 6–7
- A witness certificate has level-α error for non-improving policies and, when positive, lower-bounds policy gain using the same records for witness bounds and gain estimation.It applies to fixed policies and history-only regions with iid calibration from the deployment law.Source: §5 · p. 7
5.1 Repeated Release Control
- Repeated releases spend α_t = α/[t(t + 1)] per comparison, bounding the probability of ever accepting a non-improving release by α.Each comparison uses fresh calibration data and its assigned error budget.Source: §5.1 · p. 7
5.2 Bounded Channel Misspecification
- Under bounded channel misspecification, accepting only when L_ϵ > 0 has level-α error, and an accepted L_ϵ lower-bounds gain on the stated high-probability event.A conditional arbitrary-copy budget β combined with base deviation γ gives ϵ = γ + β − γβ; budgets are specified independently of calibration reports.Source: §5.2 · p. 7
5.3 Sparse Policy Changes
- Conditioning audits on policy disagreement can reduce finite-sample dilution, with sufficient sample size O((dµ^2)^−1) when conditional prediction masses are balanced.A binomial lower bound on disagreement probability d converts conditional evidence into an unconditional positive-gain receipt.Source: §5.3 · p. 8
6 Constructing Certifiable Revision Batches
The section constructs certifiable revision batches by preserving predicted class counts, which cancels the unknown reporting background under a shared channel. Conditional certificates then apply to the specified batch even when histories are dependent.
6 Constructing Certifiable Revision Batches
- Under a common channel C0 = ρI + u1⊤, the expected audit-contrast sum is ρNGx + u⊤a, and preserving predicted class counts (a = 0) cancels the unknown background exactly.Figure 3 illustrates a three-class cycle that conserves each predicted class count, contrasted with a one-direction proposal that creates a background term.Source: §6 · p. 8, Figure 3 · p. 8
- The background term vanishes for every admissible u if and only if a = 0; a nonempty balanced subset exists exactly when the proposal multigraph contains a directed cycle.Representing proposals as directed edges makes selecting a maximum-score balanced subset a maximum-score circulation with an integral linear-program optimum.Source: §6 · pp. 8–9
- The fixed-batch procedure conditions on histories, truths, and selection, with selected reports required to remain within TV ϵ of the same positive channel after preceding audit reports.The batch is selected before fresh reports, and records with identical incumbent and candidate predictions are excluded.Source: §6 · pp. 8–9
- For a fixed batch, accepting only when Lx > 0 yields, with probability at least 1 − α, a positive lower bound (M/N)Lx on Gx; histories and truths may be dependent.When M = 0, decline; no witness regions are required.Source: §6 · p. 9
7 Experiments
The experiments evaluate inference, policy revision, and action outcomes across controlled state tasks, MiniGrid, and human-written dialogue. They show that certification depends on audit conditions, while balanced batches trade some gain for improved acceptance under weak audits.
7 Experiments
- Policies were selected using training and development data, with comparisons fixed before calibration; controlled studies used generated or executable labels, while dialogue used published annotations.Table 2 reports study-specific data splits, counting histories, episodes, or dialogues as appropriate.Source: §7 · p. 9, Table 2 · p. 10
7.1 Mutable-State Evaluation
- Temporal inference improved on arrival LWW, and the joint-emission HMM outperformed the selected policy in the copy columns.The candidate was selected using development next-report accuracy and then evaluated with an independent current-state audit.Source: §7.1 · pp. 9–10
7.2 Memory-Dependent Action Evaluation
- The compatible-channel certificate accepted MiniGrid improvements over both incumbents, whereas the unrestricted certificate accepted none.Root-capped Nous had lower task success than the single-source baseline in each environment column.Source: §7.2 · p. 10, Table 3 · p. 10
7.3 Categorical and Repeated-Release Controls
- Across the phase grid, zero-acceptance cells reflected either nonidentifiability or inadequate finite-sample power; the unrestricted certificate accepted no changes.The grid varied report signal strength and background bias, with Figure 4 presenting acceptance frequency and identification boundaries.Source: §7.3 · p. 11, Figure 4 · p. 11
7.4 Channel Sensitivity and Learned Witnesses
- Learned witness regions increased acceptance at the intermediate deviation setting, but all methods declined at the largest setting despite positive clean gains.Witness regions were fit from design data, selected independently, and evaluated on fresh calibration data.Source: §7.4 · p. 11
7.5 Sparse-Revision Evaluation
- The disagreement-conditioned certificate accepted more beneficial proposals than the global certificate, while neither accepted null or harmful proposals.Both had zero acceptance in the smallest rare-disagreement setting, consistent with sample-size dilution at rare disagreement.Source: §7.5 · pp. 11–12
7.6 Human-Written Dialogue Evaluation
- The full-history classifier was selected by development accuracy, but balancing retained only part of its revision gain.The study used seven MultiWOZ slots; the state-change results distinguish all-turn accuracy from changing-state performance.Source: §7.6 · p. 12
- In low-signal audits, balancing increased deployed gain through more frequent certification, while the full corrected proposal retained greater gain in the high-signal region.The corrected methods rejected every harmful reverse proposal; the naive report-agreement test accepted harmful proposals.Source: §7.6 · p. 13, Figure 5 · p. 13
8 Related Work
The related work distinguishes task-specific statistical rates, noisy-label identification, constrained prediction, and agent memory. The paper's contribution is to apply these ideas to learning and certifying policy gain from reports with unknown reliability.
8.1 Task-Specific Statistical Rates
- On a shared-observation continuous-source family, the paper compares learning, powered certification, and source-estimation requirements, extending prior HMM results on unstable parameter recovery near independence.For broader classes, it gives decision rates and a source lower bound using different least-favorable subfamilies.Source: §8.1 · p. 14
8.2 Noisy Labels and Partial Identification
- Unlike noisy-label work on feasible contamination families or noisy-label classifier testing, the identified interval targets fixed-policy gain while retaining channel uncertainty and requiring no pure-class histories.The certificates combine compatible-channel bounds with finite-sample guarantees under bounded history dependence.Source: §8.2 · pp. 14–15
8.3 Constrained Prediction and Background Cancellation
- The batch construction preserves incumbent prediction counts to cancel an unknown reporting background, and its conditional certificate concerns the selected finite batch.The paper evaluates the consequences through score-regret bounds and retained gain.Source: §8.3 · p. 15
8.4 Stateful Agent Memory
- The paper's contribution is statistical task separation and auditable policy revision for agent memory, with Nous serving as the integration target.Nous provides categorical beliefs and observational updates; the related systems cited address transactional commits or evolving states.Source: §8.4 · p. 15
9 Limitations
The paper bounds its claims by model assumptions, audit contracts, and evaluation scope. It concludes that the certificates provide policy-level evidence for memory revision, while leaving broader validation and security questions open.
9 Limitations
- The learning rates apply to specified HMM families with complementary emissions, balanced states, and positive anchor orientation; they may have large constants and do not establish pointwise superiority.The broader-class lower bounds use different least-favorable subfamilies.Source: §9 · p. 15
- Population certification assumes fresh iid deployment-law records and a positively oriented shared-background channel; deviation and copying budgets need external justification and are not identified by reports.The categorical channel is restricted, and finite-sample bounds can have low power.Source: §9 · p. 15
- Fixed-batch certificates require conditional channel bounds and apply only to the audited batch, not individual revisions or future histories.Conservation may discard beneficial changes or yield no nonempty subset.Source: §9 · p. 15
- The action study uses a static cue and controlled reporting, while the dialogue study covers one benchmark and simulated audit errors; neither validates natural reporting channels.The studies also do not validate real source lineage, open-ended attributes, or online batch scheduling.Source: §9 · p. 15
10 Conclusion
- The conclusion presents compatible-channel bounds and prediction-count conservation as routes to population and conditional fixed-batch certification, respectively.The experiments show increased acceptance under weak audits and the accuracy gain sacrificed by balancing.Source: §10 · p. 16
11 AI Use Statement
- Generative AI tools assisted theory, experiments, implementation, literature analysis, interpretation, and manuscript preparation; the author is responsible for verifying the work and final content.Source: §11 · p. 16
12 Data and Code Availability
- The research archive includes protocols, data, simulator traces, audit statistics, code, replay checks, and evaluated models and predictions.It includes licensed public MultiWOZ 2.4 data and is available through the Nous reproducibility release.Source: §12 · p. 16
13 Ethics Statement
- The text experiment reused public benchmark conversations under their redistribution license, and the interface distinguishes population-policy from fixed-batch certificates to reduce individual-safety misinterpretation.Source: §13 · p. 16
Appendix A. Proof of the Shared-Experiment Theorem
The proofs separate decision learning and powered certification from source calibration on shared observed records: the first two have quadratic requirements, while source estimation has a quartic lower bound. They establish these results with task-specific constructions and extend the decision analysis to a broader model class.
A.1 Learning and a Matching Lower Bound
- An empirical orientation moment learns the sign s with O(ℓ^-2) samples, and the resulting policy is the full-history Bayes rule.The matching lower bound uses two alternatives whose record laws remain close at this scale, yielding a constant probability of nontrivial regret.Source: §A.1 · p. 17
A.2 Certification Using the Future Report
- The future report acts as a symmetric shared-background audit, with clean gain ∆ = c/16 and observable gain ∆Y = ℓ/64.A fresh-split witness certificate gives powered acceptance at O(ℓ^-2) sample scale; the fixed-proposal testing lower bound counts all records, including reused training records.Source: §A.2 · pp. 17–18
A.3 Matching Source-Estimation Bounds
- The source-estimation lower bound is quartic in ℓ^-1, even when all records are pooled and the orientation is known.The result is minimax on the continuous-source family, not a claim over the entire original model class.Source: §A.3 · p. 18
- The source lower bound distinguishes anchor parameters separated by 1/8, yet estimation error of at least 1/16 remains unavoidable below the stated quartic sample threshold.The result applies even if s is known and all learning and certification records are pooled.Source: §A.3 · p. 18
B.1 Uniform Decision Upper Bound
- For the broader finite policy class, uniform risk estimation and the positive symmetric bridge yield a decision upper bound by preserving the conditional Bayes rule.The bridge scales excess future risk by at least ℓ/9.Source: §B.1 · p. 19
B.2 Nontrivial Decision Lower Bound
- For n ≤ 1/(64a²b²ℓ²), the construction proves a quadratic decision lower bound: excess is at least .05 with probability at least 1/4.Source: §B.2 · pp. 19–20
B.3 Parameter Lower Bound without Discarding Observations
This section establishes a quartic source-parameter lower bound without discarding observations, then develops sharp compatible-channel gain bounds and finite-sample witness certificates. The proofs also identify settings where the certificates are informative and limits on their use.
B.3 Parameter Lower Bound without Discarding Observations
- Source estimation has a quartic lower bound even when all observed entries are retained; decision and parameter lower bounds arise from different least-favorable subfamilies.The full-record KL is unchanged by additional entries generated through a parameter-independent channel.Source: §B.3 · p. 20
C.1 Proof of Theorem 3
- The compatible-channel analysis proves sharp gain endpoints by optimizing over a compact box of feasible channel parameters.Every point in the box gives a compatible joint law, and connectedness makes all intermediate gains attainable.Source: §C.1 · p. 21
C.2 Proof of Theorem 4
- The finite-sample witness bound controls the worst numerator over feasible background values without conditioning on data-derived bounds.A union bound combines the witness coverage and one-sided Hoeffding control.Source: §C.2 · p. 21
C.3 Linear Program for Sharp Endpoints
- The endpoint calculation is a linear-fractional program, and its population implementation refuses constant q when the compact positive-denominator argument fails.The finite-sample witness test still applies to constant channels with ρ > 0.Source: §C.3 · p. 21
C.4 Binary Sign Condition
- A binary one-direction correction can be identifiable even when the sufficient rule ∆Y > |m| cannot accept it.In the example, the sharp numerator is positive while the unrestricted rule fails.Source: §C.4 · p. 21, §C.7 · p. 22
C.5 Exact Witness Bound; C.6 Finite-History Consistency
- Exact binomial witness bounds retain coverage with random region counts and overlapping regions, but the theorem does not cover regions learned from the same audit reports.On a finite history alphabet, independently trained minimum-frequency regions eventually accept every fixed pair with strictly positive sharp numerator.Source: §C.5 · p. 22, §C.6 · p. 22
C.7 Unrestricted-Channel Comparator; C.8 Verification
- The unrestricted-channel binary comparator requires both paired score means to be positive, making its sufficient condition unable to accept an entirely upward revision.The vertex-enumeration checks supplement, but do not replace, the analytic proof.Source: §C.7 · p. 22, §C.8 · p. 22
Appendix D. Mutable-State Protocol and Complete Results
- The mutable-state protocol selects policies using development next-anchor accuracy and replays fixed policies and receipts without changing selection.The supplied table captions identify accuracy and paired revision outcomes but do not provide cell values.Source: Appendix D · pp. 22–23, Table S1 · p. 23, Table S2 · p. 23
Appendix E. External Simulator Protocol and Paired Results
The appendices specify simulator and control protocols, expose how history-dependent reporting can reverse an apparent policy comparison, and extend certification to bounded channel misspecification. They also give power conditions for fixed and learned witness regions, subject to explicit assumptions.
Appendix E. External Simulator Protocol and Paired Results; E.1 Input and Action Audit
- The MiniGrid evaluation uses fixed policies and independently derived episode seeds, while memory policies receive reports rather than hidden state or pristine observations.The evaluation targets success or accuracy, not the simulator’s step-penalized reward.Source: Appendix E · p. 23, §E.1 · p. 24
Appendix F. Categorical Control Protocols
- The categorical lag controls weaken report evidence by construction, and the sharp interval is used diagnostically rather than supplied to the finite-sample certificate.The interval, true channel parameters, and clean states are withheld from that certificate.Source: Appendix F · p. 25
F.1 History-Dependent Channel Counterexample
- Identical report distributions across histories can coexist with policy gains of +.6 and −.6, so arbitrary history-dependent reporting defeats observable-only comparison.Candidate copying can also contaminate the paired signal and potentially reward a harmful revision.Source: §F.1 · p. 25
Appendix G. Additional Arithmetic and Assumption Scrutiny
- Witness masks chosen using calibration labels can accept a zero-gain proposal that fixed history-only witnesses reject; callers must enforce pre-calibration witness selection.Neither the low-level boolean-mask interface nor a saved mask hash proves independence.Source: Appendix G · p. 26
Appendix H. Bounded Channel Misspecification; H.2 Conditional Copying and Tight Local Constants
- Under bounded history dependence and conditional copying, the combined TV budget is γ + β − γβ, with locally optimal constants but no claim of global sharpness.The concentration radius is not claimed optimal, and budgets must be externally justified rather than inferred from noisy reports.Source: §H.2 · p. 26
Appendix I. Learned-Witness Power and End-to-End Composition
- For fixed regions with mass at least π, the bounded-misspecification certificate accepts with probability at least 1 − η when nπ ≥ 8 log(5K/η) and M > B.The result is a fixed-region power condition.Source: Appendix I · p. 27
I.2 Learned Score Regions
- Learned score regions require independently separated design, selection, and audit data; their power condition does not establish that useful large regions exist for every representation.The population-mass and approximation assumptions must be justified and are not certified by substituting calibration estimates.Source: §I.2 · pp. 27–28
I.3 Nous Composition
Nous composes candidate learning, witness selection, and fresh auditing into a policy-level receipt without identifying the channel. The appendices describe a learned-witness sensitivity study and sufficient conditions for task-specific information separation on general observation spaces.
I.3 Nous Composition
- A Nous receipt is issued only for a locked policy population after candidate and witness selection on separate data and fresh auditing under an externally justified channel budget.The guarantee controls non-improving acceptance without identifying ρ, u, or clean posteriors; it does not certify an individual memory value.Source: §I.3 · p. 28
Appendix J. Sensitivity and Continuous-Witness Protocol
- The sensitivity protocol reports all 18 calibration comparisons, including five LWW comparisons rejected at zero budget, and requires an externally bounded actual deviation.The lower bound decreases with assumed deviation; selecting a favorable numerical root does not establish a channel contract.Source: Appendix J · p. 28
J.1 Learned-Witness Design
- The learned-witness study locks a ridge-regression candidate and quantile-threshold regions before calibration, using separate design and selection data.The synthetic reference has ρ = .8 and per-history/class TV deviation at most γ by construction, not by fitting.Source: §J.1 · pp. 28–29
- The learned-witness experiment is self-reproduction rather than independent replication and includes no language model, text extraction, or new external environment.Existing MiniGrid and mutable-state results remain unchanged.Source: §J.1 · p. 29
Appendix K. Sufficient Conditions for a Three-Task Information Separation; K.1 Setting and Assumptions
- On arbitrary observation spaces, the sufficient separation result gives quadratic-in-ℓ^-r requirements for learning and powered certification, versus a source lower bound of order ℓ^-2q.The assumptions include decision and sign-valid audit margins plus distinct hard pairs; these assumptions alone imply no source upper rate.Source: §K.1 · pp. 29–30
K.2 A Sufficient Source Upper-Bound Condition; K.3 KL Bounds from Density Expansions
- A matching source upper rate requires a separate quantitative invertibility condition; it does not follow from the source-hard pair alone.The density-expansion bounds are sufficient, not necessary, and source separation, Bayes-risk decoding, and audit margins still require verification.Source: §K.2 · p. 31, §K.3 · p. 31
K.4 Insufficiency of Signal Order
The signal order alone cannot determine source recoverability or whether a policy gain is certifiable. The appendices develop sparse-revision audits and prediction-count-preserving batches, with guarantees under their stated conditions.
K.4 Insufficiency of Signal Order
- Rare-event tests distinguish the Bernoulli laws in O(ℓ^-2) samples, but identical observed laws can still prevent source estimation or powered certification of clean gain.The clean/observed link cannot be inferred from signal order.Source: §K.4 · p. 31
K.5 Continuous-Source Nous HMM as an Instantiation
- The continuous-source Nous HMM instantiation keeps persistence and source informativeness unknown, but its bounds do not establish conditions for arbitrary real memory sources.Its clipped inverse moment is uniform over a continuum rather than a two-point parameter classifier.Source: §K.5 · p. 32
Appendix L. Sparse Revisions: A Disagreement-Conditioned Audit
- Disagreement-conditioned auditing bounds positive clean gain for a preselected sparse revision, while choosing the maximum of tests after calibration is not covered.When there are no disagreements, the test declines rather than reporting a positive lower bound.Source: Appendix L · pp. 32–33
- For a balanced binary identity-channel family, fixed-candidate testing requires Ω((dµ²)^-1) full records at fixed confidence, not universally across compatible channels.The lower bound applies to the stated subfamily.Source: Appendix L · p. 33
L.1 Pre-Specified Stress Protocol
- The stress protocol varies class count, calibration size, disagreement probability, conditional gain, and copying probability, evaluating each method separately at α = .05.It uses 50 repetitions per cell and a synthetic sensitivity budget justified by construction.Source: §L.1 · pp. 33–34
L.2 Complete-Grid Outcome
- The conditioned rule accepts more beneficial revisions than the global rule, while both reject non-improving revisions and neither reports an accepted lower bound above population gain.Acceptance rises with calibration size in rare-disagreement rows where the global rule rejects every proposal.Source: Table S4 · p. 34, §L.2 · p. 34
M.1 Conservation and Circulation
- Prediction-count conservation cancels the unknown background for balanced selections; score approximation incurs regret relative to the best balanced subset, not the unconstrained best revision.Nonnegative circulations decompose into directed cycles, providing the stated existence criterion for a balanced subset.Source: §M.1 · p. 35
M.2 Conditional Receipt
- Conditional receipts use the preselected batch and conditional report-channel contract to certify positive gain without iid-history assumptions.The proof integrates over batch-selection randomness, while fixed batch counts are exactly observed rather than estimated from an iid sample.Source: §M.2 · p. 35
M.3 KL–Chernoff Version
The paper gives a KL–Chernoff alternative for batch certification, states its power and failure boundaries, and evaluates text-model revisions using simulated audits. In MultiWOZ, deployed-gain comparisons vary by report signal, and predictive-gain intervals exclude zero for each slot.
M.3 KL–Chernoff Version
- The KL–Chernoff rule inverts a bounded-variable tail to obtain a lower endpoint, which is no smaller than the Hoeffding endpoint and is not an exact binomial interval.The rule is selected before reports; the experiment does not choose between tests after observing results.Source: §M.3 · pp. 35–36
M.4 Power under Misspecification
- With a positive balanced-batch signal after misspecification correction, the Hoeffding rule has a stated sufficient condition for power; no uniform useful-batch existence condition follows.The revision graph can be acyclic, and the KL rule inherits the stated condition for exact channels at zero budget.Source: §M.4 · p. 36
M.5 Two Failure Boundaries
- Marginal total-variation guarantees do not ensure the conditional contract, and pooling groups with distinct unknown signal scales can reverse the clean-gain conclusion.The paper gives examples of false acceptance under shared copying and negative pooled clean gain despite positive proxy gain.Source: §M.5 · p. 36
N.1 Data and Features
- The evaluation uses MultiWOZ 2.4 dialogue annotations and text features from the dialogue prefix, fitting classifiers on training data and selecting policies on development data.The seven categorical slots use prespecified labels and ontology exclusions.Source: §N.1 · pp. 36–37
N.2 Memory Policies; N.3 Batch Construction
- The comparison includes native Nous and alternative aggregators, while candidate revisions change native predictions and are selected as batches before audit reports.The history classifier uses broader contextual features; a count-only circulation assigns unit scores.Source: §N.2 · p. 37, §N.3 · pp. 37–38
N.4 Audit Generation
- Simulated audits vary report signal and copying; corrected methods reject all harmful reverse proposals, and scored balanced batches accept more than full corrected batches at lower signal.Methods evaluate different subsets and do not receive the unknown channel parameters.Source: §N.4 · p. 38
N.5 Replay and Uncertainty; Appendix O. Additional Evaluation Settings and Summaries
- Dialogue-cluster bootstrap predictive-gain intervals exclude zero for every slot, while a nominal test accepted a harmful EEG revision that the unrestricted-channel certificate rejected.The unrestricted-channel certificate also rejected the beneficial occupancy revision; reported paired gain intervals are descriptive across comparisons.Source: §N.5 · p. 39, Appendix O · pp. 39, 41
Appendix O. Additional Evaluation Settings and Summaries
- At high report signal the corrected proposal has greater deployed gain, while at intermediate signal the scored balanced proposal has greater deployed gain.The reported comparison is under no-copy audits.Source: Appendix O · p. 40, Table S14 · p. 41