Source-linked AI summary
Handover of In-Context Learning State Across Session Boundaries
Masahiro Kato, Taka Kato
TL;DR
Session handover must preserve what an unfinished task still depends on, not merely reproduce earlier wording. This paper models handover as task-relative state coding and shows how predictive equivalence, memory, and query timing determine sufficient records and attainable risk.
Problem
Session handover lacks a task-centered account of which earlier information must survive when the later query is unknown at writing time.
Method
The paper formalizes handover using predictive sufficiency, information-theoretic memory analysis, and a record combining exact decisions, justified statistics, and unreplaced observations.
Results
Predictive equivalence yields the coarsest deterministic sufficient state; Gaussian regression is exact and finite-dimensional, while nonparametric regression links memory to prediction risk.
Takeaways & Limitations
The framework provides a basis for constructing and comparing handover records according to task information, memory budgets, and continuation-task requirements.
Takeaways & Limitations
Comparisons depend on the task distribution, memory units, retained channels, and continuation procedure rather than universal rankings of prompt formats.
Abstract
from arXiv · showhide
This study investigates the methodological and theoretical properties of session handover in applications that use large language models. A task may continue in a new session when the context reaches the model's input limit, when the application restarts, or when another agent is asked to finish the task. The application must then decide which information from the earlier session to pass on. We formulate handover as the transfer of a task-relative in-context learning (ICL) state and distinguish exact recovery of earlier material from preservation of the target distribution. Under an exogeneity condition, predictive equivalence characterizes the coarsest deterministic sufficient handover and gives a fixed-length bit requirement. The analysis isolates the effects of the memory constraint, the writer, and the continuation procedure, and quantifies the cost of writing before the realized downstream query is known. We propose a three-part record that stores decisions and constraints exactly, uses task-justified statistics for repeated evidence, and retains original observations whose effect is not preserved by those statistics. Gaussian linear regression gives an exact finite-dimensional handover and finite-bit perturbation bounds, while nonparametric regression gives upper and lower bounds that relate memory to squared prediction error. These results provide a theory and method for deciding what a handover must retain and how its memory requirement depends on the continuation task.
1 Introduction
The introduction frames session handover as preserving the task-relevant information needed for future prediction or action, rather than reproducing earlier wording. It develops a pre-query state-coding framework that separates memory, writer, and continuation-procedure effects and previews theoretical results for parametric and nonparametric regression.
- Motivation: Handover records are judged by whether they preserve later predictions or actions, since constraints and examples may matter even when earlier wording is omitted.The record may be delivered through prompts, files, or archives, while a KV cache does not explicitly represent objectives, commitments, or observations.
- Formulation: The framework treats handover as pre-query state coding: the writer records task information before the continuation procedure observes the realized later input.This distinguishes the problem from query-aware prompt compression and supports analysis using sufficiency, decision theory, experiment comparison, and source coding.
- Formulation and handover method: The method models the writer, handover-built prompt, continuation procedure, and task score separately, recording decisions and constraints directly while compressing repeated evidence only when task-justified.It distinguishes reproducing earlier text from preserving information needed for the task.
- Task-relative state and memory limits: Under exogeneity, predictive sufficiency yields the coarsest deterministic sufficient handover and a fixed-length bit requirement, while separating losses from memory, the writer, and the continuation procedure.Under log loss, the analysis also quantifies the cost of discarded task-relevant information.
- Statistical analysis: Gaussian linear regression provides exact finite-dimensional records and finite-bit perturbation bounds, whereas nonparametric regression relates memory to squared-loss risk through achievable and lower-bound results.The comparison identifies when handover can remain finite-dimensional and when its size must grow.
- Comparison of handover records: A deterministic ordering result compares records by information content, and equivalent representations have equal ideal risks, leaving fixed-procedure differences attributable to the continuation gap.The memory account includes every external record derived from the earlier session.
2 Overview of Session Handover
Session handover moves unfinished work across session or agent boundaries by supplying a record whose adequacy depends on the continuation task. The overview distinguishes task state and retained records from computational caches, and emphasizes that handover writers usually act before later inputs are realized.
- Boundary and retained information: Handover transfers unfinished work to a new session or another agent, and its record may be empty, incomplete, or sufficient for continuation.This differs from context carryover, which keeps earlier information within the same session.
- Boundary and retained information: An external record preserves earlier information after resumption and counts toward retained memory, whereas the current task state is observed only afterward.A KV cache stores attention state for computation rather than serving as a task record.
- Writer and continuation: A query-aware compressor has an informational advantage because it can remove information unused by the known query, unlike a handover writer acting before later inputs are realized.The writer may know the task and current goal but not the downstream query or tool output.
- Writer and continuation: The analysis focuses on information a handover must preserve for a given continuation task, especially when the writer does not know the realized later input.This focus separates handover requirements from broader systems for prompt shortening, storage, retrieval, and later use.
3 General Formulation of Session Handover
The formulation models handover as a task-relative record that determines post-boundary answers or trajectories from earlier context, task information, and downstream queries. It distinguishes exact recovery from predictive sufficiency, quantifies memory and writing costs, and separates information loss from writer and continuation effects.
- 3.1 Episode and Information Available after the Handover: An episode is ω = (C, T, X, Y ), where C is pre-handover context, T is known task information, X is the downstream query or observation, and Y is the scored target.The handover writer observes (C, T), while the application receives X after the boundary.
- 3.1 Episode and Information Available after the Handover: The handover H = (V, M) includes the first post-boundary prompt record V and retained files, indexes, or archives M; it is self-contained only when M = ∅.Because V and total stored information differ, comparisons must report both, and self-contained limits apply to all information in (V, M).
- 3.2 From the Handover to a Model Response: Fixed model parameters, decoding, prompt construction, parsing, and environment updates induce an answer, action, or trajectory from H, X, and T.Interactive tasks score trajectories, while regression is the one-step case with no environmental change before prediction scoring.
- 3.3 Recovery, Preservation of the Task, and Downstream Behavior: Under exogeneity, predictive equivalence characterizes the coarsest deterministic handover preserving downstream predictions, rather than recovering a transformer hidden vector.When there are Nt finite predictive states, every fixed-length deterministic sufficient handover needs at least ⌈log2 Nt⌉ bits, achieved by encoding the equivalence-class index.
- 3.3 Recovery, Preservation of the Task, and Downstream Behavior: Task-specific memory can be shorter than exact reconstruction because it need only distinguish contexts that induce different distributions of Y, not distinctions needed solely to reproduce earlier text.Thus predictive sufficiency separates preservation of the target task from exact recall of prior material.
- 3.3 Recovery, Preservation of the Task, and Downstream Behavior: Exact recoverability of C implies predictive sufficiency, but predictive sufficiency need not imply exact recovery; moreover, a predictively sufficient record can still yield an incorrect model action.These distinctions separate record recovery, task prediction, and realized decision performance.
- 3.4 Loss under a Memory Limit: The memory-limit loss decomposes into unavoidable budget loss, writer-induced loss, and continuation excess loss relative to the best decoder for the same record.Under logarithmic loss, removed information has an exact expression; for finite actions and bounded loss, the same information quantity controls decision loss.
- 3.4 Loss under a Memory Limit: Writing before the downstream query is known creates an information cost: a query-aware encoder can transmit CX in one bit with zero log loss, unlike a boundary writer constrained by I(C; H) ≤B bits.The example shows that a writer must preserve information useful across the query distribution rather than only the realized coordinate.
4 Proposed Handover Method
The proposed handover method builds a task-relative record that preserves exact commitments, compresses repeated evidence only with task-justified guarantees, and retains unreplaced observations. A recoverable sufficient state makes the handover predictively sufficient.
- 4 Proposed Handover Method: The three-part record stores exact commitments, justified statistical summaries, and residual observations that cannot be safely summarized.Decisions and constraints remain exact; repeated evidence may use sufficient statistics or bounded approximations; otherwise selected originals are retained.
- 4 Proposed Handover Method: The record definition follows the information’s effect on the target and fits the task-relative sufficient state within a size limit.The task and scoring rule determine which information belongs in S = ϕ(C, T), rather than any particular hidden representation.
- 4 Proposed Handover Method: A recoverable sufficient state yields a sufficient handover when S is predictively sufficient for C relative to (X, Y, T).This is Proposition 4.1: exact recoverability from (H, T) implies that H is predictively sufficient.
- 4 Proposed Handover Method: The writer validates required fields, dimensions, references, and total size before evaluating the record, while serialization changes count as part of writing.The writer acts before observing the realized later input, and deterministic transformations contribute to the record budget.
5 Statistical Analysis
Section 5 applies the handover framework to regression, showing exact finite-dimensional sufficiency and finite-bit stability for Gaussian linear models, alongside memory–risk upper and lower bounds for nonparametric regression. In both settings, the writer encodes the sample before the downstream covariate is realized.
- Regression handover setup: The writer observes the regression sample and task before the later covariate, then constructs a bit-limited handover used to predict its response.The parametric analysis targets equality of posterior predictive distributions, whereas the nonparametric analysis targets integrated squared error.
- Gaussian linear regression: Gaussian linear regression admits an exact predictively sufficient record containing d(d+1)/2+d real numbers, independent of the number of demonstrations.The record is based on the Gram matrix and response-weighted input vector; known noise variance is assumed in the primary theorem.
- Gaussian linear regression: Synthetic sufficient demonstrations reproduce the Gram matrix and response-weighted input vector, yielding the same posterior and ridge estimate as the original demonstrations.With numerical precision and parsing held fixed, differences between the representations arise after encoding rather than from different stored information.
- Gaussian linear regression: Quantizing the Gaussian sufficient statistics gives explicit covariance, mean, predictive mean, variance, and log-loss perturbation control, connecting exact sufficiency to finite-bit handovers.The finite-bit construction also distinguishes theoretical bits, serialized bytes, and tokenizer tokens.
- Nonparametric regression: For Hölder nonparametric regression, cell counts and quantized cell means attain the full-data risk order with finite coding, while upper and lower bounds expose sample and memory limits.The lower bound separates irreducible full-data error from error caused by the finite number of possible handovers.
6 Implications for Evaluating Handover Records
Handover records should be evaluated through controlled comparisons that specify retained information, continuation inputs and procedures, external channels, and task loss. The theory separates task information from decoder limitations using ideal-risk comparisons, while memory evaluations must account for all retained channels and the resulting risk curve.
- Controlled evaluation: Controlled studies must specify the writer’s boundary information, later-input distribution, retained channels, serialization, continuation procedure, and task loss.Comparing arbitrary summaries does not identify the theoretical quantities of interest.
- Reference conditions: The full context and no-information record are reference conditions, not universal bounds, so theoretical comparisons use excess risk relative to the ideal decoder.Predictive information may remain in T or X, and long inputs can reduce continuation reliability.
- Ideal-risk ordering: If H2 is a deterministic function of H1 and T, H2 cannot have lower ideal risk; maps in both directions imply equal ideal risks.The comparison fixes (X, Y, T), output space, and loss.
- Representation and decoder gaps: Equivalent records can yield different losses under a fixed continuation rule because their gaps from the common ideal risk may differ.The representation-gap identity excludes unequal task information as the cause but does not identify the source of decoder-gap differences.
- Gaussian reference: In the Gaussian setting, sufficient-statistic and exact synthetic-demonstration records have the same ideal risk, whereas lossy records combine information loss with decoder gap.Lossy representations should therefore be compared with the exact state.
- Memory accounting: Memory comparisons must report task distribution, budget unit, every retained channel, and the risk curve rather than one arbitrary operating point.External files, indexes, databases, serialized bytes, and tokenizer tokens must be distinguished from the information budget.
7 Discussion
The discussion separates information loss, memory limits, and continuation-procedure failures, and argues that handover records should retain whatever the task and scoring rule require. It also distinguishes information loss from failures to parse or use an otherwise sufficient representation.
- Discussion: Handover separates omitted task information, unavoidable memory-induced loss, and continuation procedures that fail to attain the risk allowed by received information.These are distinct properties rather than components of a single summary.
- Discussion: The record writes decisions and constraints exactly, shortens repeated observations only with task-justified guarantees, and retains exceptional observations that determine the next step.The task and its scoring rule determine what the writer may safely shorten.
- Discussion: Memory requirements depend on the continuation task: Gaussian regression has a sufficient-state dimension independent of demonstrations, while nonparametric records grow with target accuracy until the sample-size floor.Finite-precision results also connect code length to predictive log loss.
- Discussion: Proposition 6.1 identifies records with equal ideal risk, while Corollary 6.2 attributes remaining differences under a fixed procedure to the gap from the ideal decoder.This prevents representation-use failures from being misclassified as information loss by the writer.
- Discussion: Predictive sufficiency depends on the later-input law, and adaptive trajectories require trajectory-level loss because earlier actions change later observations.The stated results concern a session boundary under the continuation law.
8 Conclusion … A.3 In-Context Learning and Sufficient State
The paper defines session handover as transferring task-relative in-context learning state, distinguishing exact recovery from preserving the continuation task’s target distribution. It develops sufficiency, memory, and construction principles across handover, memory mechanisms, prompt compression, and in-context learning.
- 8 Conclusion: Handover succeeds when the passed information supports the task continuation, while separate modeling distinguishes writer omissions from losses introduced after record delivery.The framework also separates reproducing earlier text from preserving task information.
- 8 Conclusion: The proposed record stores decisions and constraints exactly, compresses repeated evidence only with task-justified loss relations, and retains irreplaceable observations.Predictive equivalence gives the coarsest deterministic sufficient state under Proposition 3.1’s exogeneity condition, while memory and writing timing constrain attainable risk.
- A Existing Memory Mechanisms and Related Work: Session handover relates to prompt compression, persistent memory, external records, runtime caches, and in-context learning, but differs in when information is written and what task-relative property it preserves.These distinctions organize the paper’s comparison with existing memory mechanisms.
- A.1 Boundary Events and Storage Channels: Context carryover concerns later availability of earlier information, whereas memory management additionally covers writing, storing, updating, and retrieving across calls.Handover records are selected at session or agent boundaries, while context management determines what enters one model call.
- A.1 Boundary Events and Storage Channels: Active prompts, compacted replacements, persistent instructions, external stores, and retrieval indexes can carry handovers, but none guarantees sufficiency for the continuation task.KV caches reduce recomputation and serving costs but are computational artifacts rather than auditable task-state records.
- A.2 Prompt Compression and Source Coding: Prompt compression preserves model outputs under representation limits, but handover writing occurs before the continuation procedure observes the realized later input.This information pattern explains why query-aware encoding advantages from rate–distortion settings do not directly characterize handover.
- A.2 Prompt Compression and Source Coding: Information bottleneck, indirect source coding, Wyner–Ziv coding, and Blackwell’s comparison provide classical tools for defining the handover formulation.Repeated compaction and structured eviction address related operations, while this analysis supplies a task-relative criterion rather than a specific algorithm.
- A.3 In-Context Learning and Sufficient State: In-context learning theory motivates analyzing the induced prediction rule after demonstrations are processed, while sufficient statistics can preserve the same posterior despite different token-level forms.Gaussian sufficient statistics and synthetic sufficient demonstrations therefore support ideal-decoder equivalence while complementing work on algorithms implemented from raw examples.
B Additional Details for the Handover Method … C Additional Results for the General Formulation and Statistical Analysis
The handover method stores only task-relevant information, including selected original observations when shorter records lose their effects, while counting all prompt and external-record content toward memory. Supporting results establish equivalences among deterministic serializations, characterize response and prediction loss, constrain repeated rewriting and query shifts, and extend memory analysis to smoother or lower-dimensional settings.
- B Additional Details for the Handover Method: The writer may keep selected observations outside the prompt while preserving their identifiers, and the size limit counts identifiers, metadata, observations, and returned content.This keeps comparisons between presentations tied to fixed information content and makes external storage part of the handover budget.
- B.1 Selected Observations Stored outside the Prompt: Selected original observations are retained only when the shorter record fails to preserve their task effect, with both the selection rule and storage cost determined by the writer.The method does not retain the entire earlier history outside the prompt.
- B.2 Serialization and Deterministic Checks: In the Gaussian setting, direct statistics and exact synthetic sufficient demonstrations determine the same (G_n, b_n), so they have the same ideal risk.More generally, deterministic processing cannot improve the best attainable risk, and mutually determining forms remain equivalent at the ideal-decoder level.
- B.2 Serialization and Deterministic Checks: The writer records decision-relevant events, computes task-required statistics, selects indispensable original examples, allocates space, and verifies schema, dimensions, targets, and size.For long-running tasks, exact fields can be updated as constraints and decisions arise instead of reconstructing commitments at the boundary.
- C.1 Response Distributions under the Two Prompts: The chain rule decomposes response relative entropy across token positions, identifying where generations under two prompts begin to differ without establishing task correctness.The result applies to autoregressive response distributions satisfying the stated absolute-continuity and factorization conditions.
- C.2 Sequential Prediction and Changes in the Distribution of Later Inputs: Under the Brier score, zero risk increase occurs if and only if the handover is predictively sufficient for the finite target.For fixed-horizon replay and related protocols, cumulative log-risk differences decompose into conditional mutual-information terms; changing-policy trajectories are outside that result.
- C.2 Sequential Prediction and Changes in the Distribution of Later Inputs: Repeated rewriting without new evidence makes ideal log risk nondecreasing, whereas new observations or tool results can improve prediction by adding unavailable information.Query-distribution transfer additionally requires the evaluation distribution to satisfy the coverage condition; without coverage, target-information loss can be arbitrary.
- C.5 Higher Smoothness and Intrinsic Dimension: When task distributions lie on a known lower-dimensional subspace, the cell construction replaces ambient dimension d with the subspace dimension, reducing the relevant memory scale.For β > 1, attaining the full Hölder rate requires local polynomial moments through order p = ⌊β⌋ rather than piecewise-constant handover.
C.3 Additional Information-Budget Results … D.3 Proof of Corollary 3.2
The appendices characterize information-budget trade-offs, finite and lower-bound handovers, and exact or truncated sufficient representations for parametric regression. Proofs establish predictive sufficiency via equivalence classes and the minimum fixed-length code for those classes.
- C.3 Additional Information-Budget Results: The full information constraint applies to both the prompt record V and externally retained information M, including storage, access requests, and returned content.The active prompt limit controls V, but the information budget covers the pair (V, M).
- C.3 Additional Information-Budget Results: Dinfo(B) is nonincreasing and convex; at zero budget it equals I(Y ; C | X, T), and discrete C is fully preserved once B ≥H(C | T).Under an exogenous future query, an additional characterization also holds, though its displayed equation is not included in the passage.
- C.3 Additional Information-Budget Results: A mixed sufficient state requires exact representation of its discrete component, while its continuous component can be approximated to a target excess-risk tolerance.The continuous term depends on target accuracy rather than transcript length.
- C.3 Additional Information-Budget Results: Information lower bounds arise when distinct histories require separated continuation policies, provided the future query is conditionally independent of the history and handover message.The separation condition can be enforced synthetically, but additional observations may reveal the history and invalidate it.
- C.4 Additional Results for Parametric Regression: Exact synthetic sufficient demonstrations use the minimum possible number of real-valued examples among exact Gram-matrix representations.The matrix and demonstration-set formats contain the same sufficient state, so differences arise downstream of stored information, subject to precision and parsing.
- C.4 Additional Results for Parametric Regression: Spectral truncation trades exact preservation for bounded ridge error and requires recording both the threshold and discarded information.A truncated record cannot be described as exact.
- D Proofs for General Handover Results: The appendix proves the general results in use order, with each subsection relying only on the assumptions of its named result; Proposition C.1 follows from autoregressive factorization and the tower property.The proof explicitly concludes the stated equation after taking expectations under P.
- D.2 Proof of Proposition 3.1: Under exogeneity, deterministic predictive sufficiency is equivalent to placing every encoder fiber within one target-equivalence class, and the quotient record is recoverable from any sufficient encoder.The proof uses identical future-query laws and shows mixtures within a fiber retain the common target distribution.
D.4 Proof of Proposition 3.3 … E.7 Proof of Corollary 5.5
The proofs establish when handover preserves predictive behavior, how information-theoretic distortion changes with memory and sequential updates, and how Gaussian regression admits exact and finite-precision records. They also provide counterexamples showing that exact recovery, preserved Bayes risk, and predictive sufficiency are distinct properties.
- D.4 Proof of Proposition 3.3: Predictive sufficiency follows when the encoder is conditionally independent of (X, Y) given (C, T) and C is recoverable from (H, T), but exact recovery alone is insufficient.The counterexample sets C = (U, V), H = U, and Y = U: prediction is preserved while V is unrecoverable; recovering O = V does not preserve prediction of Y = U.
- D.5 Proof of Proposition 4.1: Exact recoverability of a task statistic S preserves the target distribution when P(Y | C, X, T) = P(Y | S, X, T), yielding P(Y | H, X, T) = P(Y | S, X, T).The proof uses the recovered value of S to mix only contexts sharing the same target distribution.
- D.6 Proof of Proposition 3.4: Conditional mutual information equals the reduction in Bayes log risk: I(Y ; C | H, X, T) = H(Y | H, X, T) − H(Y | C, H, X, T).Bayes log risk is conditional entropy, and the encoder’s generation from (C, T) supplies the needed conditional-entropy identity.
- D.7 Proof of Proposition C.2: For finite targets, the Brier-risk gap is zero exactly when pC = pH almost surely, which is predictive sufficiency.The conditional Pythagorean identity establishes the gap, while the minimizer of the conditional Brier risk is q = p.
- D.10 Proof of Proposition C.6; D.11 Proof of Proposition 6.1: More informative records dominate decoders that can be simulated from them, while time-sharing proves convexity and the budget bound I(Y ; H | X, T) ≤ I(C; H | X, T) ≤ B.At zero budget the constant message yields Dinfo(0) = I(Y ; C | X, T); if C is discrete and B ≥ H(C | T), H = C gives zero distortion.
E.8 Proof of Corollary 5.6 … F.4 A Packing Lemma for H¨older Functions
The proofs establish finite-bit predictive-divergence bounds for Gaussian handover and derive nonparametric regression upper and lower bounds through cell-record analysis and Hölder-function packings.
- E.8 Proof of Corollary 5.6: Quantizing each stored scalar with m = ⌊(B − B0)/p⌋ bits yields a predictive-divergence bound proportional to 2^−2m.The constants depend only on fixed problem parameters, with B0 covering fixed header and coding costs.
- F Proofs for the Nonparametric Results: The nonparametric proofs first control the cell-based record and then construct a packing to obtain the memory lower bound.
- F.1 Proof of Theorem 5.7: The cell-based predictor combines Hölder bias, sampling variance, quantization, and empty-cell contributions to establish Equation (58) with explicit constants.The integrated variance contribution is at most 6B2 yM/(n + 1), while the empty-cell predictor is zero.
- F.2 Proof of Corollary 5.8: The cell-record bit count uses logarithmic-length counts and quantized means, while M = O(nd/(2β+d)) under Equation (59).Substitution into Theorem 5.7 gives the risk order stated in Corollary 5.8.
- F.3 Proof of Corollary 5.9: Choosing q2 proportional to M −2β/d makes each count and quantized mean use O(log(n + 1)) bits, enabling admissible partitions under the memory budget.
- F.4 A Packing Lemma for H¨older Functions: A standard Assouad–Fano bump construction sets M = md and h = 1/m to produce a family indexed by separated Hölder functions.The construction uses disjoint scaled bumps with amplitude a0hβ, and sufficiently small a0 keeps the family inside the Hölder ball.
F.5 Proof of Theorem 5.10
The proof establishes separate sample and memory lower bounds for handover risk. It uses packing arguments, divergence control, Assouad’s lemma, and Fano’s inequality, with the full-data decoder argument extending the sample lower bound to every handover.
- Sample floor: The sample-floor proof uses a binary-response submodel, a packing with pf(x) ∈[1/4, 3/4], and a Bernoulli divergence bound.The divergence constant depends only on By.
- Sample floor: Choosing h proportional to n−1/(2β+d) keeps adjacent divergences below one, allowing Assouad’s lemma to yield the sample lower bound.The adjacent divergence is of order nh2β+d.
- Sample floor: The sample lower bound applies to every handover because it remains valid when the decoder receives the full data.Thus restricting the decoder to a handover cannot improve the lower-bound conclusion.
- Memory floor: The memory-floor proof constructs a Varshamov–Gilbert subset with pairwise Hamming distance at least M/8 and converts this into squared L2 separation.The subset is Θ ⊆{−1, 1}M.
- Memory floor: Fano’s inequality bounds decoding error from a message alphabet of size at most 2B, after randomized decoding is represented using an independent private seed.Nearest-packing decoding links incorrect decisions to squared estimation error.
- Memory floor: The final risk lower bound follows because the risk is at least both the sample and memory bounds, with the constant reduced to obtain Equation (66).The proof selects m so that M = md is the smallest admissible order of B+1 satisfying the stated positivity condition.