Source-linked AI summary
Procedural Memory Under Change: Reuse and Interference in Controlled Web Tasks
Yanze Cao
TL;DR
The paper asks when a previously useful procedural memory becomes behaviorally disruptive after task requirements change. It combines a retrospective interface-adaptation case with controlled frozen-memory comparisons, finding no predefined interference signatures across the tested mismatches when current-task evidence was explicit and sufficient. The result marks a tested region of non-interference, not a general safety guarantee or identified mechanism.
Problem
The paper examines when a retrieved procedural memory remains applicable after task requirements change, distinguishing mismatch from observable and memory-caused error.
Method
The study combines a retrospective, human-assisted interface-adaptation case with controlled frozen-memory comparisons across four mismatch designs.
Results
Across four Batch 03 pairs, none of the predefined interference signatures appeared on diagnostic tasks with explicit and sufficient current-task evidence.
Takeaways & Limitations
A procedural memory can be mismatched without becoming behaviorally disruptive in the tested setting.
Takeaways & Limitations
The controlled observations use one local model, temperature 0, one execution per cell, and short synthetic shopping decisions with explicit evidence.
Abstract
from arXiv · showhide
Procedural memory lets language agents reuse successful routines, but reuse presumes that a stored routine remains applicable. We study what happens when that presumption is deliberately violated. The study combines a retrospective, human-assisted interface-adaptation case from BrowserGym TimeWarp with controlled frozen-memory comparisons on synthetic shopping decisions. During the documented WebShop V1-V6 development path, interface-specific code was adapted while the separately stored high-level procedure was not reported to change; this phase does not constitute an autonomous memory-agent evaluation. In the controlled phase, an early pilot produced one task on which two memory conditions selected a more expensive item while the no-memory condition selected the reference minimum. Follow-up probes did not establish a recurring row-order or identity-binding pattern. We then tested four forms of mismatch: changed quantities, a different evidence representation, a conflict between local and global optimization, and distributed promotion evidence, across 32 formal cells. Each cell used one temperature-0 generation with the same local qwen3:8b configuration and no adaptive retry. Across these pairs, none of the predefined diagnostic interference signatures appeared on the tasks for which they were defined when current-task evidence was explicit and sufficient. The result identifies a tested region of non-interference: a procedural memory can be mismatched without becoming behaviorally disruptive. It does not establish general safety or a mechanism. The remaining question is which additional conditions turn applicability mismatch into observable, memory-caused error.
1 Introduction
The paper asks when a previously successful procedural memory becomes behaviorally disruptive after task requirements change. It combines a retrospective interface-adaptation case with controlled mismatch tests, finding no predefined interference signatures when current-task evidence was explicit and sufficient.
- Motivation: A stored shopping routine can remain executable and previously successful while becoming inapplicable when a bundle discount changes the optimization problem.The mismatch concerns the procedure’s applicability, not whether the routine can still be executed.
- Study design: Phase 1 documents a retrospective, human-assisted WebShop V1–V6 adaptation path rather than an autonomous memory-agent evaluation.The retained baseline handled interface-specific search and extraction, but did not autonomously perform final selection, aggregation, or submission.
- Study design: A pilot anomaly showed two memory conditions selecting a $36.80 outcome while no memory selected the $20.00 reference minimum on one task.Follow-up probes tested row-order and identity-binding explanations without establishing a recurring pattern.
- Main result: Across the controlled mismatch pairs, none of the predefined interference signatures appeared when current-task evidence was explicit and sufficient.The tested mismatches involved quantity, representation, local versus global optimization, and distributed composition.
- Research question: The study separates interface adaptation from high-level procedural revision and procedural mismatch from observable interference.This distinction prevents an error consistent with a stale procedure from being treated as complete causal evidence.
2 Related Work
Prior work treats memory as a way to carry experience, workflows, and skills across episodes. This paper focuses downstream on whether retrieved procedures remain applicable when task requirements change, using fixed procedural text and altered task requirements rather than learning a retrieval or update policy.
- Memory across episodes: Language-agent systems store and retrieve reflections, experiences, workflows, or executable skills to improve later decisions.Examples include Reflexion, ExpeL, and ReasoningBank, which differ in how memories are produced and retrieved.
- Applicability: The central related-work gap is whether a retrieved memory remains suitable when interfaces, quantities, constraints, or objectives change.Task-family relevance does not guarantee that an embedded optimization assumption remains valid.
- Procedural representations: Procedural-memory approaches package experience as reusable action structures through skill libraries, workflows, activation conditions, or transfer mechanisms.The cited systems include Voyager, Agent Workflow Memory, ProcMEM, and AFTER.
- Position of this work: The present experiment fixes the procedural text, changes task requirements, and predefines response-level signatures of stale-procedure use.It does not evaluate a learned retrieval policy or a memory-update algorithm.
- Evaluation setting: The retrospective phase studies code adaptation under interface variation, whereas the controlled phase presents compact evidence directly and does not reproduce long, uncertain web interaction.The two phases therefore occupy different points on the web-agent evaluation spectrum.
3 Problem Setup
The paper defines mismatch as an applicability relation between a retrieved procedure and the current task, then distinguishes reference compatibility, diagnostic signatures, and causal interference. The setup uses short synthetic shopping decisions with explicit evidence, bounding the claims.
- Mismatch: A procedure is mismatched when at least one recommendation or assumption is insufficient for, or locally inappropriate to, the current task.Mismatch is defined between procedural text and task, independently of whether the agent follows the procedure.
- Behavioral diagnostics: A diagnostic signature is a predefined response pattern expected when a particular stale instruction controls behavior.Examples include omitting requested quantities or selecting independent category minima despite an eligible bundle discount.
- Causal interpretation: Observable interference is reserved for a task-level error consistent with mismatched procedure use, while causal attribution requires comparisons such as a no-memory condition and controls.The design includes a no-memory condition to assess memory-associated anomalies without claiming a mechanism.
- Evaluation fields: Reference-compatible selection, global optimization, strict task correctness, and diagnostic signatures answer different evaluation questions.Null strict-correctness values indicate missing evaluator evidence rather than success or failure.
- Task scope: The controlled setting uses two product categories and explicit candidate tables, making stale-procedure signatures transparent but limiting claims to short synthetic decisions.It is not an estimate over a broad web-task population.
4 Methods
The study uses two linked phases: a retrospective, human-assisted interface-adaptation case and frozen-prompt comparisons of procedural-memory conditions. Phase 2 tests four mismatch designs across 32 formal cells under fixed generation and evaluation procedures.
- The study separates a retrospective interface-adaptation case from frozen-prompt comparisons, retaining distinct evidential roles and scoring histories.
- Phase 2 includes a pilot, diagnostic Batch 02 probes, and Batch 03 paired tasks covering quantity, representation, local-versus-global optimization, and distributed composition mismatches.
- Phase 1 examines BrowserGym TimeWarp Task 57 across WebShop V1–V6 using retained code that searches and extracts products but does not execute stored memory or final autonomous selection.
- 32 formal Batch 03 cells cross four paired designs with four memory conditions, while diagnostic signatures apply only when the corresponding task conflict is present.
- All information required for each task is explicitly present, including the distributed-promotion matching and discount rule in T17.
- Each cell used one temperature-0 generation with the same local qwen3:8b configuration and no adaptive retry; repeated-execution and sampling robustness were not evaluated.
- Evaluation distinguishes reference compatibility, optimization, arithmetic, strict correctness, and diagnostic signatures rather than pooling them into one success rate.
5 Results
The retrospective record shows interface-specific adaptation without a reported revision to the stored high-level procedure, while the controlled tests found no predefined interference signatures under explicit and sufficient current-task evidence. The pilot anomaly and follow-up probes did not establish a recurring explanatory pattern.
- Interface-specific patches addressed parsing, controls, result boundaries, and submission across WebShop V3–V5, while the stored high-level procedure was not reported revised.
- The documented Phase 1 history is human-assisted and does not support an autonomous six-version task-success claim.
- Pilot 01 produced 2/3, 3/3, and 2/3 reference-compatible optimal outcomes for A, B, and C, respectively; on T2, A and C reported $36.80 while B reported $20.00.
- Batch 02 follow-ups did not establish recurring row-order, candidate-attraction, or identity–price binding explanations for the earlier anomaly.
- All eight Pair 01 answers were reference-compatible, and the predefined quantity-omission signature was absent.
- All eight Pair 02 answers selected A03 and R02 for the $5.10 reference total, with explicit interface confusion in 0/8 cells.
- Global optimization was correct in 8/8 Pair 03 cells, with the stale local-minimum signature occurring in 0/4 conflict-task cells.
- Pair 03 retained six null strict-correctness fields because required quantity evidence was not explicit, rather than imputing those cells as successes or failures.
6 Discussion
Across the tested mismatches, predefined behavioral failures did not appear when current-task evidence was explicit and sufficient. Earlier anomalies remain unresolved, while evaluation boundaries and alternative mechanisms limit interpretation.
- Several mismatch forms did not produce the predefined behavioral failures under the controlled batch conditions.The tested forms included quantity, representation, local-optimization, and distributed-composition mismatches.
- The earlier anomalies remain evidence of an unresolved pattern rather than a confirmed recurring interference signature.Later probes did not reproduce simple first-row, A01-attraction, or identity-binding patterns.
- Explicit, complete, and nearby current-task evidence may permit correct behavior despite a mismatched procedure.The experiments do not establish whether memory was ignored, weakly weighted, or corrected before the final response.
- Response receipt, product selection, quantities, arithmetic, and strict task correctness can diverge across evaluation boundaries.Null strict-correctness fields indicate missing evaluator-required evidence rather than an automatically inferred optimization failure or success.
- A next study should reduce current-evidence accessibility or strength while retaining solvability and a no-memory control.Longer trajectories, stronger memory salience, competing memories, and alternative models should be varied in separately frozen designs.
7 Limitations
The study’s scope is constrained by retrospective evidence in Phase 1 and narrow, one-shot controlled tests in Phase 2. These limits restrict claims about autonomy, robustness, mechanism, and generalization.
- Phase 1 is retrospective and human-assisted, so it cannot support an autonomous memory-benefit claim.The unchanged-procedure statement is provenance rather than proof from per-version cryptographic snapshots.
- Batch 03 used one local model, temperature 0, one seed, and one formal execution per cell.Repeated-execution robustness, stochastic variation, alternative decoding, and cross-model transfer were not tested.
- The controlled tasks are short synthetic shopping decisions with explicit evidence rather than long, noisy browser trajectories.They omit active evidence acquisition and accumulation of small biases over repeated actions.
- The memory conditions cover a narrow lineage, and behavioral outputs do not identify the underlying mechanism.Predefined signatures also cannot detect every subtler influence of memory.
8 Conclusion
The paper examines procedural memory under change through a retrospective interface-adaptation case and controlled frozen-memory comparisons. Across 32 one-shot cells, no predefined diagnostic interference signature appeared under explicit, sufficient current-task evidence, identifying a tested region of non-interference rather than universal safety.
- The study combines a retrospective interface-adaptation case with controlled frozen-memory comparisons.The retrospective phase documents interface-specific code adaptation without supporting an autonomous memory-agent success claim.
- Across 32 one-shot Batch 03 cells, none of the predefined diagnostic interference signatures appeared under the tested conditions.The conditions used explicit and sufficient current-task evidence over short horizons.
- The conclusion marks a tested region of non-interference rather than a universal absence of harm.Applicability mismatch did not by itself produce the expected behavior, and the mechanism remains unknown.
- The tested procedural memory includes independent category search, semantic filtering, minimum-price selection, and summation.The reproduced memory texts also specify interface and evidence assumptions, quantity handling, and constraints on stored prices and prior answers.
- The Phase 1 stored procedure’s continuity is retrospective provenance, not checksum proof across the WebShop V1–V6 path.Per-version cryptographic snapshots were not retained.
B Task Construction
The study constructs synthetic shopping tasks and evaluates them with predefined diagnostic signatures alongside multiple historical metrics. The task inventory and audits distinguish reference compatibility, strict correctness, optimization, arithmetic, and signature-based judgments.
- The task inventory covers synthetic shopping decisions whose frozen reference totals provide task-specific evaluation targets.
- The pilot includes a task where the reference minimum is $20.00, computed as 4(3.20) + 3(2.40).
- The diagnostic task designs vary quantities, evidence representation, local-versus-global optimization, and distributed promotion composition.
- The pilot and follow-up records include nine pilot cells, 18 Batch 02 cells, and replication-style probes that are not repeated random trials.
- All four T11 answers explicitly calculate 4 × 2 + 3 × 3 = 17, while no cell makes an unnecessary search request or unsupported incompleteness claim.
- The evaluation separates reference-compatible selection and totals, global optimization, arithmetic conditional on choices, strict correctness, and predefined diagnostic signatures.
E.3 Scope of reproducibility
The reproducibility scope is bounded by a frozen, archived study whose formal cells were each evaluated once. The records support reconstruction of intended configurations, but not claims of bitwise reproducibility or broader validation.
- Each formal cell was evaluated once, with repeated execution, alternative decoding, stochastic sampling, and cross-model validation outside the study.
- The repository preserves frozen tasks, memories, hashes, references, evaluator specifications, run orders, raw responses, evaluated records, and checksum reports.
- The manuscript audit created an evidence ledger and cell index to map claims to archived artifacts, but these were not prospectively frozen experimental materials.
- No frozen experiment artifact, baseline, memory, prompt, raw answer, evaluator result, or checksum manifest was changed during paper preparation.
F.1 Data and code availability
The draft describes a local repository audit trail but makes no claim of public repository upload or archival deposition.
- The local repository contains the audit trail described in the paper, but no public repository upload or archival deposition is claimed.