Beyond Context Windows: Persistent Discovery Context for Data-Centric Agents

• Discovery is a prerequisite for data-centric agents before planning or execution.Source: §1 · p. 1
• Existing agent memory systems retain conversations, observations, skills, and workflows, whereas data discovery is typically handled per task.Source: §1 · p. 1
• A discovery outcome maps task intent to the data objects that proved useful for completing the task.Source: §1 · p. 1
• Persistent discovery context stores prior intent-to-object mappings and reuses them to augment future retrieval across three structured data environments.Source: §1 · p. 1

• The registry contains tables and attributes with natural-language metadata, and standard text retrieval produces candidate objects.Source: §2 · p. 2
• A discovery memory records a task description and the data objects previously found relevant, then accumulates across tasks within one data environment.Source: §2 · p. 2
• For query q, registry retrieval scores candidate objects while similar discovery memories are retrieved to augment their rankings.Source: §2 · p. 2
• The memory weight α controls memory influence; α = 0 gives standard registry search, while setting r(q, oi) = 0 produces memory-only retrieval.Source: §2 · p. 2

Datasets and Tasks

• The study evaluates three structurally distinct environments: Finance, NYC Collisions, and Northwind.They cover banking, crash records, and trading data, respectively.Source: §3 · p. 2
• Tasks are organized into 15 families, with one seed task initializing memory and 5–12 held-out evaluation tasks per family.Source: §3 · p. 2

Experimental Conditions

• The four conditions compare raw metadata, registry descriptions, memory-only ranking, and registry retrieval augmented with discovery memory.Source: §3 · p. 2

Main Results

• F1@5 rises with TF-IDF from 0.396 → 0.499 in finance, 0.299 → 0.482 in NYC collisions, and 0.446 → 0.528 in Northwind when memory augments registry retrieval.Source: §3 · p. 3
• Neural retrieval gains are +0.167 in finance, +0.119 in Northwind, and +0.051 in NYC collisions.The NYC neural baseline is already strong, with S1⋆MRR = 0.848.Source: §3 · p. 3
• Memory-only retrieval exceeds registry retrieval in NYC collisions under TF-IDF, with F1@5 of 0.412 versus 0.299.In finance and Northwind under TF-IDF, registry retrieval remains stronger than memory-only retrieval.Source: §3 · p. 3
• Families with distinctive object vocabularies benefit most, while families whose objects recur across task types show near-zero or negative gains.Source: §3 · p. 3
• LLM-generated memories remain useful despite imperfect construction, outperforming S1 in finance and NYC and remaining above S0 in all three domains.Their object-level F1 agreement is 0.698, 0.762, and 0.735 for finance, NYC, and Northwind.Source: §3 · p. 3

• Prior agent-memory systems retain conversational context, observations, executable skills, workflows, and execution experience for data-centric tasks.Source: §4 · p. 4
• Related data systems focus on schema linking, reasoning, query generation, metadata management, or improved object representations.Source: §4 · p. 4
• Discovery memory is conceptually closest to relevance feedback but reuses agent-generated object selections across semantically related, lexically distinct tasks.Source: §4 · p. 4

Discussion and Conclusion

• Persistent discovery context consistently improves retrieval quality over metadata-only search across three structurally distinct domains.The evaluation covers 125 held-out tasks organized into 15 task families, plus seed tasks for memory initialization.Source: §5 · p. 4, A Dataset and Registry Statistics · p. 6
• The approach remains effective when initialized with automatically generated memories, although improvements vary by domain.Finance and NYC Collisions improve over registry-only search, while Northwind does not improve over S1.Source: §5 · p. 4
• Memory interference can cause semantically similar but incorrect memories to degrade retrieval below the no-memory baseline.The paper identifies interference-aware memory selection, memory aging, and broader evaluation as future work.Source: §5 · p. 5

Full Retrieval Results

• Table 3 reports R@5, R@10, MRR, and F1@5 for all retrieval conditions.The F1@5 column matches Table 1 in the main paper.Source: B Full Retrieval Results · p. 6
• The table includes results for all evaluated retrieval conditions on held-out evaluation tasks.The passage identifies the table as a full retrieval-results report.Source: B Full Retrieval Results · p. 6
• F1@5 results in Table 3 correspond to the F1@5 results reported in Table 1.This links the appendix table's F1@5 column to the main-paper presentation.Source: B Full Retrieval Results · p. 6

LLM-Generated Discovery Memories

• Generated memories are created by asking Claude Haiku to identify relevant data objects from each seed task and registry contents.This replaces manually annotated discovery memories for robustness evaluation.Source: C LLM-Generated Discovery Memories · p. 6
• Memory agreement F1 is 0.698 for Finance, 0.762 for NYC Collisions, and 0.735 for Northwind.Agreement is measured against manually annotated discovery memories using object-level F1.Source: C LLM-Generated Discovery Memories · p. 6
• Finance improves from S1 F1@5 = 0.396 to S3 F1@5 = 0.433, while NYC Collisions improves from 0.299 to 0.452.These results use automatically generated memories.Source: C LLM-Generated Discovery Memories · p. 6
• Northwind changes from S1 F1@5 = 0.446 to S3 F1@5 = 0.435, showing no improvement over registry-only retrieval.The passage attributes this to noisier object selections amid higher cross-entity vocabulary overlap.Source: C LLM-Generated Discovery Memories · p. 6
• Across all three domains, retrieval quality remains above the raw-metadata baseline S0 despite imperfect memory construction.This supports robustness to automatically generated memories relative to raw metadata.Source: C LLM-Generated Discovery Memories · p. 6

Per-Family Analysis

• Figure 1 compares F1@5 improvement from S1 to S3 across task families.The comparison is organized by task family.Source: Figure 1 · p. 7
• Temporal analysis gains +0.34, while supply chain and transaction behavior each gain +0.26.These are the three largest reported family-level gains.Source: D Per-Family Analysis · p. 7
• Loan risk gains +0.19 and account activity gains +0.17, indicating more moderate improvement.These families improve less than the top three families.Source: D Per-Family Analysis · p. 7
• Customer demographics, regional economics, and product catalog decline by −0.077, −0.102, and −0.074, respectively.Their objects appear across multiple task types, and seed memories can overlap with unrelated queries.Source: D Per-Family Analysis · p. 7
• Discovery memory is most effective when task-intent and object relationships are specific and stable, and least effective when object sets are shared.Shared object sets can make memory mildly harmful.Source: D Per-Family Analysis · p. 7

Memory Weight

• Both retrievers peak at memory weight α = 0.25 across all three domains.Setting α = 0 recovers registry-only retrieval, while α = 1 gives memory full control.Source: E Memory Weight Ablation · p. 8, Figure 2 · p. 8
• Over-weighting memory at α = 1.0 degrades retrieval performance in all settings.The main conclusions remain unchanged when using the ablation-optimal α = 0.25 instead of the default α = 0.5.Source: E Memory Weight Ablation · p. 8, Figure 2 · p. 8

Blending vs. Direct Injection

• S3 blended registry-plus-memory retrieval outperforms S4 naive injection in both MRR and F1@5 across all three domains.S4 promotes objects from the highest-scoring memory without combining memory and registry scores.Source: F Blending vs. Direct Memory Injection · p. 8, Table 4 · p. 8
• Blending preserves retrieval precision because direct injection can elevate irrelevant objects when a retrieved memory only partially matches the query.S3 combines registry and memory signals while still promoting objects associated with relevant prior discovery outcomes.Source: F Blending vs. Direct Memory Injection · p. 8

Leave-One-Out Seed Robustness

• The relative ordering of methods remains consistent across leave-one-out seed rotation and fixed-seed results.The evaluation includes 56 Finance, 51 NYC Collisions, and 36 Northwind trials.Source: G Leave-One-Out Seed Robustness · pp. 8–9

BM25 Retriever Comparison

• S3 still improves over S1 in all three domains when using BM25 instead of TF-IDF.This supports memory boosting across the evaluated underlying retrievers.Source: H BM25 Retriever Comparison · p. 9, Table 6 · p. 9
• BM25 underperforms TF-IDF at the S1 and S3 levels across all three domains, while S0 differences are negligible.The registry-level gap is attributed to BM25 term-frequency saturation being poorly matched to long, semantically rich descriptions.Source: H BM25 Retriever Comparison · p. 9

Downstream Planning

• In a downstream planning example, S3 retrieves exact transaction-related objects whereas S1 retrieves mostly demographic objects.The S3 plan references correct operational fields and produces a more executable analysis strategy for unusual cash withdrawals.Source: I Agent-in-the-Loop Demonstration · p. 9, Figure 3 · p. 10