Source-linked AI summary
Supporting Our AI Overlords: Redesigning Data Systems to be Agent-First
Shu Liu, Soujanya Ponnapalli, Shreya Shankar, Sepanta Zeighami, Alan Zhu, Shubham Agarwal, Ruiqi Chen, Samion Suwito, Shuo Yuan, Ion Stoica, Matei Zaharia, Alvin Cheung, Natacha Crooks, Joseph E. Gonzalez, Aditya G. Parameswaran
TL;DR
LLM agents may become a dominant data-system workload, but their high-throughput exploration and solution formulation can create scale and efficiency challenges. The paper proposes an agent-first architecture and research opportunities that exploit the characteristics of agentic speculation. Its case studies report higher success with more requests and fewer queries when grounding hints are provided.
Problem
Agents’ high-throughput exploration and solution formulation can strain data systems designed for more intermittent or targeted workloads.
Source: §1 · p. 1Method
Results
Success rate increased by 14%–70% with agentic speculation, while grounding hints reduced queries by >20%, depending on phase.
Source: §2 · pp. 2–3Takeaways & Limitations
The paper argues that data systems can use speculation’s characteristics to redesign interfaces, query processing, and storage for agent workloads.
Source: §1 · p. 1, §7 · pp. 6–7An agentic memory store that can remain inconsistent with current data or metadata risks stale information leading probes to mistakes.
Source: §6.1 · p. 6
Abstract
from arXiv · showhide
Large Language Model (LLM) agents, acting on their users' behalf to manipulate and analyze data, are likely to become the dominant workload for data systems in the future. When working with data, agents employ a high-throughput process of exploration and solution formulation for the given task, one we call agentic speculation. The sheer volume and inefficiencies of agentic speculation can pose challenges for present-day data systems. We argue that data systems need to adapt to more natively support agentic workloads. We take advantage of the characteristics of agentic speculation that we identify, i.e., scale, heterogeneity, redundancy, and steerability - to outline a number of new research opportunities for a new agent-first data systems architecture, ranging from new query interfaces, to new query processing techniques, to new agentic memory stores.
1 Introduction
LLM agents may become a dominant data-system workload, but their lack of grounding leads them to issue high-throughput exploratory requests. The paper asks how data systems can natively support this speculation and proposes an agent-first research vision based on its scale, heterogeneity, redundancy, and steerability.
1 Introduction
- Agents may become a dominant workload as they extract, analyze, transform, and update data on users’ behalf.Their growing capability and low cost are part of the motivation.Source: §1 · p. 1
- Agentic speculation combines metadata discovery with partial solution formulation and validation in high-throughput exploratory querying.Agents may issue hundreds or thousands of requests per second, with throughput scaling with the number of agents.Source: §1 · p. 1
- Unlike more intermittent or targeted workloads, speculation can generate enormous query volumes for a single task, making its scale and inefficiencies a prospective bottleneck.Agents may explore alternatives in parallel, including speculative updates and rolled-back database branches.Source: §1 · p. 1
- The paper asks how data systems can natively and efficiently support speculation, then proposes research opportunities organized around scale, heterogeneity, redundancy, and steerability.It previews an agent-first architecture and opportunities across interfaces, query processing, and storage.Source: §1 · pp. 1–2
2 Case Studies
Two case studies examine how agentic speculation affects task accuracy, repeated work, trace activities, and the effects of grounding hints. They report higher success with more requests and fewer queries when relevant hints are provided.
2 Case Studies
- Success rate increases with requests, with agentic speculation improving accuracy by 14%–70% in the case study.The study considered requests in parallel and in sequence.Source: §2 · p. 2, Figure 1 · p. 2
- Across 50 parallel attempts per problem, the study plots total versus unique subexpressions by size and root operator to examine repeated work.The plots aggregate GPT-4o-mini attempts across the BIRD dataset.Source: §2 · p. 2, Figure 2 · p. 2
- Agent traces show table and column exploration followed by query formulation, with the phases often overlapping.Figure 3 places normalized trace position on the x-axis and independently normalizes each activity row.Source: Figure 3 · p. 2
- Table 1 presents mean activity counts per trace with and without human expert-provided hints.The supplied passage identifies the comparison but does not report the table’s values.Source: Table 1 · p. 2
- Grounding hints reduce the queries needed for agentic speculation by >20%, depending on the phase.The study compared sequential traces with and without hints.Source: §2 · p. 3
3 Agent-First Data System Architecture
The proposed architecture lets agents send contextual probes to a database-side interpreter, which can respond with approximate answers and proactive grounding feedback. Probe optimization, agentic memory, and shared transaction management address efficiency across speculative work.
3 Agent-First Data System Architecture
- Agents issue prioritized probes that can include natural-language context and search for tokens in tables, extending beyond SQL queries.Probe context may describe phase, agent identity, accuracy needs, and overall goals.Source: §3 · p. 3
- An in-database agentic interpreter parses probes and can return approximate answers plus proactive grounding feedback to steer agents.The architecture describes this interface as supporting responses beyond direct answers.Source: §3 · p. 3
- The probe optimizer aims to satisfice by producing reasonable results without fully evaluating every query, extending multi-query optimization and approximate query processing.This is the proposed role of the optimizer for one or more probes.Source: §3 · p. 3
- An agentic memory store retains grounding for future probes, while a shared transaction manager is proposed to handle redundancy across potential transactions.The architecture also calls for changes to storage and transaction components.Source: §3 · p. 3
4 Query Interfaces
Agent-facing interfaces should let probes express intent, phase, approximation priorities, and semantic searches beyond SQL, while data systems return answers plus guidance that steers agents toward more efficient probes.
4.1 From Agents to Data Systems
- Natural-language briefs can supply a probe’s goals, phase, approximation needs, and priorities, guiding query order and accuracy.Choosing accuracy from natural-language input remains open and must account for relative query costs.Source: §4.1 · p. 3
- Probe batches can specify open-ended priorities, complete only selected queries, and use termination criteria evaluated on partial results.An example is stopping similar “needle-in-a-haystack” queries early.Source: §4.1 · pp. 3–4
- Flexible probes add semantic-similarity searches over data and metadata, helping agents find relevant tables, columns, or rows beyond SQL’s capabilities.Agents may also query or update metadata stored in agentic memory.Source: §4.1 · p. 4
4.2 From Data Systems to Agents
- Data systems can steer agents with auxiliary information and efficiency feedback, delivered alongside or instead of probe answers.On-demand sleeper agents can gather relevant information in parallel with answering probes.Source: §4.2 · p. 4
- Sleeper agents can suggest related or replacement tables and explain empty results, while cost estimates can prompt narrower or more approximate probes.They can also suggest batching sequential tasks or reusing related materialized answers.Source: §4.2 · p. 4
5 Processing and Optimizing Probes
Agent-first query processing must handle mixed exploratory workloads and optimize for enough information to guide the next agent turn, rather than simply maximizing throughput. The proposed directions use probe phase, approximation, redundancy, and interaction history to decide what to run and how to execute it.
5.1 Supporting Exploration
- Exploration mixes natural-language and SQL queries, requiring the optimizer to orchestrate both; metadata-query outputs can overwhelm an agent’s limited context.Metadata results may span thousands or hundreds of thousands of rows, and agents also need token searches across data and metadata.Source: §5.1 · p. 4
- Phase-aware optimization can prioritize exploratory queries without full evaluation and use agentic memory to avoid repeatedly querying previously learned information.This addresses exploration that may generate inapplicable queries alongside solution-formulation work.Source: §5.1 · p. 5
5.2 Probe Optimization
- The objective is minimizing total time spent answering field-agent probes, balancing approximation costs against the follow-ups that insufficient answers may trigger.Agents can state goals and approximation needs in natural-language briefs.Source: §5.2 · p. 5
5.2.1 Intra-Probe Optimization.
- Within a probe, the database can use intent, approximation needs, and information gain to prune or rank queries, then allocate accuracy incrementally across probes.It may return coarse approximations during exploration and more accurate answers during solution formulation.Source: §5.2.1 · p. 5
5.2.2 Inter-Probe Optimizations.
- Across turns, the database can drop probes that add no relevant information, spend more upfront to reduce future follow-ups, and use history to anticipate and cache useful results.It can also explore underexplored solution spaces for potential longer-term utility.Source: §5.2.2 · p. 5
6 Indexing, Storage, and Transactions
Agentic workloads evolve across phases and contain similar probes, challenging storage systems built for static, independent queries. The section proposes shared agentic memory and state-sharing transactions, with further needs around stale information, privacy, and large-scale branch handling.
6 Indexing, Storage, and Transactions
- Static layouts poorly fit probes that evolve from exploration to validation, while similar phases across probes could benefit from shared layouts and state.Traditional isolation protects consistency but can prevent cooperative sharing of intermediate state.Source: §6 · p. 6
- The proposed architecture pairs a queryable agentic memory pseudo-index with transactions centered on sharing state across probes.The transactions framework addresses probes independently attempting user-defined update sequences.Source: §6 · p. 6
6.1 Agentic Memory Store
- Agentic memory can retain prior probe results, partial solutions, and data or metadata facts such as encodings, missing values, and time or location granularities.Metadata may be embedded with tables; a vector index can support semantic search of open-ended information, though targeted structured lookups may fit less well.Source: §6.1 · p. 6
- Memory updates must address changed probes and underlying data; stale schema-related information can misdirect probes, and cross-user sharing raises privacy concerns.The authors identify knowledge-base and schema-evolution work as relevant inspiration.Source: §6.1 · p. 6
6.2 Performing Branched Updates
- Agentic speculation needs logically isolated branches that may physically overlap, including forks created by multiple agents that later must reconcile with one another.The motivating setting involves multiple “what-if” hypotheses and branches.Source: §6.2 · p. 6
- Efficient forking requires more than copy-on-write: systems must exploit branch similarity while preserving isolation and support ultra-fast rollbacks for failed branches.The target includes forking potentially thousands of near-identical snapshots and rolling back all but one.Source: §6.2 · p. 6
7 Conclusion
The conclusion characterizes agentic workloads as high-throughput, heterogeneous, and redundant discovery and validation, and presents an agent-first architecture with emergent research challenges.
7 Conclusion
- Agentic workloads combine high-throughput, heterogeneous, and redundant discovery and validation.The conclusion describes these activities as specified by probes.Source: §7 · p. 6
- The proposed probes ideally combine queries with natural language.Source: §7 · p. 7
- The paper presents an architecture for redesigned data systems and discusses emergent research challenges.Source: §7 · p. 7