Source-linked AI summary
Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines
Alimurtaza Mustafa Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui
TL;DR
Industrial agent workflows incur repeated orchestration overhead, while conventional semantic caching cannot ensure valid reuse for time- and parameter-sensitive queries. This paper combines temporal-aware caching with MCP workflow optimization, achieving 3.48× end-to-end speedup on 80 paraphrase-tier queries and 1.67× on 18 IoT queries.
Problem
Existing caching methods do not adequately address industrial queries whose answer validity depends on asset, sensor, and time-window parameters.
Method
The paper combines a temporal classifier and window-aware semantic cache with cached tool discovery, pooled servers, and dependency-aware parallel execution.
Results
3.48× end-to-end speedup was achieved on 80 paraphrase-tier queries, while MCP workflow optimization alone yielded 1.67× on 18 IoT queries.
Takeaways & Limitations
Pure semantic similarity is not a sound proxy for answer validity in parameter-rich industrial queries, even with reranking and thresholding.
Takeaways & Limitations
Cross-parameter false positives can return cached answers from different assets, sensors, or time windows despite high similarity and judge scores.
Abstract
from arXiv · showhide
Industrial asset operations workflows are latency-sensitive because a single user query may require coordination over sensor data, work orders, failure modes, forecasting tools, and domain-specific agents. We evaluate this problem on AssetOpsBench (AOB), an industrial agent benchmark whose plan-execute pipeline exposes repeated overhead from tool discovery, LLM planning, MCP tool execution, and final summarization. Existing LLM caching techniques such as KV-cache reuse and embedding-based semantic caching were designed for chatbot serving and break down when output validity depends on time, asset, or sensor parameters. We propose two complementary optimization layers for AOB plan-execute pipelines: a temporal semantic cache and a set of MCP workflow optimizations combining disk-backed tool-discovery caching and dependency-aware parallel step execution. MCP workflow optimizations corresponded to a 1.67x speedup and reduced median end-to-end latency by about 40.0% while the temporal-cache benchmark achieved a median of 30.6x speedup on cache hits. Beyond the speedup, our results expose a concrete failure mode of pure semantic caching for parameter-rich industrial queries, providing a critical analysis of how caching choices interact with evaluation correctness in MCP-backed agent benchmarks.
1 Introduction
Industrial Plan-Execute agents incur substantial orchestration latency on heterogeneous asset-operations queries, especially when repeated or parameter-shifted requests force full execution. The paper proposes temporal semantic caching and MCP workflow optimizations tailored to external state such as assets, sensors, and time windows.
- Plan-Execute agents decompose queries into tool calls before execution, but tool discovery and multistep orchestration introduce substantial wall-clock latency.
- AssetOpsBench models industrial queries spanning sensor telemetry, work orders, failure modes, and time-series forecasts across four MCP-backed domain servers.
- Repeated paraphrases, asset substitutions, and time-window shifts make naive full-cost execution prohibitively slow for systematic MCP-agent evaluation.
- Our approach: The proposed approach combines a temporal classifier and window-aware matching with MCP discovery caching, DAG-layered parallel execution, and a persistent server pool.
2 Background and Motivation
AOB evaluates industrial asset-operations agents through an MCP-backed Plan-Execute workflow spanning discovery, planning, execution, and summarization. The setting exposes sequential workflow overhead and shows why safe caching must account for external state, operational parameters, and time.
- Benchmark and workflow: AOB evaluates LLM agents on industrial asset operations using four MCP-wrapped servers for IoT telemetry, FMSR, TSFM, and work-order records.Scenarios use human-authored natural-language operational queries.
- Benchmark and workflow: Plan-Execute decomposes each query into Discovery, Planning, Execution, and Summarization rather than answering through a single LLM call.Discovery collects tool signatures, Planning creates a structured plan, Execution invokes tools, and Summarization synthesizes outputs.
- Workflow optimization: Dependency-aware execution can dispatch independent plan steps concurrently while preserving ordering across true dependencies, because Plan-Execute does not automatically provide parallelism.Many implementations consume generated plans strictly sequentially.
- Caching limitations: Pure semantic caching fails when answers depend on runtime state, operational parameters, or resolved temporal windows, despite identical or closely embedded query text.The passages identify stale work-order answers, disjoint asset-specific failure-mode results, and incorrect reuse across changing time windows.
- Motivation and positioning: These limitations motivate a temporal-aware cache that separates semantic relatedness from safe answer reuse, alongside workflow optimizations that reduce overhead on cache misses.The approach targets agent orchestration above serving engines and supports practical AOB paraphrase-tier ablations.
3 The Optimization Framework
The framework routes queries through temporal classification before semantic retrieval, making cache reuse conditional on volatility and time-window compatibility. It also reduces MCP workflow overhead through cached tool discovery and dependency-aware parallel execution.
- Temporal Semantic Cache: Four temporal buckets route queries before retrieval: Volatile bypasses caching, while Static and resolved-Anchored queries proceed to ANN retrieval and reranking.Relative expressions are resolved against the run timestamp into concrete windows.
- Temporal Semantic Cache: Temporal filtering separates live-state, time-bounded, and time-independent queries so semantic similarity does not alone determine answer reuse validity.This routing is intended to make reuse sound across different temporal regimes.
- Temporal Semantic Cache: Relative-time cache entries store concrete windows resolved at insertion, and lookup acceptance checks compatibility with the new query’s window.This prevents literal phrases such as “yesterday” from producing stale hits.
- Temporal Semantic Cache: Cosine similarity retrieves candidates, while a reranker-based judger evaluates semantic and temporal alignment for acceptance.The two-stage design independently tunes retrieval recall and judging precision.
- MCP Workflow Optimization: 2 to 3 seconds per query is spent on baseline MCP discovery, which the framework replaces with a locally persisted JSON catalog invalidated by server-source changes.The baseline spawns subprocesses, establishes stdio connections, requests catalogs, and terminates before planning.
- MCP Workflow Optimization: Independent DAG steps execute concurrently within topological layers, while dependency barriers and per-server asynchronous locks preserve ordering and server safety.A persistent MCPServerPool maintains one stdio session per required server throughout a plan.
4 Results and Evaluation
The combined cache-and-workflow pipeline substantially lowers end-to-end latency, with MCP optimizations improving the miss path and temporal caching delivering much larger gains on valid hits. Cache decisions remain imperfect, especially for parameter-shifted industrial queries.
- Full-pipeline evaluation: 3.48× median speedup reduced end-to-end latency from 34.10s to 9.80s across 80 paraphrase-tier queries.The fully optimized pipeline also reduced mean latency from 68.68s to 33.06s.
- MCP workflow evaluation: 1.67× end-to-end median speedup came from MCP workflow optimizations, including 296× lower discovery cost and 1.99× faster execution.Planning and summarization showed no statistically significant change, indicating the optimizations targeted orchestration overhead.
- Cache decision quality: F1 0.6429 for cache decisions accompanied precision 0.75, recall 0.5625, and specificity 0.7188, with residual errors concentrated on parameter-shifted queries.Compared with cache-only evaluation, precision improved from 0.667 to 0.75 while recall declined slightly.
- Full-pipeline evaluation: 31.87× median speedup occurred on 36 cache-hit rows, which bypassed plan-execute and saved a median 25.50s per row.The cache hit rate was 45.0% (36 of 80 rows).
- Full-pipeline evaluation: −3.30s median latency difference on 44 cache misses shows the optimized miss path still beats the unoptimized baseline.MCP gains more than recovered cache lookup cost, so misses incurred no net overhead.
5 Limitations and Failure Modes
The approach’s central limitation is that semantic similarity cannot reliably establish answer validity for parameter-rich industrial queries. Additional limits arise from judger inconsistency, workload structure, excluded timeouts, temporal parsing gaps, and single-machine in-memory evaluation.
- Structural limitation: Pure semantic similarity is not a sound proxy for answer validity in parameter-rich industrial queries, and no similarity threshold fully resolves this.This is identified as the approach’s most important structural limitation.
- Parameter-collision false positives: Above 0.95 cosine similarity and τjudge = 0.92, queries differing in asset, sensor, or time window can return answers from another operational context.This cross-parameter false-positive failure occurred for queries sharing a linguistic frame but differing in operational parameters.
- Judger inconsistency on legitimate paraphrases: Judger scores for semantically equivalent paraphrases ranged from 0.5 to 0.95, with no obvious residual-error pattern.The observed variance was attributed to the bounded capacity of Qwen3-Reranker-0.6B.
- Workload structure caps the achievable hit rate: 45% hit rate in the 80-row paraphrase tier reflects its constructed 60%/40% warm/cold split, while full AOB achieves roughly 15% to 30%.Most full-corpus queries are parameter-rich data fetches where pure semantic matching is fundamentally unsafe.
- Excluded queries and provider variance: Two of 20 IoT queries, Q5 and Q19, timed out across all baseline and optimized attempts and were excluded from Table 1 and Figure 7.The timeouts were upstream LLM failures rather than orchestration failures; MCP regressions included Q1 at 0.92× and Q11 at 0.67×.
- Window grammar and natural-date handling: Natural-date phrases such as “June 2020” extract as bucket=Anchored with window=None and are demoted to the Static path, losing temporal-prefilter benefits.The classifier currently handles a fixed grammar of relative phrases and explicit ISO ranges.
- Single-machine evaluation, no persistence: All experiments ran on a single Apple M-series machine with 16 GB unified memory, while in-memory caching loses warmed entries after process restart.Inter-machine variability, concurrent-load effects, checkpointing, and replay were out of scope or unavailable.
6 Future Work
Future work centers on parameter-aware hybrid retrieval to prevent cache collisions, alongside improvements to reranking, date parsing, persistence, judging, evaluation scale, and serving integration. These extensions target cache safety, restart overhead, workload drift, confidence in findings, and production deployment.
- Caching and retrieval: Parameter-aware caching would key entries by canonical_intent and parameter combinations, using exact matches before overlap-constrained semantic retrieval to eliminate observed false positives.The proposed parameters are entity, sensor, time window, and action verb.
- Model and parsing improvements: A larger or domain-adapted reranker may reduce the judger inconsistency described in Section 5.Candidates include Qwen3-Reranker-0.6B’s 4B variant and a reranker fine-tuned on AOB-style query-answer pairs.
- Deployment and evaluation: Versioned cache-state and FAISS-index persistence would avoid the 30-second pre-warm cost after process restarts and support non-trivial deployment.The passage characterizes this as mechanical engineering work necessary for deployment.
- Deployment and evaluation: Online judge-threshold recalibration, evaluation on 1000+ utterances, and integration with PagedAttention or SGLang are proposed to address workload drift, strengthen confidence, and combine system layers.The current implementation uses one offline-set threshold, while the existing AOB corpus contains 152 utterances.
7 Conclusion
The paper presents additive temporal semantic caching and MCP workflow optimizations for AOB plan-execute pipelines. It emphasizes that pure semantic similarity is unsafe for parameter-rich industrial queries, motivating parameter-aware caching as a next step.
- Optimization layers: Two complementary layers combine temporal query classification before semantic retrieval with MCP discovery elimination and parallel execution of independent plan steps.The MCP optimizations reduce latency regardless of cache state, while temporal caching adds savings for cacheable queries.
- Failure-mode analysis: Pure semantic similarity is not a sound proxy for answer validity in parameter-rich industrial queries, even with a strict reranker-based judge.Shared linguistic framing can dominate embeddings while distinct operational parameters do not.
- Failure-mode analysis: Hit-decision F1 caps near 0.64, with residual error concentrated on cross-parameter false positives that pass even at τjudge = 0.92.The paper characterizes this as structural to pure semantic caching rather than a tuning issue.
- Future direction: Parameter-aware caching is identified as the natural next step for safer deployment in MCP-backed industrial-agent evaluation pipelines.The paper lays out this direction in Section 6.
A Implementation Parameters
The implementation combines invalidation-aware discovery caching, dependency-layered parallel execution, and parameterized semantic-cache components. Experiments use specified Qwen3 models, FAISS thresholds, Llama planning infrastructure, and a single 16 GB Apple M-series machine.
- Discovery cache: Discovery-cache keys hash server paths, Python-source mtimes, and pyproject.toml’s modification time, with automatic invalidation on changes and a 24-hour TTL.The key incorporates server logic and dependency changes.
- Parallel executor: Kahn’s algorithm forms topological dependency layers, asyncio.gather() runs each layer concurrently, and persistent per-server sessions use asynchronous locks to serialize same-server calls.The MCPServerPool maintains one persistent stdio session per required server.
- Semantic cache: 1024-dim Qwen3-Embedding-0.6B embeddings support FAISS retrieval with top_k=5 and tau_sim=0.75, while Qwen3-Reranker-0.6B applies tau_jsm=0.92.Both models run with prefill-only judging on Apple Silicon MPS in fp16.
- Semantic cache: The semantic cache stores 50 entries and uses LCFU eviction.The implementation specifies a capacity of 50 entries.
B Cache-Only Configuration: Decision Breakdown
The temporal semantic cache, evaluated alone on 50 stratified queries atop the unmodified plan-execute pipeline, delivered substantial hit-speed gains but imperfect decision quality. Its 36.0% hit rate coexisted with miss overhead and balanced precision, recall, and F1 of 0.667.
- Cache performance: 36.0% hit rate was achieved across the 50-query stratified sample.The cache was evaluated as a standalone layer on the unmodified plan-execute pipeline, without MCP optimizations.
- Cache performance: 30.62× median speedup occurred on cache hits, while misses incurred +2.23s median overhead.Cache hits collapsed to near-zero cached cost regardless of baseline latency.
- Decision quality: Decision quality reached precision 0.667, recall 0.667, F1 0.667, and specificity 0.813.The reported decision metrics characterize cache-only behavior rather than the MCP-optimized pipeline.
C Per-Query Speedup and Structural Comparison
Across 18 completed IoT queries, Figure 7 reports per-query end-to-end speedup relative to a 1.0× break-even line. Query 6 illustrates that workflow optimization replaces subprocess-per-call sequencing with cached discovery, persistent servers, and concurrent dependency-layer execution.
- Per-query speedup: 18 completed IoT queries are evaluated using per-query end-to-end speedup, with 1.0× marking break-even.Figure 7 presents the comparison across the completed queries.
- Structural comparison: Query 6 uses a 5-step plan organized into two dependency layers for detecting Chiller 6 failure modes from its Chiller Efficiency sensor.The worked comparison concerns a parameter-specific industrial query.
- Structural comparison: The baseline executes one tool call per step through a fresh MCP subprocess, terminating before the next step begins.This creates sequential subprocess-per-call execution.
- Structural comparison: The optimized path starts each required server once, bypasses spawning through discovery caching, and runs independent steps concurrently within each dependency layer.Figure 8 depicts the optimized workflow with a discovery cache, parallel DAG layers, and a persistent server pool.
D Broader Impact and Societal Implications
The proposed optimizations could broaden access to LLM-backed agent systems and improve industrial workflow responsiveness, safety, and efficiency. However, caching introduces a risk of stale or incorrect answer reuse, especially in safety-critical settings.
- Positive impacts: Lower per-query latency and API cost could make LLM-backed agent systems more accessible to organizations unable to afford high-throughput commercial serving.The passage identifies reduced latency and cost as the basis for improved accessibility.
- Positive impacts: Faster, cheaper query resolution could improve response times for maintenance workflows, equipment fault detection, and work-order management.The passage links these improvements to downstream operational safety and efficiency benefits.
- Potential negative impacts: Caching can reuse stale or incorrect answers, potentially informing faulty maintenance decisions when cached sensor readings no longer reflect equipment state.The temporal classifier routes live-state queries past the cache but does not eliminate the risk, particularly for Anchored queries with truncated window grammar.