Source-linked AI summary
Mitigating Uncertainty Interactions in GenAI-based Adaptive Systems: Vision, Challenges and Preliminary Guidelines
Karthik Vaidhyanathan, Norha Villegas, Raffaela Mirandola, Nicolás Cardozo, Ivana Dusparic, Marin Litoiu, Ralf Reussner, Gabriel Tamura
TL;DR
GenAI systems introduce interacting uncertainties that existing self-adaptive systems approaches do not fully address. The paper proposes a mitigation function and applies it to three illustrative use cases, identifying tactic trade-offs, synergy, and conflict.
Problem
Existing self-adaptive systems approaches do not fully address the novel, compounding uncertainty interactions introduced by GenAI components.
Source: §I · p. 1Method
The paper proposes a mitigation function that maps uncertainty interactions, system context, criticality, and architectural tactics to quality-attribute outcomes and trade-offs.
Source: §III · pp. 2–3Results
Across LLM serving, AIOps with Software Twins, and agentic CloudOps, the framework makes tactic-to-quality-attribute trade-offs explicit and illustrates both tactic synergy and conflict.
Source: A. Tactic Patterns Across Use Cases · p. 5, B. Interaction Between Tactics · p. 5, §IX · p. 7Takeaways & Limitations
Pattern-level composition matters because individual tactics alone do not fully address compounding GenAI uncertainty.
Source: §VI · p. 5
Abstract
from arXiv · showhide
Modern software-intensive systems increasingly incorporate GenAI components, including LLM and agentic subsystems, which introduce novel and compounding sources of uncertainty across system layers. The self-adaptive systems community has made significant strides in understanding and managing uncertainty. The intrinsic characteristics of GenAI, including probabilistic outputs, hallucinations, context window limitations, and memory staleness, demand a re-examination of existing frameworks and mitigation strategies for dealing with uncertainty, and especially, the interactions among uncertainty occurrences and manifestations. This position paper posits an initial conceptual framework that establishes preliminary guidelines for characterizing and mitigating uncertainty interactions in GenAI-based software-intensive systems. We argue that mitigation must be considered across the full software lifecycle, encompassing requirements analysis, design-time decisions, and run-time adaptations, which implies the unfeasibility of decoupling it from monitoring and analysis at execution. We introduce the notion of a mitigation function that maps uncertainty interaction categories, involved uncertainty types, system design characteristics, criticality, and mitigating architectural tactics to impacted quality attributes and corresponding tradeoffs. As a proof of concept, we apply the framework to three illustrative use cases: LLM serving, AI-driven software operations, and agentic cloud management.
I. INTRODUCTION
GenAI components bring intrinsic uncertainty into increasingly autonomous software systems, and interacting uncertainty sources are not fully addressed by existing self-adaptive systems literature. The paper outlines a framework for characterizing and mitigating these interactions.
I. INTRODUCTION
- LLMs now serve as reasoning and decision components in autonomous systems, where uncertainty interactions remain insufficiently addressed by existing self-adaptive systems work,,.Multi-agent LLM systems are also used for scientific discovery and cloud operations.Source: §I · p. 1
- The paper proposes a mitigation function that connects uncertainty interactions and system context to quality-attribute outcomes through architectural tactics, and outlines a research agenda –,.The agenda includes interaction characterization, tactic selection, cross-layer propagation, trade-off evaluation, and dynamic context management.Source: §I · pp. 1–2
A. Uncertainty in Self-Adaptive Systems
- Uncertainty can be described by its location, epistemic or aleatory nature, level, and emergence time; interactions include independence, dominance, augmentation, and conflict,,.Dominance subsumes another source, augmentation amplifies sources, and conflict produces contradictory impacts.Source: A. Uncertainty in Self-Adaptive Systems · p. 2
B. Uncertainty in GenAI Systems
- GenAI uncertainty includes variable outputs, hallucinations, context-window limits, stale memory, and variable tool orchestration, with risks also propagating across agents.A hallucinated result can enter shared memory, be retrieved by another agent, and affect a tool call, creating a cascade that single-agent analysis may miss.Source: B. Uncertainty in GenAI Systems · p. 2
III. THE UNCERTAINTY MITIGATION FRAMEWORK
The framework characterizes uncertainty interactions and uses system context, goals, criticality, and candidate tactics to guide mitigation. Its outputs inform architectural decisions and are refined through monitoring and feedback.
III. THE UNCERTAINTY MITIGATION FRAMEWORK
- The process identifies uncertainties in managing and managed systems, characterizes their interactions, then uses context, goals, and tactics to guide mitigation and quality-attribute trade-offs.Monitoring and feedback support ongoing refinement of characterization and mitigation decisions.Source: §III · p. 2
- The mitigation function considers interaction category, uncertainty nature, system design and context variables, criticality, and quality attributes under risk.Interaction categories include dominance, augmentation, conflict, and independence; uncertainty nature may be epistemic, aleatory, or both.Source: §III · p. 2
- Candidate tactics specify interventions and mechanisms, while the framework also represents intervention scope and lifecycle timing.Scope ranges from component to ecosystem; lifecycle timing includes design-time, deployment-time, and runtime.Source: §III · p. 2
- For the same uncertainty interaction and system conditions, different tactics can produce different quality-attribute outcomes, making tactic choice the architect’s degree of freedom.The formalization maps characterized interactions to mitigating tactics and their associated trade-offs.Source: §III · pp. 2–3
A. Trade-off Characterization
The framework represents each tactic’s quality-attribute trade-offs as the attributes it assures or improves and those it may degrade, enabling comparison between tactics.
A. Trade-off Characterization
- A tactic’s trade-off profile pairs quality attributes it assures or improves with those it may degrade, allowing architects to compare tactics for the same uncertainty interaction.The pair is (Q+, Q−).Source: A. Trade-off Characterization · p. 3
B. Composition of Tactics
Composing tactics can provide emergent assurance beyond what any individual tactic achieves, but the composition can also introduce new quality-attribute degradation.
B. Composition of Tactics
- Composed tactics may assure quality attributes that no individual tactic achieves, while also creating degradation absent from any individual tactic.Composition therefore requires considering both emergent assurance and new conflicts.Source: B. Composition of Tactics · p. 3
IV. USE CASES AND TACTIC EXEMPLIFICATION
In LLM serving, workload and output uncertainty augment one another to inflate tail latency. The use case pairs this interaction with runtime tactics that balance throughput, latency, cost, and resource use.
A. Use Case 1: SLO Compliance in LLM Serving System
- Variable request lengths shift the KV-cache and batching trade-off, while stochastic output lengths make the decode horizon uncertain at admission.Longer responses can make decoding memory-bandwidth bound, so wider batches may worsen p95/p99 despite high average utilization.Source: A. Use Case 1: SLO Compliance in LLM… · p. 3
- Workload and output uncertainties augment one another, jointly inflating tail latency in a way neither source alone would cause.Source: A. Use Case 1: SLO Compliance in LLM… · p. 3
- Prompt batching amortizes GPU launch overhead and improves throughput, but excessively large batches can increase per-request latency.The listed trade-offs include tail latency (p95/p99) and fairness for short requests.Source: A. Use Case 1: SLO Compliance in LLM… · pp. 3–4
- Routing requests among LLM instances by estimated complexity balances cost against quality, with output-quality variance and routing overhead as trade-offs.Source: A. Use Case 1: SLO Compliance in LLM… · p. 4
- Response-length prediction before decoding supports informed scheduling and resource allocation, but incurs prediction overhead and misprediction cost.Source: A. Use Case 1: SLO Compliance in LLM… · p. 4
- Parallel-batch scheduling balances memory-bandwidth use against decode progress, trading throughput and SLO compliance for memory contention and implementation complexity.Source: A. Use Case 1: SLO Compliance in LLM… · p. 4
B. Use Case 2: Agentic System for Autonomous CloudOps
In CloudOps, stale or incomplete memory dominates context-window limitations, making available context unreliable even when the window is sufficient. The tactics address retrieval staleness, retrieval gaps, and bounded context through runtime measures.
B. Use Case 2: Agentic System for Autonomous CloudOps
- CloudOps agents face memory staleness or retrieval gaps and bounded context windows that cause partial observability of system state and prior reasoning.Source: B. Use Case 2: Agentic System for… · p. 4
- Memory staleness dominates context-window limits because stale or incomplete memory makes available context unreliable even when the window is sufficient.The interaction combines epistemic uncertainty in stored beliefs with aleatory variability in retrieval quality.Source: B. Use Case 2: Agentic System for… · p. 4
- Drift-aware retrieval monitors stored-metric drift and triggers re-indexing or cache invalidation when staleness exceeds thresholds.Source: B. Use Case 2: Agentic System for… · p. 4
- External knowledge augmentation compensates for retrieval gaps, while hierarchical summarization fits more relevant history into the bounded context window.Augmentation can improve context completeness and decision accuracy but adds integration overhead and potential noise; summarization aims to preserve critical information.Source: B. Use Case 2: Agentic System for… · p. 4
C. Use Case 3: Proactive Continuous Operations Using LLMs and AIOps
For digital-twin-driven operations, uncertainty in the LLM’s learned representations dominates variability in monitored operational data. The use case proposes synthetic data, ensembles, and knowledge augmentation to address model uncertainty and data imbalance.
C. Use Case 3: Proactive Continuous Operations Using LLMs and AIOps
- Digital twins use simulations to assess operational-change risks before production, while LLMs support experimentation, analysis, and risk explanations.The use case identifies epistemic uncertainty in the LLM and aleatory uncertainty in noisy, variable operational data and monitoring.Source: C. Use Case 3: Proactive Continuous… · p. 4
- LLM model uncertainty dominates operational-data uncertainty because model errors systematically bias predictions regardless of data variability and quality.Source: C. Use Case 3: Proactive Continuous… · p. 4
- Design-time synthetic operational data labeled with root causes and fix recommendations expands training coverage to reduce model uncertainty about failure modes.The listed quality gains are diagnostic accuracy and failure coverage, with distribution-shift risk and generation cost as trade-offs.Source: C. Use Case 3: Proactive Continuous… · p. 5
- Runtime ensembles aggregate predictions from multiple models or LLM configurations to reduce the impact of any single model’s epistemic errors through diversity.The listed trade-offs include inference cost, latency, and resource consumption.Source: C. Use Case 3: Proactive Continuous… · p. 5
- Knowledge augmentation supplements the LLM with knowledge hyper-graphs to compensate for imbalanced data on component and failure-source relationships.Source: C. Use Case 3: Proactive Continuous… · p. 5
A. Tactic Patterns Across Use Cases
Across the use cases, the framework links uncertainty interactions and tactics to quality-attribute trade-offs. The analysis finds that interaction type and whether the LLM is managed or managing shape tactic selection, while most tactics operate at runtime.
A. Tactic Patterns Across Use Cases
- Table I summarizes tactic-to-quality-attribute mappings across all three use cases and makes interaction categories, tactics, and trade-offs explicit.Source: A. Tactic Patterns Across Use Cases · p. 5
- Tactics that improve correctness, accuracy, performance, or reliability tend to add computational, temporal, or system-complexity costs.The formalization represents these cost-quality trade-offs through Q+ and Q–.Source: A. Tactic Patterns Across Use Cases · p. 5
- Interaction category shapes tactic design: augmentation tactics reduce amplification, while dominance tactics target the dominant uncertainty source.Source: A. Tactic Patterns Across Use Cases · p. 5
- UC1 and UC2 treat the LLM-based system as managed, whereas UC3 treats the LLM as the managing system; tactics accordingly target inputs and runtime behavior or model knowledge and robustness.Source: A. Tactic Patterns Across Use Cases · p. 5
- Most tactics operate at runtime because GenAI uncertainty is largely irreducible at design time; synthetic-data generation is the stated design-time exception.That exception reduces epistemic uncertainty by expanding the model’s training distribution.Source: A. Tactic Patterns Across Use Cases · p. 5
B. Interaction Between Tactics
Tactic combinations can produce synergy, conflict, or enablement, motivating reusable mitigation patterns that connect uncertainty interactions and design contexts to quality-attribute outcomes and trade-offs.
B. Interaction Between Tactics
- In LLM serving, response-length prediction combined with parallel batch scheduling improves SLO compliance over either tactic alone.Predicted lengths inform batch-composition decisions.Source: B. Interaction Between Tactics · p. 5
- In agentic CloudOps, knowledge augmentation may conflict with hierarchical summarization when compression discards newly injected knowledge.External knowledge expands the context that must be compressed.Source: B. Interaction Between Tactics · p. 5
- In AIOps, synthetic data generation enables ensemble diversity by supplying labeled examples; without adequate data, the robustness benefit is reduced.Source: B. Interaction Between Tactics · p. 5
VI. TOWARDS MITIGATION PATTERNS
- A mitigation pattern captures its motivating interaction and uncertainty types, applicable design contexts, prescribed tactics, and quality-attribute trade-off profile.The proposed catalog would make such combinations reusable.Source: §VI · p. 5
- Illustrative patterns compose tactics for LLM serving, CloudOps, and AIOps, addressing performance and cost, user satisfaction, and functional correctness trade-offs.Examples include Predictive Batch Orchestration, Memory-Aware Agent, and Robust AIOps Predictor; the paper argues individual tactics do not fully address compounding uncertainty.Source: §VI · p. 5
- Table I summarizes uncertainty interactions, tactics, and quality-attribute trade-offs across the use cases.Source: Table I · p. 6
VII. VISION AND OPEN CHALLENGES
The paper identifies open challenges in characterizing and mitigating interacting GenAI uncertainties across architectures, quality attributes, and runtime context. It places these challenges within a staged research roadmap and relates its framework to prior uncertainty work.
VII. VISION AND OPEN CHALLENGES
- GenAI uncertainty interactions need explicit, architecture-aware characterization because they can be implicit, emergent, and visible only through downstream system effects.The proposed direction extends existing frameworks with GenAI-specific concepts and architectural abstractions.Source: §VII · p. 6
- Architects lack practical guidance for selecting mitigation strategies across model techniques, architectural tactics, and operational interventions.The mitigation function is a conceptual foundation that needs to support principled selection and comparison using interaction categories, criticality, and quality-attribute impacts.Source: §VII · p. 6
- In layered, dynamic GenAI architectures, uncertainty can propagate across components and subsystems, sometimes amplifying or transforming.The paper calls for cross-layer control strategies spanning design-time, deployment-time, and runtime.Source: §VII · p. 6
- Mitigations can improve reliability, safety, or correctness while harming performance, cost, or usability, yet systematic evaluation under interacting uncertainties is limited.The proposed direction evaluates broader system qualities in realistic operational conditions.Source: §VII · p. 6
- Runtime context management must adapt monitoring to changing context relevance; stale memory can dominate context-window limitations when retrieved information is outdated or irrelevant.The lifecycle of context variables and monitoring strategies must be managed as uncertainty interactions change.Source: §VII · pp. 6–7
- The roadmap moves from conceptual taxonomies and architecture-aware representations, to mitigation patterns and composition support, then to rigorous evaluation and adaptive runtime mitigation.Source: §VII · p. 7
VIII. RELATED WORK
- The framework builds on uncertainty definitions, taxonomies, and self-adaptation research,,,, while addressing uncertainty combinations and newer LLM sources.The cited work also identifies inherent AI-model uncertainty as a performance-evaluation challenge [9].Source: §VIII · p. 7
IX. CONCLUSION
- The conclusion presents an interaction formalization that makes architectural tactics an explicit input and applies it to LLM serving, AIOps with Software Twins, and agentic CloudOps.The authors say the use cases demonstrate reasoning about trade-offs, tactic composition, and mitigation patterns.Source: §IX · p. 7