PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty Propagation in LLM-Based Systems

Boming Xia, Liming Zhu, Erdun Gao, Qinghua Lu, Minhui Xue, Dino Sejdinovic

arXiv:2604.23505v1cs.SEcs.AI

TL;DR

Research on LLM uncertainty often focuses on single outputs, while deployed systems reuse uncertainty across components and human or organisational processes. This survey frames propagation as a cross-boundary problem and organizes it into P1–P3 mechanisms. It finds the literature is most developed for P1, moderately covered for P2, and substantially thinner for P3.

  • Problem

    Research often studies uncertainty at the level of a single model output, leaving the cross-boundary reuse of uncertainty in compound systems without a primary organizing framework.

    Source: §1 · pp. 1–2

  • Method

    The survey introduces a contextualised-signal framing and uses it to organize propagation mechanisms across intra-model P1, system-level P2, and socio-technical P3 boundaries.

    Source: §1 · p. 2

  • Results

    P1 mechanisms are the most technically developed, P2 mechanisms are moderately covered, and P3 mechanisms have substantially thinner systematic study.

    Source: §4.1 · p. 20

  • Takeaways & Limitations

    Engineering progress depends on representing, propagating, evaluating, and governing uncertainty across the system lifecycle, rather than adding only local signals.

    Source: §6 · p. 24

    Evidence on how communication topology affects uncertainty in multi-party systems is preliminary and has not yet motivated a dedicated taxonomy entry.

    Source: §5.4 · p. 23

Abstract

from arXiv · show

Uncertainty in large language model (LLM)-based systems is often studied at the level of a single model output, yet deployed LLM applications are compound systems in which uncertainty is transformed and reused across model internals, workflow stages, component boundaries, persistent state, and human or organisational processes. Without principled treatment of how uncertainty is carried and reused across these boundaries, early errors can propagate and compound in ways that are difficult to detect and govern. This paper develops a systems-level account of uncertainty propagation. It introduces a conceptual framing for characterising propagated uncertainty signals, presents a structured taxonomy spanning intra-model (P1), system-level (P2), and socio-technical (P3) propagation mechanisms, synthesises cross-cutting engineering insights, and identifies five open research challenges.

1 Introduction

Deployed LLM applications reuse uncertain intermediate evidence across components and people, so early errors can compound as evidence shapes later actions. The survey defines propagation as the cross-boundary trajectory of uncertainty signals and organizes its review around that trajectory.

1 Introduction

  • Weakly evidenced intermediate artefacts can enter state, tools, or later decisions, allowing errors to compound in deployed LLM systems.The survey distinguishes these compound systems from isolated predictors and notes that practical deployment has outpaced principled frameworks for analysing their failure modes.Source: §1 · p. 1
  • Propagation tracks how an uncertainty signal’s scope, form, consumer, and decision role change as it crosses boundaries, unlike estimation, calibration, or communication alone.The organizing question is the signal’s cross-boundary trajectory and downstream reuse.Source: §1 · pp. 1–2
  • The survey classifies propagation by consumption locus: P1 is within a model-facing request, P2 is technical-component use, and P3 is use by people or organisations beyond the deployed system.The system boundary excludes end users, organisational processes, and real-world outcomes; a single model-facing request includes internal generation, scoring, or sampling resolved before a non-model component acts.Source: §1 · p. 2

Organisation

  • The survey contributes a boundary-analysis framework, a P1–P3 taxonomy, cross-cutting engineering insights, and five open research challenges.The paper’s sections develop the framing, taxonomy, engineering implications, challenges, and concluding claims in that order.Source: §1 · pp. 2–3, Organisation · p. 3

Positioning relative to adjacent surveys

  • Adjacent surveys cover uncertainty estimation, compound-system architectures, and reliability methods, but do not make uncertainty’s cross-boundary reuse their primary organizing question.This survey complements that work by examining how signals are carried forward, re-expressed, retained, and consumed across model, system, and socio-technical boundaries.Source: Positioning relative to adjacent surveys · p. 3

2 Conceptual Framing for Uncertainty Propagation

The conceptual framing treats uncertainty as a contextualised statement, not an isolated score: its meaning depends on what it concerns, how it guides action, where it is available, and who expresses and consumes it. The trace illustrates how those fields can change as a signal moves from a model request through system control to human review and governance.

2 Conceptual Framing for Uncertainty Propagation

  • The tuple u = ⟨s, o, d, r, (a, c)⟩ represents a contextualised uncertainty statement: signal, object, decision role, reuse locus, attribution, and consumer.It is a descriptive abstraction for comparing heterogeneous systems, not a full epistemic model.Source: §2 · p. 3

2.1 Uncertainty signals and their scope

  • The signal field s records what is available for downstream use, from probabilities and sample disagreement to diagnostics, scores, or richer intervals and credal sets.Signals may be scalar, interval, label, disagreement pattern, or structured diagnostic; aggregation or re-expression can bridge the scope at which a signal is produced and the level at which a decision is made.Source: §2.1 · pp. 3–4

2.2 Object of concern

  • The object field o identifies what the uncertainty is treated as being about, including an output artefact, a decision prerequisite, or operating conditions and controls.The object can shift during propagation even when the signal itself does not change.Source: §2.2 · p. 4

2.3 Decision role

  • The decision-role field d captures how uncertainty is used at a decision point, distinguishing diagnosis, runtime control, cross-run adaptation, communication, and assurance support.The same signal may suit one role but not another; roles attach to use events and can change independently of the object of concern.Source: §2.3 · p. 4

2.4 Reuse locus

  • The reuse locus r records where an uncertainty statement becomes actionable: model-internal, system-held, human-facing, or an assurance artefact.P1–P3 placement follows the highest locus reached, so a statement not externalised to system state cannot steer P2 execution, and one not surfaced to people or assurance artefacts cannot support P3 governance.Source: §2.4 · pp. 4–5

2.5 Attribution and consumption

  • Attribution a identifies whose uncertainty is expressed, while consumer c identifies who acts on it; these actors need not be the same or the signal’s computing component.A model-derived score can be attributed to the model and consumed by the system, while a system summary may later be consumed by a human or assurance actor.Source: §2.5 · p. 5

2.6 Illustrative Trace of Uncertainty Propagation

  • In the trace, an internal model confidence signal is externalised into system-held state, changing its reuse locus and consumer so it can steer branching, abstention, or escalation.The illustrative trace starts from model confidence over a drafted response and re-expresses it as a system-usable control signal.Source: §2.6 · p. 5
  • When weak support reaches a human warning or governance record, the object and decision role can shift, while the trace’s reuse locus advances from model-internal to system-held, human-facing, and assurance.The human warning concerns whether the decision can proceed given missing prerequisites; an aggregated governance record instead supports assurance about operating conditions.Source: §2.6 · pp. 5–6

3 Taxonomy of Uncertainty Propagation in LLM Systems

The taxonomy classifies recurring uncertainty-propagation patterns by the fields that change, the boundary crossed, and downstream use. It separates mechanisms within a model-facing request, across the deployed technical system, and beyond that system.

3 Taxonomy of Uncertainty Propagation in LLM Systems

  • Propagation patterns are classified by which fields of u = ⟨s, o, d, r, (a, c)⟩ change, which boundary is crossed, and how the resulting statement is used downstream.Figure 2 provides a high-level overview of the taxonomy.Source: §3 · p. 6
  • P1 covers propagation within a single model-facing request; P2 covers components and execution steps within the deployed technical system.P1–P2 distinctions follow where uncertainty statements are taken up after production.Source: §3 · p. 6
  • P3 covers propagation beyond the technical system, where users, operators, or organisational and governance processes take up uncertainty statements.The taxonomy distinguishes recurring propagation classes within each level.Source: §3 · p. 6

3.1 Within-request uncertainty propagation (P1)

P1 covers uncertainty that stays within a single model-facing request, including signals carried across internal positions, transformed into new proxies, or used to control inference.

3.1.1 Uncertainty transition across sequential internal positions (P1.1)

  • P1.1 tracks how uncertainty signals evolve across ordered internal positions, distinguishing temporal, structural, and intervention-conditioned transitions.These transitions remain within one request; their analysis is distinct from transforming a signal into a new proxy or consuming it to control computation.Source: §3.1.1 · p. 7
  • Depth-wise carry-over describes stage-to-stage changes in signals, while depth-wise dominance identifies layers or modules that disproportionately shape their trajectory.Treating all stages as equally influential can misrepresent the dynamics.Source: §3.1.1 · p. 8
  • Intervention-conditioned transition treats divergence between a baseline and deliberately perturbed variants under the same request as the uncertainty signal.The perturbation can probe continuation sensitivity or attribute downstream divergence to a source or interaction.Source: §3.1.1 · pp. 8–9

3.1.2 Uncertainty transformation across abstraction levels (P1.2)

  • P1.2 re-expresses uncertainty within the request by aggregating local traces, reading out internal features, or consolidating parallel realisations.Multi-realisation methods vary in how samples are generated and whether their uncertainty is reduced to a scalar or retained in interval- or set-valued form.Source: §3.1.2 · pp. 9–10
  • Aggregation combines local token- or step-level signals into a coarser proxy, with the combination rule and aggregation scope as distinct design choices.The resulting proxy can be defined at token, span, entity, or response level.Source: §3.1.2 · p. 9

3.1.3 Uncertainty-conditioned inference control (P1.3)

  • P1.3 uses uncertainty as a within-request control input to alter computation, including whether generation continues, how much compute is allocated, or how the next token is selected.Documented patterns include uncertainty-gated stopping and adaptive compute reduction or expansion.Source: §3.1.3 · p. 11

3.2 System-level uncertainty propagation across the technical system (P2)

P2 examines how uncertainty becomes system-held state that technical components carry forward, use to control actions, re-express across boundaries, or retain across runs. It covers how uncertainty fields may be preserved, transformed, or re-specified, and how persistent state can shape later decision logic or the artefact pool inherited by future runs.

3.2 System-level uncertainty propagation across the technical system (P2)

  • P2 begins when uncertainty leaves model-internal computation and enters system-held state, where component boundaries may preserve, transform, or silently re-specify its fields.The taxonomy distinguishes carry-forward within a workflow, action control, boundary re-expression, and cross-run adaptation.Source: §3.2 · p. 12

3.2.1 Uncertainty carried across workflow steps (P2.1)

  • P2.1 carries uncertainty as workflow state across steps on a realised run path, updating it rather than estimating each stage independently.Carry-over may follow a single path or separate branches; branch signals are later reconciled by aggregation into a post-merge score or selection of one surviving candidate.Source: §3.2.1 · pp. 12–13

3.2.2 Uncertainty-guided workflow control (P2.2)

  • P2.2 turns diagnostic uncertainty into control that changes the next system action through planning and search, route selection, or assistance and escalation gates.Assistance gates withhold autonomous continuation pending clarification, confirmation, or additional input, including when next actions or required arguments remain ambiguous.Source: §3.2.2 · pp. 13–15

3.2.3 Uncertainty re-expression across component boundaries (P2.3)

  • P2.3 re-expresses uncertainty for downstream components through tool or environment interfaces, model handoffs, structured intermediates, or inter-agent messages.Boundary changes can shift the signal’s form and consumer, while silently re-specifying its object or decision role; preserving scope and structure-bound attachment helps prevent overclaiming or flattening.Source: §3.2.3 · pp. 15–17

3.2.4 Cross-run uncertainty propagation through system adaptation (P2.4)

  • Across runs, P2.4 retains uncertainty in persistent system state so it changes later executions, unlike P2.1–P2.3, which concern propagation within one run.The signal may be stored, aggregated, or translated into an update rule that changes persistent decision logic or the retained artefact pool.Source: §3.2.4 · p. 17
  • Cross-run uncertainty changes either persistent decision logic or the artefacts available to later runs, rather than only affecting a current execution.Decision-logic adaptation revises rules for later runs; artefact curation changes the persistent pool of examples, labels, rewards, outputs, or supervision.Source: §3.2.4 · pp. 17–18
  • Decision-logic adaptation uses retained uncertainty to revise future training, selection, validation, acceptance, review, or escalation controls.Learning-based methods use reward, preference, or trajectory uncertainty to constrain updates or reweight data [54] [84]; validation evidence configures controls such as model-tier selection, gates, audits, and escalation triggers [85].Source: §3.2.4 · pp. 17–18
  • Artefact curation uses uncertainty to retain representative, diverse examples or exclude unreliable artefacts from the pool inherited by future runs.Coverage-aware selection supports annotation, feedback, or later training, while reliability-gated filtering removes unreliable reward feedback, training pairs, generated summaries, or synthetic outputs [54] [84].Source: §3.2.4 · p. 18

3.3 Socio-technical uncertainty propagation beyond the technical system (P3)

P3 tracks uncertainty whose main effect occurs beyond the deployed technical system, through uptake by people, organisations, or institutions. It distinguishes human response to communicated uncertainty from governance use of documented uncertainty.

3.3 Socio-technical uncertainty propagation beyond the technical system (P3)

  • P3 concerns uncertainty taken up outside the deployed system, where it can shape reliance, verification, organisational decisions, or governance actions.If externally taken-up uncertainty is later fed back into the system, the outward segment is P3 and the return is a new P2 episode.Source: §3.3 · p. 18
  • P3 distinguishes human uptake of communicated uncertainty from institutional uptake of documented uncertainty used in assurance and governance.The former concerns immediate interpretation of outputs; the latter concerns evidence used to justify, qualify, or constrain system use.Source: §3.3 · p. 18, §3.3.1 · p. 18, §3.3.2 · p. 19

3.3.1 Human uptake of communicated uncertainty (P3.1)

  • P3.1 follows human-interpretable uncertainty as it affects people’s reliance, verification, attention, or oversight of system outputs.Disclosure may use natural-language hedges or interface visualisations; automated actions triggered by a display belong in P2 when their main effect is inside the system.Source: §3.3.1 · pp. 18–19

3.3.2 Governance and assurance uptake of documented uncertainty (P3.2)

  • P3.2 treats documented uncertainty as governance evidence used to qualify system claims or inform operating constraints and authorisation decisions.Uncertainty’s object of concern, estimation conditions, and limitations must remain visible; merely logging or monitoring uncertainty within the system is P2, not P3.2.Source: §3.3.2 · p. 19

4 Insights and implications

Uncertainty propagation matters through consequential reuse, especially at handoffs, control points, and persistence points where meaning or scope can change. The survey therefore calls for engineering uncertainty for downstream consumers and evaluating its propagation across system and governance boundaries.

4.1 Cross-cutting insights

  • Handoff, control, and persistence points are consequential because signals can be reformulated, translated into action, or retained to shape later behaviour.These sites are natural targets for engineering intervention and evaluation.Source: §4.1 · p. 20
  • Propagation failures are often semantic: signals can drift from their original object, support stronger decisions than warranted, or become overgeneralised into system-wide guarantees.Local signal quality alone will miss failures in preserving intended scope and use.Source: §4.1 · p. 20
  • P1 mechanisms are best developed, P2 moderately covered, and P3 substantially thinner, leaving consequential human and institutional uptake least understood.The paper links this asymmetry to the greater difficulty of empirically studying socio-technical outcomes.Source: §4.1 · p. 20
  • Richer P1 signals preserve distinctions that scalar scores collapse, but can be harder to encode in forms downstream components can use.Interface design must therefore be treated as an engineering concern alongside estimation quality.Source: §4.1 · p. 20
  • Uncertainty-conditioned adaptations can reshape artefacts and decision logic across runs, creating feedback loops that single-run or fixed-set evaluations miss.Accumulated adaptations may shift deployment behaviour despite snapshot calibration.Source: §4.1 · pp. 20–21

4.2 Implications: from estimating uncertainty to engineering uncertainty

  • System design should preserve a signal’s object and decision role at handoffs, match it to control decisions, and govern what uncertainty persists in state.Uncertainty belongs in the system interface contract, not as incidental metadata.Source: §4.2 · p. 21
  • Evaluation should follow uncertainty through handoffs, control, persistence, and disclosure, testing interpretability, justified action, drift, and appropriate reliance.Cross-run mechanisms require longitudinal evaluation designs.Source: §4.2 · p. 21
  • Assurance should preserve uncertainty’s scope, provenance, and estimation conditions so decision-makers can judge what evidence does and does not support.Uncertainty should constrain assurance claims rather than merely accompany them.Source: §4.2 · p. 21

5 Open challenges and future directions

The research agenda centres on preserving uncertainty’s meaning across boundaries, evaluating propagation rather than only local estimates, and specifying how signals should shape technical and socio-technical decisions. It also calls for formal foundations suited to heterogeneous system boundaries.

5.1 Preserving semantic fidelity across boundaries

  • Preserving semantic fidelity requires preventing semantic drift, object drift, and role mismatch when uncertainty signals are transformed or reused.Signals meaningful locally may be consumed downstream for a different object, abstraction level, or purpose.Source: §5.1 · pp. 21–22

5.2 Evaluating propagation quality

  • Propagation evaluation should test whether uncertainty remains interpretable and scoped after reuse, while observability records transformations, assumptions, and downstream consumers.The agenda joins boundary-aware evaluation with tracing and documentation across connected technical and socio-technical boundaries.Source: §5.2 · p. 22

5.3 Operationalising propagated uncertainty into system decisions

  • Systems need explicit uncertainty-to-action semantics specifying when signals support acceptance, deferment, refinement, or refusal.Proposed work includes comparative system studies, decision criteria, and workflow contracts for acceptance and execution commitment.Source: §5.3 · pp. 22–23

5.4 Beyond the system boundary: human and institutional uptake

  • Human uptake research needs studies of reliance, accuracy, expertise, stakes, and modality, while institutional uptake needs guidance for using uncertainty in assurance and governance.The human evidence base is clearer and more cumulative than the thinner institutional evidence; multi-party interaction structures are also underexplored, with preliminary evidence that communication topology can amplify, damp, or distort uncertainty [94].Source: §5.4 · p. 23

5.5 Formal foundations for uncertainty propagation

  • Formal foundations must establish when calibration and scope survive aggregation, handoffs, transformation, re-expression, and retention across heterogeneous boundaries.Existing guarantees described here apply within a single formalism over a homogeneous state space, leaving cross-component extension open.Source: §5.5 · pp. 23–24

6 Conclusion

The survey frames uncertainty in compound LLM deployments as a boundary problem: signals are transformed and reused across technical and socio-technical contexts. Its taxonomy reveals uneven understanding across those regions and motivates engineering uncertainty across its lifecycle.

6 Conclusion

  • Uncertainty in contemporary LLM deployments is transformed and reused across model internals, workflows, component boundaries, persistent state, and socio-technical processes.The central question is whether meaning, scope, and decision relevance survive reuse across boundaries.Source: §6 · p. 24
  • The survey organises propagation into P1 intra-model, P2 system-level, and P3 socio-technical regions, finding model-internal mechanisms better understood than those affecting human and institutional decisions.Signals may be re-expressed, combined, retained, disclosed, or documented as they cross boundaries.Source: §6 · p. 24
  • The field’s challenge is to keep propagated uncertainty interpretable, decision-appropriate, and evidentially bounded through representations, evaluation, observability, control, and assurance.The survey concludes that uncertainty should be engineered across the system lifecycle, not treated only as a local estimate.Source: §6 · p. 24

Related papers

Loading 2604.23505v1…