PaperlayerView exact revision on arXiv

Source-linked AI summary

A Mathematical Theory of Pragmatic Information

Kai Niu, Ping Zhang

arXiv:2609.10986v1cs.ITcs.AIcs.ROeess.SY

TL;DR

Information used to guide decisions and actions matters because those choices determine system performance. The paper develops a pragmatic information framework and derives a cross-layer Lagrangian form balancing pragmatic utility and information cost.

Abstract

from arXiv · show

We propose a pragmatic information theory unifying communication, control, and decision-making. Its core is the isoteleia mapping, formalizing equifinality: distinct semantic paths leading to the same optimal action are pragmatically equivalent. This induces a three-tier hierarchy of syntactic, semantic, and pragmatic information, each abstraction discarding task-irrelevant distinctions. We develop pragmatic entropy, up/down mutual information, channel capacity, and rate-distortion, and prove three coding theorems generalizing Shannon's classical results. We introduce pragmatic value (VoI) and cost (CoI) of information as decision-theoretic duals to rate-distortion and capacity, respectively, and formulate a Lagrangian dual framework for cross-layer optimization. The pragmatic efficiency bound $\mathcal{E}_p(λ)=\sup_R[Φ_p(R)-λ\,\mathrm{CoI}_p(R)]$ quantifies the maximum net utility any resource-constrained intelligent system can extract, thereby establishing a fundamental behavioral capacity limit---generalizing Shannon's symbol-level capacity to goal-directed action. Extensions to continuous messages yield closed-form Gaussian expressions, while dynamic settings are addressed via a Bellman equation for sequential decision-making. This framework provides a rigorous foundation for task-oriented communication, networked control, autonomous systems, and embodied AI, shifting focus from symbol fidelity to the effectiveness of information in guiding actions, and offers a unified mathematical language for next-generation intelligent systems.

I. INTRODUCTION · A. Background and Motivation · B. Contributions and Organization

The introduction motivates a pragmatic information framework that unifies communication, control, and decision-making by evaluating information through its guidance of effective action rather than symbol fidelity. It presents syntactic, semantic, and pragmatic information and connects information-theoretic measures to decision value, resource cost, coding, and communication-control decisions, including continuous and dynamic settings.

A. Background and Motivation

  • Communication, control, and decision theory developed largely independently, while related information-theoretic control work emphasized signal fidelity or action complexity rather than pragmatic effects.The introduction identifies limited cross-fertilization among the three fields and notes that prior work focused on syntactic information or action complexity rather than physical communication resources.Source: A. Background and Motivation · pp. 2, 4
  • Semantic information theory formalized synonymy and developed entropy, mutual-information, channel-capacity, and rate-distortion concepts with corresponding coding theorems.Its coding theorems demonstrated performance gains over the preceding approach.Source: A. Background and Motivation · p. 3
  • Prior work lacked a unified framework for pragmatic information that could quantify, compress, and reliably transmit it in engineered communication systems.Earlier quantitative work focused on action-distribution changes in biological evolution, without establishing coding theorems, rate-distortion limits, or channel-capacity characterizations.Source: A. Background and Motivation · p. 3
  • Task-oriented communication systems transmit task-relevant semantic features, moving beyond bit-level fidelity toward meaning-level effectiveness.The introduction notes performance improvements in image transmission, speech communication, and text understanding, and describes systems that prioritize the end goal over exact symbol reconstruction.Source: A. Background and Motivation · pp. 3–4
  • In perception-communication-decision-execution loops, information guides decisions and actions, so its pragmatic dimension is tied to system performance.The introduction presents this loop as a shared structure across information-driven systems.Source: A. Background and Motivation · p. 4
  • An LLM response is valuable through its help toward a user’s goal, while the same sensor readings can imply different driving actions depending on context.For autonomous driving, the relevant pragmatic information is the optimal action class, which supports safe and efficient navigation.Source: A. Background and Motivation · pp. 4–5
  • The same sensory input can support different actions depending on the task goal, making pragmatic information context-dependent and measurable by action effectiveness.The introduction describes pragmatic information as the bridge between perception and effective physical action.Source: A. Background and Motivation · p. 6
  • Across AI, autonomous systems, robotics, and human cognition, information flows through a PCDE loop, and success depends on the effectiveness of resulting actions.The introduction identifies a lack of a unified mathematical framework for analyzing and optimizing pragmatic information across these domains.Source: A. Background and Motivation · pp. 7–8

B. Contributions and Organization

  • The theory unifies communication, control, and decision-making through a three-tier information hierarchy, with isoteleia grouping distinct semantic paths that lead to the same optimal action.It treats classical and semantic information theories as special cases within the broader framework.Source: B. Contributions and Organization · pp. 8–9
  • Pragmatic information concerns effectiveness in guiding action, while reification bridges pragmatic purposes to syntactic signals and the PCDE loop connects information to physical action.The framework subsumes semantic and classical information theories as special cases when the corresponding mappings are trivial.Source: B. Contributions and Organization · pp. 9–10
  • The contribution overview names pragmatic channel coding, static joint optimization, pragmatic source coding, and pragmatic source-channel coding theorems.It also identifies a unified pragmatic information system framework.Source: B. Contributions and Organization · p. 10

Information-Theoretic Measures for Pragmatic Information

  • It defines pragmatic entropy, up/down mutual information, channel capacity, and rate-distortion, relating them hierarchically to semantic and syntactic counterparts.The paper also identifies performance gains achievable through pragmatic abstraction.Source: B. Contributions and Organization · p. 11

Pragmatic Value of Information and Pragmatic Cost of Information

  • Pragmatic value of information measures utility gains from task-relevant information, while pragmatic cost measures the minimum resources needed to convey it at a given rate.The paper establishes dualities between value and rate-distortion, and between cost and channel capacity, with single-sided extensions for perceptual and expressive paths.Source: B. Contributions and Organization · p. 11

Lagrangian Dual Framework and Pragmatic Shadow Price for Cross-Layer Optimization

  • A unified Lagrangian framework combines rate-distortion/value and capacity/cost dualities, yielding a pragmatic efficiency bound as behavioral capacity for resource-constrained systems.The framework shifts the capacity focus from symbol fidelity to how effectively information guides actions.Source: B. Contributions and Organization · p. 11

Coding Theorems for Pragmatic Communication

  • Three coding theorems set minimum lossless and lossy pragmatic rates to pragmatic entropy and rate-distortion, respectively, and maximum reliable transmission rate to pragmatic capacity.The paper presents these theorems as generalizations of classical and semantic coding theorems.Source: B. Contributions and Organization · p. 12

Continuous-Domain Pragmatic Information Measures

  • For continuous messages, the paper introduces isoteleic volumes and derives closed-form Gaussian pragmatic capacity and rate-distortion expressions, along with corresponding cost and value functions.It also derives band-limited capacity results.Source: B. Contributions and Organization · p. 12

Joint Optimization of Communication, Control, and Decision

  • Joint communication, control, and decision optimization yields a posterior-maximizing Bayes rule and marginal value-cost balance in static settings, and a Bellman equation for dynamic settings.The dynamic formulation supports co-design of communication and control in sequential decision-making systems.Source: B. Contributions and Organization · p. 12

Generalization and Unification

  • The framework subsumes classical communication, closed physical, embodied and interactive, and open social and cognitive systems, serving as a reference model for goal-directed information processing under uncertainty.These settings differ in whether utility functions are explicit, implicit, or evolving.Source: B. Contributions and Organization · pp. 12–13

II. PRAGMATIC INFORMATION SYSTEM AND ISOTELEIA MAPPING … 3) Axiomatic Foundations:

This section presents a pragmatic information system unifying communication and control across syntactic, semantic, and pragmatic tiers, with isoteleia treating distinct semantic paths to the same optimal action as equivalent. It grounds the framework in utility-driven action and Markov dynamics and applies a shared information-processing model from classical communication to open social and cognitive settings.

II. PRAGMATIC INFORMATION SYSTEM AND ISOTELEIA MAPPING

  • The three-tier hierarchy separates syntactic, semantic, and pragmatic information, while isoteleia groups distinct semantic paths that lead to the same optimal action.Source: §II · p. 13

A. Notation Conventions

B. Pragmatic Information System

  • The PCDE system is a closed loop that integrates Shannon communication with cybernetic control so received information guides terminal actions maximizing task utility.Source: B. Pragmatic Information System · p. 15

1) System Structure and Processing Flow:

  • Ten modules form a sequential perception–communication–decision–execution cascade, with pragmatic symbols specifying optimal action classes, semantic symbols justifying them, and syntactic symbols transmitting them.Reification composes isoteleia, mapping each pragmatic symbol to semantic justifications, with synonymy, mapping each semantic symbol to syntactic realizations.Source: 1) System Structure and Processing Flow · pp. 15–16

2) Functional Subsystems:

  • The Cognitive-Regulation Subsystem handles intent and information-value evaluation, while the Symbolic-Conduction Subsystem handles encoding, transmission, and decoding; reification bridges them.The CRS groups intent generation, pragmatic representation, destination reconstruction, and execution; the SCS carries meanings through syntactic and physical-signal stages.Source: 2) Functional Subsystems · p. 18

3) Axiomatic Foundations:

  • The probability axiom models observations, states, and intentions as uncertain variables with probability distributions, including a channel conditional distribution P(Yt|Xt).This provides the probabilistic basis for information-theoretic quantities including entropy, mutual information, channel capacity, rate-distortion, and VoI/CoI.Source: 3) Axiomatic Foundations · p. 19
  • Utility maximization groups semantic states that yield the same optimal action, grounding isoteleia, VoI, and the trade-off formalized by the Pragmatic Lagrangian Functional.The utility function assigns scalar reward or cost to state-action pairs; VoI measures utility gain from additional information.Source: 3) Axiomatic Foundations · p. 19
  • The Markov property permits POMDP modeling and temporal decomposition into a static communication and coding problem, enabling coding theorems to characterize transmission limits.The decomposition treats time steps independently while preserving the pragmatic system’s overall structure.Source: 3) Axiomatic Foundations · pp. 19–20

Case I: Classical Communication Systems

  • When utility depends only on symbol fidelity, the mappings become identities and classical source coding, channel coding, and rate-distortion theorems emerge as special cases.The three-tier source then collapses to a syntactic source and source-channel separation architecture.Source: 3) Axiomatic Foundations · p. 20

Case II: Closed Physical Systems

  • In closed physical systems, VoI quantifies the trade-off between communication resources and closed-loop performance, while isoteleia groups sensory inputs yielding the same control or navigation command.These systems have explicit measurable utility and strict physical and temporal constraints.Source: 3) Axiomatic Foundations · pp. 20–21

Case III: Embodied and Interactive Systems

  • For embodied and interactive systems, isoteleia captures distinct perceptual or contextual states leading to the same joint action, while VoI assesses information sharing under asymmetric information and conflicting objectives.The framework also models signaling games, negotiation protocols, and shared intentionality.Source: 3) Axiomatic Foundations · p. 21

Case IV: Open Social and Cognitive Systems

  • For open social and cognitive systems, the framework models information exchange without assuming complete rationality, common knowledge, or stable utility functions.Isoteleia captures generalization across experiences, and VoI concerns communication that improves decisions under subjective, evolving objectives.Source: 3) Axiomatic Foundations · p. 21

The Unity of the Framework

  • Across the four cases, a common core combines state space, utility, a three-tier information hierarchy, communication constraints, and VoI within a unified reference model spanning physical to social systems.The framework presents the cases as a continuum rather than a rigid partition.Source: 3) Axiomatic Foundations · pp. 21–22

C. Design Principles of Pragmatic Information Systems … 6) Quotient-Space Structure:

This section presents design principles for pragmatic information systems centered on utility and effective actions, exploiting semantic equivalence while preserving compatibility with classical communication and control. It defines mappings among pragmatic purposes, semantic meanings, and syntactic signals, and characterizes the three layers as successive quotients that discard distinctions irrelevant to meaning or action.

C. Design Principles of Pragmatic Information Systems

  • Pragmatic information is evaluated through utility, so systems should prioritize the quality of resulting actions over symbol-reconstruction fidelity.Communication resources should be allocated according to the marginal Value of Information they provide.Source: C. Design Principles of Pragmatic… · p. 22
  • Semantic synonymy allows different syntactic realizations of the same meaning, supporting lossy syntactic compression when semantic content is preserved.The design principle also calls for contextual decoding and transmitting task-relevant semantic features rather than reconstructing the original signal.Source: C. Design Principles of Pragmatic… · p. 23
  • The framework is a strict generalization that recovers Shannon communication when semantic and pragmatic mappings are identities and reduces to control theory when communication is ignored.Its three layers should be modular and independently analyzable, optimizable, or bypassed.Source: C. Design Principles of Pragmatic… · p. 23

D. Pragmatic Information and Isoteleia Mapping

1) The Nature of Pragmatic Information:

  • Pragmatic information concerns what information achieves: messages with different symbols or meanings are equivalent when they lead to the same optimal terminal action.This equifinality principle, Isoteleia, makes the receiver’s goal guidance for action rather than exact message reconstruction.Source: 1) The Nature of Pragmatic Information · p. 24

3) Formal Definitions of the Mappings:

  • The synonymous mapping associates each semantic symbol with a set of syntactic realizations, expressing one-to-many correspondence from meaning to symbols.Source: 3) Formal Definitions of the Mappings · p. 25
  • Isoteleia groups semantic meanings yielding the same optimal action, while reification composes it with synonymy to connect a purpose to its syntactic realizations.For sequential decisions, the dynamic mapping accounts for future consequences; when γ = 0, it recovers the static definition.Source: 3) Formal Definitions of the Mappings · pp. 26–27

4) The Five Relations Between Layers:

  • The mappings relate one pragmatic class to multiple semantic classes and one meaning to multiple syntactic symbols, with inverse mappings assigning each lower-layer class uniquely.Each semantic class must be homogeneous with respect to optimal action; otherwise, it is split into action-homogeneous classes before isoteleia is uniquely defined.Source: 4) The Five Relations Between Layers · p. 27

6) Quotient-Space Structure:

  • The syntactic, semantic, and pragmatic layers are successive quotients of the same signal set, discarding distinctions irrelevant to meaning and then to optimal action.The pragmatic quotient is the coarsest, most task-relevant description and provides the mathematical foundation for the subsequent information-theoretic development.Source: 6) Quotient-Space Structure · p. 28

III. PRAGMATIC ENTROPY · A. Formal Definition of Pragmatic Entropy · B. Pragmatic Joint Entropy and Conditional Entropy

This section defines pragmatic entropy as uncertainty relevant to optimal action and develops its joint, conditional, and sequential forms. It describes how task-irrelevant distinctions are abstracted away, yielding chain-rule inequalities and entropy bounds across pragmatic, semantic, and syntactic representations.

A. Formal Definition of Pragmatic Entropy

  • Pragmatic entropy measures uncertainty about the optimal terminal action, grouping semantic symbols that induce the same optimum and quantifying irreducible uncertainty to resolve through communication and control.Its unit is the pragmatic bit (Prabit).Source: A. Formal Definition of Pragmatic… · pp. 28–29
  • For an alphabet of size L, 0 ≤ Hp(W) ≤ log L; the lower bound occurs with one pragmatic symbol and the upper bound with a uniform distribution.Source: A. Formal Definition of Pragmatic… · pp. 29, 31
  • Entropy follows Hp(W) ≤ Hs(Ŵ) ≤ H(W); equality between pragmatic and semantic entropy holds when the isoteleia mapping is injective.Each abstraction discards distinctions irrelevant at its level, with the pragmatic layer discarding distinctions that do not affect the optimal terminal action.Source: A. Formal Definition of Pragmatic… · p. 30
  • In the vehicle example, H(W) ≈ 1.295 bits, Hs(Ŵ) ≈ 0.469 sebits, and Hp(W) ≈ 0.469 prabits.The semantic and pragmatic entropies match because each semantic class maps to a distinct pragmatic class, while syntactic entropy is higher.Source: A. Formal Definition of Pragmatic… · p. 32
  • In the traffic-light example, a decision requires distinguishing “Go” from “Stop,” not green from yellow, illustrating entropy as the minimum decision uncertainty communication and control must resolve.The measure is presented as a foundation for resource allocation and system design in pragmatic information systems.Source: A. Formal Definition of Pragmatic… · p. 33

B. Pragmatic Joint Entropy and Conditional Entropy

  • The joint Isoteleia mapping groups semantic pairs that induce the same pair of optimal actions, and pragmatic joint entropy is the Shannon entropy of the resulting joint pragmatic distribution.This extends pragmatic entropy to dependencies and information flows between components of a system.Source: B. Pragmatic Joint Entropy and… · pp. 33–34
  • Pragmatic conditional entropy has syntactic- and pragmatic-conditioning variants: the former measures residual decision uncertainty after the exact signal, the latter uncertainty given the other decision.The choice of conditioning variable depends on the application.Source: B. Pragmatic Joint Entropy and… · pp. 34–35
  • Pragmatic joint entropy lies between marginal pragmatic entropy plus conditional uncertainty given the syntactic signal and syntactic entropy plus that same conditional term.Unlike classical entropy, pragmatic entropy does not satisfy an exact additive chain rule; its inequalities reflect information coarsening.Source: B. Pragmatic Joint Entropy and… · pp. 35–36
  • Joint entropy is ordered from pragmatic to semantic to syntactic, with equality at the pragmatic-semantic step only for an injective Isoteleia mapping and at the semantic-syntactic step only for an injective synonymous mapping.These conditions respectively mean no distinct semantic pairs merge into one pragmatic pair, and no distinct syntactic pairs share the same semantic meaning.Source: B. Pragmatic Joint Entropy and… · pp. 36–37
  • The sequential pragmatic chain rule establishes inequalities relating full-sequence entropy to partial entropies and syntactic entropy.The proof iteratively applies the binary chain rule to adjacent variables, replacing pragmatic conditioning with syntactic conditioning.Source: B. Pragmatic Joint Entropy and… · p. 38
  • Across sequences, pragmatic joint and partial conditional entropies are no greater than semantic entropy, which is no greater than syntactic entropy.The hierarchy follows from the deterministic mappings between syntactic, semantic, and pragmatic sequences.Source: B. Pragmatic Joint Entropy and… · p. 39
  • The hierarchy interprets syntactic entropy as retaining all symbol distinctions, semantic entropy as removing meaningless variations, and pragmatic entropy as removing decision-irrelevant distinctions.Each successive coarsening reduces joint entropy, supporting multi-level source coding and compression.Source: B. Pragmatic Joint Entropy and… · p. 39
  • 3.6318 bits of syntactic joint entropy falls to 2.7565 sebits semantically and 1.7509 prabits pragmatically in the vehicle example.The example also verifies Hp(W) + Hp(V|W) = 1.9051 ≥ Hp(W,V) = 1.7509.Source: B. Pragmatic Joint Entropy and… · pp. 40–41, Table VI · p. 41
  • Merging semantically distinct stop states with the same optimal action makes pragmatic entropy strictly lower than semantic entropy.The short-red and long-red states both imply “Stop,” so their decision-irrelevant distinction can be discarded.Source: B. Pragmatic Joint Entropy and… · p. 41

IV. PRAGMATIC RELATIVE ENTROPY AND PRAGMATIC MUTUAL INFORMATION · A. Pragmatic Relative Entropy

Pragmatic relative entropy evaluates distributional divergence after grouping symbols by the terminal actions they induce, while the section introduces pragmatic mutual information as the corresponding action-level measure of statistical dependence. The relative-entropy subsection defines three aggregation forms and establishes their basic properties and decision-oriented interpretation.

IV. PRAGMATIC RELATIVE ENTROPY AND PRAGMATIC MUTUAL INFORMATION

  • Pragmatic relative entropy and mutual information evaluate divergence and statistical dependence, respectively, at the level of terminal actions rather than symbols or meanings.This frames the measures introduced in the following subsection.Source: §IV · p. 42

A. Pragmatic Relative Entropy

  • Three pragmatic relative entropies aggregate over classes of syntactic symbols that induce the same optimal action: full, partial Type I, and partial Type II.The full form compares pragmatic marginal distributions; Type I compares aggregated p with fine-grained q, while Type II compares fine-grained p with aggregated q.Source: A. Pragmatic Relative Entropy · pp. 42–43
  • Partial pragmatic relative entropy Dp(p∥qp) may be negative, so the authors suggest using its positive part, (Dp(p∥qp))+.Source: A. Pragmatic Relative Entropy · p. 43
  • The full pragmatic divergence Dp(pp∥qp) is nonnegative and equals zero exactly when the two pragmatic aggregated distributions coincide.The pragmatic information inequality also gives equality conditions for comparisons involving the fine-grained distributions.Source: A. Pragmatic Relative Entropy · pp. 43–44
  • Pragmatic coarse-graining reduces divergence when p is fine-grained and q is aggregated, but increases it when p is aggregated and q is fine-grained.The paper also states ordering chains among the three pragmatic forms and relationships with classical and semantic relative entropies.Source: A. Pragmatic Relative Entropy · pp. 44–46
  • Pragmatic relative entropy is convex in the pair of distributions (p, q) for each of its three forms.The proof uses linearity of pragmatic aggregation and joint convexity of classical KL divergence.Source: A. Pragmatic Relative Entropy · p. 45
  • These measures quantify differences between decision-oriented distributions and support comparison of an action-level belief, such as a control policy, with raw observations.Pragmatic abstraction discards distinctions that do not affect the optimal action.Source: A. Pragmatic Relative Entropy · p. 46

B. Pragmatic Mutual Information … 3) Sequential Chain Rules for Pragmatic Mutual Information:

The section develops pragmatic mutual information as complementary measures of decision-uncertainty reduction across levels of coarse-graining, including cases where neither, one, or both sides are abstracted to task-relevant actions. It extends their ordered hierarchies and chain rules to sequences, illustrating their use with a two-vehicle communication example.

1) Definitions of Up and Down Pragmatic Mutual Information:

  • Up pragmatic mutual information adds gains across abstraction layers, while down pragmatic mutual information subtracts losses, with classical mutual information as the reference point.The section also establishes a hierarchy among pragmatic, semantic, and classical mutual-information measures.Source: 1) Definitions of Up and Down… · pp. 48, 50

2) Single-Sided Pragmatic Mutual Information:

  • Four single-sided pragmatic mutual information measures address asymmetric links where only the source or observation is coarsened.The section motivates these cases with perception, expression, and reconstruction tasks.Source: 2) Single-Sided Pragmatic Mutual… · p. 50

1) Perceptual Pragmatic Mutual Information (PPMI):

  • PPMI quantifies how much information a syntactic observation carries about the pragmatic action.It combines the pragmatic entropy of the action with the syntactic observation and joint entropies.Source: 1) Perceptual Pragmatic Mutual… · pp. 50–51

2) Prospective Perceptual Pragmatic Mutual Information (PPPMI):

3) Expressive Pragmatic Mutual Information (EPMI):

  • EPMI quantifies how much information about the syntactic source is contained in the pragmatic label.Its definition uses the pragmatic entropy of the label alongside syntactic source and joint entropies.Source: 3) Expressive Pragmatic Mutual… · p. 51

4) Prospective Expressive Pragmatic Mutual Information (PEPMI):

  • PEPMI provides an upper bound on expressive information, and joins classical and bilateral measures in a lattice covering all coarse-graining combinations.The lattice covers no coarse-graining, coarse-graining on one side, and coarse-graining on both sides.Source: 4) Prospective Expressive Pragmatic… · p. 51
  • Single-sided measures with pragmatic marginal entropy are bounded above by classical mutual information, while those with syntactic marginal entropy are bounded below by it.Perceptual and expressive measures are generally incomparable with each other; either lower-side measure is bounded above by either upper-side measure.Source: 4) Prospective Expressive Pragmatic… · p. 52

3) Sequential Chain Rules for Pragmatic Mutual Information:

  • For sequences, down pragmatic mutual information obeys a chain rule, while up pragmatic mutual information increases through successive partial blocks.Sequential single-sided measures also have chain rules; their up versions reverse the inequalities, and all seven measures retain the single-letter partial order for fixed n.Source: 3) Sequential Chain Rules for… · pp. 52–54
  • The up bilateral pragmatic mutual information is 2.2203 prabits, and the passage identifies this value as one of the reported results before introducing their summary.The supplied passages report PPMI as 0.2458 bits, PPPMI as 1.2386 bits, and EPMI as 0.2458 bits.Source: 3) Sequential Chain Rules for… · pp. 54–55
  • The example's single-sided measures lie between down bilateral and classical mutual information or between classical and up bilateral mutual information, illustrating the lattice structure.PPMI and EPMI are equal in this symmetric example, but the passage notes that they generally differ; single-sided measures quantify information change when only one side is pragmatically coarsened.Source: 3) Sequential Chain Rules for… · p. 55

V. PRAGMATIC CHANNEL CAPACITY AND PRAGMATIC RATE DISTORTION … 3) Prospective Perceptual Pragmatic Achievable Rate:

This section develops pragmatic capacity and rate-distortion limits for reliable communication and task distortion, including measures for asymmetric information flows. It also defines perceptual and expressive rate-distortion functions and a prospective perceptual achievable rate, distinguishing action-focused inference, source reconstruction, and resource-limited extraction of actionable information.

A. Pragmatic Channel Capacity

  • Pragmatic channel capacity measures the maximum rate for reliably conveying terminal actions and can exceed the syntactic bit rate by allowing errors that preserve meaning or action.It is maximized over input distributions and joint reification mappings; the section also gives an additive decomposition of pragmatic capacity gains relative to classical and semantic capacity.Source: A. Pragmatic Channel Capacity · pp. 56–58
  • Pragmatic capacity is at least semantic capacity because reification can merge semantic classes that lead to the same optimal action.The section establishes a hierarchy among pragmatic, semantic, and syntactic capacities.Source: A. Pragmatic Channel Capacity · pp. 56–57

B. Pragmatic Rate Distortion

  • Pragmatic rate-distortion gives the minimum rate for a source under a task-utility-loss threshold, measuring distortion on terminal actions rather than symbols.The section establishes an additive decomposition of pragmatic rate-distortion and states that Rp(D) can be significantly lower than R(D) when errors do not affect the terminal action.Source: B. Pragmatic Rate Distortion · pp. 58–60

C. Single-Sided Pragmatic Achievable Rates and Rate-Distortion Functions

  • For asymmetric links where only one side is pragmatically abstracted, the section introduces perceptual and expressive rate-distortion functions and prospective perceptual and expressive achievable rates.These measures analyze task-oriented information flows when either the observation or the action label is coarsened.Source: C. Single-Sided Pragmatic Achievable… · p. 60

1) Perceptual Pragmatic Rate-Distortion Function:

  • Perceptual pragmatic rate-distortion captures the fundamental limit of compressing an observation V to extract information.Source: 1) Perceptual Pragmatic… · p. 60
  • The perceptual pragmatic rate-distortion function gives the minimum rate needed for a target distortion when estimating the pragmatic action from an observation.It uses distortion on pragmatic symbols and focuses on the receiver’s optimal action class, allowing sensory compression that exploits action equivalence.Source: 1) Perceptual Pragmatic… · p. 61

2) Expressive Pragmatic Rate-Distortion Function:

  • The expressive pragmatic rate-distortion function gives the minimum rate for reconstructing a syntactic source from a pragmatic action label, with distortion measured on syntactic symbols or corresponding semantics.It applies when the action label is the only source information available, as in low-bandwidth actuation or control.Source: 2) Expressive Pragmatic… · p. 61

3) Prospective Perceptual Pragmatic Achievable Rate:

  • The prospective perceptual achievable rate is the maximum pragmatic information extractable from an uncoarsened observation under a resource budget, bounding actionable information conveyed by a sensor.The maximization is over conditional distributions p(v|w) satisfying a cost constraint on the pragmatic source, such as sensing power or bandwidth.Source: 3) Prospective Perceptual Pragmatic… · p. 62

4) Prospective Expressive Pragmatic Achievable Rate: … C. Lagrangian Duality

The section defines a prospective expressive achievable rate under a source-cost constraint and characterizes pragmatic value and cost of information as decision-level measures and duals of rate-distortion and capacity. It combines these ideas in a Lagrangian framework that uses rate to link decision and transmission layers for cross-layer optimization.

4) Prospective Expressive Pragmatic Achievable Rate:

  • The prospective expressive achievable rate is the maximum rate at which a pragmatic action label conveys source information under a resource budget.It optimizes over conditional distributions of the label given the syntactic source, subject to a source cost constraint.Source: 4) Prospective Expressive Pragmatic… · p. 62
  • Coarsening only one side preserves more syntactic distinctions, requiring higher rates for the same distortion and yielding lower achievable rates.Together with bilateral counterparts, these measures support task-oriented communication design in asymmetric systems.Source: 4) Prospective Expressive Pragmatic… · p. 63

VI. PRAGMATIC COST OF INFORMATION AND PRAGMATIC VALUE OF INFORMATION

  • Pragmatic value and cost of information are benefit and resource-cost duals of rate-distortion and capacity, combined in a Lagrangian framework for cross-layer optimization.Source: §VI · p. 63

A. Pragmatic Value of Information

  • Pragmatic value of information measures expected utility gain from observations after coarsening both states and observations to distinctions relevant to optimal actions.Classical value of information is a special case when the isoteleia mappings are identities.Source: A. Pragmatic Value of Information · pp. 64–65
  • Pragmatic value of information is nonnegative, increases weakly under refinement of the observation partition, and its maximum achievable value is non-decreasing and concave in rate.It is zero exactly when the pragmatic observation provides no useful information for improving the decision.Source: A. Pragmatic Value of Information · p. 65
  • Pragmatic VoI is dual to pragmatic rate-distortion through a variational Legendre–Fenchel relationship between the value-rate and rate-distortion functions.The value-rate function is the concave conjugate of the rate-distortion function.Source: A. Pragmatic Value of Information · pp. 66–67
  • ≈0.5 utils is the pragmatic VoI, compared with about 4.2 utils for classical VoI using the full syntactic signal.The pragmatic observation is binary Go/Stop and resolves little decision uncertainty.Source: A. Pragmatic Value of Information · p. 67

B. Pragmatic Cost of Information

  • Pragmatic CoI is the minimum resource needed for a given rate and depends on the channel and pragmatic abstraction, not utility or task.Resources may include power, bandwidth, time, or channel uses.Source: B. Pragmatic Cost of Information · pp. 67, 69
  • Pragmatic capacity is non-decreasing and concave in available resource, while CoI is nonnegative, strictly increasing for positive rates, and convex.CoI is the inverse of the capacity-resource function.Source: B. Pragmatic Cost of Information · pp. 68–70
  • CoI may be convex but not strictly convex when zero-cost capacity plateaus yield linear segments.Strict convexity requires strictly concave capacity and globally strict increase.Source: B. Pragmatic Cost of Information · p. 70

C. Lagrangian Duality

  • The Lagrangian framework combines the value-rate and capacity-cost dual pairs, using rate as the common variable linking decision and transmission sides.This enables cross-layer optimization.Source: C. Lagrangian Duality · p. 70

1) Source-Distortion-Value Lagrangian (Down Loop): … 5) Relationships and Duality Structure:

The section develops a Lagrangian and duality framework linking decision quality, communication rate, information value, capacity, and resource costs across symmetric and asymmetric systems.

1) Source-Distortion-Value Lagrangian (Down Loop):

  • At the down-loop optimum, the marginal value of rate equals its marginal rate cost, balancing reduced distortion against the rate required.The down-loop Lagrangian combines the pragmatic rate-distortion function with value-rate; λd represents the marginal cost of rate in utility units.Source: 1) Source-Distortion-Value Lagrangian… · p. 71

2) Channel-Power-Cost Lagrangian (Up Loop):

  • At the up-loop optimum, the resource shadow price equals the marginal cost of increasing rate, balancing achievable rate against physical resource expenditure.The Lagrangian uses pragmatic capacity and CoIp(R) as the resource cost; γ is the resource shadow price in utility units per unit resource.Source: 2) Channel-Power-Cost Lagrangian (Up… · p. 71

3) Global Cross-Layer Lagrangian and Behavioral Capacity:

  • If pragmatic value is concave and pragmatic cost convex, the global Lagrangian is concave, so a differentiable first-order optimum is globally maximizing.Its rate condition equates marginal value to the resource shadow price times marginal resource consumption per bit.Source: 3) Global Cross-Layer Lagrangian and… · pp. 72–73
  • At the global optimum, the global multiplier equals the up-loop resource shadow price, while down-loop marginal rate cost equals the marginal value of rate.These relationships establish duality between the decision-theoretic and physical-layer perspectives.Source: 3) Global Cross-Layer Lagrangian and… · p. 73
  • The pragmatic efficiency bound is the maximum net utility an agent can extract under resource pricing, serving as behavioral capacity for resource-constrained goal-directed systems.The definition takes the supremum of net benefit over achievable rates and accounts for sensing, communication, and computation costs.Source: 3) Global Cross-Layer Lagrangian and… · pp. 73–74

D. Single-Sided Pragmatic Cost of Information and Pragmatic Value of Information

  • Single-sided pragmatic VoI and CoI capture utility gains and resource expenditures when only one side of a communication link is pragmatically abstracted.They provide finer-grained analysis for asymmetric communication systems.Source: D. Single-Sided Pragmatic Cost of… · p. 75

1) Perceptual Pragmatic Value of Information:

  • Perceptual pragmatic VoI gives the maximum utility gain when an observation carries at most R bits about the pragmatic action, and is non-decreasing and concave in R.It is the dual of perceptual pragmatic rate-distortion and has a variational duality.Source: 1) Perceptual Pragmatic Value of… · p. 75

2) Expressive Pragmatic Value of Information:

  • Expressive pragmatic VoI quantifies the maximum expected utility gain from a pragmatic label when reconstructing the syntactic source, and is dual to expressive pragmatic rate-distortion.Source: 2) Expressive Pragmatic Value of… · p. 76
  • Expressive pragmatic value measures utility gained by representing a source with a pragmatic label under a rate limit; it is non-decreasing, concave in R, and dual to Rexpr.The value is evaluated with respect to the reconstructed source.Source: 2) Expressive Pragmatic Value of… · p. 76

3) Prospective Perceptual Pragmatic Cost of Information:

  • Prospective perceptual pragmatic cost is the minimum resource expenditure to extract a given perceptual pragmatic information rate; it is non-decreasing, convex, and the strict inverse of achievable rate.The source cost can represent sensing power.Source: 3) Prospective Perceptual Pragmatic… · p. 76

4) Prospective Expressive Pragmatic Cost of Information:

  • Prospective expressive pragmatic cost is the minimum resource expenditure to convey a given expressive pragmatic information rate; it is non-decreasing, convex, and the strict inverse of achievable rate.The source cost can represent transmit power.Source: 4) Prospective Expressive Pragmatic… · p. 77

5) Relationships and Duality Structure:

  • The framework pairs value with rate-distortion through Legendre–Fenchel duality, and pairs cost with capacity through the inverse of achievable rate for both perceptual and expressive paths.These are the two fundamental dual pairs in the single-sided framework.Source: 5) Relationships and Duality Structure · pp. 77–78
  • Perceptual measures coarsen the source and expressive measures the reconstruction; prospective variants provide upper bounds on achievable rates, and single-sided measures are bounded by bilateral counterparts.The measures support joint design when sensing and actuation have different resource constraints and utility contributions.Source: 5) Relationships and Duality Structure · pp. 77–79

VII. PRAGMATIC LOSSLESS SOURCE CODING … 2) An encoding function that operates in three stages:

This section develops pragmatic lossless source coding around preserving full decision utility rather than exact sequence reconstruction, using three-tier AEP results and hierarchical typical sets. It establishes pragmatic entropy as the rate threshold for preserving action-relevant distinctions while discarding distinctions that do not affect the optimal action.

3) Properties of the Typical Sets:

1) Three-Tier AEP Theorems:

  • The syntactic, semantic, and pragmatic AEP theorems establish convergence in probability; semantic and pragmatic convergence follows from the weak law applied to aggregated class probabilities.The aggregations are over synonymous and reified equivalence classes, respectively.Source: 1) Three-Tier AEP Theorems · p. 81

2) Definitions of the Hierarchical Typical Sets:

  • The typical-set hierarchy refines syntactic sequences into synonym classes, then groups those classes by pragmatic action, with Hp(W) ≤ Hs(˜W) ≤ H(W).The reified typical set is the union of synonymous typical sets across semantic sequences in the corresponding isoteleia typical set.Source: 2) Definitions of the Hierarchical… · pp. 83–84

B. Pragmatic Lossless Source Coding Theorem

  • Rates above pragmatic entropy are achievable with vanishing error, whereas rates below it fail; the theorem also relates this limit to down pragmatic value of information.The coding architecture transmits only the pragmatic index, allowing arbitrary syntactic and semantic realizations.Source: B. Pragmatic Lossless Source Coding… · p. 87

1) Three-Tier Pragmatic Source Code Model:

  • Because pragmatic equivalence groups realizations by action purpose, decoding may return a representative sequence without reconstructing exact syntactic or semantic details.The decoder selects a syntactic sequence from the corresponding reified typical set; the pragmatic purpose is preserved.Source: 1) Three-Tier Pragmatic Source Code… · p. 87

2) An encoding function that operates in three stages:

  • The encoder selects a semantic interpretation within an action class, a synonymous syntactic realization, and a reification index identifying a sequence in its reified typical set.Equivalently, it can directly map the input sequence to its pragmatic class and encode that index; the reification index is optional for preserving the action.Source: 2) An encoding function that operates… · p. 88
  • Pragmatic lossless coding achieves vanishing error above Hp(W), fails below it, and preserves full VoI if and only if R ≥ Hp(W).The theorem states achievability for R > Hp(W) and a converse for R < Hp(W).Source: 2) An encoding function that operates… · pp. 89–90
  • With no within-class redundancy preserved, total rate approaches Hp(W); at its maximum, it approaches H(W), recovering the classical lossless limit.Intermediate redundancy rates preserve different levels of syntactic detail within each pragmatic equivalence class.Source: 2) An encoding function that operates… · p. 90
  • When R < Hp(W), unresolved uncertainty about the optimal action class makes achievable VoI strictly lower than under perfect pragmatic knowledge.Source: 2) An encoding function that operates… · p. 91
  • Full pragmatic value requires and is achieved at rate R ≥ Hp(W), making Hp(W) the exact minimum rate.Source: 2) An encoding function that operates… · p. 91
  • The entropy hierarchy Hp(W) ≤ Hs(˜W) ≤ H(W) quantifies compression gains from pragmatic coding relative to semantic and syntactic coding.Source: 2) An encoding function that operates… · p. 91
  • The encoding partitions typical sequences into isoteleia sets for the same action, then synonymous sets for the same meaning; additional rate Rp preserves syntactic detail and recovers the classical limit.Source: 2) An encoding function that operates… · p. 91
  • At a given rate, preserving pragmatic equivalence classes maximizes achievable utility by discarding syntactic and semantic distinctions that do not affect the optimal action.Source: 2) An encoding function that operates… · p. 92

VIII. PRAGMATIC CHANNEL CODING · A. Jointly Asymptotic Equipartition Property and Jointly Reified Typical Set

This section states a pragmatic channel coding theorem and develops joint typicality by reifying channel input-output sequences, grouping syntactic pairs by pragmatic equivalence. For independently selected syntactic sequences, pragmatic joint typicality has probability about 2^-nI_p(X;Y), reflecting the additional syntactic freedom when preserving pragmatic purpose.

VIII. PRAGMATIC CHANNEL CODING

  • Pragmatic capacity equals the maximum achievable rate and extends classical capacity by tolerating errors that do not affect pragmatic purpose.The theorem is established using random coding and joint typical decoding.Source: §VIII · p. 92

A. Jointly Asymptotic Equipartition Property and Jointly Reified Typical Set

  • Joint reification composes joint synonymous and joint isoteleia mappings to partition syntactic input-output sequences by pragmatic pair and optimal terminal action.The resulting jointly reified typical set is a union of synonymous typical sets across semantic pairs in the corresponding isoteleia typical set.Source: A. Jointly Asymptotic Equipartition… · pp. 92, 95
  • The pragmatically joint AEP follows by applying the weak law of large numbers to aggregated pragmatic probabilities.The reified typical set's conditions combine syntactic joint typicality, pragmatic joint typicality, and reification typicality.Source: A. Jointly Asymptotic Equipartition… · pp. 95–96
  • A jointly reified typical set has cardinality approximately 2^n(H(X,Y)-Hp(X,Y)), larger than a single synonymous typical set.The reification mapping merges semantically distinct pairs that are pragmatically equivalent, while synonymous mapping merges syntactic differences that do not affect meaning.Source: A. Jointly Asymptotic Equipartition… · p. 98
  • The probability gain reflects additional syntactic freedom when only the pragmatic purpose needs to be preserved.The passage gives a factor of 2^n(H_s(tilde X, tilde Y)−H_p(X,Y)).Source: A. Jointly Asymptotic Equipartition… · p. 98
  • The probability that independently selected syntactic sequences are pragmatically jointly typical is about 2^-nI_p(X;Y), larger than the classical 2^-nI(X;Y).The probability refers to the sequences falling into the same jointly reified typical set.Source: A. Jointly Asymptotic Equipartition… · p. 98

B. Pragmatic Channel Coding Theorem · IX. PRAGMATIC LOSSY SOURCE CODING · A. Pragmatic Distortion and Jointly Typical Set

These sections develop pragmatic coding limits by focusing on action-relevant distinctions, moving from channel capacity to lossy source coding that preserves pragmatic utility. They formalize pragmatic distortion and jointly typical source–reconstruction sets, establishing entropy, set-size, and typicality-probability results at the pragmatic level.

B. Pragmatic Channel Coding Theorem

  • The two-level code assigns an index to an optimal-action class and a reification index to its syntactic codeword; the decoder recovers only the pragmatic index.Random coding and jointly typical decoding use the jointly reified typical set.Source: B. Pragmatic Channel Coding Theorem · pp. 98–100
  • Reliable pragmatic transmission is achievable below capacity Cp, while above Cp the error probability is bounded away from zero.The theorem therefore identifies Cp as the maximum achievable total rate.Source: B. Pragmatic Channel Coding Theorem · pp. 99–100
  • Pragmatic information cost is finite for rates at or below Cp and unbounded above it, making Cp the threshold for reliable communication with finite resources.The theorem connects this threshold to the pragmatic cost of information.Source: B. Pragmatic Channel Coding Theorem · pp. 100, 102
  • By tolerating errors that do not affect the optimal action, pragmatic coding achieves strictly larger capacity.The theorem generalizes classical and semantic channel coding in the trivial-isoteleia and synonymous-mapping cases, respectively.Source: B. Pragmatic Channel Coding Theorem · p. 102

IX. PRAGMATIC LOSSY SOURCE CODING

  • Pragmatic lossy source coding compresses a source while preserving pragmatic utility rather than exact syntactic or semantic content.It extends the asymptotic equipartition property to source-reconstruction pairs at the pragmatic level, using reification mappings and a jointly reified typical set.Source: §IX · pp. 102–103
  • Random coding and jointly typical encoding establish the pragmatic rate-distortion function Rp(D) as the fundamental limit for task-oriented compression.The encoding uses the jointly reified typical set.Source: §IX · p. 103

A. Pragmatic Distortion and Jointly Typical Set

  • Pragmatic distortion measures the utility loss when a pragmatic symbol is represented by a reconstruction, or another nonnegative degradation in decision quality.The construction uses the reification mapping, composed of synonymous and isoteleia mappings, to aggregate syntactic sequences into pragmatic equivalence classes.Source: A. Pragmatic Distortion and Jointly… · p. 103
  • Reification gives the entropy hierarchy Hp(X, Xhat) ≤ Hs(Xtilde, Xhattilde) ≤ H(X, Xhat), reflecting successive syntactic, semantic, and pragmatic coarsening.The isoteleia mapping merges semantic distinctions that do not affect optimal actions, while synonymous mapping merges syntactic realizations of the same meaning.Source: A. Pragmatic Distortion and Jointly… · pp. 105–106
  • For sufficiently large n, a pragmatically jointly typical pair has probability between 2^-n(Hp(X, Xhat)+ε) and 2^-n(Hp(X, Xhat)−ε) under i.i.d. test-channel sampling.The pragmatic joint AEP applies to sequences under the joint reification mapping.Source: A. Pragmatic Distortion and Jointly… · pp. 106–107
  • A jointly reified typical set has cardinality approximately 2^n(H(X, Xhat)−Hp(X, Xhat)), exceeding a synonymous typical set by a factor of 2^n(Hs(Xtilde, Xhattilde)−Hp(X, Xhat)).This additional size reflects the syntactic freedom available when only pragmatic purpose must be preserved.Source: A. Pragmatic Distortion and Jointly… · p. 108
  • For syntactic sequences that fall into the same jointly reified typical set (that is, are pragmatically jointly typical), the probability is about 2^-nIp(X; Xhat), larger than classical 2^-nI(X; Xhat).The passage identifies this difference as reflecting the pragmatic rate-distortion gain.Source: A. Pragmatic Distortion and Jointly… · p. 108

B. Pragmatic Rate-Distortion Coding Theorem

The section states a rate threshold for pragmatic lossy coding, relating achievable distortion to preservation of the reconstruction’s full pragmatic value. It also describes how preserving additional syntactic or semantic detail recovers classical or semantic limits, reflecting a modular approach to pragmatic information-system design.

B. Pragmatic Rate-Distortion Coding Theorem

X. PRAGMATIC INFORMATION MEASURE OF CONTINUOUS MESSAGE … B. Pragmatic Capacity of Gaussian Channel and Pragmatic Cost of Information

This section extends pragmatic information measures to continuous messages and variables, characterizing entropy and mutual information and applying the framework to Gaussian channels and sources. It derives pragmatic capacity and rate-distortion results and relates them to information cost and value, including how bilateral abstraction and isoteleic volume affect the power required to achieve a pragmatic rate.

X. PRAGMATIC INFORMATION MEASURE OF CONTINUOUS MESSAGE

  • For Gaussian channels and sources, the framework derives pragmatic capacity and rate-distortion functions; CoI is the inverse of achievable rates, while VoI is the Legendre-Fenchel dual of rate-distortion.Source: §X · p. 113

A. Pragmatic Entropy and Pragmatic Mutual Information for Continuous Message

  • Quantizing continuous space and taking the limit yields isoteleic volumes for signals sharing an optimal action; the hierarchy H_p ≤ H_s ≤ H persists, and continuous characterizations inherit the discrete coding theorems’ operational meaning.Source: A. Pragmatic Entropy and Pragmatic… · p. 114

1) Pragmatic Entropy for Continuous Messages:

2) Pragmatic Mutual Information for Continuous Variables:

  • When E[log(ΩwΩv)] ≥ 0, up pragmatic mutual information is at least classical mutual information, while down pragmatic mutual information is at most classical mutual information and may be negative.Source: 2) Pragmatic Mutual Information for… · p. 117
  • The continuous-measure extension is applied to derive Gaussian-channel pragmatic capacity and Gaussian-source rate-distortion results with corresponding cost and value measures.Source: 2) Pragmatic Mutual Information for… · p. 118

B. Pragmatic Capacity of Gaussian Channel and Pragmatic Cost of Information

C. Pragmatic Rate Distortion of Gaussian Source and Pragmatic Value of Information · XI. JOINT OPTIMIZATION OF PRAGMATIC INFORMATION SYSTEM · A. Pragmatic Source-Channel Coding

For Gaussian sources, pragmatic rate-distortion and value of information characterize the rate–utility trade-off after action-irrelevant distinctions are abstracted. The section extends source-channel coding to reliable preservation of pragmatic intention and action-relevant information, with feasibility governed by pragmatic entropy or rate-distortion relative to pragmatic channel capacity.

C. Pragmatic Rate Distortion of Gaussian Source and Pragmatic Value of Information

  • For Gaussian sources, pragmatic rate-distortion measures the minimum rate for a threshold on action-level distortion, and pragmatic VoI is its Legendre-Fenchel dual.With quadratic utility and symmetric pragmatic abstraction, bilateral coarsening yields greater utility gain at a given rate than single-sided abstraction because it discards more decision-irrelevant information. Closed forms assume continuous Euclidean action spaces; discrete-action tasks require suitable discrete distortion measures and generally lack such simple closed forms.Source: C. Pragmatic Rate Distortion of… · pp. 122–124
  • At R = 1, bilateral coarsening yields 0.984 utils and distortion 0.0156, versus 0.9375 utils and distortion 0.0625 for single-sided coarsening.This is a fourfold distortion reduction at the same rate; the table covers Gaussian-source VoI and distortion for P = 1 and Ω = 2.Source: C. Pragmatic Rate Distortion of… · p. 124, Table X · p. 124

XI. JOINT OPTIMIZATION OF PRAGMATIC INFORMATION SYSTEM

  • The joint-optimization framework co-designs communication rate, control law, and pragmatic abstraction under resource constraints.It derives optimality conditions through a pragmatic Lagrangian for communication and control in closed-loop systems.Source: §XI · pp. 125–126

A. Pragmatic Source-Channel Coding

  • Pragmatic source-channel coding targets recovery of the receiver’s pragmatic intention, allowing symbol errors that do not change the optimal action.The joint-optimization section states that reliable transmission is possible iff pragmatic entropy or the pragmatic rate-distortion function does not exceed pragmatic channel capacity, equivalently achieving full pragmatic VoI within the resource budget.Source: §XI · p. 125, A. Pragmatic Source-Channel Coding · p. 126
  • The architecture separately designs a pragmatic source code and pragmatic channel code while preserving asymptotic optimality.The receiver can decode a pragmatic index while ignoring syntactic and semantic variations that do not affect the terminal action.Source: A. Pragmatic Source-Channel Coding · p. 127
  • If Hp(W) < Cp, lossless pragmatic source-channel codes exist; if Rp(D) < Cp, codes can meet distortion D, whereas rates above Cp are unattainable.For lossless coding, Hp(W) > Cp leaves error probability bounded away from zero for sufficiently large block lengths.Source: A. Pragmatic Source-Channel Coding · p. 127
  • Pragmatic value is non-decreasing and saturates at the rate needed to preserve all pragmatic distinctions, while pragmatic cost is non-decreasing and convex.The saturation threshold is Hp(W) for lossless transmission or Rp(D) for lossy transmission; cost represents the minimum resource required to achieve a rate.Source: A. Pragmatic Source-Channel Coding · pp. 128–129
  • Because Hp(W) ≤ H(W) and Cp ≥ C, the pragmatic feasibility condition is weaker than the classical one, tolerating errors that do not change the optimal action.This allows reliable operation in regimes where classical communication would fail.Source: A. Pragmatic Source-Channel Coding · p. 129
  • With H(W) ≈ 1.9856 bits per symbol, Hp(W) ≈ 0.9928 prabits per symbol, and C = 1.5 bits per channel use, classical coding fails while pragmatic coding can reliably transmit Go/Stop when Cp ≥ 0.9928.The example states exact traffic-light symbols cannot be reliably transmitted under this condition.Source: A. Pragmatic Source-Channel Coding · pp. 129–130

1) Separation Principle and the Imperative of Joint Optimization: … Lagrangian relaxation of the constrained utility maximization problem

This section moves from the pragmatic separation principle’s asymptotic condition to joint optimization for finite-length or resource-constrained systems, extending Lagrangian methods across communication, control, and decision-making. It characterizes static optimality by balancing marginal utility against information cost and examines how pragmatic abstraction changes utility and resource trade-offs in Gaussian settings.

1) Separation Principle and the Imperative of Joint Optimization:

  • Reliable pragmatic transmission holds if and only if Hp(W) ≤ Cp in the lossless case or Rp(D) ≤ Cp in the lossy case.This is the pragmatic separation principle, established by concatenating the pragmatic source and channel codes.Source: 1) Separation Principle and the… · p. 130
  • Separation is strictly optimal asymptotically without encoding or decoding complexity constraints, but can be suboptimal at finite blocklength or under resource constraints.In those practical regimes, the encoder must jointly account for source statistics, channel characteristics, and decision utility.Source: 1) Separation Principle and the… · pp. 130–131
  • Joint optimization couples source-side abstraction and channel-side error protection through the rate–resource trade-off, which must be optimized for net benefit.The same rate controls both the isoteleia coarsening level and error-protection strength; the Lagrangian framework is presented as a tool for resource-constrained systems.Source: 1) Separation Principle and the… · p. 131

2) Gaussian Source over Gaussian Channel:

  • 0.996 VoI is achieved with Ω = 2 and Ptx = 0, exceeding the target VoI of 0.9 through pragmatic abstraction alone.The passage gives VoI = 1 − 1/256 ≈ 0.996 and concludes that zero transmit power suffices.Source: §25.6 · p. 134
  • For finite Ω > 1 and Ptx, one-sided coarsening yields lower VoI than bilateral coarsening, with SNR amplification Ω^4 rather than Ω^8.The passage attributes the difference to coarsening only the source or reconstruction, versus coarsening both sides of the link.Source: §25.6 · p. 134

B. Communication-Control-Decision Joint Optimization

  • The framework jointly optimizes communication, control, and decision-making, with formal theorems characterizing optimal trade-offs in static and dynamic settings.The theorems are described as bridging information-theoretic limits and decision-theoretic objectives.Source: B. Communication-Control-Decision… · pp. 134–135

Lagrangian relaxation of the constrained utility maximization problem

  • For asymmetric links with only one side coarsened, single-sided Lagrangians substitute perceptual sensing cost or expressive transmit cost, with corresponding optimality conditions.The stated conditions are identical to those for the global Lagrangian with the relevant cost function.Source: Lagrangian relaxation of the… · p. 136
  • In the LQG example with perceptual cost, optimal power decreases as the shadow price λ rises, while pragmatic abstraction amplifies effective signal-to-noise ratio.The example presents a closed-form trade-off between control performance and communication cost.Source: Lagrangian relaxation of the… · p. 136
  • In the Gaussian channel, pragmatic abstraction raises net benefit without changing R*, amplifies effective SNR by Ω^4, and reduces power needed for a given net benefit.Figure 17 shows the upward net-benefit shift with R* unchanged; the text states that Ω > 1 amplifies effective SNR by Ω^4.Source: Figure 17 · p. 137, Lagrangian relaxation of the… · pp. 138–139
  • At the static optimum, the Bayes decision rule applies, marginal pragmatic value equals λ times marginal CoI, and maximized net benefit is behavioral capacity.These are the stated optimality conditions for the constrained utility problem.Source: Lagrangian relaxation of the… · p. 138
  • Behavioral capacity rises with rate at diminishing returns and saturates at perfect-information utility P; it falls as resources become costlier, while abstraction raises capacity and flattens that decline.Communication ceases at the zero-capacity threshold; larger Ω provides robustness against rising resource costs.Source: Lagrangian relaxation of the… · p. 138, Figure 18 · p. 139

2) Dynamic Joint Optimization: Sequential Decision Making: … C. Limitations and Future Directions

This section presents a belief-state POMDP formulation that jointly optimizes actions and information rates in communication-constrained control, accounting for communication costs and future utility. It summarizes the theory’s contributions and engineering relevance, outlines a path for learning and deploying context-sensitive pragmatic mappings, and identifies theoretical and computational limitations for future work.

2) Dynamic Joint Optimization: Sequential Decision Making:

  • Theorem 48 formulates communication-constrained POMDP control as discounted utility maximization with a pragmatic rate cost, using a Lagrangian multiplier for the resource constraint.The observation kernel depends on the communication rate, which affects observation quality.Source: 2) Dynamic Joint Optimization:… · pp. 139–140
  • The Bellman operator is a contraction with a unique fixed point; the resulting dynamic isoteleia mapping depends on future value and recovers the static mapping when γ = 0.In the steady-state case, the hierarchy’s entropy inequalities remain valid as limiting bounds.Source: 2) Dynamic Joint Optimization:… · p. 141
  • The optimal rate adapts to belief uncertainty and sensitivity of future rewards to estimation, while actions and rates couple through belief updates and observation quality.The framework balances communication cost against long-term utility gains from improved state information.Source: 2) Dynamic Joint Optimization:… · pp. 141–142
  • In stationary scalar LQG with linear rate cost, the optimal rate is constant over time, decreases with shadow price λ, and is found using an augmented Riccati equation.The example illustrates tractable joint design of communication and control.Source: 2) Dynamic Joint Optimization:… · p. 142

A. Summary of Contributions

  • The theory defines pragmatic entropy, mutual information, capacity, and rate-distortion hierarchies, and introduces VoI and CoI as decision-theoretic duals supporting cross-layer optimization.The stated hierarchies are Hp ≤ Hs ≤ H, Cp ≥ Cs ≥ C, and Rp ≤ Rs ≤ R.Source: A. Summary of Contributions · p. 143
  • The pragmatic efficiency functional defines behavioral capacity under resource constraints, while three coding theorems generalize Shannon’s results and extensions provide Gaussian expressions and static and dynamic joint optimization.The three theorems cover lossless source coding, channel coding, and rate-distortion coding.Source: A. Summary of Contributions · p. 143

B. Engineering Implications and Deployment Pathways

  • By focusing on how information guides actions rather than symbol fidelity, the framework offers a mathematical language for task-oriented communication, control, autonomous systems, and embodied AI.VoI and CoI provide a basis for allocating resources to balance utility gains against costs.Source: B. Engineering Implications and… · p. 143
  • The Lagrangian dual framework is presented as a principled means of cross-layer optimization that bridges information-theoretic limits and engineering practice.In deployment, information should be evaluated by its effectiveness in achieving goals rather than by syntactic accuracy.Source: B. Engineering Implications and… · p. 143
  • Deep networks can learn isoteleia mappings from data without an analytic utility model, with context-conditioned inference and online refinement supporting changing environments.The proposed pipeline combines offline task-specific training, context-adaptive inference, and online adaptation through reinforcement learning or fine-tuning.Source: B. Engineering Implications and… · pp. 144–145
  • For safety-critical systems, observation-to-action mappings should be deterministic to ensure consistent outputs, reproducibility, and easier fault diagnosis.The passage suggests deterministic inference policies or temperature-annealed Gumbel-Softmax as implementation options.Source: B. Engineering Implications and… · p. 144

C. Limitations and Future Directions

  • The framework assumes stationarity and ergodicity, uses asymptotic coding theorems, and primarily treats single-agent or centralized settings, limiting its scope.Future work includes time-varying entropies and online adaptive coding, finite-blocklength dispersion bounds, and game-theoretic extensions for multi-agent settings.Source: C. Limitations and Future Directions · p. 145
  • High-dimensional belief-state Bellman optimization is computationally intractable, while neural approximations introduce errors and generalization gaps outside current guarantees.Approximate dynamic programming is proposed for the belief-state problem; integration with pragmatic coding also calls for provably efficient algorithms with finite-sample guarantees and convergence rates.Source: C. Limitations and Future Directions · p. 145
  • Future applications include multi-agent networks and cognitive science, linguistics, and economics, where information exchange is goal-directed and context-dependent.The paper presents pragmatic information as a unified language for intelligent systems that extract value from information under resource constraints.Source: C. Limitations and Future Directions · pp. 145–146

APPENDIX … F. Proof of Theorem 43

The appendix proves Theorems 33, 34, 36, 39, 40, and 43 by deriving stated properties from sequence-set definitions, probability inequalities, and independence or large-number arguments. The proofs of Theorems 39 and 43 additionally use aggregated probabilities over reified equivalence classes and entropy-hierarchy bounds.

A. Proof of Theorem 33

  • For Theorem 33, property (1) follows from A(n)’s definition, and the epsilon limit and bounds from properties (1) and (2) establish property (3).The proof states that epsilon tends to 1 as n tends to infinity and that an n0 exists for any epsilon > 0.Source: A. Proof of Theorem 33 · p. 149

B. Proof of Theorem 34

  • For Theorem 34, property (1) follows from E(n)’s definition, and the proof derives the stated inequalities using P(wn) > 0.The passage describes dividing by P(wn) and then establishing the right inequality.Source: B. Proof of Theorem 34 · p. 150

C. Proof of Theorem 36

D. Proof of Theorem 39

  • For Theorem 39, properties (1)–(3) follow from A(n)’s definition; property (4) uses the law of large numbers, independence, entropy bounds, and A(n)’s size bound.The large-number argument concerns an i.i.d. pragmatic sequence pair with probabilities aggregated over reified equivalence classes.Source: D. Proof of Theorem 39 · p. 151, F. Proof of Theorem 43 · p. 152

E. Proof of Theorem 40

F. Proof of Theorem 43

  • For Theorem 43, properties (1)–(3) follow from A(n)’s definition, while property (4) uses independence, aggregated probabilities, entropy bounds, and the size bound on A(n).The probability bounds invoke Hp(X) ≤ Hs(˜X) ≤ H(X).Source: F. Proof of Theorem 43 · p. 152

Related papers

Loading 2609.10986v1…