Source-linked AI summary
A Mathematical Theory of Pragmatic Information
Kai Niu, Ping Zhang
TL;DR
Information-driven systems use information to guide decisions and actions, yet distinctions that do not affect the optimal action need not matter pragmatically. The paper develops a framework for this task-relevant view, and its Gaussian channel example reports that bilateral abstraction yields the greatest power savings for a given pragmatic rate.
Problem
Information-driven systems use information to guide decisions and actions, making the pragmatic effect of information central to understanding system performance.
Source: A. Background and Motivation · p. 4Method
The paper models a perception-communication-decision-execution loop and uses pragmatic equivalence classes to focus information measures on distinctions that affect optimal actions.
Source: B. Pragmatic Information System · p. 15, A. Pragmatic Value of Information · p. 66Results
Bilateral abstraction yields the greatest power savings for a given pragmatic rate in the cited Gaussian channel example.
Source: B. Pragmatic Capacity of Gaussian… · p. 121Takeaways & Limitations
Pragmatic value offers a more parsimonious, task-relevant measure by treating only state and observation distinctions that affect the optimal action as valuable.
Source: A. Pragmatic Value of Information · p. 66The stated future direction is a unified language for goal-directed agents extracting value from information under resource constraints.
Source: C. Limitations and Future Directions · pp. 145–146
Abstract
from arXiv · showhide
We propose a pragmatic information theory unifying communication, control, and decision-making. Its core is the isoteleia mapping, formalizing equifinality: distinct semantic paths leading to the same optimal action are pragmatically equivalent. This induces a three-tier hierarchy of syntactic, semantic, and pragmatic information, each abstraction discarding task-irrelevant distinctions. We develop pragmatic entropy, up/down mutual information, channel capacity, and rate-distortion, and prove three coding theorems generalizing Shannon's classical results. We introduce pragmatic value (VoI) and cost (CoI) of information as decision-theoretic duals to rate-distortion and capacity, respectively, and formulate a Lagrangian dual framework for cross-layer optimization. The pragmatic efficiency bound $\mathcal{E}_p(λ)=\sup_R[Φ_p(R)-λ\,\mathrm{CoI}_p(R)]$ quantifies the maximum net utility any resource-constrained intelligent system can extract, thereby establishing a fundamental behavioral capacity limit---generalizing Shannon's symbol-level capacity to goal-directed action. Extensions to continuous messages yield closed-form Gaussian expressions, while dynamic settings are addressed via a Bellman equation for sequential decision-making. This framework provides a rigorous foundation for task-oriented communication, networked control, autonomous systems, and embodied AI, shifting focus from symbol fidelity to the effectiveness of information in guiding actions, and offers a unified mathematical language for next-generation intelligent systems.
I. INTRODUCTION
The paper proposes a unified theory linking communication, control, and decision-making through syntactic, semantic, and pragmatic levels focused on how information guides actions. It summarizes coding-theorem results, continuous-domain measures, and a joint optimization framework, presenting these contributions as extensions and unification of communication and semantic coding frameworks for goal-directed information systems.
A. Background and Motivation
- The introduction identifies a gap: pragmatic information lacks a comprehensive mathematical framework, while prior formulations did not establish coding theorems, rate-distortion limits, or channel capacity [27].Earlier semantic information work includes Niu and Zhang’s framework based on synonymy and corresponding coding theorems [21–23].Source: A. Background and Motivation · pp. 3, 8
- The proposed PCDE framework treats information as valuable for its effects on decisions and actions, across settings such as autonomous driving and other information-driven systems.In autonomous driving, pragmatic objectives include safety, efficiency, and comfort, and the same sensor readings can call for different actions depending on context.Source: A. Background and Motivation · pp. 4, 5, 7
B. Contributions and Organization
- The isoteleia mapping groups distinct semantic paths that lead to the same optimal action, completing a three-tier hierarchy that subsumes semantic and classical information theory as special cases.The synonymous mapping connects syntactic realizations to meaning, while reification bridges pragmatic purposes to syntactic signals.Source: B. Contributions and Organization · pp. 8–10
- The listed results include pragmatic channel and source coding theorems, a pragmatic source-channel coding theorem, and a static joint optimization theorem.The introduction identifies these as part of the paper’s contributions but does not state their theorem results in these passages.Source: B. Contributions and Organization · p. 10
- The system framework organizes ten core modules into cognitive-regulation and symbolic-conduction subsystems, with reification connecting pragmatic intent to syntactic signals.The cognitive-regulation subsystem handles reasoning, intent formulation, and value/cost evaluation; the symbolic-conduction subsystem handles encoding, transmission, and decoding.Source: B. Contributions and Organization · pp. 10–11
- The paper defines pragmatic entropy, up/down mutual information, channel capacity, and rate-distortion, alongside pragmatic VoI and CoI and their dualities with rate-distortion and capacity.VoI measures utility gained from task-relevant information; CoI measures the minimum resources needed to convey information at a given pragmatic rate.Source: B. Contributions and Organization · p. 11
- The paper develops a unified Lagrangian framework for cross-layer optimization and a pragmatic efficiency bound that characterizes the behavioral capacity of resource-constrained intelligent systems.The shadow price λ converts resource use such as power, bandwidth, and latency into utility-equivalent units, balancing marginal value and cost in static settings and adapting to scarcity in dynamic settings.Source: B. Contributions and Organization · pp. 11–12
- The three pragmatic coding theorems set minimum lossless rate to Hp(W), maximum reliable rate to Cp, and minimum lossy rate at distortion D to Rp(D), generalizing Shannon [1, 2] and Niu and Zhang.The lossless source, channel, and rate-distortion theorems establish these respective bounds.Source: B. Contributions and Organization · p. 12
- For continuous messages, isoteleic volumes Ω yield Gaussian channel capacity Cp = 1/2 log(Ω^4(1 + P/σ^2)) and source rate-distortion Rp(D) = 1/2 log(P/(Ω^4D)), with corresponding CoI and VoI functions.Isoteleic volumes are described as continuous analogues of pragmatic equivalence classes.Source: B. Contributions and Organization · p. 12
- In static joint optimization, the optimal policy is the posterior-maximizing Bayes decision rule and the optimal rate equates marginal value with marginal cost; dynamic optimization uses a Bellman equation.The framework addresses communication, control, and decision-making in both static and sequential settings.Source: B. Contributions and Organization · p. 12
- The framework is presented as subsuming classical communication, closed physical, embodied and interactive, and open social and cognitive systems involving goal-directed information processing under uncertainty.These cases correspond to trivialized semantic and pragmatic layers, explicit utility functions, multiple agents, and implicit or evolving utility functions, respectively.Source: B. Contributions and Organization · pp. 12–13
II. PRAGMATIC INFORMATION SYSTEM AND ISOTELEIA MAPPING
The section defines pragmatic information through the decisions and actions it enables, treating distinct semantic paths that lead to the same optimal action as equivalent. It formalizes this equifinality through mappings and successive quotients from syntax to semantics to pragmatics, and presents a PCDE architecture connecting communication with utility-driven decisions.
1) System Structure and Processing Flow:
- The PCDE system’s ten-module cascade turns task intent into transmitted signals, reconstructs pragmatic purpose, and executes actions, evaluating information through VoI and CoI.VoI quantifies expected utility gain; CoI quantifies the minimum resource expenditure required to convey pragmatic information at a given rate.Source: B. Pragmatic Information System · p. 15, 1) System Structure and Processing Flow · pp. 15, 17, 18
2) Functional Subsystems:
- The CRS formulates intent and evaluates its value, while the SCS encodes and transmits it; reification g = f ◦e bridges intent to syntactic signals.The CRS’s isoteleia mapping groups semantic justifications leading to the same optimal action, while the SCS’s synonymous mapping links meanings to multiple syntactic realizations.Source: 2) Functional Subsystems · p. 18
3) Axiomatic Foundations:
- The framework assumes probabilistic uncertainty, expected-utility-maximizing actions, and Markov dynamics, enabling POMDP modeling and temporal decomposition into static coding problems.The decision axiom underpins isoteleia equivalence classes and VoI; the Markov property supports applying coding theorems at each time step.Source: 3) Axiomatic Foundations · pp. 19–20
- The framework spans classical communication, closed physical control, embodied interaction, and open social or cognitive systems, sharing a common core across a continuum of applications.When utility depends only on symbol fidelity, the semantic and pragmatic layers reduce to identity mappings and classical information-theoretic results emerge as special cases.Source: 3) Axiomatic Foundations · pp. 20–22
C. Design Principles of Pragmatic Information Systems
- Isoteleia groups distinct semantic interpretations—and syntactic realizations—when they yield the same optimal terminal action under the system’s utility.Pragmatic information is not directly observable; Value of Information measures the utility gain from information.Source: §II · p. 13, C. Design Principles of Pragmatic… · p. 22
- Pragmatic information is inferred through utility rather than read from a signal, so system design and resource allocation should prioritize resulting action quality and marginal VoI.Different meanings and syntactic forms are pragmatically equivalent when they produce the same optimal action.Source: C. Design Principles of Pragmatic… · p. 22
- Semantic information is inferred from symbols, and synonymy permits multiple syntactic realizations of one meaning; the framework also preserves compatibility with classical communication and control.Syntactic compression is acceptable when semantic content is preserved; trivializing pragmatic and semantic layers recovers Shannon communication, while ignoring communication recovers classical control.Source: C. Design Principles of Pragmatic… · p. 23
1) The Nature of Pragmatic Information:
- Different symbols or meanings can carry the same pragmatic information when they lead to the same optimal action, while identical meanings can differ pragmatically across contexts.Isoteleia names this equifinality: the receiver seeks guidance for the desired outcome, not exact reconstruction of the sender’s message.Source: 1) The Nature of Pragmatic Information · p. 24
3) Formal Definitions of the Mappings:
- Isoteleia groups semantic classes by their induced optimal action; synonymous mapping links meanings to syntactic realizations, and their composition links pragmatic purposes to possible signals.For sequential decisions, the dynamic extension uses a value function over belief states and recovers the static definition when γ = 0.Source: 3) Formal Definitions of the Mappings · pp. 26–27
4) The Five Relations Between Layers:
- Each pragmatic class corresponds to multiple semantic classes and their syntactic signals, while inverse mappings assign each symbol to one class; semantic classes must be homogeneous in optimal action.If a semantic class contains symbols leading to different optimal actions, it must be split by action before isoteleia is uniquely defined.Source: 4) The Five Relations Between Layers · pp. 27–28
6) Quotient-Space Structure:
- The three layers are successive quotients of the original signal set: semantics discards distinctions that do not affect meaning, and pragmatics discards those that do not affect optimal action.The pragmatic layer is the coarsest, most task-relevant description.Source: 6) Quotient-Space Structure · p. 28
III. PRAGMATIC ENTROPY
This section develops pragmatic entropy as a measure of uncertainty about optimal actions, grouping meanings that induce the same action. It extends the measure to joint and conditional dependencies and to successive abstraction from syntactic and semantic distinctions into action-relevant classes, including sequences.
A. Formal Definition of Pragmatic Entropy
- Pragmatic entropy measures uncertainty about the optimal terminal action, not transmitted symbols or conveyed meanings, and is measured in pragmatic bits (Prabits).It quantifies the minimum average pragmatic bits needed to specify the optimal action and the irreducible decision uncertainty to resolve through communication and control.Source: A. Formal Definition of Pragmatic… · pp. 28–29
- Syntactic, semantic, and pragmatic entropy form a descending hierarchy as each layer discards distinctions irrelevant to symbols, meanings, or optimal actions, respectively.The hierarchy is H_p(W) ≤ H_s(˜W) ≤ H(W); the first inequality is equality when the isoteleia mapping is injective.Source: A. Formal Definition of Pragmatic… · p. 30
- For a pragmatic alphabet of size L, entropy is at most log L, attained by a uniform distribution; merging semantic classes can lower this bound, reaching zero for one optimal action.The maximum pragmatic entropy is strictly below the semantic maximum when the isoteleia mapping is non-injective.Source: A. Formal Definition of Pragmatic… · p. 31
- In the vehicle example, syntactic entropy is 1.295 bits while semantic and pragmatic entropy are each 0.469, because green and yellow both lead to Go.The two semantic classes induce distinct optimal actions, so semantic and pragmatic entropy coincide, while pragmatic entropy avoids distinguishing green from yellow.Source: A. Formal Definition of Pragmatic… · pp. 32–33
B. Pragmatic Joint Entropy and Conditional Entropy
- Joint and conditional pragmatic entropies aggregate semantic pairs by their jointly induced optimal actions and distinguish residual uncertainty given an exact syntactic signal from uncertainty given another decision.Joint pragmatic entropy applies Shannon entropy to the aggregated joint pragmatic distribution.Source: B. Pragmatic Joint Entropy and… · pp. 33–35
- Pragmatic joint entropy obeys inequality-based chain bounds rather than an exact additive chain rule, and lies between a pragmatic lower bound and a syntactic upper bound.The joint-entropy hierarchy likewise places pragmatic entropy no higher than semantic entropy, which is no higher than syntactic entropy; equality depends on injectivity of the corresponding mappings.Source: B. Pragmatic Joint Entropy and… · pp. 35–37
- Joint syntactic and pragmatic entropy are equal only when the corresponding mapping is injective, so no distinct input pairs are merged.For the joint synonymous mapping, equality likewise holds if and only if that mapping is injective.Source: B. Pragmatic Joint Entropy and… · p. 37
- Sequential pragmatic entropy obeys a chain of inequalities relating full-sequence entropy to partial entropies conditioned on pragmatic or syntactic prefixes.The proof iteratively applies the binary chain rule while replacing pragmatic conditioning with syntactic conditioning.Source: B. Pragmatic Joint Entropy and… · p. 38
- For sequences, joint entropies follow the same pragmatic ≤ semantic ≤ syntactic hierarchy, and sequential partial conditional entropies satisfy the same ordering.This follows from the deterministic element-wise mappings and the partial entropies’ linear-combination structure.Source: B. Pragmatic Joint Entropy and… · p. 39
- Pragmatic entropy further removes decision-irrelevant distinctions, yielding the hierarchy pragmatic ≤ semantic ≤ syntactic entropy.The layers respectively retain action-relevant distinctions, remove meaningless syntactic variation, and capture symbol-level distinctions.Source: B. Pragmatic Joint Entropy and… · pp. 39, 41
IV. PRAGMATIC RELATIVE ENTROPY AND PRAGMATIC MUTUAL INFORMATION
This section develops pragmatic divergence and dependence measures by aggregating over equivalence classes that preserve terminal actions, distinguishing up/down and single-sided measures for task-oriented communication. It organizes classical and pragmatic mutual informations into a lattice and extends their chain rules and hierarchy to sequences, including information changes under one-sided pragmatic coarsening.
A. Pragmatic Relative Entropy
- Pragmatic relative entropy compares distributions over terminal-action equivalence classes in full and two partial forms.The full form compares pragmatic marginals; Type I compares p aggregated pragmatically with syntactic q, while Type II reverses which distribution is aggregated.Source: A. Pragmatic Relative Entropy · pp. 42–43
- The three pragmatic relative entropies obey information inequalities, fit a hierarchy with semantic and classical relative entropies, and are jointly convex in p and q.The full pragmatic divergence is nonnegative, with equality when the pragmatic marginals match.Source: A. Pragmatic Relative Entropy · pp. 43–45
1) Definitions of Up and Down Pragmatic Mutual Information:
- Up and down pragmatic mutual information capture decision-uncertainty reduction from different perspectives, and lie in an inequality hierarchy with semantic and syntactic mutual information.Up measures information about terminal actions extractable from syntactic variables; down measures pragmatic dependence remaining after coarse-graining.Source: 1) Definitions of Up and Down… · pp. 47–48
- Additive decompositions express up pragmatic mutual information as semantic information plus a pragmatic increment, and down pragmatic mutual information as semantic information minus a pragmatic loss.The increments and losses are nonnegative, making the classical mutual information a reference point for the abstraction gains and losses.Source: 1) Definitions of Up and Down… · pp. 48–50
2) Single-Sided Pragmatic Mutual Information:
- Four single-sided measures address links where only the source or observation/reconstruction is coarsened pragmatically.Perception coarsens the source to an action while retaining a syntactic observation; expression or reconstruction retains the syntactic source and coarsens the other side.Source: 2) Single-Sided Pragmatic Mutual… · p. 50
1) Perceptual Pragmatic Mutual Information (PPMI):
- PPMI quantifies how much a syntactic observation carries about a pragmatic action.The measure is defined as the mutual information between the pragmatic action and syntactic observation.Source: 1) Perceptual Pragmatic Mutual… · pp. 50–51
2) Prospective Perceptual Pragmatic Mutual Information (PPPMI):
- PPPMI provides an upper bound on perceptual information when the source’s syntactic entropy is retained in the marginal term.Source: 2) Prospective Perceptual Pragmatic… · p. 51
3) Expressive Pragmatic Mutual Information (EPMI):
- EPMI quantifies how much information about a syntactic source is contained in the pragmatic label.Source: 3) Expressive Pragmatic Mutual… · p. 51
4) Prospective Expressive Pragmatic Mutual Information (PEPMI):
- The seven information quantities form a complete lattice: perceptual and expressive measures are ordered within their chains, while cross-chain comparisons are generally incomparable except for pragmatic-to-syntactic bounds.Source: 4) Prospective Expressive Pragmatic… · pp. 51–52
3) Sequential Chain Rules for Pragmatic Mutual Information:
- For sequences, down, up, and single-sided pragmatic mutual informations obey chain inequalities; up variants reverse direction, and all seven measures retain the single-letter partial order for fixed n.Source: 3) Sequential Chain Rules for… · pp. 52–54
- In the two-vehicle example, classical mutual information is 0.3394 bits, down bilateral is 0 (positive part), and up bilateral is 2.2203 prabits; PPMI/EPMI are 0.2458 bits and PPPMI/PEPMI 1.2386 bits.Single-sided measures quantify information loss or gain when only one side is pragmatically coarsened, providing finer-grained analysis than bilateral measures alone; PPMI and EPMI coincide here by symmetry but generally differ.Source: 3) Sequential Chain Rules for… · pp. 54–55, Table VII · p. 55
V. PRAGMATIC CHANNEL CAPACITY AND PRAGMATIC RATE DISTORTION
The section defines pragmatic capacity and rate-distortion limits around reliable transmission of optimal actions and task-relevant fidelity, then extends them to asymmetric systems where only one side is pragmatically abstracted.
A. Pragmatic Channel Capacity
- Pragmatic channel capacity measures the maximum reliable rate for conveying optimal actions and can exceed syntactic capacity by treating action-equivalent messages as equivalent.It is optimized over input distributions and joint reification mappings, and the section gives an additive decomposition of its gains.Source: A. Pragmatic Channel Capacity · pp. 56–58
B. Pragmatic Rate Distortion
- Pragmatic rate-distortion gives the minimum rate for a task-distortion threshold, measuring fidelity through terminal actions rather than syntactic reconstruction.The section establishes a hierarchy of rate-distortion functions and states that pragmatic rate-distortion can be significantly lower than classical rate-distortion when errors leave terminal actions unaffected.Source: B. Pragmatic Rate Distortion · pp. 58–60
C. Single-Sided Pragmatic Achievable Rates and Rate-Distortion Functions
- For asymmetric systems, four single-sided measures cover perceptual and expressive rate-distortion and their corresponding prospective achievable rates.They apply when either the observation or action label is coarsened, complementing bilateral measures for task-oriented information flows.Source: C. Single-Sided Pragmatic Achievable… · p. 60
1) Perceptual Pragmatic Rate-Distortion Function:
- Perceptual pragmatic rate-distortion quantifies the rate needed to compress observations while estimating the pragmatic action, with the source remaining syntactic.It is suited to perception where sensory data can be compressed by exploiting action-equivalence classes.Source: 1) Perceptual Pragmatic… · pp. 60–61
2) Expressive Pragmatic Rate-Distortion Function:
- Expressive pragmatic rate-distortion measures the rate needed to represent a syntactic source through a pragmatic action label while meeting reconstruction distortion.It applies when a low-bandwidth command must convey enough information to reconstruct the intended state or message.Source: 2) Expressive Pragmatic… · p. 61
3) Prospective Perceptual Pragmatic Achievable Rate:
- Prospective perceptual achievable rate bounds the pragmatic information extractable from an uncoarsened observation under a resource budget.It concerns a receiver that has the full syntactic observation but uses only pragmatic information.Source: 3) Prospective Perceptual Pragmatic… · pp. 61–62
4) Prospective Expressive Pragmatic Achievable Rate:
- Prospective expressive achievable rate measures how much source information a pragmatic command can convey under a resource budget.The single-sided bounds imply that coarsening only one side preserves more syntactic distinctions, requiring higher rates for the same distortion and yielding lower achievable rates for the same resource.Source: 4) Prospective Expressive Pragmatic… · pp. 62–63
VI. PRAGMATIC COST OF INFORMATION AND PRAGMATIC VALUE OF INFORMATION
This section develops pragmatic value and cost of information as decision-benefit and resource-cost measures, linking them through a cross-layer Lagrangian framework to rate-distortion and capacity. It also defines an efficiency bound for net utility under resource pricing and single-sided measures for asymmetric communication.
C. Lagrangian Duality
- The cross-layer framework uses rate R to connect the source-side rate-distortion/value duality with the channel-side capacity/cost duality.It formulates optimization through Lagrangians that balance decision value against transmission resources.Source: §VI · p. 63, C. Lagrangian Duality · p. 70
A. Pragmatic Value of Information
- Pragmatic VoI evaluates utility gain after coarsening both states and observations into isoteleia classes that preserve distinctions relevant to optimal actions.It is nonnegative, increases under refinement of the pragmatic observation, and its maximum achievable value is non-decreasing and concave in rate.Source: A. Pragmatic Value of Information · pp. 64–65
- The value-rate function and pragmatic rate-distortion function form a Legendre–Fenchel dual pair.When isoteleia mappings are identity mappings, pragmatic VoI reduces to classical VoI.Source: A. Pragmatic Value of Information · pp. 65–67
- In the two-vehicle example, pragmatic VoI is approximately 0.5 utils, compared with about 4.2 utils for classical VoI using the full syntactic signal.The pragmatic mapping merges “Stop-short” and “Stop-long” into one “Stop” action; the baseline utility is 11 for both vehicles.Source: A. Pragmatic Value of Information · p. 67
B. Pragmatic Cost of Information
- Pragmatic CoI is the minimum physical-channel resource required for a pragmatic rate, independent of the task utility, and is the inverse of pragmatic capacity.Capacity is non-decreasing and concave in resource; CoI is nonnegative, strictly increasing for positive rates, and convex, though not necessarily strictly convex.Source: B. Pragmatic Cost of Information · pp. 68–70
1) Source-Distortion-Value Lagrangian (Down Loop):
- The down-loop Lagrangian balances value gained by reducing distortion against the marginal cost of rate, with λd measuring rate cost in utility units per bit.Its optimality condition equates marginal value of rate with marginal rate cost.Source: 1) Source-Distortion-Value Lagrangian… · p. 71
2) Channel-Power-Cost Lagrangian (Up Loop):
- The up-loop Lagrangian balances rate against physical resource cost, with γ representing the resource’s shadow price in utility per resource unit.At optimality, the shadow price equals the marginal cost of increasing rate.Source: 2) Channel-Power-Cost Lagrangian (Up… · p. 71
3) Global Cross-Layer Lagrangian and Behavioral Capacity:
- The global pragmatic Lagrangian maximizes net value minus resource cost over rate, converting physical cost into utility with shadow price λ.Its concavity is established under the stated conditions that the value function is concave and the cost function is convex in rate.Source: 3) Global Cross-Layer Lagrangian and… · p. 72
- The global cross-layer Lagrangian balances marginal information value against resource-priced marginal cost, and its supremum defines pragmatic efficiency as maximum net utility under resource price λ.Under the stated differentiability and concavity conditions, the first-order condition suffices for a global maximum; the global multiplier equals the up-loop resource shadow price.Source: 3) Global Cross-Layer Lagrangian and… · pp. 73–74
D. Single-Sided Pragmatic Cost of Information and Pragmatic Value of Information
- Single-sided value and cost measures capture utility gains and resource expenditures when only one side of a communication link is pragmatically abstracted, supporting asymmetric sensing-to-action and command-to-execution analysis.Source: D. Single-Sided Pragmatic Cost of… · p. 75
1) Perceptual Pragmatic Value of Information:
- Perceptual pragmatic value of information is the maximum expected utility gain from observation V when decisions use pragmatic action W, subject to a rate constraint; it is non-decreasing and concave in R.Source: 1) Perceptual Pragmatic Value of… · p. 75
2) Expressive Pragmatic Value of Information:
- Expressive pragmatic value of information measures utility gain from using pragmatic label V as a rate-limited compressed representation of source W, and is non-decreasing and concave in R.Source: 2) Expressive Pragmatic Value of… · p. 76
3) Prospective Perceptual Pragmatic Cost of Information:
- Prospective perceptual pragmatic cost of information is the minimum resource expenditure to obtain at least R bits of pragmatic information from an observation; it is non-decreasing, convex, and the strict inverse of achievable rate.The source cost function can represent sensing power.Source: 3) Prospective Perceptual Pragmatic… · p. 76
4) Prospective Expressive Pragmatic Cost of Information:
- Prospective expressive pragmatic cost of information is the minimum resource expenditure to convey at least R bits of source information through a pragmatic label; it is non-decreasing, convex, and the strict inverse of achievable rate.The source cost function can represent transmit power.Source: 4) Prospective Expressive Pragmatic… · p. 77
5) Relationships and Duality Structure:
- The single-sided measures complete a duality framework: value functions are Legendre–Fenchel duals of rate-distortion functions, and prospective costs invert achievable rates across perceptual and expressive paths.Perceptual and expressive paths coarsen opposite sides of communication; prospective variants use up mutual information to provide upper bounds on achievable rates.Source: 5) Relationships and Duality Structure · pp. 77–78
VII. PRAGMATIC LOSSLESS SOURCE CODING
The section extends asymptotic typicality across syntactic, semantic, and pragmatic layers and uses their hierarchy to establish pragmatic lossless coding that preserves action rather than exact symbols. It characterizes the rate threshold for vanishing error and full pragmatic value, while showing how additional rate can preserve syntactic redundancy and connect pragmatic compression to the classical limit.
VII. PRAGMATIC LOSSLESS SOURCE CODING
- The section develops pragmatic lossless source coding by extending AEP and typical-set methods from symbol and meaning layers to action-preserving equivalence classes.Its coding result makes pragmatic entropy the rate threshold for preserving pragmatic purpose with vanishing error.Source: §VII · p. 79, B. Pragmatic Lossless Source Coding… · p. 87
A. Asymptotic Equipartition Property and Pragmatic Typical Set
- Pragmatic abstraction merges semantic distinctions that leave the optimal action unchanged, reducing typical-sequence counts beyond semantic abstraction and enabling more efficient pragmatic lossless compression.The section introduces pragmatic, isoteleia, and reified typical sets and analyzes their cardinality and probability properties.Source: §VII · p. 79, A. Asymptotic Equipartition Property… · p. 80
1) Three-Tier AEP Theorems:
- The syntactic, semantic, and pragmatic AEPs converge in probability, with the latter two following by aggregating probabilities over synonymous and reified equivalence classes.These AEP results provide the basis for the corresponding pragmatic and cross-layer typical sets.Source: 1) Three-Tier AEP Theorems · p. 81
2) Definitions of the Hierarchical Typical Sets:
- The typical sets form nested partitions: synonymous mapping groups syntactic sequences into semantic classes, then isoteleia groups those classes into reified classes for pragmatic actions.This hierarchy entails Hp(W) ≤ Hs(˜W) ≤ H(W), and each reified typical set is the union of synonymous typical sets associated with its isoteleia typical set.Source: 2) Definitions of the Hierarchical… · pp. 83–84
3) Properties of the Typical Sets:
- Typical-set cardinalities follow the same hierarchy, and a reified typical set has size approximately 2^n(H(W)−Hp(W)), providing additional compression freedom over a synonymous typical set.That freedom is larger by a factor of 2^n(Hs(˜W)−Hp(W)).Source: 3) Properties of the Typical Sets · p. 86
B. Pragmatic Lossless Source Coding Theorem
- Rates above pragmatic entropy achieve vanishing error, while rates below it fail; full decision utility is available only at the pragmatic-entropy rate.The code transmits a pragmatic index, allowing arbitrary syntactic and semantic realizations that preserve the action.Source: B. Pragmatic Lossless Source Coding… · p. 87
1) Three-Tier Pragmatic Source Code Model:
- The source-code model assigns each pragmatic index to an optimal-action class, and decoding may output any syntactic sequence in its corresponding reified typical set.Exact syntactic or semantic reconstruction is unnecessary to recover the pragmatic purpose.Source: 1) Three-Tier Pragmatic Source Code… · p. 87
2) An encoding function that operates in three stages:
- The encoder can identify the pragmatic class directly through reification, transmit its index, and let the decoder choose a representative sequence from that class.The three-stage description selects a semantic interpretation, a synonymous syntactic realization, and optionally a reification index for syntactic fidelity.Source: 1) Three-Tier Pragmatic Source Code… · p. 87, 2) An encoding function that operates… · p. 88
- For R > Hp(W), pragmatic lossless codes can have arbitrarily small error, whereas for R < Hp(W), every such code’s error probability tends to 1.The converse follows because fewer than 2^nR indices cannot cover the approximately 2^nHp(W) pragmatic typical sequences.Source: 2) An encoding function that operates… · pp. 89–90
- The compressed index achieves full pragmatic VoI if and only if R ≥ Hp(W), the minimum rate needed to recover all pragmatically relevant distinctions.Below this rate, unresolved uncertainty about the optimal action class makes achievable VoI strictly lower than its maximum.Source: 2) An encoding function that operates… · pp. 89, 91
- The theorem generalizes classical and semantic lossless source coding, with Hp(W) ≤ Hs(˜W) ≤ H(W) quantifying compression gains from pragmatic abstraction.The classical case has Hp(W) = H(W); the semantic case has Hp(W) = Hs(˜W).Source: 2) An encoding function that operates… · p. 91
- The total syntactic rate Rsyn = R + Rp ranges from the pragmatic limit Hp(W) at Rp = 0 to the classical limit H(W) as Rp reaches H(W) − Hp(W).Intermediate Rp values preserve varying amounts of syntactic redundancy within pragmatic equivalence classes.Source: 2) An encoding function that operates… · p. 91
- The optimal strategy encodes pragmatic equivalence classes directly, discarding syntactic and semantic distinctions that do not affect the optimal action.For a given rate, preserving those classes maximizes achievable utility.Source: 2) An encoding function that operates… · p. 92
VIII. PRAGMATIC CHANNEL CODING
This section extends channel coding to pragmatic equivalence, grouping input-output sequences by their optimal actions and defining pragmatic capacity as the maximum rate achievable while tolerating action-irrelevant errors. It characterizes reliable communication at rates at or below Cp, with a two-level code separating pragmatic action classes from syntactic codewords.
A. Jointly Asymptotic Equipartition Property and Jointly Reified Typical Set
- The joint reification mapping partitions jointly typical syntactic pairs into pragmatic equivalence classes, merging semantically distinct pairs that share an optimal action.A reified typical set has approximate cardinality 2^n(H(X,Y)-Hp(X,Y)); independent pairs fall in the same set with probability about 2^-nIp(X;Y), versus 2^-nI(X;Y) classically.Source: A. Jointly Asymptotic Equipartition… · pp. 92, 96, 98
B. Pragmatic Channel Coding Theorem
- The two-level code selects a pragmatic index, then a syntactic codeword from its reified typical set; decoding recovers the pragmatic index while ignoring syntactic variations.Random coding and jointly typical decoding based on the jointly reified typical set establish reliable transmission up to pragmatic capacity.Source: B. Pragmatic Channel Coding Theorem · pp. 98–100
- Rates below Cp are achievable, rates above Cp have error bounded away from zero, and CoIp(R) becomes unbounded above Cp.Thus Cp is the threshold for resource-limited pragmatic communication.Source: B. Pragmatic Channel Coding Theorem · p. 100
- Reliable pragmatic communication is possible for R ≤ Cp, while any Rtot > Cp prevents the error probability from tending to zero.The converse uses the bound on up pragmatic mutual information by nCp.Source: B. Pragmatic Channel Coding Theorem · p. 102
- CoIp(R) is finite for R ≤ Cp and increases to infinity as R approaches Cp, making Cp the threshold for finite-resource reliable communication.The passage characterizes Cp as the point beyond which no finite resource can guarantee reliable pragmatic communication.Source: B. Pragmatic Channel Coding Theorem · p. 102
- Pragmatic coding generalizes Shannon and semantic coding, and tolerating errors that do not affect the optimal action achieves a strictly larger capacity.The passage identifies the special cases Cp = C for trivial isoteleia and synonymous mappings, and Cp = Cs when only synonymous mapping is nontrivial.Source: B. Pragmatic Channel Coding Theorem · p. 102
- The two-level code assigns pragmatic indices to optimal actions and reification indices to syntactic codewords, while the decoder recovers only the pragmatic index.The total rate decomposes as Rtot = Rp + Rr; the structure can be refined to include synonymous classes when intermediate semantic information is of interest.Source: B. Pragmatic Channel Coding Theorem · p. 102
IX. PRAGMATIC LOSSY SOURCE CODING
This section develops pragmatic lossy source coding by defining task-relevant distortion and joint typicality, then establishing the rate-distortion threshold through achievability and converse results that connect coding rate, distortion, and preservation of decision-relevant information.
A. Pragmatic Distortion and Jointly Typical Set
- The reification mapping composes synonymous and isoteleia mappings to group source–reconstruction pairs by optimal-action equivalence, while pragmatic distortion measures utility or decision-quality loss.Pragmatic distortion can be defined as utility loss or another non-negative function reflecting degraded decision quality.Source: A. Pragmatic Distortion and Jointly… · p. 103
- Isoteleia merges semantic differences that do not affect optimal actions, synonymy merges syntactic differences that do not affect meaning, and Hp(X, Xhat) ≤ Hs(Xtilde, Xhattilde) ≤ H(X, Xhat).This hierarchical coarsening reduces the number of equivalence classes while preserving pragmatic purpose.Source: A. Pragmatic Distortion and Jointly… · pp. 106, 108
- For sufficiently large n, the pragmatic joint AEP bounds typical source–reconstruction pair probabilities using pragmatic joint entropy Hp(X, Xhat).The bounds are 2^-n(Hp(X, Xhat)+ε) ≤ P(x^n, xhat^n) ≤ 2^-n(Hp(X, Xhat)−ε).Source: A. Pragmatic Distortion and Jointly… · pp. 106–107
- Independent sequence pairs share a pragmatic typical set with probability about 2^-nIp(X; Xhat), versus the classical joint-typicality probability 2^-nI(X; Xhat).The passage identifies this higher probability as reflecting the pragmatic rate-distortion gain.Source: A. Pragmatic Distortion and Jointly… · p. 108
B. Pragmatic Rate-Distortion Coding Theorem
- Rp(D) is the task-oriented compression limit: above it a code exists, below it average distortion exceeds D−ε, and full pragmatic VoI occurs iff R≥Rp(D).The VoI obtained from the reconstruction equals the full pragmatic value at or above the threshold and is strictly lower below it.Source: B. Pragmatic Rate-Distortion Coding… · p. 109
- The coding argument establishes achievability for sufficiently large blocklengths, with the encoding error probability tending to zero.The passage states that the error probability vanishes and that this proves achievability.Source: B. Pragmatic Rate-Distortion Coding… · p. 111
- For R < Rp(D), any sufficiently long code must have average distortion greater than D − ϵ.Assuming average distortion at most D leads to a contradiction, completing the converse.Source: B. Pragmatic Rate-Distortion Coding… · p. 112
- Full pragmatic value of information is attainable iff R ≥ Rp(D); below that threshold, lost pragmatic information reduces achievable utility.The reconstruction must preserve all pragmatic distinctions to attain the full VoI available from direct observation.Source: B. Pragmatic Rate-Distortion Coding… · pp. 112–113
- The rate-distortion hierarchy is Rp(D) ≤ Rs(D) ≤ R(D), with identity reification recovering classical rate-distortion and nontrivial synonymous mapping recovering semantic rate-distortion.Setting extra reification rates Rr and Rc to zero achieves pure pragmatic compression; increasing them preserves additional syntactic or semantic detail.Source: B. Pragmatic Rate-Distortion Coding… · p. 113
X. PRAGMATIC INFORMATION MEASURE OF CONTINUOUS MESSAGE
This section extends pragmatic information measures to continuous messages, using isoteleic volumes to characterize signals leading to the same optimal action. It develops Gaussian channel and source rate functions and bilateral and single-sided information cost and value measures, including value-of-information measures as rate-distortion duals.
1) Pragmatic Entropy for Continuous Messages:
- Quantization extends the entropy hierarchy to continuous alphabets, where isoteleic volumes measure signal-space regions leading to the same optimal action.The resulting pragmatic differential entropy includes an expected log-volume term, is defined relative to a reference measure, and requires E[log Ω] ≤ h(W) for non-negativity.Source: A. Pragmatic Entropy and Pragmatic… · p. 114, 1) Pragmatic Entropy for Continuous… · pp. 115–116
A. Pragmatic Entropy and Pragmatic Mutual Information for Continuous Message
- The continuous formulation replaces finite-alphabet equivalence classes with isoteleic volumes, obtained as the quantization limit of the discrete theory.The passage states that the entropy hierarchy is preserved and that later Gaussian channel and source characterizations inherit the operational meaning of the discrete coding theorems.Source: A. Pragmatic Entropy and Pragmatic… · p. 114
2) Pragmatic Mutual Information for Continuous Variables:
- Continuous up and down pragmatic mutual information differ from classical mutual information by abstraction terms: under E[log(ΩwΩv)] ≥ 0, up is no smaller and down is no greater.Down pragmatic mutual information may be negative; the passage notes that its positive part can be used, and Gaussian examples show abstraction can enhance or reduce measured information depending on whether coarsening affects joint or marginal entropy.Source: 2) Pragmatic Mutual Information for… · pp. 117–118
B. Pragmatic Capacity of Gaussian Channel and Pragmatic Cost of Information
- For Gaussian channels, pragmatic capacity and its inverse cost characterize achievable pragmatic rates; bilateral abstraction requires less power than single-sided abstraction.In the band-limited example, at 3 prabits/s, bilateral abstraction requires zero power while the perceptual path requires 1 W.Source: B. Pragmatic Capacity of Gaussian… · pp. 119–121
C. Pragmatic Rate Distortion of Gaussian Source and Pragmatic Value of Information
- For Gaussian sources, pragmatic rate-distortion measures the minimum rate under action-level distortion, with equality for a Gaussian test channel and an optimal isoteleia partition.Pragmatic value of information is the Legendre-Fenchel dual of rate-distortion functions.Source: §X · p. 113, C. Pragmatic Rate Distortion of… · pp. 122–123
- For Gaussian quadratic utility, bilateral VoI is the Legendre-Fenchel dual of the rate-distortion function, while symmetric perceptual and expressive paths have identical single-sided expressions.Either single-sided measure is termed single-sided VoI.Source: C. Pragmatic Rate Distortion of… · pp. 123–124
- Bilateral pragmatic abstraction yields higher utility gain than single-sided abstraction at a given rate because coarsening both ends discards more decision-irrelevant information.The three VoI measures are ordered by the hierarchy stated for Ω ≥ 1.Source: C. Pragmatic Rate Distortion of… · pp. 123–124
- At R = 1, bilateral coarsening yields VoI of 0.984 utils and distortion 0.0156, versus single-sided coarsening’s 0.9375 utils and 0.0625 distortion.The bilateral distortion is fourfold lower at the same rate.Source: Table X · p. 124, C. Pragmatic Rate Distortion of… · p. 124
- As R →∞, bilateral and single-sided VoI approach 1 while distortion tends to 0; for Ω = 1, classical VoI is 1 − 2^-2R, strictly below pragmatic VoI at finite R.At P = 1 and Ω = 2, bilateral coarsening reaches the single-sided distortion of 0.0625 at R = 0, saving 1 prabit.Source: C. Pragmatic Rate Distortion of… · p. 125
- The closed-form CoI and VoI expressions assume a continuous Euclidean action space with MSE distortion; discrete actions require measures such as Hamming distance or utility loss.The general method remains: CoI is the inverse of achievable rate functions, and VoI is the Legendre-Fenchel dual of rate-distortion functions, including single-sided cases.Source: C. Pragmatic Rate Distortion of… · p. 125
- Gaussian-channel and Gaussian-source CoI and VoI complete the bilateral and single-sided duality spectrum for continuous task-oriented communication with explicit resource-utility trade-offs.The section frames these measures as a comprehensive design framework.Source: C. Pragmatic Rate Distortion of… · p. 125
XI. JOINT OPTIMIZATION OF PRAGMATIC INFORMATION SYSTEM
This section characterizes pragmatic source-channel coding limits and develops joint rate, control, and abstraction optimization under resource constraints. It derives Gaussian power–utility trade-offs and extends joint optimization to static and dynamic communication-control-decision settings, identifying optimal trade-offs for single-step and multi-step systems.
XI. JOINT OPTIMIZATION OF PRAGMATIC INFORMATION SYSTEM
- The section formulates closed-loop co-design of communication rate, control law, and pragmatic abstraction through a pragmatic Lagrangian.The unified optimization targets communication and control under resource constraints.Source: §XI · pp. 125–126
A. Pragmatic Source-Channel Coding
- Reliable pragmatic transmission is possible iff Hp(W) ≤ Cp or Rp(D) ≤ Cp, enabling recovery of pragmatic intention without exact syntactic reconstruction.The value-cost characterization links this feasibility to achieving full pragmatic value within the available resource budget.Source: A. Pragmatic Source-Channel Coding · pp. 126, 129, 1) Separation Principle and the… · p. 130
1) Separation Principle and the Imperative of Joint Optimization:
- Asymptotically, pragmatic source and channel codes can be designed independently, but finite blocklength or complexity, delay, and memory constraints make separation suboptimal.Joint optimization couples the rate that sets source abstraction and channel error protection to the resource trade-off.Source: 1) Separation Principle and the… · pp. 130–131
2) Gaussian Source over Gaussian Channel:
- In the Gaussian-source/AWGN-channel case, closed-form pragmatic quantities yield a feasibility trade-off linking transmit power and abstraction to distortion and target VoI.Under equal source and channel isoteleic volumes, the Ω^8 factor combines pragmatic gains from the source–reconstruction and input–output pairs.Source: 2) Gaussian Source over Gaussian Channel · pp. 132–133
- The Gaussian value-cost relationship shows utility gain increasing with transmit power but saturating at P, while pragmatic abstraction amplifies effective SNR and lowers power needed for a given utility.When Ω = 1, the expression reduces to the classical value-cost relationship.Source: 2) Gaussian Source over Gaussian Channel · p. 133
25.6 −1 ≈−0.96 (zero power suffices).
- At Ω = 2, pragmatic abstraction achieves VoI ≈ 0.996 with Ptx = 0, while one-sided coarsening yields less VoI than bilateral coarsening for finite Ω > 1 and Ptx.The asymmetric case amplifies effective SNR by Ω^4 rather than Ω^8.Source: §25.6 · p. 134
- The Gaussian example provides a closed-form model for jointly optimizing communication power, decision performance, and pragmatic abstraction.Its expressions support principled trade-offs in task-oriented communication system design.Source: §25.6 · p. 134
B. Communication-Control-Decision Joint Optimization
- The framework extends Lagrangian joint optimization to closed-loop systems and gives optimality conditions for static single-step and dynamic multi-step settings.It targets joint design of communication rate, control policy, and pragmatic abstraction under resource constraints.Source: B. Communication-Control-Decision… · pp. 134–135
1) Static Joint Optimization: Lagrangian Approach and Single-Sided Extensions:
- The static joint optimization theorem considers a state, action, utility, and rate-constrained pragmatic observation, with communication cost specified by CoI_p(R).The observation satisfies I_p(S; Y) ≤ R, and the receiver maps it to an action through a policy.Source: 1) Static Joint Optimization:… · p. 135
Lagrangian relaxation of the constrained utility maximization problem
The static Lagrangian framework characterizes optimal communication by balancing marginal information value against resource cost, with the optimum yielding behavioral capacity. Its dynamic extension uses belief-state optimization to jointly allocate communication and control resources over time.
Lagrangian relaxation of the constrained utility maximization problem
- At the static optimum, the Bayes decision rule applies, marginal value equals shadow-priced marginal cost, and the Lagrangian value is behavioral capacity.The necessary conditions are presented as sufficient under the stated framework.Source: Lagrangian relaxation of the… · pp. 135, 138
- Single-sided perceptual and expressive extensions substitute their respective costs while retaining the same optimality conditions; in perceptual LQG, optimal power decreases as λ rises.The perceptual cost is sensing power, and the expressive cost is transmit power.Source: Lagrangian relaxation of the… · p. 136
- For bilateral Gaussian communication, increasing pragmatic abstraction raises the net-benefit curve without changing the optimal rate.Figure 17 illustrates the marginal-value/marginal-cost balance and the abstraction effect.Source: Figure 17 · p. 137, Lagrangian relaxation of the… · p. 138
- Communication stops when P ≤ λσ2; otherwise behavioral capacity rises with rate with diminishing returns, saturates at P, and falls as resource costs increase.Larger Ω raises capacity and makes its decline with resource cost less steep.Source: Lagrangian relaxation of the… · pp. 138–139, Figure 18 · p. 139
2) Dynamic Joint Optimization: Sequential Decision Making:
- For a discounted POMDP, the dynamic Lagrangian yields a belief-state Bellman optimization that couples action and rate through their effects on future beliefs.The per-step communication cost is weighted by λ, while future value captures gains from improved state information.Source: 2) Dynamic Joint Optimization:… · pp. 139, 140, 142
- In stationary LQG systems with linear rate cost, the optimal rate is constant over time, decreases with λ, and is found using a Riccati equation augmented by the rate penalty.The example illustrates tractable joint design of communication and control.Source: 2) Dynamic Joint Optimization:… · p. 142
XII. CONCLUSIONS · APPENDIX
The paper presents pragmatic information theory as a framework unifying communication, control, and decision-making through action-based equivalence and a hierarchy of syntactic, semantic, and pragmatic information. Its conclusions outline measures and coding results, engineering pathways, and limitations that motivate future work.
A. Summary of Contributions
- The isoteleia mapping treats distinct semantic paths leading to the same optimal action as pragmatically equivalent, inducing a hierarchy that removes task-irrelevant distinctions.The framework unifies communication, control, and decision-making.Source: A. Summary of Contributions · pp. 142–143
- The theory defines pragmatic entropy, mutual information, capacity, and rate-distortion, alongside VoI, CoI, a cross-layer Lagrangian, and three coding theorems.It also gives the behavioral-capacity functional, extends to closed-form Gaussian expressions, and formulates joint optimization in static and dynamic settings.Source: A. Summary of Contributions · p. 143
B. Engineering Implications and Deployment Pathways
- The framework targets task-oriented communication and intelligent systems by evaluating information for its effectiveness in guiding actions and balancing utility gains against resource consumption.Applications named include networked control, autonomous systems, embodied AI, human-machine interaction, and social and cognitive systems.Source: B. Engineering Implications and… · p. 143
- When utility is unspecified, deep networks can learn an implicit isoteleia mapping by approximating optimal actions from data using task-oriented losses.The proposed three-stage pipeline trains on task-specific data, adapts inference to context, then refines the mapping through reinforcement learning or online fine-tuning for non-stationary environments.Source: B. Engineering Implications and… · pp. 144–145
- For deployment, the isoteleia mapping should adapt to context and produce deterministic actions, supporting consistent behavior in safety-critical systems.Context-gating or conditional networks can adjust mappings to environmental conditions; deterministic inference or temperature-annealed Gumbel-Softmax can yield hard decisions.Source: B. Engineering Implications and… · p. 144
C. Limitations and Future Directions
- The framework assumes stationarity and ergodicity, uses asymptotic coding theorems, centers on single-agent or centralized settings, and faces intractable high-dimensional belief-state optimization and neural approximation gaps.Future work includes non-stationary coding, finite-blocklength dispersion bounds, game-theoretic multi-agent methods, finite-sample learning guarantees, and approximate dynamic programming.Source: C. Limitations and Future Directions · p. 145
- The authors envision pragmatic information theory as a unified language for next-generation goal-directed systems that extract value from information under resource constraints.They also identify cognitive science, linguistics, and economics as cross-disciplinary application areas.Source: C. Limitations and Future Directions · pp. 145–146
A. Proof of Theorem 33
The proof derives property (1) from the definition of A(n), then uses the stated epsilon limit and properties (1)–(2) to establish property (3).
A. Proof of Theorem 33
- Property (1) follows directly from the definition of A(n).Source: A. Proof of Theorem 33 · p. 149
- The proof states that epsilon tends to 1 as n tends to infinity and that for any epsilon > 0, there exists n0.Source: A. Proof of Theorem 33 · p. 149
- Using property (2) and the bounds from property (1), the proof establishes property (3).Source: A. Proof of Theorem 33 · p. 149
B. Proof of Theorem 34
The proof establishes property (1) from the definition of E(n), then derives property (2) through probability-normalized and right-inequality steps.
B. Proof of Theorem 34
- Property (1) follows directly from the definition of E(n).The proof then turns to the inequality in property (2).Source: B. Proof of Theorem 34 · p. 150
- The cited text says dividing by P(wn) > 0 yields a result, and introduces a step for the right inequality, without specifying in these excerpts what either step concludes.Source: B. Proof of Theorem 34 · p. 150
C. Proof of Theorem 36
The proof of Theorem 36 derives property (1) directly from the definition of B(n) and proceeds through the inequality in property (2).
C. Proof of Theorem 36
- Property (1) follows directly from the definition of B(n), and the proof proceeds to the inequality in property (2).Source: C. Proof of Theorem 36 · p. 150
- The proof divides by P(wn) > 0, considers the right inequality, and then concludes.Source: C. Proof of Theorem 36 · p. 151
D. Proof of Theorem 39
The proof derives properties (1)–(3) from the definition of A(n) and establishes property (4) using independence and bounds from the entropy hierarchy and the size of A(n).
D. Proof of Theorem 39
- Properties (1)–(3) follow directly from the definition of A(n).The proof invokes the law of large numbers for the i.i.d. pragmatic sequence pair, aggregating probabilities over reified equivalence classes.Source: D. Proof of Theorem 39 · p. 151
- Property (4) follows from independence of the reified variables and bounds on sequence probabilities from the entropy hierarchy and the size of A(n).The entropy hierarchy used is Hp(X) ≤ Hs(˜X) ≤ H(X).Source: D. Proof of Theorem 39 · p. 151
E. Proof of Theorem 40 · F. Proof of Theorem 43
The proof sections establish theorem properties from the definitions and derive probability bounds using positivity, independence, the law of large numbers, and the entropy hierarchy.
E. Proof of Theorem 40
- For Theorem 40, property (1) follows directly from the definition.Source: E. Proof of Theorem 40 · p. 152
- For Theorem 40, dividing by P(xn, yn) > 0 yields a lower bound, and reversing the inequality gives the corresponding upper bound.Source: E. Proof of Theorem 40 · p. 152
F. Proof of Theorem 43
- For Theorem 43, properties (1)–(3) follow directly from the definition of A(n).Source: F. Proof of Theorem 43 · p. 152
- The law of large numbers is applied to the i.i.d. pragmatic sequence pair, aggregating probabilities over reified equivalence classes.Source: F. Proof of Theorem 43 · p. 152
- For property (4), the proof uses independence of ˙Xn and ˆ˙Xn.Source: F. Proof of Theorem 43 · p. 152
- The probability bounds use the entropy hierarchy Hp(X) ≤ Hs(˜X) ≤ H(X).Source: F. Proof of Theorem 43 · p. 152
- The proof also invokes the size bound on A(n).Source: F. Proof of Theorem 43 · p. 152