Source-linked AI summary
Innovation-Based Sampling for New Information
Jiping Luo, Anthony Ephremides, Nikolaos Pappas
TL;DR
Communicating raw observations can be wasteful, motivating sampling that selects sufficiently new information relative to prior samples. The paper introduces innovation-based sampling and characterizes it for a Wiener process; its silence-aware MMSE estimator uses AoI and reduces asymptotic MSE relative to ZOH.
Problem
Communicating every piece of raw data can be wasteful or infeasible, raising the question of how much to sample and what information is worth conveying.
Source: §I · p. 1Method
Innovation-based sampling generates samples when the process enters a sufficiently new region relative to previous samples, and the paper characterizes its process and estimation behavior.
Source: 1) Innovation-Based Sampling · p. 1, §I · p. 1Results
82.14% asymptotic MSE reduction relative to ZOH is achieved by the silence-aware MMSE estimator, while the sampling rate vanishes asymptotically.
Source: 3) Age-Aware Estimation · p. 2, §VI · p. 15Takeaways & Limitations
The absence of an innovation is informative, so the MMSE estimate evolves with AoI rather than simply retaining the latest received sample.
Source: 3) Age-Aware Estimation · p. 2
Abstract
from arXiv · showhide
This work introduces innovation-based sampling of continuous stochastic processes, in which a sample is generated only when the process reveals ``sufficiently new'' information relative to all previous samples. The resulting innovation process admits a lattice structure and a three-dimensional state representation. For a Wiener process, we characterize the direction, timing, and frequency of innovations. We show that direction reversals become increasingly rare and establish limit theorems for their number. Innovations also become progressively sparser: the expected number of innovations grows only as the square root of time, and hence the sampling rate vanishes asymptotically. We then study remote estimation from sparsely received innovations. Notably, the absence of an innovation carries information: the minimum mean-square error (MMSE) estimate evolves with the age of information (AoI) and achieves a substantial MSE reduction over the conventional silence-ignorant zero-order hold (ZOH) estimator. Finally, we develop tractable affine-age and exponential-age approximations for practical use. Overall, information is conveyed not only by the content of innovations, but also by their direction and timing.
I. INTRODUCTION
The introduction motivates sampling that transmits novel, task-relevant information rather than repeated observations, and presents innovation-based sampling as a history-dependent alternative. For a Wiener process, the paper characterizes innovation directions, reversals, timing, and rate, then uses informative silence for remote estimation.
1) Innovation-Based Sampling:
- Innovations form a lattice process with a three-dimensional state representation, so the sampler need not retain its full history.The state representation accompanies a spread that grows linearly with the number of innovations.Source: 1) Innovation-Based Sampling · p. 1
2) Characterization of the Innovation Process:
- For a Wiener process, reversal probability at innovation n is 1/(n+1), and independent reversal indicators support strong-law and central-limit theorems for their count.The direction sequence is a time-inhomogeneous binary Markov chain.Source: 2) Characterization of the Innovation… · p. 1
- The expected number of innovations grows on the order of √t, while the sampling rate decays on the order of t^-1/2 and vanishes asymptotically.Expected inter-sampling time grows linearly with n, whereas expected sampling time grows quadratically.Source: 2) Characterization of the Innovation… · p. 2
3) Age-Aware Estimation:
- 82.14% asymptotic MSE reduction relative to ZOH is achieved by the silence-aware MMSE estimator under innovation-based sampling.The estimator evolves with AoI; under deviation-based sampling, the MMSE estimator reduces to silence-ignorant ZOH.Source: 3) Age-Aware Estimation · p. 2
A. Goal-Oriented Semantic Communications
- Goal-oriented communication seeks to quantify information value and selectively transmit what contributes to a system goal, while freshness alone may not indicate usefulness.The introduction motivates innovations as sufficiently new information relative to prior samples, conveying information through content, direction, and timing.Source: §I · p. 1, A. Goal-Oriented Semantic Communications · p. 2
B. Sampling Rules
- Innovation-based sampling triggers on excursions beyond the previously sampled range, unlike deviation-based sampling, which references the latest sample.The innovation rule is history-dependent and its sample sequence is a pathwise thinning of deviation-based samples.Source: B. Sampling Rules · pp. 2–3
B. Two Classical Sampling Rules
The section contrasts classical sampling rules with an innovation-based sampler, which samples only when the process exits the range of previously sampled values by a threshold. For a Wiener process, this rule yields a lattice-structured innovation process with a compact state representation.
B. Two Classical Sampling Rules
- Periodic sampling uses equally spaced times, while deviation-based sampling triggers when the process moves a fixed threshold from its last sampled value.Both retain less history than the innovation-based rule: periodic sampling ignores the trajectory, and deviation-based sampling retains only the latest sample.Source: B. Two Classical Sampling Rules · p. 4
C. Innovation-Based Sampling Rules
- An innovation occurs when the process exceeds the recorded maximum or minimum by at least δ, so samples outside that previously explored range are communicated.The rule uses the recorded extrema as references and generates samples only at innovation times.Source: C. Innovation-Based Sampling Rules · p. 4
- Direction reversals mark a change in exploration and may warrant additional system inspection or intervention.The section defines upper and lower innovations by whether a sample sets a new recorded maximum or minimum.Source: C. Innovation-Based Sampling Rules · pp. 4–5
D. Lattice Structure of the Innovation Process
- The innovation process lies on the lattice δZ, and its sampled-value spread is nδ after n innovations.The lattice structure supports a three-dimensional state representation (n, Ξn, Dn), avoiding the need to retain the full sample history.Source: D. Lattice Structure of the Innovation… · p. 5
A. First-Exit Time Analysis
- For a Wiener process, each inter-sampling time is a first exit from an interval whose bounds depend on innovation direction and index n.The strong Markov property makes the two-dimensional state (n, Dn) sufficient to characterize the sampler; Ξn is needed to decode the innovation value.Source: A. First-Exit Time Analysis · p. 5
B. Binary Chain Representation of Innovation Directions
Under innovation-based sampling, directions form a time-inhomogeneous binary Markov chain whose reversals become less likely as innovations accumulate. Reversals are independent, increasingly informative events, and their count grows logarithmically with a Gaussian limiting fluctuation; deviation-based sampling instead has frequent reversals.
B. Binary Chain Representation of Innovation Directions
- The innovation directions form a time-inhomogeneous binary Markov chain, with reversal probability 1/(n+2), unlike the independent, equiprobable directions under deviation-based sampling.Fig. 3 presents the binary-chain representation of innovation directions.Source: B. Binary Chain Representation of… · pp. 5–6, Figure 3 · p. 6
C. Independence and Limit Theorems for Direction Reversals
- Reversal indicators are mutually independent, with P(In = 1) = 1/(n+1), so reversal surprisal grows as log2(n + 1).The passage relates increasingly rare reversals to their increasing informativeness.Source: C. Independence and Limit Theorems for… · p. 6
- The reversal count grows logarithmically almost surely, while its standardized fluctuations are asymptotically Gaussian.Source: C. Independence and Limit Theorems for… · p. 6
- Under deviation-based sampling, reversal probability is 1/2, each reversal carries one bit of surprisal, and the expected reversal count grows linearly.Source: C. Independence and Limit Theorems for… · p. 6
D. Distributions of the Inter-Sampling and Sampling Times
For innovation-based sampling of a Wiener process, inter-sampling intervals lengthen as innovations accumulate, making the sampler progressively less active. The analysis characterizes interval distributions and moments, and contrasts the resulting sparsity with deviation-based sampling.
D. Distributions of the Inter-Sampling and Sampling Times
- Theorem 2 characterizes the inter-sampling-time distribution by its Laplace transform and gives its mean and variance; the inter-sampling times are mutually independent.Source: D. Distributions of the Inter-Sampling… · pp. 6–7
- The expected inter-sampling time grows linearly with the number of innovations, while the expected sampling time grows quadratically.Innovations therefore become progressively sparser as the range of communicated values expands.Source: D. Distributions of the Inter-Sampling… · p. 7
- Deviation-based sampling has inter-sampling times with mean δ^2, so its sampling activity does not decrease as more samples are generated.Source: D. Distributions of the Inter-Sampling… · p. 7
E. Expected and Asymptotic Sampling Rates
For a Wiener process, innovation sampling becomes progressively sparser: the expected innovation count grows on the order of √t while the expected sampling rate decays on the order of t^-1/2 and vanishes. This contrasts with deviation-based sampling, whose asymptotic rate is a positive constant that can become excessive when δ is small.
- Theorem 3 characterizes the expected number of innovations and the expected sampling rate for a Wiener process.The supplied theorem passages identify the quantities and their Gaussian-tail-function notation but do not expose the full formulas.Source: E. Expected and Asymptotic Sampling… · p. 7
- The theorem shows that the expected number of innovations grows on the order of t, while the expected sampling rate decays on the order of t^-1/2 and vanishes asymptotically.Source: E. Expected and Asymptotic Sampling… · p. 8
- Deviation-based sampling instead has a positive asymptotic sampling rate of 1/δ^2, which can become excessively large when δ is small.Source: E. Expected and Asymptotic Sampling… · p. 8
V. ESTIMATORS UNDER INNOVATION-BASED SAMPLING
This section develops remote estimation from innovations received over time. It derives an MMSE estimator that uses silence, then presents affine-age and exponential-age approximations with tractable MSE analysis.
- The section derives an MMSE estimator that exploits the absence of new innovations, then introduces affine-age and exponential-age approximations.The affine-age class includes ZOH, LSI, and OAC estimators; the exponential-age correction approaches the MMSE long-silence limit.Source: §V · p. 8
- The proposed estimator classes are analyzed through their performance, with closed-form MSE expressions for affine-age estimators.Source: §V · p. 8
- The section summarizes its main estimator results in Table III.Source: §V · p. 8
A. MMSE Estimator and Limiting Behavior
Under innovation-based sampling, the MMSE estimate uses the time since the latest sample because silence is informative. This differs from deviation-based sampling, where the MMSE estimator reduces to ZOH.
- Under innovation-based sampling, silence informs the MMSE estimate, which evolves with the age of information.The estimate decomposes into the last received sample and an AoI-dependent correction inferred from the absence of an innovation.Source: A. MMSE Estimator and Limiting Behavior · pp. 8–9
- Under deviation-based sampling, the MMSE estimator reduces to the silence-ignorant ZOH estimator.For this sampling rule, the continuation region is symmetric and the estimator remains at the last received sample.Source: A. MMSE Estimator and Limiting Behavior · pp. 8–9
- As AoI tends to infinity, the MMSE correction approaches the midpoint of the continuation interval.This long-silence limit motivates the LSI estimator.Source: A. MMSE Estimator and Limiting Behavior · p. 9
- Lemma 5 characterizes the MMSE estimator's MSE over a sampling interval using its AoI-dependent correction and killed transition density.Source: A. MMSE Estimator and Limiting Behavior · p. 10
B. Affine-Age Estimators
Affine-age estimators approximate the AoI-dependent MMSE correction with a tractable form and closed-form MSE analysis. The section describes the LSI and OAC variants and reports their gains relative to ZOH.
B. Affine-Age Estimators
- An affine-age estimator applies a correction that varies linearly with AoI, and its MSE has a closed-form expression.The estimator coefficients are fixed during each sampling interval; ZOH, LSI, and OAC are included in this class.Source: B. Affine-Age Estimators · pp. 10–11
J LSI
- The LSI estimator uses the long-silence limit correction of the MMSE estimator.Source: B. Affine-Age Estimators · p. 11
B. Affine-Age Estimators
- The MMSE correction converges exponentially to the LSI correction as AoI increases, whereas the OAC correction varies linearly and is unbounded when its slope is nonzero.Fig. 5 compares estimator correction terms.Source: B. Affine-Age Estimators · pp. 11–12, Figure 5 · p. 12
J OAC
- For OAC, the section derives asymptotic MSE behavior and compares its performance gain over ZOH with LSI and MMSE.Source: J OAC · pp. 12–13
- The LSI and OAC gains over ZOH converge to 50% and 74.29%, respectively, while the numerically evaluated MMSE gain approaches 82.14%.Fig. 6a reports the relative performance gains.Source: J OAC · p. 13, Figure 6 · p. 13
C. Exponential-Age Estimators
Exponential-age corrections address the possibility that affine-age estimators produce unbounded corrections. The estimator’s correction approaches the MMSE long-silence limit, and its optimized form captures much of the benefit of silence over ZOH.
C. Exponential-Age Estimators
- Exponential-age corrections address the unbounded corrections that affine-age estimators may produce by converging to a finite limit.The correction approaches the MMSE long-silence limit as AoI increases.Source: C. Exponential-Age Estimators · p. 13
- The exponential-age estimator has a closed-form MSE over each sampling interval and a closed-form asymptotic gain over ZOH.The theorem gives the MSE using E_n,0 and a correction term for r > 0.Source: C. Exponential-Age Estimators · p. 13
J EXP
- An optimized exponential-age estimator with ρ∗≈16.82 achieves an asymptotic MSE gain of approximately 81.85% over ZOH.The correction starts at zero and approaches the MMSE long-silence limit; ρ controls how quickly it approaches that limit.Source: C. Exponential-Age Estimators · p. 13, J EXP · p. 15
- The asymptotic gain depends on ρ: small values correct too slowly, while large values move the estimate toward the long-silence limit prematurely.The paper numerically optimizes ρ for the reported gain.Source: J EXP · p. 15
VI. CONCLUSION
The conclusion summarizes innovation-based sampling and its estimation implications. For a Wiener process, innovations become sparser and their directions more persistent, while silence informs the receiver’s AoI-dependent MMSE estimate.
VI. CONCLUSION
- Innovation-based sampling generates samples only when the source provides new information relative to all previous samples.The innovation process lies on a lattice and admits a three-dimensional state representation despite its history dependence.Source: §VI · p. 15
- For a Wiener process, innovations become progressively sparser, their sampling rate vanishes asymptotically, and their directions become increasingly persistent.The paper also establishes limit theorems for the number of direction reversals.Source: §VI · p. 15
- Silence carries information under innovation-based sampling, so the MMSE estimate evolves with AoI.The paper introduces affine-age and exponential-age approximations that recover most of the MMSE gain.Source: §VI · p. 15
APPENDIX
The appendix collects mathematical background used in the analysis, including filtrations, Wiener-process properties, and first-exit-time results.
APPENDIX
- The appendix provides background on Wiener processes and stochastic integrals used in the analysis.It directs readers to and for further details.Source: APPENDIX · p. 15
A. Filtrations, Stopping Times, and Martingales
- A filtration represents information available over time, and a stopping time is determined by information available up to the time in question.The appendix also states optional stopping for uniformly integrable martingales at almost surely finite stopping times.Source: A. Filtrations, Stopping Times, and… · p. 15
B. Wiener Process and Its Properties
- The appendix describes Wiener-process properties including independent Gaussian increments and the strong Markov property.The strong Markov property means the process starts afresh after a stopping time.Source: B. Wiener Process and Its Properties · pp. 15–16
C. First Exit Time of the Wiener Process
- For exit from (−α, β), the probability of exiting at β is α/(α + β), while the probability of exiting at −α is β/(α + β).The exit time is almost surely finite, and its moments are finite.Source: C. First Exit Time of the Wiener Process · p. 16
D. Stochastic Integrals
The appendix establishes properties of stochastic integrals and uses Itô’s formula and martingale arguments to evaluate time-integrated Wiener-process moments up to a first exit time.
D. Stochastic Integrals
- The stochastic integral accumulates Wiener increments weighted by an adapted process, and its expectation is zero under the stated conditions.Its second moment is expressed as an ordinary time integral by Itô’s isometry.Source: D. Stochastic Integrals · p. 16
E. Itˆo’s Formula and Polynomial Martingales
- Itô’s formula yields a martingale-based method for evaluating time-integrated moments of a Wiener process up to a first exit time.The derivation applies optional stopping to the resulting martingale.Source: E. Itˆo’s Formula and Polynomial… · p. 17