Source-linked AI summary
When Fresh Information Is Not Enough: Age-of-Actuation-Optimal Scheduling for Energy-Harvesting Actuators
Ali Nikkhah, Anthony Ephremides, Nikolaos Pappas
TL;DR
Fresh received data may not produce a fresh actuation when energy is unavailable. The paper models average-AoA scheduling as an MDP with an AoA-threshold optimal policy and compares it with always-transmit AoI optimization. AoA-aware scheduling can lower average AoA while reducing transmissions, though average AoI may rise, and the policies coincide for some channel and energy probabilities.
Problem
State-dependent scheduling has optimized received-data freshness, but prior AoA work used state-independent policies, leaving the state-dependent AoA-optimal policy and its differences from AoI optimization unresolved.
Source: 2) Dynamic Policies · p. 2Method
The paper formulates average-AoA scheduling as an MDP and establishes an AoA-threshold property for an optimal policy.
Source: D. Structural Property of the Results · p. 6, §VI · p. 9Results
Average AoA under the AoA-optimal policy is no greater than under the AoI-optimal policy, while average AoI under the AoI-optimal policy is no greater than under the AoA-optimal policy.
Source: B. Analysis of the Results · p. 8Takeaways & Limitations
State-dependent scheduling can improve average AoA and reduce transmission attempts, although average AoI may increase; the policies coincide for some channel-success and energy-arrival probabilities.
Source: §VI · p. 9, B. Analysis of the Results · p. 9
Abstract
from arXiv · showhide
We study transmission scheduling in an energy-harvesting actuation system where an actuation requires both a received update and available energy. While age of information (AoI) measures information freshness, age of actuation (AoA) measures the time elapsed since the latest completed actuation. The transmitter observes the receiver's AoA and battery state and decides whether to transmit a fresh update in each slot. We formulate average-AoA minimization as a Markov decision process and establish an AoA-threshold property for an optimal policy. We also derive the average AoA in closed form under the AoI-optimal policy, which is always-transmit. Numerical results show that AoA-aware scheduling can improve actuation freshness while reducing the number of transmissions, although information freshness may deteriorate. The two policies coincide for some pairs of channel success and energy-arrival probabilities. For a one-unit battery, the transmission rate under the AoA-optimal policy can vary nonmonotonically with the energy-arrival probability.
I. INTRODUCTION
The paper addresses a gap between freshness of received data and freshness of completed actuations when actuation also depends on harvested energy. It models this setting and compares state-dependent scheduling with AoI-focused policies.
I. INTRODUCTION
- AoI can reset on reception even when an empty battery prevents actuation, so fresh information alone does not capture how fresh the latest action is.AoL assumes a fixed periodic actuation cycle, unlike actuation constrained by random energy availability.Source: §I · p. 1
1) Randomized Policies:
- Randomized policies make state-independent transmission decisions and can support closed-form timeliness analysis.Dynamic policies instead use system-state feedback and can be formulated as MDPs; prior state-dependent work optimized received-data freshness, not actuation freshness.Source: 1) Randomized Policies · p. 1, 2) Dynamic Policies · p. 2
2) Dynamic Policies:
- The paper studies whether an AoA-optimal state-dependent policy differs from the AoI-optimal policy, a question left open by prior work.Prior state-dependent policies addressed received-data freshness, while AoA had been studied under state-independent randomized policies.Source: 2) Dynamic Policies · p. 2
B. Contribution
- The authors formulate average AoA as an average-cost MDP, solve it with relative value iteration, and prove an AoA-threshold structure.A threshold structure in battery state is observed only numerically.Source: B. Contribution · p. 2
- The paper derives a closed-form average AoA for the always-transmit AoI-optimal policy and compares the two policies’ average AoA and AoI.It also analyzes operating regions defined by comparing the policies and their resulting average metrics.Source: B. Contribution · p. 2
- With battery capacity one, the AoA-optimal policy’s transmission rate can vary nonmonotonically with energy-arrival probability.Source: B. Contribution · p. 2
A. System Dynamics
- The transmitter chooses whether to send a packet each slot; successful reception occurs with probability µ, while the receiver harvests one energy unit with probability λ.The battery stores discrete energy units up to capacity Bm, and an energy unit can be used in its harvest slot.Source: A. System Dynamics · pp. 2–3
- Actuation requires a transmission, successful packet delivery, and available energy, including energy harvested in that same slot when the battery was empty.The transmitter’s state-dependent decisions use receiver AoA and battery feedback over a separate noiseless link.Source: A. System Dynamics · p. 3
B. Considered Timeliness Metrics: AoI and AoA
The section distinguishes information freshness from actuation freshness, then formulates average-AoA optimization as an MDP over age and battery state.
B. Considered Timeliness Metrics: AoI and AoA
- AoI tracks time since the freshest received data, whereas AoA tracks time since the destination’s most recent actuation.Actuation requires both data and a possibility, modeled here as energy availability.Source: B. Considered Timeliness Metrics: AoI… · p. 3
A. MDP Characterization
- The average-AoA problem is modeled as an MDP whose state records AoA and battery level, with transmission or no transmission as the available actions.The state is measured at each slot’s end; the instantaneous cost is the state’s AoA and does not depend on the action.Source: A. MDP Characterization · p. 4
- The finite MDP models battery-level-dependent transitions and truncates AoA at Am to keep the state space finite.The truncation value is chosen sufficiently large to represent infinity in practice.Source: A. MDP Characterization · p. 4
B. Problem Formulation
The problem is an infinite-horizon, finite-state, finite-action average-cost MDP, with AoA as its per-stage cost. The analysis establishes conditions for optimality and convergence of relative value iteration.
B. Problem Formulation
- The objective is to minimize long-run average AoA over policies that map each state to a transmission decision.The state comprises AoA and battery level, and the transition probabilities encode the system dynamics.Source: B. Problem Formulation · p. 4
- The transmission rate under the AoA-optimal policy is the long-run fraction of slots scheduled for transmission and represents transmission cost.The formulation also defines the probability of an empty battery under that policy.Source: B. Problem Formulation · p. 4
C. Optimality Analysis
- The MDP is infinite-horizon with finite state and action sets, and its Bellman equation is solved using relative value iteration.The optimal cost function is associated with each state.Source: C. Optimality Analysis · pp. 4–5
- The per-stage cost equals the current state’s AoA and is independent of the action and policy.Source: C. Optimality Analysis · p. 5
- For 0 < µ, λ < 1, the MDP is weakly accessible and every policy’s induced Markov chain has exactly one ergodic recurrent class.Source: C. Optimality Analysis · p. 5
- Relative value iteration converges to the optimal average cost, equivalent to average AoA, independently of the starting state.The theorem also states that the optimal solution is achieved from any initial state.Source: C. Optimality Analysis · pp. 4–5
- If Am is large enough for the optimal policy to transmit in at least one state at every battery level, its induced chains have no transient states and are fully irreducible.Source: C. Optimality Analysis · p. 5
B. Problem Formulation
- The probabilities defined for the AoA-optimal policy exist and are independent of the starting state.Source: B. Problem Formulation · p. 4, C. Optimality Analysis · p. 5
D. Structural Property of the Results
The optimal AoA policy has a threshold structure in age for each battery level. A threshold structure in battery level cannot be established analytically, though it is observed numerically.
D. Structural Property of the Results
- The structural argument uses cost-function properties that hold at each relative value iteration and for the converged optimal cost function.The supporting proof is referred to Appendix B.Source: D. Structural Property of the Results · p. 5
- For a fixed battery level, the optimal policy is threshold-based in AoA: once transmission is chosen at an age, it is chosen at all higher ages.The threshold property is stated for pairs of states with the same battery level.Source: D. Structural Property of the Results · p. 6
- The battery-level monotonicity results do not establish a threshold-based structure in battery level.The battery-state threshold pattern is observed numerically, but remains an analytical question for this model.Source: D. Structural Property of the Results · p. 6
IV. AGE OF INFORMATION (AOI) OPTIMIZATION
For average AoI, always transmitting is optimal, but fresh updates can arrive when the battery is empty and fail to produce an actuation. The paper derives the resulting average AoA in closed form and compares the freshness metrics under both policies.
IV. AGE OF INFORMATION (AOI) OPTIMIZATION
- Always-transmit is optimal for average AoI, with transmission rate 1 and average AoI 1/µ.The policy is optimal because each transmission carries the freshest possible sample and attempts do not create a queue, interfere with future opportunities, or incur a cost or constraint in the AoI optimization.Source: §IV · p. 6
- The average AoA under the AoI-optimal policy is given in closed form as a function of service probability µ, arrival probability λ, and battery size Bm.The result is stated in Theorem 3.Source: §IV · p. 6
- Under always-transmit, the gap between average AoA and average AoI is an empty-battery penalty; the average AoA is 1/λ for µ = 1 and 1/µ for λ = 1.When µ = λ = 1, the average AoA is 1.Source: §IV · p. 6
- Average AoI under the AoI-optimal policy is no greater than under the AoA-optimal policy, while average AoA under the AoA-optimal policy is no greater than under the AoI-optimal policy.The comparisons follow from each policy optimizing its respective metric.Source: §IV · p. 6
V. RESULTS
The section compares average AoA and AoI under the AoA-Opt. and AoI-Opt. policies, while also considering transmission rate. It presents numerical results across channel success and energy-arrival probabilities and battery capacities.
V. RESULTS
- The AoA-Opt. average AoA is obtained by solving the MDP with Relative Value Iteration and validated through simulations; related metrics are computed by simulation or numerical expressions.The RVI uses ϵ = 0.01 and truncation level Am = 300.Source: §V · pp. 6–7
- Under AoI-Opt., the optimal average AoI is 1/µ, and the policy always transmits, giving a transmission rate of 1.Source: §V · pp. 6–7
A. Illustration of the Results
- Fig. 2 displays AoA-Opt. actions and both policies’ AoA, AoI, transmission rate, and empty-battery probability for Bm = 1 and Bm = 10.For Bm = 1 it shows six (µ, λ) pairs, and for Bm = 10 it shows two; green filled squares mark transmission and hollow red squares mark no transmission.Source: A. Illustration of the Results · p. 7, Figure 2 · p. 7
- The figures compare average AoA and AoI across λ and µ, with different battery capacities; Fig. 3 also includes AoA-Opt. transmission-rate insets.Source: A. Illustration of the Results · pp. 7–8, Figure 3 · p. 7
- Fig. 5 maps AoA-Opt. transmission rate over µ and λ, while Fig. 6 marks boundaries between transmission-rate and battery-probability regions.Fig. 7 simulates the µ = 1 transmission rate over 10^7 slots and includes an average-AoA inset for λ ∈ [0.3, 0.5].Source: A. Illustration of the Results · p. 8
- For Bm = 1, the boundaries separating the transmission-rate regions and the energy-related regions coincide.Source: A. Illustration of the Results · p. 8
B. Analysis of the Results
AoA-aware scheduling can improve actuation freshness while using fewer transmissions, though AoI may worsen; the policies coincide in some parameter regions. Battery capacity also shapes energy use and the AoA-optimal transmission rate.
B. Analysis of the Results
- Transmission rate is treated as a cost of sensing and transmission, but it is not explicitly modeled in the optimization and is not directly comparable to actuation energy cost.The analysis measures timeliness using average AoA and average AoI.Source: B. Analysis of the Results · p. 8
- In the greedy region, AoA-Opt. transmits at every slot, so both policies' average AoA and average AoI coincide.This regime generally occurs when the energy-arrival rate is relatively high compared with channel success; for Bm = 1, the boundary is symmetric around µ = λ = 0.5.Source: B. Analysis of the Results · pp. 8–9
- The energy-unlimited region under AoA-Opt. contains that under AoI-Opt.; for Bm = 1 their energy-limitation regions coincide, unlike for Bm = 10.The regions identify where average AoA and average AoI coincide or differ; the conclusion also notes that policy coincidence depends on channel-success and energy-arrival probabilities.Source: B. Analysis of the Results · p. 9, §VI · p. 9
- With µ = 1, AoA-Opt. yields lower average AoA than AoI-Opt., and its average AoA decreases as battery capacity increases.At λ = 1, average AoA equals average AoI and is independent of battery capacity; for Bm = 10, AoA-Opt. keeps sufficient energy for actuation and avoids emptying the battery.Source: B. Analysis of the Results · p. 9
- For µ = 1, the transmission rate approaches λ as Bm increases, but with Bm = 1 it rises, falls, then rises sharply before saturating as λ increases from 0.3 to 0.5.Over that λ range, optimal average AoA decreases monotonically; with a one-unit battery, transmission rate is generally above λ.Source: B. Analysis of the Results · p. 9
VI. CONCLUSION
- The AoA-optimal policy can lower average AoA and transmission attempts while increasing average AoI; across the policies, AoA and AoI favor their respective optimizers.The conclusion reports the transmission trade-off, while the plotted comparisons show average AoA is no higher under AoA-Opt. and average AoI is no higher under AoI-Opt.Source: B. Analysis of the Results · p. 8, §VI · p. 9
APPENDIX A PROOF OF THEOREM 1
The appendix establishes properties of the scheduling MDP and the convergence of its optimization procedure. It also gives conditions under which the optimal-policy chains have stronger recurrence properties.
A. Proof of Theorem 1(i)
- Every stationary policy induces a single recurrent class, making the presented MDP unichain.The recurrent class depends on the policy's transmission decisions across battery levels.Source: A. Proof of Theorem 1(i) · pp. 9–10
B. Proof of Theorem 1(ii)
- Because all stationary policies are unichain, the MDP satisfies the weak accessibility condition.The proof invokes the stated MDP result that unichain stationary policies imply weak accessibility.Source: B. Proof of Theorem 1(ii) · p. 10
C. Proof of Theorem 1(iii)
- The induced chains are ergodic and have unique stationary distributions because their recurrent classes are aperiodic.The state (Am, Bm) belongs to each recurrent class and can transition to itself, establishing aperiodicity.Source: C. Proof of Theorem 1(iii) · p. 10
D. Proof of Theorem 1(iv)
- Weak accessibility, unichain policies, and aperiodic recurrent classes ensure RVI converges to the optimal average AoA independently of its starting state.The proof uses these conditions to establish convergence of Algorithm 1.Source: D. Proof of Theorem 1(iv) · p. 10
E. Proof of Theorem 1(v)
- For optimal policies with Am = 300 and 0 < µ, λ < 1, the assumed age threshold at each battery level leaves no transient states.Under those stated conditions, the induced chains are fully irreducible.Source: E. Proof of Theorem 1(v) · p. 10
F. Proof of Theorem 1(vi)
- Ergodicity ensures the stationary distributions and probabilities (16) and (17) exist and do not depend on the starting state.The proof applies this result to all policies, including optimal ones.Source: F. Proof of Theorem 1(vi) · p. 10
A. Proof of Lemma (i)
- The proof of Lemma (i) uses induction over RVI iterations to establish the claimed inequality through the converged optimal cost function.It compares update terms for transmission and no-transmission actions across battery cases.Source: A. Proof of Lemma (i) · pp. 10–11
B. Proof of Lemma (ii)
The proof establishes that inequality (25) holds throughout the iterations, including at convergence.
B. Proof of Lemma (ii)
- Inequality (25) holds for every iteration, including the converged cost function.The proof compares update terms for transmission and no-transmission actions across battery-level cases.Source: B. Proof of Lemma (ii) · p. 11
APPENDIX C PROOF OF THEOREM 2 AND DISCUSSION ON REMARK 4
The proof establishes that the optimal policy has an age-threshold structure.
APPENDIX C PROOF OF THEOREM 2 AND DISCUSSION ON REMARK 4
- The optimal policy has a threshold structure in age because the relevant difference is non-increasing in A across all battery cases.This establishes the threshold-based structure stated in (28).Source: A. Proof of Theorem 2 · pp. 11–12
B. Discussion on Remark 4
The battery-state threshold argument requires showing that Δθ is non-increasing with battery level, equivalently that Δβ*₂ − Δβ*₁ ≤ 0. The authors report they cannot establish this inequality analytically across the considered expressions.
B. Discussion on Remark 4
- A battery-state threshold would follow if Δθ were non-increasing with battery level, which requires Δβ*₂ − Δβ*₁ ≤ 0.The argument considers states with battery levels B1 and B2.Source: B. Discussion on Remark 4 · p. 12
- The comparison is considered across four relative battery-level cases, including zero, interior, and maximum battery levels.The passage specifies cases such as B1 = 0 with 0 < B2 < Bm, B2 = Bm, and interior battery levels.Source: B. Discussion on Remark 4 · p. 12
- The authors cannot establish Δβ*₂ − Δβ*₁ ≤ 0 from the available lemmas and mathematical relations for expressions (59)–(62).They report the inequality remains unestablished regardless of how the terms are arranged or grouped.Source: B. Discussion on Remark 4 · p. 12
APPENDIX D AVERAGE AOA UNDER AOI-OPT. (GREEDY) POLICY
For the always-transmit policy, the appendix derives stationary battery probabilities and uses the stationary distribution to obtain average AoA. It also reports numerical observations concerning the threshold structure of the optimal policy.
APPENDIX D AVERAGE AOA UNDER AOI-OPT. (GREEDY) POLICY
- Numerical observations show Δθ is non-increasing in three battery-level cases; in the first, it can be non-monotonic without changing sign, and the policy still has a threshold structure.The passage reports the first-case behavior as occasional.Source: Appendix D · p. 12
- When µ = λ, all battery levels have stationary probability 1/(Bm + 1).For µ ≠ λ, the battery-level probabilities follow the geometric expression in (66b), with r = y/x.Source: Appendix D · pp. 12–13
- The appendix obtains the average AoA from the stationary distribution, giving expressions (70b) and (71) for µ ≠ λ and µ = λ, respectively.The transition matrix defines a Markov chain whose stationary distribution supplies the state probabilities.Source: Appendix D · pp. 12–13