PaperlayerView exact revision on arXiv

Source-linked AI summary

One Agent per Zone: Decentralised Active Inference under Partial Observability for Multi-Zone HVAC Control on Edge Hardware

Panagiotis Kasnesis, Lazaros Toumanidis, Christos Chatzigeorgiou, Amalia Contiero Syropoulou, Miguel De Prado

arXiv:2610.19790v1cs.MAcs.AI

TL;DR

Shared air loops couple zone demands, but independent zone agents cannot observe the loop or the building-wide peak. The paper assigns each zone an active-inference agent that infers hidden coupling and uses other-floor power as a congestion signal. On unseen noisy weather years, the fleet achieves the lowest mean annual peak and the lowest coincidence factor among the tested controllers, while its energy use is about 10% higher than a reactive rule.

  • Problem

    Shared air-loop capacity couples zone demands, yet each decentralised zone agent lacks direct access to the loop state and cannot observe the collective building peak.

    Source: §I · p. 1

  • Method

    Each zone gets an independent active-inference agent that models unobserved coupling as a latent disturbance and uses metered power from other floors as a congestion signal.

    Source: §I · p. 2

  • Results

    88.3 kW was the lowest mean annual peak among nine tested controllers, and the 0.929 coincidence factor was lower than any baseline on five unseen noisy weather years.

    Source: §I · p. 2

  • Takeaways & Limitations

    Live load information combined with agents that respond differently is needed for the observed staggering, and MADDPG also benefits from the same meters.

    Source: §I · p. 2

    The AIF fleet does not outperform the rule-based controllers on comfort and cost simultaneously; its energy use is about 10% higher than the reactive rule.

    Source: §VII · p. 8, §VIII · p. 8

Abstract

from arXiv · show

Multi-zone HVAC control is a shared-resource problem: zones on a floor share one fan and cooling coil, so each zone's demand affects its neighbours and the building's peak power. One controller per zone scales well and isolates faults, but each controller then observes only its own zone, not the shared air loop. In this work, we present a decentralised controller that assigns each zone an independent active-inference agent, which plans by minimising expected free energy and learns its zone dynamics online. The agents exchange no messages, so none depends on peers or a coordinator; each accounts for the unobserved coupling through a latent disturbance and a congestion signal derived from the metered power of the other floors. Fifteen agents are evaluated in EnergyPlus on a Sinergym office building against three rule-based controllers and three MADDPG fleets, one given the same floor meters. On five noisy weather years unseen during training, the fleet attains the lowest mean building peak (88.3 kW, 3 to 17 % below the baselines) and staggers the floor peaks better than any baseline, with a coincidence factor (building peak over the sum of the floor peaks) of 0.929 versus 0.936 to 0.994. Compared with a reactive rule at similar comfort (0.912 versus 0.916), it lowers the sum of monthly peaks, on which demand charges are billed, by 33 kW (4 %) in every year, at the cost of about 10 % more energy. It costs 9% less per year than the proactive rule (at lower comfort) and 1% less than the MADDPG fleet given the same floor meters, in every test year. The whole fleet runs on a single low-cost Raspberry Pi 5, closing the observation-to-action loop over MQTT in 442 ms at p95, far below the 15-minute control step.

I. INTRODUCTION

Shared HVAC infrastructure couples zone demands, but independent zone controllers cannot observe the air loop or see the building-wide peak. This work uses decentralised active-inference agents and reports lower, better-staggered demand on unseen noisy weather years, alongside analyses of coordination, cost, and edge deployment.

I. INTRODUCTION

  • Because monthly tariffs charge for peak power as well as energy, building controllers must manage comfort while shaping demand.The coincidence factor captures whether loop peaks occur together; lower values indicate more staggering and can reduce the building peak without necessarily reducing each loop’s energy.Source: §I · p. 1
  • Shared air-loop capacity couples zones, while a single joint controller can scale poorly, create a single point of failure, and require redesign when zoning changes.Each decentralised agent sees only its own zone, so other-zone demand and supply-air conditions remain latent influences.Source: §I · p. 1
  • The controller assigns an independent active-inference agent to each zone; agents infer unobserved air-loop coupling as latent disturbances and use other-floor meters as a congestion signal.The approach is presented as a zone-level decentralised AIF application to shared-air-loop HVAC, evaluated against rule-based and DRL controllers on demand metrics.Source: §I · p. 2
  • 88.3 kW was the lowest mean annual peak among nine tested controllers, and the fleet’s 0.929 coincidence factor was lower than any baseline on five unseen noisy weather years.The agents staggered demand across floors without exchanging messages.Source: §I · p. 2
  • Live information about the rest of the building’s load combined with heterogeneous agent responses is needed for the observed staggering; constant or previous-day signals remove the effect.A MADDPG fleet given the same meters also lowers its peak, supporting the role of live load information rather than active inference alone.Source: §I · p. 2
  • The paper compares against well-tuned rule-based controllers, identifies metrics where simple rules prevail, and quantifies the demand-charge level that offsets the fleet’s additional energy use.Source: §I · p. 2
  • The 15-agent fleet runs concurrently on one Raspberry Pi 5, with 442 ms p95 observation-to-action latency over the MQTT round trip.Source: §I · p. 2

II. BACKGROUND

Discrete active inference represents hidden-state dynamics probabilistically, updates beliefs from observations, and selects actions using expected free energy. Its objective combines preference satisfaction with information gain, while online transition learning refines the model during control.

A. Active Inference in Discrete State Spaces

  • An agent uses a probabilistic generative model with likelihood A, transition model B, and preference distribution C over observations.Factoring the state space gives each factor, such as temperature or occupancy, its own likelihood and transition terms, keeping the model small enough for real-time updates.Source: A. Active Inference in Discrete State… · p. 2
  • At each step, the agent infers hidden states from its prior prediction and new observation, then evaluates candidate policies by expected free energy.The objective has pragmatic and epistemic terms, making information seeking part of the objective; Dirichlet counts let the agent learn transitions online.Source: A. Active Inference in Discrete State… · p. 2

III. RELATED WORK

Prior HVAC reinforcement-learning studies largely control buildings with a single agent, while the paper’s testbed models zones sharing air loops. The evaluation uses a controlled five-year noisy-weather test set and also includes unseen climates.

III. RELATED WORK

  • Earlier HVAC reinforcement-learning work on EnergyPlus benchmarks includes single-agent studies, while prior work identifies sample efficiency as a central obstacle.The cited studies compare algorithms across buildings, climates, and reward designs; single-agent approaches do not explicitly model coupling among zones sharing an air loop.Source: §III · pp. 2–3

A. Building and Simulation

  • The study uses the 18-zone ASHRAE OfficeMedium prototype in EnergyPlus, with 15 conditioned zones across three floors and one shared packaged air loop per floor.The custom Sinergym environment exposes per-zone setpoints, extends observations, and adds one electricity meter per air loop; the observation vector has 95 values and the action vector has 30 setpoints.Source: A. Building and Simulation · p. 3

B. Evaluation Protocol

  • The five-seed W1 test set perturbs the training weather year with noise, pairs controllers on identical weather, and is evaluated with learning switched off.The protocol also tests unseen hot and cool climates, and scores all controllers against the same seasonal comfort bands.Source: B. Evaluation Protocol · p. 3

C. Controllers

Controllers share a 15-minute observation-and-setpoint interface, but differ in their control rules, learned policies, and treatment of shared demand. The active-inference design combines an online zone model with a metered congestion signal and heterogeneous energy weights.

C. Controllers

  • Every controller receives observations and returns 30 setpoints every 15 minutes, with learning controllers deployed one agent per zone.Source: C. Controllers · p. 3

1) Rule-based controllers:

  • The reactive and proactive rule-based controllers differ in anticipation and structural awareness, while the proactive rule’s cooling margin spans a comfort-cost range.A demand-limiting variant staggers floor pre-conditioning and sheds setpoints above a building-demand threshold.Source: 1) Rule-based controllers · p. 3

2) MADDPG:

  • Three MADDPG fleets use 15 independent actors trained with a centralized twin critic; one fleet additionally receives floor meters, matching the AIF agents’ information.The actors use zone, weather, time, demand, and energy features, with a GRU and multilayer perceptron.Source: 2) MADDPG · p. 4

3) Active inference:

  • Each AIF agent models temperature, occupancy, outdoor temperature, and an unobserved disturbance, selecting among 25 heating-and-cooling setpoint pairs.A physical prior initializes transitions, then Dirichlet updates learn from experience; preferences reward comfort, relaxed by 4 °C when unoccupied, and penalize HVAC effort.Source: 3) Active inference · p. 4
  • Agents score 25 actions over an assumed two-hour horizon, and a reactive safety rule takes over when a zone is more than 2 °C outside its comfort band.Source: 3) Active inference · p. 4
  • Agents exchange no messages but use other-floor power to form a congestion signal, while individually adapted energy weights prevent agents on the same floor from reacting in lockstep.The fixed reference is the main configuration; the running-reference variant is a sensitivity check whose growing, deployment-history-dependent maximum can initially overstate congestion.Source: 3) Active inference · p. 4
  • The batched implementation has no coupling across agents, so each row produces the same decisions as an independent single-agent model.The fleet adapts for five years on W0 and is frozen for testing.Source: 3) Active inference · p. 4

V. EVALUATION METRICS

The evaluation scores occupied-hour comfort, building demand and peak coincidence, and operating cost; MADDPG is trained with a shared shaped reward.

V. EVALUATION METRICS

  • Occupied-zone comfort score averages temperature-band scores on weekdays from 07:00 to 19:59; zone comfort rate is the share of zone-steps strictly inside the band.The comfort score is 1 inside the band and decreases with the distance outside it.Source: §V · p. 4
  • Demand metrics are annual building peak, the sum of twelve monthly peaks billed by the tariff, and coincidence factor, where 1 means all three floors peak together.Lower coincidence factors indicate floor peaks spread out in time.Source: §V · pp. 4–5
  • The simplified PG&E Schedule B-19 tariff uses time-of-use energy prices of $0.22, $0.32, and $0.45/kWh, plus a $26/kW monthly-peak demand charge.Source: §V · p. 5
  • MADDPG agents share a per-step shaped reward combining time-of-use-scaled electricity, comfort violations, deadband, peak increases, and midpoint bonuses.The deadband term penalizes heating and cooling setpoints less than 2 °C apart; the peak term applies only when demand sets a new monthly peak.Source: §V · p. 5

A. Main Comparison on the Noisy Test Years

Across five noisy test years, the AIF fleet achieves the lowest mean annual peak and coincidence factor among the baselines. Comfort and cost comparisons vary by rule-based baseline, while the additional analyses show sensitivity to test-year conditions and peak timing.

A. Main Comparison on the Noisy Test Years

  • 88.3 kW mean annual peak was lowest among baselines at 90.9–106.0 kW, and the AIF coincidence factor of 0.929 was lower than any baseline.Its peak was below the reactive RBC and every MADDPG fleet in all five years, but below the proactive RBC in only three.Source: A. Main Comparison on the Noisy Test… · p. 5
  • At similar comfort to reactive RBC (0.912 versus 0.916), AIF uses about 10% more energy and costs $1.6k more yearly; it costs $5.7k less than proactive RBC at lower comfort.Source: A. Main Comparison on the Noisy Test… · p. 5
  • 91–93 kW was the proactive RBC peak range on noisy years, while its 2.5 °C setting reached 82.8 kW on the deterministic year but averaged 92.8 kW on noisy years.The passage cautions that a single favorable day can substantially change an annual peak.Source: A. Main Comparison on the Noisy Test… · p. 5
  • 97.2 kW was the demand-limiting RBC peak on noisy years, versus 90.9 kW for plain proactive RBC; the rule reacts after demand spikes when zones are already above setpoints.A tighter limit raised the peak further, likely through rebound after shedding ended.Source: A. Main Comparison on the Noisy Test… · p. 5
  • 83.8 kW was the AIF design-day peak versus 91.6 kW under proactive RBC; when the bottom floor rose, the middle floor stayed near 20 kW.AIF began conditioning earlier, with a short spike around 04:30.Source: A. Main Comparison on the Noisy Test… · p. 5

B. Deterministic Year and First-Year Behaviour

On the deterministic year, AIF beat every baseline in its first year from a physical prior, and learning alone reduced its peak without the congestion signal. On noisy-year cold starts, it was cheaper in every year, though it beat its first-year peak in fewer years.

B. Deterministic Year and First-Year Behaviour

C. Unseen Climates

With the same floor meters, AIF outperformed MADDPG on peak, cost, and energy in every noisy year, while its hot-climate results exposed a comfort trade-off. Ablations indicate that low peaks depend on live load information combined with differentiated agent responses.

C. Unseen Climates

  • In the cool climate, AIF matched the proactive RBC peak (48.5 kW) and cost $4.6k less; in the hot climate, its CSocc was 0.798, lowest among controllers, and its peak exceeded both main RBCs.Source: C. Unseen Climates · p. 7
  • In the hot climate, online learning raised CSocc from 0.798 to 0.846, while the peak rose to 115.6 kW.Source: C. Unseen Climates · p. 7

D. AIF component contribution

  • A constant same-mean signal or a 24-hour-delayed signal never reached the low-peak regime, even at weight 8; other-floor meters did so at lower comfort cost than own-floor or whole-building signals.The latter signals required weight 8 and cost about 0.03 more in comfort.Source: D. AIF component contribution · p. 7
  • Neither congestion nor adaptive energy weights alone lowered the peak; their combination did so only from congestion weight 5, while weight 4 left it at 92 kW.The peak change behaves like a switch, consistent with the coarse action set.Source: Table V · p. 7, D. AIF component contribution · p. 7
  • Fixed random agent weights reached the low-peak regime in four of eight draws, whereas adaptive weights found a working combination every time.Adaptive weights often sat at a bound and switched frequently, leaving some agents to save energy while others protected comfort.Source: D. AIF component contribution · p. 7
  • With the same floor meters, AIF had an 8.7 kW lower peak, lower cost, and lower energy consumption than MADDPG in every noisy year; its CF advantage was 0.007, within year-to-year variability.Source: D. AIF component contribution · p. 7
  • Adding the signal did not change MADDPG's final reward or comfort; comfort stopped improving after about 30 to 40 training years while shaped reward still rose slowly at year 50.Source: D. AIF component contribution · p. 7

E. Deployment on Edge Hardware

The 15-agent fleet runs on a Raspberry Pi 5 with 442 ms p95 observation-to-action latency. Its lower, better-staggered demand trades off against higher energy use, and the evaluation has limits in building, tariff, comfort, and test scope.

E. Deployment on Edge Hardware

  • 442 ms p95 latency for all 15 agents on a Raspberry Pi 5 includes the full MQTT observation-to-action round trip.Latency was measured over 90 decisions after discarding 10 warm-ups; the control step is 15 minutes.Source: E. Deployment on Edge Hardware · p. 8

VII. DISCUSSION AND LIMITATIONS

  • 33 kW lower annual sum of monthly peaks at similar comfort comes with 11 MWh more energy, and the evaluated tariff makes the trade-off uneconomic.At the $26/kW-month demand charge, break-even with the reactive RBC would require about $75/kW-month.Source: §VII · p. 8
  • The AIF fleet adds lower, better-staggered demand, an adjustable congestion weight spanning the RBC range, and an inspectable decision mechanism.The proactive RBC is most comfortable while the reactive RBC is cheapest; neither learning approach beats both on comfort and cost simultaneously.Source: §VII · p. 8
  • Single-year peak results can mislead: one RBC setting looked better on the deterministic year, but that advantage disappeared on noisy years.The authors recommend testing several noisy years, comparing controllers year by year, and reporting monthly-peak sums.Source: §VII · p. 8
  • The study evaluates one building and one tariff, varying weather but not the building or occupancy patterns.Source: §VII · p. 8
  • The fleet is the least comfortable of the strong controllers, and its comfort degrades considerably in hot climates when adaptation is disallowed.Source: §VII · p. 8
  • Comfort scoring covers weekdays from 07:00 to 19:59, although EnergyPlus reports occupants outside those hours, including evenings and Saturdays.Source: §VII · p. 8
  • Each MADDPG variant was trained once, limiting evidence about variability across training runs.Source: §VII · p. 8

VIII. CONCLUSION

The conclusion presents a decentralized fleet of 15 message-free active-inference agents that coordinates demand across shared floor air loops. Across five unseen noisy weather years it achieves the lowest mean annual peak and coincidence factor among tested controllers, with higher energy use than a reactive rule; all agents run on a Raspberry Pi 5 at 442 ms p95 latency.

VIII. CONCLUSION

  • Across five unseen noisy weather years, the fleet achieved the lowest mean annual peak and lower coincidence factor than any tested baseline.Coordination is attributed to a live load signal combined with agents that weigh energy differently; MADDPG given the same meters also improves.Source: §VIII · p. 8
  • Fifteen active-inference agents, one per zone and exchanging no messages, coordinate demand in a building with one shared air loop per floor.Source: §VIII · p. 8
  • The demand behavior costs about 10% more energy than a simple reactive rule and pays off only where demand is charged heavily.Source: §VIII · p. 8
  • All 15 agents run concurrently on a Raspberry Pi 5 with 442 ms p95 latency over the full MQTT round trip.Source: §VIII · p. 8
  • Planned work includes comfort-gating the congestion signal, refining the action set, and testing more buildings and tariffs.The authors also propose seeding the transition model with structural knowledge used by the proactive RBC.Source: §VIII · p. 8

Related papers

Loading 2610.19790v1…