PaperlayerView exact revision on arXiv

Source-linked AI summary

EnergyStar++: Towards more accurate and explanatory building energy benchmarking

Pandarasamy Arjunan, Kameshwar Poolla, Clayton Miller

arXiv:1910.14563v2stat.APcs.LGeess.SY

TL;DR

Energy Star benchmarking uses interpretable but potentially inaccurate linear models and limited, infrequently updated survey data. This paper tests interaction-enhanced linear regression and gradient-boosted trees, adds SHAP explanations, and reports lower prediction error than the status quo while noting that generalizability beyond the tested U.S. datasets remains untested.

  • Problem

    Energy Star’s weighted linear models may not capture complex nonlinear relationships, and its CBECS reference dataset is limited and infrequently updated.

  • Method

    The paper evaluates interaction-enhanced multiple linear regression and gradient-boosted trees, using SHAP values and visualizations to interpret predictions.

  • Results

    MLRi and GBT models reduced prediction error by 7.0% and 13.7%, respectively, relative to the Energy Star status quo.

  • Takeaways & Limitations

    Feature-interaction analysis and SHAP visualizations provide a way to expose factors influencing benchmarking scores, while public datasets can support accurate prediction for most building types.

  • Takeaways & Limitations

    The reported generalizability applies only to the three U.S. datasets tested; implementation in other countries was not tested.

Abstract

from arXiv · show

Building energy performance benchmarking has been adopted widely in the USA and Canada through the Energy Star Portfolio Manager platform. Building operations and energy management professionals have long used a simple 1-100 score to understand how their building compares to its peers. This single number is easy to use, but is created by inaccurate linear regression (MLR) models. This paper proposes a methodology that enhances the existing Energy Star calculation method by increasing accuracy and providing additional model output processing to help explain why a building is achieving a certain score. We propose and test two new prediction models: multiple linear regression with feature interactions (MLRi) and gradient boosted trees (GBT). Both models have better average accuracy than the baseline Energy Star models. The third order MLRi and GBT models achieve 4.9% and 24.9% increase in adjusted R2, respectively, and 7.0% and 13.7% decrease in normalized root mean squared error (NRMSE), respectively, on average than MLR models for six building types. Even more importantly, a set of techniques is developed to help determine which factors most influence the score using SHAP values. The SHAP force visualization in particular offers an accessible overview of the aspects of the building that influence the score that non-technical users can readily interpret. This methodology is tested on the 2012 Commercial Building Energy Consumption Survey (CBECS)(1,812 buildings) and public data sets from the energy disclosure programs of New York City (11,131 buildings) and Seattle (2,073 buildings).

1. Introduction

Building energy benchmarking compares non-residential buildings with peers to identify inefficient behavior, and disclosure programs have expanded across cities. The paper asks whether Energy Star models, interpretability, and public-data benchmarking can be improved.

  • 1. Introduction: Benchmarking systems compare buildings with peers to identify inefficient behavior, and their use has expanded across national and city-scale programs.Energy Star covers substantial portions of the U.S. commercial building stock, while disclosure ordinances require selected buildings to report energy data publicly.
  • 1. Introduction: Energy Star benchmarking relies on peer groups and predictive models, but the paper identifies questions about model accuracy, interpretability, and public-data alternatives.The study asks whether linear or nonlinear interaction models can improve prediction, whether interpretability can support decisions, and whether public datasets can replace CBECS.
  • 1. Introduction: The paper compares interaction-enhanced linear regression and nonlinear gradient boosting with the baseline ordinary regression used by Energy Star.It also applies explainable-AI methods and combines New York City and Seattle disclosure data with tax assessment records.

2. Background and novelty

Energy Star provides interpretable peer benchmarking but uses linear models that may miss complex relationships and relies on a limited, infrequently updated survey. The paper proposes interaction models, gradient boosting, and SHAP-based explanations to address these gaps.

  • 2. Background and novelty: Energy Star’s weighted linear regression can miss complex nonlinear relationships, while its CBECS reference data contain 6,720 buildings and are updated approximately every four years.The paper frames nonlinear modeling and city-specific disclosure data as responses to these limitations.
  • 2. Background and novelty: MLR models are interpretable and widely used, but their linear and additive structure limits representation of complex relationships between building attributes and energy use.The paper emphasizes that feature interactions can represent combined attribute effects while preserving interpretability.
  • 2. Background and novelty: The study introduces explicit interaction terms in MLR models and XGBoost models that implicitly capture higher-level interactions for building benchmarking.XGBoost is presented as accurate, scalable, and less data-intensive than advanced alternatives such as deep learning.
  • 2. Background and novelty: The paper presents SHAP values and visualizations as an explainability approach for building performance rating systems.Its evaluation compares the proposed approach with Energy Star and four recent methods across approximately 15,000 buildings and six building types.

3. Open source data sets

The study assembles CBECS and city-scale public datasets to evaluate benchmarking models across common building types. It applies Energy Star filtering and grouping rules while combining public energy and property records for New York City and Seattle.

  • 3. Open source data sets: The study uses CBECS data containing 6,720 buildings across cities and climates to represent the U.S. building stock and validate alternative predictive models.FINALWT supplies the full-sample weight used in Energy Star model development.
  • 3. Open source data sets: New York City and Seattle public datasets combine energy disclosure information with building characteristics and property records.Seattle records were matched using tax parcel identifiers, while New York City records used Borough Block and Lot numbers.
  • 3. Open source data sets: Energy Star building-type, program, data-limitation, and analytical filters were applied to create comparable peer groups.The same building-grouping definitions were used for the public datasets, including grouping offices, banks, and courthouses as office buildings.
  • 3. Open source data sets: Six building-use types were selected because predictive-model details were available and they represented 94% of New York City and 82% of Seattle buildings.Cleaning removed duplicates, inconsistent attributes, and extreme source-EUI observations before grouping.

4. Methodology

The methodology combines interpretable linear regression with feature interactions and nonlinear XGBoost models, then tunes and evaluates them for building-energy prediction. Interaction terms relax MLR’s additivity and linearity assumptions, while boosted trees model complex relationships through sequentially combined CARTs.

  • Overview: Feature-interaction MLR and XGBoost were developed to improve building-energy prediction while retaining or adding interpretability.MLR uses explicit interaction terms; XGBoost uses nonlinear tree ensembles augmented with SHAP values.
  • Multiple linear regression: MLR is interpretable because each predictor has a separable effect, but its additivity and linearity assumptions can inaccurately represent real-world relationships.The model assumes predictor effects are independent and constant across predictor values.
  • Multiple linear regression with feature interactions: Interaction terms represent joint predictor effects, exposing compounded relationships and potentially improving accuracy by better modeling dependencies among building attributes.The paper hypothesizes that building attributes influence energy use beyond their individual effects.
  • Multiple linear regression with feature interactions: An interaction-augmented EUI model adds a GFA·OCC term, making the effect of either predictor depend on the value of the other.The interaction coefficient measures the increase in GFA’s effect on EUI for a one-unit increase in OCC, or vice versa.
  • Gradient boosted trees - XGBoost: XGBoost sequentially adds CART models trained on harder-to-predict samples, and sums their leaf predictions into the final score.The example contains four height-two trees, while model parameters were tuned by grid search with repeated 10-fold cross-validation using RMSE.

5. Model implementation

The study evaluates interaction-enhanced linear and gradient-boosted models against Energy Star’s ordinary MLR using CBECS and public New York City and Seattle data. Across CBECS building types, interaction models improve predictive metrics, while GBT models provide the strongest overall performance; public-data results vary by building type and sample size.

  • 5. Model implementation: The implementation predicted source energy for six building types using Energy Star predictors, while public datasets used total energy and all available building attributes.The evaluation compared ordinary MLR, interaction MLR, and GBT models using adjusted R2, RMSE, NRMSE, and MAPE.
  • 5.2. CBECS data set: GBTi3 achieved the lowest average NRMSE among the evaluated proposed and baseline models, while MLRi4 was lowest except for Random Forest.The comparison included MLRi2, MLRi3, MLRi4, GBTi2, GBTi3, ordinary MLR, and recent machine-learning models.
  • 5.2. CBECS data set: All interaction models achieved higher adjusted R2 and lower NRMSE than ordinary MLR across the six CBECS building types.MLRi2 improved adjusted R2 by 2.3% and reduced NRMSE by 3.2% on average; MLRi3 generally performed better than MLRi2.
  • 5.2. CBECS data set: 24.9% increase in adjusted R2 and 13.7% decrease in NRMSE were achieved by GBTi3 versus baseline MLR across six CBECS building types.GBT models performed better overall than ordinary and interaction MLR models, with GBTi3 strongest on both metrics.
  • Public data sets: On New York City and Seattle data, GBTi2 generally had the highest adjusted R2, but it underperformed on worship buildings and K-12 schools.The worship-building result was associated with only 69 Seattle samples, compared with 249 CBECS samples, and GBT requires more training data than MLR.
  • Public data sets: MAPE was 129% for offices with the baseline comparison and fell to 70% when interaction terms were included.The authors attributed the high office error and low adjusted R2 to diverse activities within the office category.

6. Model explainability

The paper combines interaction-aware models with SHAP visualizations to improve both prediction interpretation and explanation of building energy use. These tools reveal nonlinear and interacting effects that ordinary feature summaries can miss.

  • 6. Model explainability: Nine second-order interaction terms were statistically significant for office-building source energy modeling, including interactions involving GFA.The interaction model showed that GFA’s effect depended on other building attributes, including employee count.
  • 6. Model explainability: GFA, computers, occupants, and operating hours were the most influential office-building features in SHAP-based GBT analysis.Unlike MLR coefficients, SHAP values show each attribute’s influence for individual buildings.
  • 6. Model explainability: SHAP dependence plots showed that computer count could either lower or raise predicted energy use, with wider positive-value spread indicating feature interactions.Few computers generally had negative influence, while many computers generally had positive influence, although other features could reverse that pattern.
  • 6. Model explainability: SHAP force plots present each feature’s contribution to an individual building’s predicted output in an accessible visual format.The force chart was introduced because interaction and SHAP visualizations were considered too technical for average building performance analysts.

7. Discussion

The proposed explainable benchmarking framework is discussed as a bridge between data-driven operational ratings and asset-rating approaches. Its visual explanations may help users interpret performance differences, assess inputs, and identify energy-saving opportunities.

  • 7.1. Bridging the gap with asset rating methodologies: SHAP analysis identifies the interactions and features contributing to office-building predictions, including GFA-related interaction terms.The top three reported interactions were GFA:ComputersCnt, GFA:WorkersCnt, and GFA:OpernHours.
  • 7.1. Bridging the gap with asset rating methodologies: Explainable data-driven models could bridge operational and asset ratings by providing diagnostic information without requiring full physical-attribute data collection.The paper frames this as a potential way to improve scalability and cost-effectiveness when deciding whether deeper analysis is warranted.
  • 7.2. Energy savings interventions: Benchmarking has influenced ownership decisions more than energy-saving implementation, motivating explanations that expose potential intervention opportunities.The paper gives occupancy-related inputs as an example of information that could inform operational decisions.
  • 7.3. Increased trust in performance rating due to explainable quality assessment: A SHAP force plot can show why a building’s predicted EUI is below its peer average by separating factors pushing energy use higher or lower.In the example, lower workers, computers, and operating hours outweighed the upward effects of floor area and full air conditioning.
  • 7.3. Increased trust in performance rating due to explainable quality assessment: Greater visibility into model calculations may support user trust, guidance-following, and quality control through engineering review of inputs and outputs.The paper connects explainability with the possibility that decision-makers will better understand and use rating results.

8. Conclusion

The paper concludes that interaction-aware MLR and GBT models reduce prediction error relative to Energy Star’s status quo while SHAP tools expose influential features and interactions. The authors also report implementation across multiple U.S. building-energy datasets, with generalizability beyond tested data remaining unverified.

  • 8. Conclusion: 7.0% and 13.7% lower error rates were reported for the MRLi and GBT models, respectively, than the Energy Star benchmarking status quo.The authors state that these reductions improve representation of the discrepancy between actual and predicted consumption.
  • 8. Conclusion: Feature-interaction analysis and SHAP visualizations provided a view into the prediction models’ internal influences.The conclusion identifies interpretability as a separate outcome from predictive accuracy.
  • 8. Conclusion: The methodology was accurate and implementable on large building-energy datasets from different U.S. locations.The tested data included three large datasets, while broader international applicability was not tested.
  • 8. Conclusion: The authors suggest that the framework could support additional energy savings, retrofit and operations decisions, and trust in rating-system values.These are presented as potential benefits of gaining model insight with less physical-data collection than physics-based modeling.
  • 8.1. Limitations and reproducibility: Generalizability is limited to the tested input data, and applicability to other countries remains untested.The authors identify deployment in additional city, state, and country contexts as an important next step.

10. Author Credit Statement

The author-credit statement assigns conceptualization across the team while distributing methodology, data work, analysis, software, supervision, and writing responsibilities among the three authors.

  • 10. Author Credit Statement: Anjunan led methodology, data curation, formal analysis, software, and original drafting, while Poolla handled conceptualization and review; Miller contributed conceptualization, investigation, methodology, data curation, drafting, and supervision.The statement lists distinct contributions for all three authors.

Appendix A. Model hyper-parameters

Appendix A lists the models and hyper-parameters used in the study, including their validation procedure and predictor-count notation.

  • The appendix lists the models and their selected hyper-parameters.
  • All models were validated with 10-fold cross-validation over two repeated rounds to select final parameters for each building type.
  • The notation p denotes the total number of predictors, while the method symbol identifies the model passed to caret's train function in R.

Appendix B. Energy Star variables

Appendix B lists the building attributes used in the Energy Star system across different building types.

  • The appendix catalogs building attributes used in the Energy Star system.
  • The listed attributes are organized for use across different building types.
  • The table provides the variable reference for the Energy Star system's building-type-specific inputs.
Loading 1910.14563v2…