PaperlayerView exact revision on arXiv

Source-linked AI summary

From Grey-Box to Green-Box: When can Physics-Informed Machine Learning Reduce Carbon Footprints in Structural Health Monitoring?

Daisy R. Bradley, Nathan A. Hinchliffe, Daniel J. Pitchforth, Matthew R. Jones, Elizabeth J. Cross

arXiv:2609.33387v1cs.LG

TL;DR

Engineering machine learning can add computing energy demand and carbon emissions, and the paper asks whether physics-informed models can reduce that impact. Across SHM case studies, the emissions trade-off varies with the model and performance target, so the authors frame model choice around required performance and environmental cost.

  • Problem

    The paper asks whether embedding physical knowledge in engineering machine-learning models can reduce their environmental impact, given rising computing energy demand and emissions.

    Source: §2 · pp. 1–2

  • Method

    The authors compare physics-informed models across SHM regression cases, relating training-data coverage, error, runtime, and emissions.

    Source: §3.1 · p. 4

  • Results

    Emissions trade-offs vary by case and target error: physics-informed models can reduce training emissions, but can also increase them relative to black-box models.

    Source: §5.1 · p. 9, §7.2 · p. 14, §9.1 · p. 19

  • Takeaways & Limitations

    Model choice should account for the required performance threshold and training time, since a grey-box model may be faster at similar performance while other cases favor black-box models.

    Source: §6.1 · p. 11, §9.1 · p. 20

    The carbon cost of data collection and storage cannot be quantified generically because it depends on the problem and storage setup.

    Source: §4.2 · p. 6

Abstract

from arXiv · show

Machine learning plays an increasingly vital role in engineering, but the corresponding increase in compute time is not without environmental cost. Physics-informed machine learning or "grey-box" models have been developed to overcome some of the limitations of traditional black-box learners, utilising the physical insight that an engineer would have about the structure they are modelling and have shown promising results in the structural engineering field among many others. This work explores whether an additional advantage could be a reduced environmental impact, considering the relationship between training data quantity and training time, linking this duration to carbon emissions from computing. In a structural health monitoring context, four physics-informed machine learning approaches - spanning Gaussian processes and neural networks - are evaluated: residual modelling, input augmentation, hybrid modelling, and constrained learning. The emissions for training each of the models to reach a given error threshold is compared, and in most examples, shown to be lower for the physics-informed models (with input augmented models being an exception). This reduction in training emissions further compounds the environmental savings achieved by collecting and storing less data. Although promising results, we cannot expect a silver bullet and the case studies demonstrate that a trade-off is needed between the increased complexity that comes from introducing physics into a machine learner, against the gain from reduced training data requirements.

2 Introduction

The paper asks whether embedding physical knowledge in engineering machine-learning models can reduce their environmental impact. It examines training data, runtime, and emissions in structural health monitoring, with performance thresholds framing the comparison.

2 Introduction

  • The paper frames rising engineering use of machine learning and AI as increasing computing energy demand and associated carbon emissions.The environmental impact is difficult to quantify and project because major AI providers are not currently required to share energy and water use.Source: §2 · p. 1
  • The study asks whether embedding physics in machine-learning models can reduce environmental impact in structural health monitoring regression tasks.Prior work cited by the authors reports that physics-informed models can reduce training-data burden and outperform black-box models in SHM.Source: §2 · p. 2
  • The authors relate training-data quantity to training time and computing emissions, while also considering the carbon cost of collecting and storing data.They examine the hypothesis that reduced data requirements can lower impact through shorter training duration.Source: §2 · p. 2
  • The paper compares four physics-informed approaches across Gaussian processes and neural networks, assessing emissions required to reach a given error threshold.The approaches are residual modelling, input augmentation, hybrid modelling, and constrained learning.Source: §2 · p. 2
  • The authors position the work as the first environmental comparison of grey-box and black-box models aimed at meeting required performance with minimal impact.They emphasize balancing added model complexity against reduced training-data requirements rather than assuming physics-informed models are always greener.Source: §2 · p. 2

3 Context and literature

The paper situates its question within sustainable machine learning and physics-informed modeling, then compares approaches that introduce physical insight to different degrees. Its case studies relate data coverage, model error, runtime, and emissions, while acknowledging limits in scope and dataset scale.

3 Context and literature

  • Emissions-tracking tools support relative model comparisons when applied consistently, although benchmark studies report variance and systematic under-reporting.The paper distinguishes static estimation tools from dynamic utilities that capture real-time CPU and GPU use.Source: §3 · p. 3
  • Efficiency research has developed methods such as pruning, quantisation, and mixture-of-experts architectures, but is not generally focused on reducing emissions.The paper also notes Jevons’ paradox: efficiency gains can coincide with increased total resource consumption.Source: §3 · p. 3
  • Prior SHM work reports that grey-box models can outperform black-box models when training data do not span all environmental and operational conditions.The paper investigates whether reduced training-data requirements also affect environmental impact.Source: §3 · p. 4
  • PINNs can increase computational cost through physics-loss evaluation and optimization difficulties, so this study focuses on other physics-informed methods.The cited examples report higher per-epoch costs and slow convergence for PINNs.Source: §3 · p. 4

3.1 Our Approach

  • The case studies assess how model error and runtime change with training-data coverage across physics-informed methods, relying heavily on Gaussian processes.The authors examine whether coverage reductions translate into lower training emissions in SHM applications.Source: §3.1 · p. 4
  • Residual modelling assigns a physical model the main prediction and uses a learner to account for the difference between its prediction and the target.The study applies Gaussian processes and a multilayer perceptron to this approach.Source: §3.1 · p. 4
  • Input augmentation adapts model inputs to reflect physical laws before regression, while hybrid modelling embeds physical knowledge through GP kernel design.The hybrid case studies include kernel designs that can switch between physical states.Source: §3.1 · pp. 4–5
  • Constrained GPs incorporate known problem boundaries into the covariance function using a sparse approximation.The initial study selects examples where physics-informed models have shown performance gains, assumes physics without performance value will not reduce emissions, and uses datasets considered very small for deep learning.Source: §3.1 · p. 5

4 Methods

The methods use Gaussian processes and neural networks to compare black-box and physics-informed models, with performance measured by NMSE and emissions estimated from runtime. The accounting also recognizes that collecting and storing training data can carry substantial but context-dependent carbon costs.

4.1 Models

  • Gaussian processes are the primary model because their mean and covariance functions allow physical insight to be incorporated at varying levels.The paper also uses neural networks as black-box models and learns GP hyperparameters by optimizing negative marginal log likelihood.Source: §4.1 · p. 5
  • GP inference uses covariance relationships between training and test inputs, while physics can enter through mean functions, kernel selection, or constraints.Known system behavior can guide kernel selection, and hyperparameters may be bounded or fixed to reduce the search space.Source: §4.1 · pp. 5–6
  • The black-box neural network is a three-hidden-layer multilayer perceptron trained with ADAM, early stopping, and a maximum budget of 1,000 epochs.Each hidden layer has 64 neurons, and validation performance triggers stopping after 20 consecutive epochs without at least 10^-5 improvement.Source: §4.1 · p. 6
  • Model performance is measured with normalized mean squared error, and the authors compare models at performance thresholds where applicable.The error uses test-set predictions and the variance of true values.Source: §4.1 · p. 6

4.2 Model Carbon Emissions

  • Emissions estimates assume CPU-only runs at 50% of thermal design power and 20 W for 32 GB of RAM, making estimated energy linear in runtime.The authors separately note that data-collection emissions vary by problem and storage emissions depend on storage location and infrastructure, making generic quantification difficult.Source: §4.2 · p. 6

5 Residual Modelling

Residual modelling combines a linear physics model with a GP or neural network that learns the remaining deflection error. It can outperform black-box models at low training-data coverage, but the GP comparison depends on the required NMSE threshold and the physics model’s fit to the data.

5 Residual Modelling

  • Residual modelling first subtracts a physics-based model’s output from observations, then trains a data-driven model on the resulting residual.Here, a linear temperature–deflection model is fit to the first 500 data points, and a GP or NN learns the residual.Source: §5 · p. 7
  • Training periods were increased in 10% temporal-coverage increments; GP and NN training times exclude the one-time linear-model preprocessing step.The data span September to January, so early training periods omit the colder temperatures encountered later.Source: §5 · p. 7

5.1 Results and Discussion

  • At low training coverage, residual models outperform black-box models by extrapolating to colder temperatures; performance levels as coverage increases, while the GP black-box outperforms at 40–60% coverage.The authors attribute the GP exception to the linear physics model’s incomplete fit and the greater complexity of the residual relationship.Source: §5.1 · p. 8, Figure 1 · p. 8, Figure 2 · p. 8
  • At NMSE <30, the grey-box GP reaches the threshold with 10% coverage and 0.06 gCO2e, versus 30% coverage and 0.12 gCO2e for the black-box; at NMSE <10, it emits 0.43 gCO2e versus 0.31 gCO2e.Thus, which GP model uses fewer training emissions depends on the selected performance threshold.Source: Figure 1 · p. 8, §5.1 · p. 9
  • For both model types, training time increases with training-data size; GP and residual-GP training times are nearly identical, while NN training emissions at NMSE <30 fall by 10.7% (0.009 gCO2e).The NN residual model requires less than half the black-box training coverage at that threshold.Source: §5.1 · pp. 8–9, Figure 2 · p. 8
  • Residual-model performance depends on whether the physics-based model captures the underlying relationships, although engineering knowledge can make that possible in many cases.The Tamar bridge linear model does not fully capture the temperature–deflection relationship.Source: §5.1 · p. 9, Figure 3 · p. 9

6 Input Augmentation

Input augmentation turns raw sensor channels into physics-based quantities before model training, with potential carbon savings when those inputs simplify the learning problem. In this case study, however, it did not produce performance gains, and the models showed little or no training-time advantage.

6 Input Augmentation

  • Input augmentation converts raw sensor measurements into physically meaningful model inputs, which may simplify input–target relationships and reduce the time needed for good performance.The grey-box inputs include forces derived using Newton’s second law, dynamic pressure, and directional accelerations.Source: §6 · p. 10

6.1 Results and Discussion

  • For GP input augmentation, the black-box had lower NMSE below 500 training points, but then deteriorated while the grey-box stayed near NMSE 5; training times were similar.Because both models performed well with minimal data, the comparison yielded no emissions savings at a performance threshold; the grey-box was more consistent across data quantities.Source: §6.1 · p. 10, Figure 4 · p. 11
  • For NN input augmentation, the models performed similarly, with the black-box slightly outperforming the grey-box across training data quantities.The grey-box trained slightly faster, but the difference was not significant; Figure 5 presents the NMSE and training-time results.Source: §6.1 · p. 11, Figure 5 · p. 11
  • In this case, some black-box inputs were already close to linearly related to strain, while several augmented inputs had no clear target correlation.The authors did not calculate carbon savings because GP performance diverged and NN performance did not overlap; they note that augmentation may help in other cases where it simplifies input–output relationships.Source: §6.1 · p. 11

7 Physics-Informed Kernel Selection

This section tests whether physics-informed GP kernels can reduce training-data needs enough to offset their added optimization complexity. Across a toy surface and two GARTEUR data layouts, Grey-2 needed the least data to meet the NMSE threshold, but emissions outcomes varied with model complexity and data layout.

7 Physics-Informed Kernel Selection

  • Physics-informed kernels encode known behaviors such as periodicity, but their additional hyperparameters can increase training time and create a trade-off against reduced data needs.The toy-box periodic surface was designed to explore this trade-off; Grey-1 bounded the period near its true value, while Grey-2 fixed it at the true value and learned one fewer hyperparameter.Source: §7 · pp. 12–13

7.1 An engineering test case

  • The engineering test used a lab-based modal test of the GARTEUR aircraft structure, predicting acceleration response across space and time from data collected at different coverage levels.The two layouts used accelerometer positions with every tenth data point, or an upsampled dataset with 200 training points per 10% of surface coverage.Source: §7.1 · p. 13

7.2 Results and Discussion

  • At a given coverage, Black-1 ran fastest, followed by Grey-2 and then Grey-1, consistent with their relative numbers of hyperparameters.The authors describe GP runtime as approximately O(N^3) + O(HN^2), with hyperparameter optimization more prominent at small training-data sizes.Source: §7.2 · p. 14
  • At NMSE <10, Grey-2 required the least training data in all three cases, followed by Grey-1 and Black-1.Grey-box NMSE dropped sharply relative to the black-box as coverage increased.Source: §7.2 · p. 14
  • At NMSE <10 on the toy-box, emissions were 0.45 gCO2e for Black-1, 0.42 gCO2e for Grey-1, and 0.12 gCO2e for Grey-2.These corresponded to emissions reductions of 6.7% for Grey-1 and 73.3% for Grey-2 compared with Black-1.Source: §7.2 · p. 14
  • On original GARTEUR data, emissions were 0.07 gCO2e for Black-1, 0.12 gCO2e for Grey-1, and 0.10 gCO2e for Grey-2.Compared with Black-1, Grey-1 and Grey-2 increased emissions by 71.4% and 42.9%, respectively.Source: §7.2 · p. 14
  • On upsampled GARTEUR data, emissions were 2.04 gCO2e for Black-1, 3.38 gCO2e for Grey-1, and 0.21 gCO2e for Grey-2.Grey-1 increased emissions by 65.7%, while Grey-2 reduced them by 89.7% compared with Black-1.Source: §7.2 · p. 14
  • The authors conclude that reduced data requirements alone do not guarantee lower emissions; optimization complexity also matters, and savings may be more prominent at larger data sizes.Larger training-data savings may also reduce data collection and storage emissions.Source: §7.2 · pp. 14–15

8 Switching Kernels

The switching-kernel grey-box model extrapolates beyond the training range and consistently outperforms the black-box model, but its more complex optimization can increase training time and emissions. The case study highlights trade-offs among performance, robustness, and computational cost.

8 Switching Kernels

  • The switching-kernel hybrid embeds a lift-force polynomial activated under high north-south winds, alongside an SE kernel for other conditions.A regime-dependent heteroscedastic noise term models increased deck-acceleration variance at high wind speeds.Source: §8 · p. 16

8.1 Results and Discussion

  • Across all training-data quantities, the grey-box consistently outperforms the black-box in NMSE, including beyond the <30 mph training range.The physics-informed mixture kernel enables extrapolation, while the black-box model cannot extrapolate.Source: §8.1 · pp. 16–17, Figure 12 · p. 17
  • The grey-box takes longer to train, with the gap growing as data quantity increases, because its 11 hyperparameters and unstable kernel switch complicate optimization.QPSO needs more generations to converge, and each generation requires a covariance-matrix inversion.Source: §8.1 · p. 17
  • Even with fewer training points, the grey-box may have higher training emissions, and equivalent emissions cannot be compared because the black-box never reaches its performance.At 300 points, the grey-box performance is strong while training time remains relatively low, so savings may still be possible.Source: §8.1 · p. 17
  • Assess target performance and training emissions before collecting data; avoiding unnecessary samples can also reduce collection and storage emissions.Marginal performance gains may come with substantial runtime increases.Source: §8.1 · pp. 17–18
  • Across six random training-data selections, the black-box shows greater NMSE variation and sensitivity to data layout, while the grey-box is more robust.The authors suggest that repeated black-box training runs may be needed to ensure good performance, creating additional emissions.Source: §8.1 · p. 17, Figure 13 · p. 18

9 Constrained Gaussian Processes

The constrained GP encodes known plate boundaries and is evaluated when training data cover only a limited cross-section. It outperforms both black-box models with little data and can reduce training emissions substantially when performance matches the standard GP.

9 Constrained Gaussian Processes

  • The constrained GP fixes the model domain to known plate boundaries, incorporating geometry that can restrict the function space and reduce data needs.The case study concerns damage localisation from acoustic-emission maps, where holes or bolts disrupt wave propagation and sparse data may be challenging.Source: §9 · p. 18
  • The comparison uses data from a plate cross-section and contrasts a standard SE-kernel GP, a sparse GP with a wide domain, and the constrained grey-box GP.The sparse kernel approximation uses 256 bases and reduces GP computational complexity from O(n^3) to O(nm^2).Source: §9 · pp. 18–19, §9.1 · p. 19

9.1 Results and Discussion

  • With little training data, the grey-box outperforms both black-box models; as data quantity rises, it converges with the standard GP, while the sparse black-box performs worst.The sparse approximation can reduce emissions, but without physical knowledge it can cost performance; at some thresholds its performance is sufficient.Source: §9.1 · p. 19
  • At NMSE <10, the grey-box reduces emissions 35.5% versus the standard black-box (0.31 vs 0.20 gCO2e), while the sparse black-box reduces them 58.1% (0.13 gCO2e).At 600 training points, grey-box and standard black-box NMSE are indistinguishable, while the grey-box trains 25.9 times faster and emits 0.12 rather than 3.11 gCO2e.Source: §9.1 · pp. 19–20, Figure 15 · p. 20

10 Discussion and conclusions

Across five structural-health-monitoring case studies, physics-informed models often reached target errors with lower emissions, especially when training data were limited. Their added complexity and variable savings mean the benefits require case-by-case assessment and cannot yet be generalized across domains or techniques.

Discussion and conclusions

  • At given target errors, the explored physics-informed approaches showed emissions gains except input augmentation, where adapting GP and neural-network inputs made little difference.Their strongest performance advantages generally appeared at low training-data quantities.Source: §10 · p. 20
  • Savings vary by target-error threshold, and added model complexity can lengthen training enough that reduced data requirements are essential for net benefit.In GPs, extra hyperparameters lengthened optimization; less navigable cost surfaces may also increase runtime.Source: §10 · pp. 20–21
  • Grey-box models often perform more consistently as data quantity or layout changes, making them more reliable in the cases shown, even with a performance penalty.In some applications, reliability may matter more than peak performance.Source: §10 · p. 21
  • Physics-informed models can reduce training-data needs, with potential savings in data collection, storage, and transmission as well as training emissions.The authors identify reduced data requirements as the main result and expect gains to grow for larger datasets, if the physics remains valid.Source: §10 · pp. 20–21
  • The authors recommend setting a required performance target and aligning model strategy and dataset size with carbon-reduction goals, while evaluating savings case by case.The findings suggest lower emissions at target thresholds in many cases, but the savings are problem-dependent.Source: §10 · p. 21
  • Because each PIML method was evaluated in a single engineering context, the findings cannot yet be generalized across all applications or techniques.The authors call for broader domain evaluations and additional PIML methods.Source: §10 · p. 21

Related papers

Loading 2609.33387v1…