PaperlayerView exact revision on arXiv

Source-linked AI summary

AI-based Prediction of Independent Construction Safety Outcomes from Universal Attributes

Henrietta Baker, Matthew R. Hallowell, Antoine J. -P. Tixier

arXiv:1908.05972v3cs.LGstat.ML

TL;DR

Predicting safety outcomes from incident reports requires laborious data collection, and earlier attribute-based work extracted predictors and outcomes with the same NLP tool. This study extracts attributes with NLP but uses independent human-entered outcomes to train machine-learning models; attributes remain highly predictive across four outcomes, including injury severity.

  • Problem

    Earlier attribute-based prediction extracted both attributes and outcomes from the same reports with the same NLP tool, leaving possible artificial predictor–outcome correlation amid laborious data collection.

    Source: §1 · p. 2

  • Method

    NLP extracts fundamental attributes from incident reports, and machine-learning models predict four safety outcomes supplied as independent fields entered by safety professionals.

    Source: §1 · pp. 2–3, §4.1 · p. 7

  • Results

    Attributes predict all four independent outcomes—incident type, injury type, bodypart, and injury severity—with high skill, and severity is well predicted unlike in.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The approach may complement current construction-safety methods, but whether it improves human decision-making remains for future research.

    Source: §9 · p. 24, §10 · p. 25

    The 80-attribute set may need adjustment or additions for more nuanced companies and tasks, requiring time and resources.

    Source: §9 · p. 24

Abstract

from arXiv · show

This paper significantly improves on, and finishes to validate, an approach proposed in previous research in which safety outcomes were predicted from attributes with machine learning. Like in the original study, we use Natural Language Processing (NLP) to extract fundamental attributes from raw incident reports and machine learning models are trained to predict safety outcomes. The outcomes predicted here are injury severity, injury type, body part impacted, and incident type. However, unlike in the original study, safety outcomes were not extracted via NLP but were provided by independent human annotations, eliminating any potential source of artificial correlation between predictors and predictands. Results show that attributes are still highly predictive, confirming the validity of the original approach. Other improvements brought by the current study include the use of (1) a much larger dataset featuring more than 90,000 reports, (2) two new models, XGBoost and linear SVM (Support Vector Machines), (3) model stacking, (4) a more straightforward experimental setup with more appropriate performance metrics, and (5) an analysis of per-category attribute importance scores. Finally, the injury severity outcome is well predicted, which was not the case in the original study. This is a significant advancement.

1. Introduction

The paper addresses the challenge of predicting construction safety outcomes from incident reports and validates an attribute-based NLP-plus-ML approach using outcomes independent of attribute extraction. Attributes predict all four outcomes with high skill, including injury severity, improving on.

1. Introduction

  • The study builds on, where attributes predicted outcomes better than random but shared an NLP extraction process that could induce artificial correlation.Source: §1 · p. 2
  • Using safety-professional outcome fields independent of attribute extraction removes potential artificial predictor–outcome correlation, increasing prediction validity and difficulty.Source: §1 · pp. 2–3
  • Attributes predict all four independent safety outcomes—incident type, injury type, bodypart, and injury severity—with high skill, confirming.Source: §1 · p. 3
  • Unlike, injury severity was well predicted, a significant improvement over its earlier hypothesis that attributes alone could not predict it.Source: §1 · p. 3
  • The study expands the dataset to more than 90,000 reports and adds XGBoost and linear SVM models.Source: §1 · p. 3
  • Additional contributions include model stacking, a more straightforward setup with appropriate performance metrics, and per-category attribute-importance scores for interpreting predictions.Source: §1 · p. 3

2. Background

Construction machine-learning research broadened from early prediction applications to diverse methods and tasks, while construction-safety prediction developed from lagging-statistics analysis toward forecasting. Prior safety prediction studies vary in predictor validity, outcome complexity, and methodology.

2.1. Machine learning in construction

  • Construction machine learning expanded from early applications to widespread use of SVMs and more diverse tasks, including equipment recognition and productivity or ergonomic assessment.Source: §2.1 · p. 4

2.2. Machine learning for construction safety

  • Before 1995, construction-safety research mainly analyzed lagging statistics and correlations rather than forecasting safety events or future trends.Source: §2.2 · p. 4
  • Several construction-safety prediction studies used outcomes as predictors, which are not valid pre-incident attributes; was an exception among the surveyed work.Source: §2.2 · pp. 4–5
  • The studies differ in prediction complexity and methods, while uses leading indicators that the authors say cannot be directly compared with fundamental attributes.Source: §2.2 · p. 5

3. Attribute framework and NLP method

The paper represents construction situations with universal, pre-incident attributes and uses a rule-based NLP system to extract them from injury reports. It contrasts this interpretable, validity-oriented feature pipeline with deep learning on raw text, noting information loss and extraction errors as trade-offs.

3. Attribute framework and NLP method

  • The attribute framework describes construction work through finite, universal descriptors of means, methods, and environmental conditions that are observable before an accident.Source: §3 · p. 5
  • For example, the system maps a report about tripping on a cable while carrying a “2x4” to attributes including object on the floor, lumber, and light vehicle.Source: §3 · p. 5
  • Eight coders manually inspected 1000+ accident reports, with weekly peer reviews, calibration meetings, and voting, to produce 80 attributes; NLP rules and keyword dictionaries extracted them, reaching 95% agreement against external manual coding.The agreement threshold was achieved after several weeks of iterations and fine-tuning.Source: §3 · pp. 5–6
  • Unlike deep learning's direct raw-text mapping, the two-step approach extracts expert-defined attributes before supervised prediction; these features are interpretable and pre-incident-valid, but omit information and can be extracted with errors.Source: §3 · pp. 6–7

4. Dataset

The study uses more than 90,000 oil-and-gas incident reports from five continents, spanning the early 2000s to 2018. It extracts attributes with NLP and prepares outcome-specific training and test data.

4.1. Text and outcomes

  • The dataset contains more than 90,000 incident reports from a global industrial partner, covering five continents and the early 2000s through 2018.Source: §4.1 · p. 7
  • Severity is measured by immediate medical care required; the four outcomes also include injury type, bodypart, and incident type, with category counts in Table 2.Source: §4.1 · p. 7, Table 2 · p. 8

4.2. Attribute extraction with Natural Language Processing

  • Attributes were extracted with the NLP method of, developed and fine-tuned on industrial, energy, infrastructure, and mining reports, then applied to oil-and-gas reports.Source: §4.2 · p. 7

4.3. Train/test splits and class imbalance

  • Each outcome used a 90/10 train/test split, with 10% of training reserved for validation; multi-level cases were duplicated across labels only after splitting to prevent leakage.Source: §4.3 · p. 8
  • Inverse-frequency weights from each training set were used to keep the models from neglecting minority categories.Source: §4.3 · p. 8

5. Machine learning models

The paper uses CART, Random Forest, XGBoost, and linear SVM models to classify safety outcomes from construction attributes. It also describes model-specific attribute importance measures and tests stacking models to combine their forecasts.

5. Machine learning models

  • The study compares Random Forest, XGBoost, and linear SVM, with the tree ensembles built from CART trees and all models used for classification.The models were implemented with scikit-learn for Random Forest and linear SVM, and the original Python library for XGBoost.Source: §5 · pp. 8–9

5.1. CART

  • CART greedily partitions the 80-attribute feature space with recursive binary splits, assigning each leaf its most frequent category using Gini decrease and a depth stopping rule.Deep trees can capture nonlinear, high-order interactions but may overfit and have high variance.Source: §5.1 · pp. 9–10

5.2. Random Forest

  • Random Forest grows trees on bootstrap samples and considers a random predictor subset at each split, adding variety to reduce correlated forecasts and variance.Bagging aggregates deep trees by majority vote; it reduces variance but is less interpretable and may retain correlated upper tree structures.Source: §5.2 · pp. 10–12

5.3. XGBoost

  • XGBoost builds shallow trees sequentially on pseudo-residuals and adds a regularization term to penalize model complexity while speeding training through optimization tricks.Unlike parallel Random Forest trees that reduce variance, boosting sequentially reduces the bias of high-bias, low-variance weak learners.Source: §5.3 · p. 12

5.4. Linear SVM

  • Linear SVM classifies observations by the side of a maximum-margin separating hyperplane, with parameter C controlling tolerance for misclassification and the margin trade-off.For targets with more than two categories, one-versus-rest trains one binary SVM per category; its training time can scale as O(n^2) or O(n^3), limiting large-data use.Source: §5.4 · pp. 12–13

5.5. Attribute importance measures

  • Random Forest importance is the increase in out-of-bag error after permuting a predictor, while XGBoost attributes weighted node-purity reductions to split predictors.For linear SVM, coefficient magnitudes indicate contribution strength and signs indicate attraction to or rejection from a category.Source: §5.5 · p. 14

5.6. Model stacking

  • Stacking logistic regression was trained on summed forecasts, but adding SVM did not outperform any single model for any outcome, and SVM stacking was unavailable for incident type.SVC did not scale in practice to datasets over 10,000 observations, while incident type had more than 80,000 training examples.Source: §5.6 · p. 15

6. Experimental setup

The experiments tune Random Forest, XGBoost, and linear SVM on validation data, then evaluate final models on the test set. Because outcomes are imbalanced, performance is assessed with class-level precision, recall, and F1 rather than accuracy.

6.1. Hyperparameter optimization

  • Grid searches selected model- and outcome-specific hyperparameters, with XGBoost tree count controlled by early stopping and SVM C searched across 800 values.Final models were trained on combined training and validation sets; category or sample weights inversely proportional to category size addressed imbalance.Source: §6.1 · pp. 15–16

6.2. Configuration

  • XGBoost grid searches used GPU-accelerated gpu_hist, but final training and testing used the exact algorithm.The grid-search configuration yielded an 8X speedup.Source: §6.2 · p. 17

6.3. Performance metrics

  • Large class imbalance made accuracy inadequate, so the study computed class-level precision, recall, and F1 from confusion matrices.Precision divides correct predictions by predictions assigned to a category; recall divides them by observations in that category, and F1 is their harmonic mean.Source: §6.3 · p. 17

7. Results

The three models outperform random classification across four independently annotated safety outcomes, with severity predicted better than in prior work. Per-category importance scores highlight both intuitive attribute patterns and less expected rankings.

7.1. Quantitative results

  • All models predict severity with skill much better than random, improving on prior work that found it unpredictable from attributes alone.This supports the approach using independently extracted outcomes.Source: §7.1 · p. 18, Table 10 · p. 19
  • Stacking improves results for three of four outcomes, but SVM leads for injury type and severity.For severity, stacking XGBoost and Random Forest improves on either alone but does not surpass SVM; for injury type, stacking is slightly worse than XGBoost alone.Source: §7.1 · p. 18
  • All three models perform comparably and are largely better than random across incident type, injury type, bodypart, and severity.The tables report precision, recall, and F1 for each outcome class and model, alongside a random baseline.Source: §7.1 · pp. 17–18, Table 7 · p. 18, Table 8 · p. 19

7.2. Qualitative results: attribute importance scores

  • The importance plots show the six most and least predictive attributes for each outcome category; slip/trip/fall and cut/puncture examples align with intuitive hazards.The examples include slippery surfaces, objects on the floor, uneven surfaces, cold, and stairs for falls, and sharp edges, nails, and powered tools for cuts.Source: §7.2 · p. 20, Figure 4 · p. 21, Figure 5 · p. 21
  • Improper PPE ranks low for four of six incident types and for upper and lower extremities, while severity rankings associate medical cases with high-energy attributes.First-aid severity rankings instead emphasize attributes associated with lower-energy tasks.Source: §7.2 · p. 20, Figure 6 · p. 22, Figure 7 · p. 22

8. Academic Contributions and Application to Industry

The study validates attribute-based prediction using safety outcomes independently recorded by human annotators. It also describes ways construction organizations could use report data and attribute mappings in work planning and analytics.

8. Academic Contributions and Application to Industry

  • High predictive performance on four independently annotated outcomes validates the NLP-plus-machine-learning approach beyond outcomes extracted from the same text.The results also support using the NLP tool outside its original oil-and-gas domain, including industrial, energy, infrastructure, and mining settings.Source: §8 · pp. 22–23
  • Organizations could systematically record report information or capture attributes with checkboxes to use injury databases for prediction.Structured attribute capture could support mappings between attribute combinations and outcomes without relying on rule-based NLP extraction.Source: §8 · p. 23
  • During work planning, attributes may augment human judgment because they are objective descriptors that can be identified without specific training or hazard-forecasting expertise.The paper suggests possible application to design and advanced work packaging as well.Source: §8 · p. 23
  • The study adds linear SVM, XGBoost, model stacking, and per-category attribute importance analysis to the prior approach.It also demonstrates high-skill prediction of injury severity, previously considered unpredictable from attributes alone.Source: §8 · pp. 23–24

9. Limitations and Recommendations

The approach is bounded by the coverage of its attribute set and predicts an incident’s outcome only after an incident has occurred. The paper also identifies future research on decision-making and on predicting incident occurrence.

9. Limitations and Recommendations

  • The 80 attributes suit general construction but may need adjustment or additions for more nuanced companies and tasks.Such expansion takes time and resources, potentially offset by collecting attributes through report-entry checkboxes.Source: §9 · p. 24
  • The method predicts an incident outcome conditional on an incident having occurred, not whether an incident will occur.Incident-occurrence prediction would require descriptions of non-safety events; near-misses and hazard observations are not ideal substitutes.Source: §9 · p. 24
  • The two-step approach can use future machine-learning advances once attributes are extracted, and researchers should test whether it improves human decision-making.The paper specifically proposes investigating its use during work planning to address judgment-based bias with empirical data.Source: §9 · p. 24

10. Conclusion

The study validates the NLP + ML approach of using external, independent safety outcomes and finds high predictive power, including for injury severity. It also reports methodological improvements and potential to complement construction safety methods dominated by opinion-based judgments.

10. Conclusion

  • Attributes had high predictive power for external, independent safety outcomes, validating the NLP + ML approach of.Injury severity was also well predicted, unlike in.Source: §10 · p. 25
  • The study added a larger dataset, XGBoost and linear SVM, model stacking, more appropriate performance metrics, and per-category attribute-importance analysis.Source: §10 · p. 25
  • The NLP tool of performed well outside its original domain, and the approach can complement construction safety methods dominated by opinion-based judgments.Source: §10 · p. 25

12. Author Contributions

The authors divided responsibility across study design and execution, background and contributions writing, review, and submission leadership.

12. Author Contributions

  • Antoine Tixier designed the study, curated the dataset, implemented models, ran experiments, interpreted and visualized results, and wrote the paper's core.Source: §12 · p. 25
  • Henrietta Baker wrote the background and academic contributions sections, reviewed the paper, and led its submission.Source: §12 · p. 25
  • Matthew Hallowell reviewed the paper.Source: §12 · p. 25
Loading 1908.05972v3…