Source-linked AI summary
IndustryAssetEQA: A Neurosymbolic Operational Intelligence System for Embodied Question Answering in Industrial Asset Maintenance
Chathurangi Shyalika, Dhaval Patel, Amit Sheth
TL;DR
Industrial maintenance assistants often provide fluent but weakly grounded explanations, with missing provenance and non-testable counterfactuals. IndustryAssetEQA addresses this gap through episodic telemetry, an FMEA-KG, provenance enforcement, and simulator-grounded intervention reasoning, achieving the strongest reported reliability results while remaining advisory and human-supervised.
Problem
Industrial maintenance QA systems can produce generic explanations that lack telemetry grounding, verifiable provenance, and testable counterfactual or action-oriented reasoning.
Method
IndustryAssetEQA combines episodic telemetry facts, an FMEA-KG, a parametric intervention risk estimator, and provenance-aware evaluation for embodied asset-maintenance QA.
Results
The full IndustryAssetEQA achieves the best reported results, including counterfactual direction accuracy of 0.88 for GPT and 0.91 for Claude, with provenance enforcement producing the largest deployable-reliability gain.
Takeaways & Limitations
The findings support treating industrial maintenance QA as an embodied decision problem requiring complementary episodic grounding, provenance, knowledge-graph context, and simulation.
Takeaways & Limitations
Counterfactual outputs are surrogate risk estimates rather than proven causal effects, while the domain-level FMEA-KG may miss asset-specific variations and rare failure patterns.
Abstract
from arXiv · showhide
Industrial maintenance environments increasingly rely on AI systems to assist operators in understanding asset behavior, diagnosing failures, and evaluating interventions. Although large language models (LLMs) enable fluent natural-language interaction, deployed maintenance assistants routinely produce generic explanations that are weakly grounded in telemetry, omit verifiable provenance, and offer no testable support for counterfactual or action-oriented reasoning that undermine trust in safety-critical settings. We present IndustryAssetEQA, a neurosymbolic operational intelligence system that combines episodic telemetry representations with a Failure Mode Effects Analysis Knowledge Graph (FMEA-KG) to enable Embodied Question Answering (EQA) over industrial assets. We evaluate on four datasets covering four industrial asset types, including rotating machinery, turbofan engines, hydraulic systems, and cyber-physical production systems. Compared to LLM-only baselines, IndustryAssetEQA improves structural validity by up to 0.51, counterfactual accuracy by up to 0.47, and explanation entailment by 0.64, while reducing severe expert-rated overclaims from 28% to 2% (approximately 93% reduction). Code, datasets, and the FMEA-KG are available at https://github.com/IBM/AssetOpsBench/tree/IndustryAssetEQA/IndustryAssetEQA.
1 Introduction
Industrial maintenance QA must move beyond fluent but weakly grounded language generation toward time-situated, evidence-grounded, risk-constrained, and knowledge-grounded decision support. IndustryAssetEQA operationalizes this embodied framing through neurosymbolic integration, structured episodes, FMEA-KG grounding, intervention risk estimation, and provenance-aware evaluation.
- Industrial maintenance is shifting toward continuous, data-driven decisions based on multivariate telemetry, alerts, and maintenance records.
- LLM-based maintenance assistants often provide generic explanations, omit verifiable provenance, and produce non-testable counterfactuals or action suggestions.
- Embodied maintenance QA requires answers to be time-situated, evidence-grounded, risk-constrained, and knowledge-grounded within a perception–prediction–reasoning–decision loop.
- IndustryAssetEQA combines neural learning with symbolic reasoning for transparent, knowledge-aligned Embodied Question Answering over industrial asset operations.
- The framework contributes structured episode facts, an FMEA-KG, an intervention-style risk estimator, and provenance- and structure-aware reliability evaluations.
2 Related Work
Prior industrial QA and embodied-AI systems address documents, datasets, observations, or simulators, but they generally do not verify operational decisions for correctness or risk consistency. IndustryAssetEQA targets this gap with an architecture integrating episodic evidence, failure knowledge, simulation, and verification.
- Industrial knowledge-enhanced LLM systems typically operate on static documents and global datasets.
- Time-series QA research supports structured reasoning over numerical sequences, multimodal signals, and temporal events.
- Even strong LLMs struggle with long-range dependencies, causal interpretation, parallel event reasoning, and zero-shot temporal generalization.
- Prior embodied systems largely evaluate understanding from observations or simulators without explicitly verifying operational-decision correctness or risk consistency.
- IndustryAssetEQA’s architecture integrates a Fact Extractor, Episodic Store, FMEA-KG, Causal Simulator, Verifier, and Safety Gate.
3 Industrial Setting and QA Task Taxonomy
The framework targets time-sensitive industrial maintenance queries centered on a specific asset and episode window. Its five QA types align with a perception → reasoning → prediction → decision pipeline and require episodic, knowledge-graph, and intervention-risk grounding when relevant.
- The setting covers manufacturing, utilities, and process plants where operators use telemetry, alerts, and maintenance logs for time-sensitive decisions.
- Each query is episode-centric, pairing an asset with a concrete time window and evaluating semantic grounding in episodic and FMEA-KG evidence.
- Relevant queries also assess consistency with a simple, data-driven intervention risk model across five QA types aligned to perception → reasoning → prediction → decision.
4 IndustryAssetEQA System Overview
IndustryAssetEQA converts heterogeneous telemetry into provenance-rich episode facts, retrieves symbolic failure knowledge, estimates intervention risks, and applies structured generation and safety checks. Its components form a closed-loop service that routes problematic recommendations to human review.
- FMEA Knowledge Graph and Neurosymbolic Fusion: The FMEA-KG contains 63 failure modes mapped to 9 asset categories and represents failure semantics, sensor indicators, severity, and admissible interventions.
- Fact Extractor: The Fact Extractor aggregates configurable historical telemetry windows into summary statistics, contextual features, labels, FMEA enrichment, and JSONL records with provenance.
- Episodic Store: The Episodic Store persists episode metadata and decomposed feature values in SQLite to support indexed retrieval, threshold queries, and balanced label sampling.
- Risk Modeling and Causal Simulator: The risk simulator fits multinomial logistic regression over episode features and estimates pre- and post-intervention risks after explicit feature substitution.
- EQA Builder: The EQA Builder constructs task-specific questions, answers, reasoning traces, provenance, and FMEA-KG enrichment from episodic facts.
- Runtime Deployment Flow: The deployed flow retrieves episodic and FMEA knowledge, invokes simulation for counterfactuals, generates provenance-enforced answers, and validates recommendations before surfacing them.
5 Datasets and QA Task Design
The evaluation uses four industrial datasets with SME-validated episodes and QA instances spanning multiple maintenance task categories under a closed-world assumption.
- Table 1 organizes datasets by asset statistics and QA counts across descriptive, temporal, diagnostic, counterfactual, and action-oriented categories.
- The four datasets cover predictive maintenance, degradation modeling, cyber–physical systems, and hydraulic systems.
- Episodes and QA instances were validated by subject-matter experts and treated as ground truth for experiments.
- The ground-truth interpretations apply a closed-world assumption to the provided evidence.
6 Evaluation Methodology
IndustryAssetEQA is evaluated as an operational QA system using episodic facts, multiple system configurations, and metrics focused on reliability, grounding, counterfactual validity, and explanation faithfulness.
- Each evaluation instance contains one QA query grounded in a specific episodic fact.
- The comparison includes LLM-only, episodic-evidence, KG-grounding, provenance-enforced, and full IndustryAssetEQA configurations.
- Metrics assess structural validity, evidence grounding, counterfactual accuracy, answer correctness, and semantic faithfulness of explanations.
7 Experiments and Results
Experiments assess reliability, counterfactual reasoning, architectural components, FMEA-KG quality, and expert-rated failure modes across the evaluated QA system configurations.
- Experiments and Results: GPT and Claude IndustryAssetEQA configurations achieve Struct.OK values of 0.88 and 0.90, respectively.
- Statistical analysis: McNemar’s tests find significant differences for descriptive, diagnostic, and counterfactual QA, but not for temporal and action-oriented QA.
- Counterfactual reasoning: GPT and Claude achieve CF Acc. values of 0.88 and 0.91, while baselines without simulator access perform near chance at approximately 0.45.
- Ablation analysis: Removing the risk simulator reduces CF Acc. from 0.88 to 0.49, while disabling provenance enforcement reduces Full Pass from 0.89 to 0.19.
- FMEA-KG validation: 96.0% of 1004 candidate FMEA-KG triples are judged valid, while structurally weaker relations show higher noise.
- Expert evaluation: IndustryAssetEQA outputs are judged answerable in 97% of cases versus 46% for LLM-only outputs, with severe overclaims at 2% versus 28%.
8 Conclusion
IndustryAssetEQA reframes industrial QA as an embodied decision problem and is engineered for live deployment.
- IndustryAssetEQA is being integrated as a reliability layer into enterprise agents and extended toward multi-asset production deployment.
Limitations
The paper identifies limitations in counterfactual modeling, knowledge-graph coverage, episode windowing, evaluation scope, and deployment overhead.
- Counterfactual modeling: Counterfactual outputs are surrogate risk estimates rather than proven causal effects.The module is a lightweight parametric intervention estimator, not a fully identified structural causal model.
- FMEA-KG quality and coverage: The FMEA-KG has noisy relation types and may miss asset-specific variations and rare failure patterns.Its coverage is domain-level, with structurally weaker links exhibiting greater noise.
- Fixed episode windowing: Fixed episode windows may miss long-horizon or multiscale precursors.Adaptive windowing remains future work.
- Evaluation scope and pilot plan: Evaluation is limited to offline experiments on four benchmark datasets.A controlled pilot is planned to measure operational impact and calibrate counterfactual estimates empirically.
- System overhead and hosting: The full pipeline adds engineering and runtime overhead relative to LLM-only systems.The design supports on-premises, hybrid, or VPC deployment, but the passage does not quantify the overhead.
Ethical Considerations
The system is presented as advisory decision support with deterministic evidence handling, provenance, and domain-grounded components for maintenance QA.
- Ethical and safety safeguards: IndustryAssetEQA routes low-confidence or inconsistent outputs to human review instead of automating maintenance actions.The advisory design addresses safety- and cost-critical settings where incorrect or overconfident recommendations could create operational risk.
- Knowledge grounding: The FMEA-KG links failure semantics to assets, components, sensors, and maintenance actions using an asset-centric schema.The graph is constructed from ISO-style specifications and expert-curated maintenance documentation.
- Traceability: The episodic representation retains temporal scope, engineered telemetry features, observed labels, and provenance identifiers.Facts include asset and dataset identifiers, episode type, timestamps, telemetry summaries, error indicators, and maintenance recency features.
E.0.4 Counterfactual (with KG fields in provenance)
The counterfactual example evaluates how an immediate maintenance intervention targeting comp3 changes estimated failure risk. The enriched QA instance combines episodic telemetry, provenance, FMEA-KG context, and a risk estimate to produce a testable intervention outcome.
- Counterfactual QA: The counterfactual question resets hours_since_last_maint_comp3 from 477.0 to 0 by assuming maintenance immediately before the episode window.The episode concerns machine_56, comp3, and the window from 2015-01-01 03:00:00 to 2015-01-02 03:00:00.
- Counterfactual result: Baseline P(failure) is approximately 1.000, while post-intervention P(failure) is approximately 0.000, with Δ = -1.000.The output also reports risk_after=9.256e-06 and delta_risk=-0.99999.
- Counterfactual result: The estimated risk direction is decrease, with confidence 1.0 and predicted label comp3.The output includes probabilities before and after the intervention.
- Action-oriented QA: The related action-oriented example recommends opening a maintenance work order for rotating_equipment machine_56 rather than continuing monitoring.Its reasoning cites estimated failure probability approximately 1.00 against a 0.50 threshold, high severity, and pressure as the most informative sensor.
H.3 Ablation Design
The ablation design removes individual architectural components to isolate their effects on model behavior. All variants are evaluated on identical QA instances, making the comparison target architectural constraints rather than dataset differences.
- Ablation variants: The study evaluates ablations removing episodic memory, the FMEA knowledge graph, provenance enforcement, or the risk simulator.Each ablation withholds a distinct evidence, semantic-grounding, verification, or counterfactual-reasoning component.
- Ablation variants: Removing episodic memory provides free-form telemetry summaries without fact-level indexing or verifiable provenance.This tests the contribution of explicit episode identifiers and the episodic store.
- Ablation variants: Removing the FMEA knowledge graph eliminates failure-mode metadata, associated sensors, and recommended actions from the model context.The model must then rely on episodic telemetry and prompts alone.
- Ablation variants: Removing provenance enforcement permits free-text explanations without structured-output requirements or machine-verifiable provenance.This isolates the effect of output constraints and verifier checks.
- Ablation variants: Removing the risk simulator requires counterfactual answers without simulator guidance.This isolates the simulator’s role in counterfactual reasoning.
- Evaluation control: All models are evaluated on the same QA instances to isolate architectural constraints rather than dataset differences.