PaperlayerView exact revision on arXiv

Source-linked AI summary

When More Modalities Hurt: Modality Dropout for Heavy-Duty Vehicle Engine Diagnostics

Adeel Zafar, Slawomir Nowaczyk, Hamid Sarmadi, Saeed Gholami Shahbandi

arXiv:2608.23161v1cs.LG

TL;DR

The paper asks whether disconnected text, sensor, and DTC streams improve heavy-duty vehicle engine-component classification despite multilingual complaints, missing telemetry, and mismatched timing. It evaluates modality combinations with cross-validation and finds that modality dropout makes text+DTC fusion the strongest overall approach, while per-class results show fault-dependent modality complementarity.

  • Problem

    The study addresses whether fusing technician complaints, sensor telemetry, and DTCs improves engine-component classification when these modalities differ in language coverage, completeness, and temporal relation.

  • Method

    The authors compare logistic regression, gradient-boosted trees, and MLP configurations across modality combinations using 5-fold cross-validation on 885 samples spanning five engine-component classes.

  • Results

    68.8% accuracy makes text+DTC fusion with modality dropout the best result across methods, compared with 65.3% for text-only; modality utility varies by fault class.

  • Takeaways & Limitations

    Diagnostic fusion is most useful when modalities are treated as fault-dependent complementary signals rather than combined naively, with sensors strongest for intake/exhaust and fusion strongest where text is weakest.

  • Takeaways & Limitations

    Results come from one manufacturer’s 885-sample matched population, use an English-centric text encoder, and rely on pragmatically chosen dropout and DTC-window settings.

Abstract

from arXiv · show

Heavy-duty vehicle diagnostics generate three disconnected data modalities: unstructured multi- lingual service complaints, high-dimensional sensor telemetry with over 80% missing values, and Diagnostic Trouble Codes (DTCs). We investigate whether fusing these modalities improves engine component classification on a proprietary dataset from a major truck manufacturer. Through 5-fold cross-validation across multiple model configurations spanning three model families on five engine component classes (885 samples, the full cross-database matched population for this manufacturer), we find that naive fusion provides modest gains over text alone (65.3%). However, modality dropout during training, which randomly disables entire modalities per batch, forces the network to exploit weaker inputs and achieves 68.8% accuracy on text+DTC fusion (weighted F1: 0.67), a 3.5-point improvement over text-only (65.3%, weighted F1: 0.64) and the best result across all methods including logistic regression and gradient-boosted trees. Per-class analysis shows that the dominant modality varies by fault type: text describes symptoms, DTCs encode structured fault signals, and sensors measure physical state. On intake/exhaust faults, sensors alone reach 93% where text achieves 80%. On fuel system faults, fusion with modality dropout nearly triples accuracy from 15% to 38% over text alone. To our knowledge, this is the first application of three-way modality fusion combining text, sensors, and fault codes in industrial vehicle diagnostics.

1 Introduction

The paper asks whether connecting multilingual complaints, sparse sensor telemetry, and DTCs improves engine-component classification. It evaluates this question systematically and reports modality dropout as the strongest fusion approach.

  • Motivation: Three diagnostic streams describe the same fault from different perspectives but are analyzed in isolation in current practice.Complaints describe symptoms, sensors capture physical state, and DTCs provide structured fault indicators.
  • Challenges: 10+ languages, mostly non-diagnostic complaints, and over 80% missing sensor data make cross-modal vehicle diagnostics challenging.The modalities also have causal rather than correlational timing because complaints follow failures while sensors capture earlier state.
  • Study design: The study evaluates five-class engine-component classification on 885 samples using 5-fold cross-validation across logistic regression, gradient-boosted trees, and MLP models.The experiment compares all modality combinations in a systematic ablation.
  • Main contribution: 68.8% accuracy from text+DTC fusion with modality dropout surpasses text-only accuracy of 65.3% and all classical baselines.Per-class results show sensors reaching 93% on intake/exhaust faults, while dropout fusion raises fuel-system accuracy from 15% to 38%.

2 Related Work

Prior work aligns modalities mainly in vision-language, regular time-series, or isolated maintenance-text settings. The paper positions industrial vehicle diagnostics as a distinct sparse, irregular, and previously unconnected three-modality application.

  • Multimodal alignment: Vision-language and related multimodal methods demonstrate cross-modal alignment but typically rely on abundant paired data and dense signals.The paper applies modality-level dropout from prior multimodal settings to industrial diagnostics.
  • Time-series methods: Time-series-language systems target regular-frequency sensor signals, unlike the irregular, pre-aggregated ECU diagnostics used here.The paper states that these approaches are not directly applicable to its over 80%-missing sensor data.
  • Maintenance diagnostics: Manufacturing-maintenance NLP and technician-text methods generally process complaint text without connecting it to sensors or fault codes.Other temporal-text methods target forecasting with clean signals rather than engine fault classification.

3 Data

The dataset links complaints, sensor readouts, and DTC logs by vehicle and time, then restricts analysis to 885 matched engine-domain triplets across five classes. Cross-database coverage and temporal alignment substantially constrain the usable population.

  • Sources: The three modalities are linked by vehicle identifier and timestamp in data from a major heavy-duty vehicle manufacturer.This creates triplets from independently maintained operational databases.
  • Complaints: 68% of technician complaints are non-diagnostic, including parts requests and campaign notes, making the text corpus noisy for fault classification.Complaints span global markets and 10+ languages.
  • DTCs: DTCs are threshold-triggered structured fault signals encoded as multi-hot vectors over the 500 most frequent codes within a 30-day window.The underlying database contains over 51M records across 7,339 vehicles and 1,677 unique codes.
  • Analysis population: The study focuses on 885 triplets across five engine classes: mechanical, fuel system, cooling, intake/exhaust, and electronics.Class sizes range from 60 fuel-system samples to 367 intake/exhaust samples.

4 Method

The method encodes text, sparse sensor telemetry, and DTCs through parallel branches, concatenates their embeddings, and classifies five engine components. Modality dropout independently removes branches during training to reduce reliance on a single strong modality.

  • Task: The task classifies which of five engine-component classes is affected from a complaint, sensor readout, and associated DTC set.The architecture and modality-dropout strategy are evaluated through ablation.
  • Text: Text is encoded as a frozen 384-dimensional sentence embedding and compressed into a 64-dimensional representation.The encoder remains frozen to limit overfitting on 885 samples and retain general multilingual-corpus representations.
  • Sensors: Each sensor group combines six feature values with six observation indicators, and a per-group MLP produces health scores summarized across observed groups.A subsequent two-layer MLP maps the global sensor statistics to a 64-dimensional embedding.
  • Architecture: Three parallel encoders produce 64-dimensional embeddings from text, sensors, and DTC inputs before concatenation and classification.The sensor input has 525 groups × 12 features, while DTC input is a 500-dimensional multi-hot vector.
  • Classifier: The classifier is a three-layer ReLU MLP with standard dropout p = 0.3, trained with cross-entropy loss.Input dimensions vary with the modality combination, and simple concatenation is used because the dataset is too small for more complex fusion.
  • Modality dropout: Modality dropout independently zeros each branch with probability p = 0.2 during training, while inference activates all branches.If all branches are dropped, text is forced active; the masks are applied before concatenation.
  • Evaluation: Stratified 5-fold cross-validation reports mean ± standard deviation across the evaluated configurations.Classical baselines use the same input features, with hand-crafted 13-dimensional sensor summaries for fixed-vector models.

5 Results

Modality dropout makes text+DTC fusion the strongest overall configuration, while per-class results show that the most useful modality depends on the engine fault.

  • Overall results: 0.67 weighted F1: modality dropout fusion improves on text-only's 0.64, confirming the accuracy trend across common and rare classes.
  • Fusion behavior: 67.2% accuracy: naive three-modality MLP fusion only modestly exceeds text-only at 65.3%, while adding sensors to text reduces accuracy to 64.4%.The sensor embeddings are described as noisy and high-dimensional, diluting the stronger text signal.
  • Overall results: 68.8% accuracy: text+DTC with modality dropout achieves the best result, exceeding text-only at 65.3% by 3.5 points.The improvement is reported with low variance (±1.6%), though it is not statistically significant across five paired folds.
  • Per-class analysis: The dominant modality varies by component class, so no single input source is uniformly best across engine faults.
  • Per-class analysis: 93%: sensors achieve the best intake/exhaust result, exceeding text at 80% and DTCs at 72%.Boost-pressure and air-intake measurements directly quantify physical condition.
  • Per-class analysis: 38%: text+DTC with modality dropout nearly triples fuel-system accuracy from text-only's 15%, whereas sensors alone reach 0%.The passage attributes the gain to structured DTC signals supplementing vague complaint text.
  • Per-class analysis: Text+DTC with modality dropout improves fuel-system recall by 23 percentage points, intake/exhaust by 5, and cooling by 2, while electronics declines by 6.

6 Discussion

The discussion finds that modality utility varies by fault type and that modality dropout improves fusion by reducing reliance on a dominant modality. Class-specific gains, especially for fuel-system faults, matter more than aggregate improvement alone.

  • Class-specific complementarity: Sensors dominate intake/exhaust faults, text dominates electronics, and fusion helps fuel-system and cooling faults.Each modality captures a different diagnostic dimension, so the strongest input depends on the fault class.
  • Evaluation: Text+DTC fusion with modality dropout outperforms text-only in 4 of 5 folds, with a mean improvement of 3.5 percentage points.Figure 5 compares fold-level accuracy across 5-fold cross-validation.
  • Modality dropout: Modality dropout prevents dominant-modality collapse by forcing the model to learn from modality subsets.Naive fusion achieves 67.2% versus 65.3% for text-only, while text+DTC with dropout reaches 68.8%.
  • Class-specific complementarity: Fusion improves fuel-system accuracy from 15% with text-only to 37% with text+DTC.DTCs alone reach 58.0% using a raw binary vector, indicating substantial class-specific value beyond text.

7 Conclusion

The study shows that three-way modality fusion benefits heavy-duty vehicle engine diagnostics when training prevents modality collapse. Its strongest gains are fault-specific, with sensors, DTCs, and text contributing different diagnostic information.

  • Main findings: Modality dropout on text+DTC achieves 68.8%, the best result across all methods, while sensors add limited value beyond text+DTC.Naive fusion does not improve over text, but dropout produces complementary modality use.
  • Per-class complementarity: Sensors dominate intake/exhaust faults at 93%, while fusion nearly triples fuel-system accuracy from 15% to 38%.The combined model improves most on classes where text alone is weakest.
  • Per-class complementarity: Text describes symptoms, DTCs encode structured faults, and sensors measure physical state.These distinct diagnostic dimensions explain why modality utility varies across fault classes.

Limitations

The study is bounded by a small, manufacturer-specific and incompletely matched dataset, pragmatic hyperparameter choices, and a simple fusion architecture. These constraints limit generalization and leave alternative encoders and fusion strategies untested.

  • Scope and data: The 885-sample dataset represents one manufacturer’s full matchable population and may not generalize to other OEMs or vehicle types.Only 74% of vehicles have complete coverage across the three aligned databases.
  • Modeling choices: The English-centric text encoder may yield different modality rankings for the corpus spanning 10+ languages.A multilingual encoder was not evaluated.
  • Modeling choices: The DTC window and pdrop = 0.2 were chosen pragmatically, while alternative dropout strategies were not explored.These settings were not optimized in the reported study.
  • Modeling choices: The study used simple concatenation fusion, leaving dynamic per-sample modality weighting for future work.The dataset was considered too small to reliably train more complex fusion mechanisms without overfitting.

A Pilot Studies

Two pilot studies preceded the main experiment and informed the study’s approach.

  • Pilot studies: Two pilot studies were conducted before the main experiment.They served as preliminary investigations preceding the primary evaluation.
  • Pilot studies: The pilot studies informed the approach used in the main experiment.The passage identifies their role but does not specify their individual designs.
  • Pilot studies: The main experiment followed the pilot studies.The pilots are presented as preparatory work rather than the primary evaluation.

A.1 Decoupled Retrieval

Decoupled retrieval independently matches historical complaints to cases and flags sensor anomalies for expert review. Retrievals were often relevant, but flagged sensors were not judged directly diagnostic.

  • Top-5 historical complaints were retrieved with a sentence transformer, while sensor-group anomalies were computed independently using z-scores.The pipeline separates complaint retrieval from sensor anomaly detection before expert evaluation.
  • Experts rated retrieved complaints for relevance and classified flagged sensors as directly diagnostic, consequential, or not relevant.Table 3 used a four-point relevance scale: 3 high, 2 partial, 1 not relevant, and 0 cannot assess.
  • In 4 of 5 cases, at least one retrieved complaint was partially or highly relevant, but no flagged sensor was directly diagnostic.

A.2 Contrastive Alignment

Contrastive alignment embeds complaints and sensor data in a shared space for cross-modal retrieval. Its results motivated category-level classification and adding DTCs as structured supervision between text and sensors.

  • Contrastive Alignment: A dual-encoder trained with symmetric InfoNCE loss mapped complaints and sensor data into a shared 128-dimension embedding space.The text encoder was frozen, while a custom sensor GRU contained 861K trainable parameters.
  • Contrastive Alignment: 13× improvement over random was achieved at Top-1 retrieval on 553 test pairs, with mean rank 164/553.The mean rank placed results in the top 30%.
  • Contrastive Alignment: Diagnostic complaints had mean rank 171 versus 160 for non-diagnostic complaints in the cross-modal retrieval evaluation.Diagnostic complaints comprised 31.5% of the set, compared with 68.5% non-diagnostic complaints.
  • Method Decisions: The studies motivated category-level classification and adding DTCs as a third modality supplying structured supervision between text and sensors.
Loading 2608.23161v1…