Quantile-Led Feature Extraction for Multi-Horizon Predictive Maintenance in Industrial Manufacturing Systems

David J Poland; Daniele Ravi; Na Helian

Source-linked AI summary

Quantile-Led Feature Extraction for Multi-Horizon Predictive Maintenance in Industrial Manufacturing Systems

David J Poland, Daniele Ravi, Na Helian

arXiv:2609.07533v1Sep 7, 2026cs.AI

TL;DR

Predictive-maintenance feature extraction is often reused unchanged even when forecasting horizons change, raising questions about representation stability and scalability. The paper proposes a dual-stage MLP–QRNN extractor and evaluates horizon-conditioned variants, finding that four-quantile refinement improves short-horizon performance while adapted extractors outperform unmodified transfer at longer horizons. The results support treating feature extraction as a horizon-dependent representational stage.

Problem
PdM commonly reuses fixed descriptors as downstream models or forecasting horizons change, motivating evidence on horizon-scalable representations.
Method
A dual-stage MLP–QRNN hierarchy learns broad ten-quantile channel distributions, then refines retained mid-tail quantiles into compact distribution-aware features.
Results
Across 1-hour, 70-hour, and 30-day regimes, horizon-conditioned extractors outperform unmodified transfer as temporal separation increases.
Takeaways & Limitations
Feature extraction should be treated as horizon-dependent because transfer requires scaling feature capacity, temporal embedding, activation strategy, and sensor breadth.
Takeaways & Limitations
The evaluation is bounded to one broad manufacturing asset and product family, and joint QRNN1–QRNN2 fine-tuning was not used.

Abstract

from arXiv · show

In data-driven predictive maintenance (PdM), feature extraction is usually treated as fixed preprocessing: a descriptor set is chosen once and reused while the downstream model or forecasting horizon changes. This paper isolates the representation-learning stage and presents a quantile-led feature-extraction framework based on a dual-stage MLP-QRNN hierarchy. QRNN1 learns a broad ten-quantile conditional distribution for each sensor channel, while skip-connected QRNN2 refines a retained mid-tail quantile set into compact, channel-resolved, distribution-aware features. A fixed thirteen-pipeline ablation spans 1-hour, 70-hour, and 30-day regimes across 72 machines in 9 industrial facilities, with the downstream temporal classifier held fixed within each regime. Increasing the retained mid-tail set from two to four quantiles improves 30- and 60-minute F1-score, reaching 75.92% and 72.44% with attention enabled. The results also show that representations do not transfer reliably beyond their design horizon unless feature capacity, temporal embedding, activation strategy, and sensor breadth are scaled with the forecasting task. The unmodified short-horizon extractor falls to 42.90% F1 at 70 hours, whereas horizon-conditioned extractors reach 60.38% at 70 hours and 79.97% at 30 days. The framework therefore supports treating PdM feature extraction as a horizon-dependent representational stage rather than fixed preprocessing.

1. Introduction

Predictive maintenance depends on representations that preserve distributional changes in multivariate sensor data, yet feature extraction is commonly fixed while forecasting horizons change. This paper instead evaluates a dual-stage quantile-led extractor across horizons and studies how its design scales.

Motivation: Incipient degradation often appears as dispersion widening, asymmetric tails, or shifts in operating distributions rather than large point anomalies.Point-estimate features can obscure these distributional effects through averaging.
Research gap: Most PdM systems reuse a fixed descriptor set while varying downstream models or extending forecasting horizons.The paper isolates representation learning to examine its stability, sensitivity, and scalability across horizons.
Approach: The proposed dual-stage MLP–QRNN hierarchy learns broad ten-quantile channel distributions before refining retained mid-tail quantiles into compact feature vectors.The downstream classifier is deliberately separated from the feature-extraction methodology.
Evaluation: The evaluation covers 72 machines across 9 facilities and three regimes spanning 10 seconds–60 minutes, 36–70 hours, and 7–30 days.A fixed staged thirteen-pipeline programme quantifies the cost of transferring representations beyond their design horizon.
Main finding: Four retained mid-tail quantiles improve robustness as temporal separation grows, supporting horizon-conditioned feature extraction rather than fixed preprocessing.The study varies feature capacity and other representation choices across forecasting regimes.

2. Related Work

Related work spans hand-crafted, classical, deep, and hybrid feature extraction, while quantile regression provides distribution-aware summaries and Transformers model long-range temporal dependencies. This paper combines these ideas by using quantile-state features upstream of a fixed temporal decision stage.

Feature-extraction families: Industrial feature extraction includes manual, classical data-driven, deep representation-learning, and hybrid or multimodal families.These approaches differ in whether representations are engineered, shallowly transformed, learned end to end, or combined across modalities.
Quantile regression: Quantile regression summarizes spread and asymmetry for downstream prognostic or classification tasks rather than only estimating an expected value.The paper adopts distributional feature extraction rather than direct probabilistic forecasting.
Temporal modelling: Transformer attention captures long-range industrial-sequence dependencies but becomes computationally demanding on raw high-rate streams as sequence length grows.The cited motivation is the quadratic cost of full attention over sequence length.
Paper positioning: Here the Transformer consumes hourly quantile-state vectors, reducing its effective sequence length from raw measurements to hourly words.The downstream decision stage is held fixed within each regime.
Scope: The study evaluates the upstream representation and does not claim the downstream classifier as a feature-extraction contribution.This scope distinction separates representation learning from temporal decision modelling.

3. Industrial Data Foundation and Temporal Representation

The dataset combines heterogeneous multi-rate sensors from 72 machines across 9 facilities with causal temporal alignment and horizon-specific representations. Hourly sensor windows become compact feature vectors that are sequenced for downstream forecasting.

Industrial deployment: The deployment contains 72 machines across 9 facilities within one broad high-speed manufacturing asset and product family.Consistent sensor placement supports controlled multi-site evaluation while exposing variation in operating and maintenance conditions.
Sensor foundation: Sensors cover vibration, flow, thermal, pressure, tooling, and acceleration groups, with 43 channels for short and extended horizons and 81 for day-scale forecasting.The day-scale regime expands sensor breadth for slow-evolving degradation modes.
Temporal representation: Causal alignment places multi-rate streams on a common 20 ms grid, preventing future information from entering features.Lower-frequency channels are forward-aligned, and each hourly word contains roughly 180,000 reference-grid steps per channel.
Temporal representation: The extractor maps each hourly word to a compact quantile-state vector, and the downstream classifier processes the resulting ordered document of vectors.For a 30-day document, the sequence contains K = 720 hourly words.
Horizon conditioning: Beyond one hour, an 18-dimensional temporal embedding adds time-of-day, shift, and schedule information to the extractor input.The effective input increases from 43 to 61 dimensions at 70 hours and from 81 to 99 dimensions at 30 days.

4. Quantile-Led Feature Extraction

The extractor uses channel-resolved quantile regression to learn conditional distributions and preserve sensor-specific distributional information. Its objective applies asymmetric pinball loss across quantile levels and channels while enforcing ordered downstream quantiles.

Inputs and targets: Each sensor channel has a 12-bit target, and the multichannel observation is represented as a vector across channels.The model predicts conditional quantiles at levels α ∈ (0, 1).
Training objective: Pinball loss penalizes over- and under-estimation asymmetrically for each predicted quantile.The loss weighting depends on the quantile level α.
Training objective: The aggregate stage objective averages channel–quantile errors over a mini-batch and the selected quantile set.The objective is applied separately to each channel and quantile pair before averaging.
Channel resolution: Intermediate MLP embeddings mix cross-channel information, but supervised outputs remain channel-resolved for every quantile level.Global context conditions predictions while quantile errors remain sensor-specific.
Quantile structure: Predicted quantiles are ordered before feature construction to prevent quantile crossing.This preserves a non-crossing quantile structure for downstream representation building.

4.2. Dual-Stage Hierarchy

The extractor uses a dual-stage hierarchy that first learns broad channel-wise conditional distributions, then refines selected mid-tail quantiles into compact features for anomaly-sensitive prediction.

The hierarchy separates broad distribution learning from targeted mid-tail refinement.QRNN1 models the overall conditional distribution, while QRNN2 produces the retained mid-tail representation.
A 64-dimensional MLP embedding is concatenated with the raw input, conditioning QRNN1 on measured and learned cross-channel context.
QRNN1 predicts channel-resolved outputs across lower-tail, central, and upper-tail quantile levels to learn each channel’s conditional distribution shape.
MLP2 compresses the broad 430-dimensional representation into a 32-dimensional latent space emphasizing mid-tail structure and filtering less critical detail.
Skip-connected QRNN2 refines the retained mid-tail predictions while preserving coarse context and supporting gradient flow.
The baseline two-quantile representation retains A2 = {0.25, 0.75}, producing an 86-dimensional feature vector that balances coverage, sensitivity, and latency.

4.3. Two-Tier Training Procedure

Training proceeds in two stages: QRNN1 learns calibrated broad distributions first, then its frozen representation supports mid-tail refinement by MLP2 and QRNN2.

QRNN1 is trained first over ten quantiles to calibrate multi-quantile coverage across 43 channels.This produces 43×10 = 430 channel–quantile loss terms per example.
After QRNN1 parameters are frozen, MLP2 and QRNN2 are trained over the retained mid-tail quantiles.
The staged procedure first captures broad distributional structure and then concentrates capacity on pre-failure distributional widening relevant to the downstream classifier.
Table 1 lists the horizon-conditioned extractor configurations used to scale the hierarchy across forecasting regimes.

4.4. The Four-Quantile Mid-Tail Variant

The four-quantile variant keeps the dual-stage pipeline and training procedure unchanged while increasing refined mid-tail resolution to test sensitivity to gradual distributional drift.

The four-quantile variant increases the retained mid- and near-tail set from two to four levels while preserving the MLP1 → QRNN1 → MLP2 → QRNN2 pipeline.
Output dimensionality rises from 43×2 = 86 to 43×4 = 172 in Pipeline P5.
Because all other components remain fixed, performance differences isolate the effect of refined mid-tail representation resolution.
The tested hypothesis is that denser mid-tail sampling improves sensitivity to gradual distributional drift, especially as prediction horizons lengthen.

4.5. Horizon-Conditioned Scaling

The framework preserves its dual-stage hierarchy across horizons but scales temporal context, model capacity, and input representation with the forecasting horizon.

The same dual-stage hierarchy extends from the 1-hour regime to TQRNN70h and TQRNN30d rather than replacing the extractor.
An 18-dimensional temporal embedding augments the physical input to expose periodic and slow-evolving patterns at longer horizons.The resulting inputs are 61-dimensional at 70 hours and 99-dimensional at 30 days.
Longer-horizon configurations widen MLP encoders and deepen QRNN stacks to increase representational capacity.

5. Experimental Design

The experimental design uses a fixed staged thirteen-pipeline ablation to compare feature representations across progressively longer forecasting regimes. Within each regime, the downstream temporal decision model is held fixed, while the strongest configuration is transferred forward to measure degradation beyond its design horizon.

Staged Ablation Protocol: The strongest configuration from each regime is carried unchanged into the next as a transferred representational baseline.This staged design directly measures transfer degradation rather than conflating it with re-optimization.
Staged Ablation Protocol: The thirteen-pipeline ablation compares broad-quantile, mid-tail, Transformer-only, and integrated representations across three forecasting regimes.The integrated hierarchy is MLP1 → QRNN1 → MLP2 → QRNN2; configurations are instantiated across short-, extended-hour, and day-scale settings.
Short Horizon: Short-horizon pipelines P1–P5 vary representation type and retained quantile cardinality, with P4 using two quantiles and P5 using four under otherwise fixed conditions.The short-horizon input contains 43 sensor channels, and the final P5 row reports additive attention enabled.
Extended Hour: Extended-hour pipelines P6–P9 retrain horizon-specific representations while retaining P5 unchanged as the transferred reference.The comparison includes isolated QRNN1, isolated QRNN2, Transformer-only, and fully integrated extended-hour extractors.
Day Scale: Day-scale pipelines P10–P13 retrain the representation stages on 81-channel input while retaining P9 as the transferred bridge configuration.The day-scale comparison again includes isolated broad-quantile, isolated mid-tail, Transformer-only, and integrated extractors.
Decision Stage: Within each forecasting regime, the temporal decision stage remains fixed so measured differences primarily reflect the upstream representation.Quantile features feed a Transformer-based temporal decision stage, with a multi-stream temporal-fusion stage used at the longest horizon.

6. Results and Discussion

The ablations show that quantile-integrated representations are strongest at short horizons, but reliable forecasting at extended and day-scale horizons requires horizon-conditioned architectural scaling rather than direct transfer.

Short-Horizon Ablation: The four-quantile extractor outperforms the Transformer-only baseline and other matched configurations as temporal separation increases.Quantile-integrated configurations are consistently stronger, while the Transformer-only baseline is relatively stronger at longer short-horizon windows.
Short-Horizon Ablation: 75.92% and 72.44% F1 are achieved at 30 and 60 minutes by the attention-enabled four-quantile extractor.The four-quantile configuration improves over two quantiles by 3.19 and 4.76 percentage points at those horizons.
Extended-Hour Ablation: 42.90% F1 is reached by the unmodified short-horizon extractor at 70 hours, whereas the fully integrated extended-hour extractor reaches 60.38%.The extended-hour configuration combines coarse and refined quantile structure with increased feature capacity.
Day-Scale Ablation: 61.49% to 52.41% F1 is the transferred bridge’s decline from 7 to 30 days without day-scale adaptation.The day-scale evaluation expands sensor-level input dimensionality from 43 to 81 channels.
Day-Scale Ablation: 79.97% F1 is achieved at 30 days by the complete integrated day-scale extractor, which improves from 76.09% at 7 days while alternatives decline.The day-scale configuration also reports 80.18% recall, 81.82% precision, and 82.39% accuracy at 30 days.
Cross-Regime Interpretation: The cross-regime progression P5 → P9 → P13 shows that feature extraction must be scaled with horizon rather than transferred unchanged.The reported scaling includes feature capacity, temporal embedding, activation, and sensor breadth.

7. Conclusion

The paper presents a quantile-led MLP–QRNN framework for multi-horizon predictive maintenance and shows that robust cross-horizon representation transfer requires horizon-specific scaling.

The dual-stage MLP–QRNN hierarchy converts multivariate sensor streams into compact, channel-resolved, distribution-aware feature vectors.It uses broad ten-quantile learning followed by skip-connected mid-tail refinement under channel-resolved pinball-loss supervision.
Four-quantile mid-tail refinement improves robustness as temporal separation increases across the thirteen-pipeline evaluation.The retained levels are α ∈{0.25, 0.40, 0.60, 0.75}.
60.38% F1 at 70 hours and 79.97% F1 at 30 days are reached by horizon-conditioned extractors.The transferred P9 bridge declines to 52.41% F1 at 30 days without full day-scale adaptation.
Representation transfer across horizons depends on scaling feature capacity, temporal embedding, activation strategy, and sensor breadth to the forecasting task.