Source-linked AI summary
Delay-Guided Attention for Quality Prediction in Multistage Manufacturing
Jiarui Zhang, Zhaoyang Guan, Zhixuan Xia, Naichen Shi
TL;DR
Equal timestamps in multistage manufacturing may represent different process states, and existing approaches do not identify which upstream history is relevant to each downstream observation. DAMF estimates inter-stage lag distributions and uses them to guide cross-stage alignment for quality prediction. It outperforms compared baselines on final-quality prediction and recovers more accurate, stable lag paths on controlled-delay benchmarks.
Problem
Equal timestamps may represent different process states, and final quality depends on which upstream histories are associated with current downstream conditions.
Source: §1 · p. 2Method
DAMF estimates sample-dependent lag distributions for adjacent stages, uses them as priors for cross-stage attention in quality forecasting, and decodes coherent lag paths for interpretation.
Source: §1 · p. 3Results
DAMF outperforms compared baselines on final-quality prediction and recovers more accurate and stable lag paths on controlled-delay benchmarks.
Source: §5 · p. 20Takeaways & Limitations
Estimated inter-stage delays provide interpretable diagnostic outputs and useful temporal guidance for aligning upstream and downstream process representations.
Source: §5 · p. 20The framework focuses on adjacent-stage delays and a discrete candidate lag space, leaving non-adjacent interactions, branching, feedback, and continuous delay representations for future work.
Source: §5 · p. 21
Abstract
from arXiv · showhide
Quality prediction in multistage manufacturing is challenging when inter-stage lags vary over time, as measurements taken at the same timestamp may reflect different process states. Existing approaches often preserve production order but fuse timestamp-aligned features or model delayed dependencies implicitly, without identifying the upstream history relevant to each downstream observation. We propose Delay-Guided Attention for Multi-stage Forecasting (DAMF), which explicitly estimates inter-stage lag distributions and uses them to guide cross-stage information transfer for final quality prediction. For each adjacent stage pair, a lag predictor compares the current downstream representation with candidate lagged upstream representations. An occurrence-magnitude factorization separately estimates the probability of a positive lag and its conditional magnitude, yielding a sample- and stage-pair-specific distribution over discrete lag states without requiring plant-specified delays at inference. DelayAlignment incorporates this soft distribution as a temporal prior into differentiable cross-stage attention, while gated residual fusion regulates information propagation across production stages. For interpretation, segment-wise Viterbi decoding converts pointwise lag distributions into temporally coherent lag paths, whereas forecasting retains the soft distributions for alignment. On the real-world LiquidSugar dataset, DAMF improves final quality prediction over the compared baselines. Evaluations on two controlled-delay benchmarks show more accurate and stable lag recovery, and ablation studies support the contributions of lag prediction, delay-guided attention, and structured temporal decoding.
1. Introduction
Multistage quality prediction must connect downstream observations to relevant upstream histories because equal timestamps may represent different process states and inter-stage lags can vary. DAMF estimates those lags and uses them to guide final-quality prediction while providing interpretable lag paths.
1. Introduction
- Equal timestamps may represent different process states, while stage-pair lags vary with production conditions and can be difficult to estimate from noisy observations.Estimating offsets can help identify when and at which stage a quality-relevant deviation first becomes observable.Source: §1 · pp. 2–3
- DAMF jointly models inter-stage delays and final quality by using estimated lag distributions to guide cross-stage alignment and quality prediction.The model retains soft lag distributions for forecasting and uses Viterbi decoding to produce temporally coherent lag paths for interpretation.Source: §1 · p. 3
- Its lag estimator compares current downstream representations with candidate upstream histories for each adjacent stage pair, estimating both lag occurrence and magnitude.A Viterbi filter stabilizes lag estimates.Source: §1 · p. 3
- DelayAlignment uses the estimated lag distribution as a temporal prior for attention, and gated residual fusion integrates aligned information for final-quality prediction.The framework is evaluated on the real LiquidSugar dataset and two controlled-delay benchmarks, with ablations of its components.Source: §1 · p. 3
2. Related Work
Prior work models stage structure, general temporal patterns, or lagged dependencies, but does not combine sample-dependent stage-pair lag distributions with final-quality forecasting. DAMF connects lag estimation to cross-stage alignment and decodes coherent lag paths for interpretation.
2. Related Work
- Stage-aware quality models represent production order through shared representations, gated transfer, graph message passing, or sequential modules, but leave cross-stage temporal relationships implicit.They do not expose and use a sample-dependent lag distribution for each adjacent stage pair to align observations.Source: §2 · p. 4
- General forecasting architectures extract temporal patterns but do not explicitly use production order and adjacent-stage relationships to associate downstream observations with upstream history.Inter-stage offsets are absorbed into learned representations rather than represented as explicit, interpretable variables.Source: §2 · p. 4
- Lag-identification methods estimate dependencies over variables or analysis windows, while sequence decoders smooth score sequences; neither connects sample-dependent stage-pair lag distributions to forecasting.DAMF estimates those distributions, uses them to guide quality-prediction alignment, and decodes coherent lag paths for interpretation.Source: §2 · pp. 4–5
3. Methodology
DAMF estimates discrete, adjacent-stage lag distributions and uses them to guide cross-stage information transfer for final-quality forecasting. It combines occurrence and magnitude prediction, smooths lag paths for recovery, and retains soft lag distributions for forecasting alignment.
3.1. Lag modeling and DAMF
- For each adjacent stage pair, DAMF estimates a lag distribution and uses DelayAlignment to aggregate candidate upstream representations before gated fusion with the current stage.Repeated across stages, the updates form the representation used for the final-quality forecast.Source: §3.1 · p. 6, §3.3 · pp. 9–10
3.2.1. Lag matching and estimation
- The lag predictor encodes upstream and downstream windows with GRUs, then scores semantic correspondence between the downstream context and candidate lagged upstream states.A lag-specific bias is included in the similarity score.Source: §3.2.1 · p. 7
3.2.2. Lag-predictor training
- Lag occurrence and conditional magnitude are predicted separately, then combined into a categorical distribution over candidate lags.Training uses occurrence and positive-lag magnitude losses, with auxiliary Top-N, large-lag, and selector objectives; the selector is not used to construct the inference distribution.Source: §3.2.1 · p. 7, §3.2.2 · pp. 7–8
3.2.3. Denoised inference of the lag
- Viterbi decoding smooths fluctuating pointwise lag predictions by balancing agreement with predicted lag scores against penalties for abrupt changes.The transition score also applies specified corrections for transitions involving large-lag and zero-lag states.Source: §3.2.3 · p. 8
- The decoded path is used for lag recovery and interpretation, while forecasting uses the pointwise lag distribution as a soft alignment prior.DelayAlignment combines lag priors with representation-based matching, allowing multiple candidate lags to contribute to the upstream message.Source: §3.2.3 · p. 9, §3.3 · pp. 9–10
3.3. Lag-guided forecasting
- Forecasting proceeds in two phases: fit the lag predictor, then train the quality model with its distributions as alignment priors while holding the predictor fixed.The forecasting objective is mean absolute error; it updates the forecasting backbone, DelayAlignment, gated fusion modules, and prediction head.Source: §3.3 · p. 10
4. Experiments
Experiments evaluate DAMF’s lag recovery on controlled-delay benchmarks and final-quality prediction on LiquidSugar. The benchmarks include both injected delays in LiquidSugar and synthetic time-varying delays.
4. Experiments
- The evaluation covers lag recovery, real-data quality prediction, and ablations of model components.The ablations examine lag-predictor architecture, loss components, structured decoding, and delay priors.Source: §4 · p. 10
4.1. Datasets
- LiquidSugar quality prediction uses 22,854 observations, 24 process variables, and yield flow as the quality target.Data are split chronologically into training and test sets at approximately 80/20 after regularizing continuous segments to a 15-minute grid.Source: §4.1 · p. 11
- The delay-injected LiquidSugar benchmark uses six lag states from 0 to 75 minutes and retains the injected trajectory as ground truth.Half of eligible segments are randomly selected for lag injection; the remainder retain zero lag.Source: §4.1 · pp. 11–12
- The CausalAI synthetic benchmark tests prescribed time-varying lags across seven states, including persistent block and transient smooth-bump trajectories.Its explicitly specified ground-truth lag trajectory supports direct evaluation of recovery and tracking of changing delays.Source: §4.1 · p. 12
4.2. Experimental Settings
The experiments compare DAMF with machine-learning, sequential, and multistage quality-prediction baselines, and with two lag-recovery methods. Evaluation metrics distinguish quality prediction from lag occurrence, magnitude, pointwise accuracy, and large-lag recovery.
4.2. Experimental Settings
- Quality-prediction comparisons include traditional machine-learning, deep sequential, and multistage baselines.The baselines are Linear Regression, XGBoost, Random Forest, LightGBM, LSTM, GRU, Transformer, TimeXer, PEGNN-SS, and MultistageNet.Source: §4.2 · p. 12
- Lag recovery is compared with MNet’s MultistageNet hidden-lag predictions and WCC’s windowed cross-correlation followed by Viterbi decoding.The controlled-delay benchmarks include persistent block-shaped and transient smooth bump-shaped lag patterns.Source: §4.2 · p. 12
4.2.1. Evaluation metrics
- Quality prediction is assessed with RMSE, MAE, and R2; lower error and higher R2 indicate better performance.Results are reported as mean ± standard deviation over three random seeds.Source: §4.2.1 · p. 13
- Lag-recovery measures distinguish occurrence detection from magnitude error, peak-height error, pointwise accuracy, and large-lag recovery, with large lags defined as true lags of at least 4 sampling steps.Measures include In-bump MAE on true positive-lag points, Outside mean delay on true zero-lag points, Peak Value Error, exact-match Acc, and one-step-tolerance accuracy. Occurrence reporting includes precision, false alarm rate, and Occ F1; Occ F1 labels positives at probability ≥0.5.Source: §4.2.1 · pp. 13–14
- Large-lag Recall checks whether true large lags remain classified as large, while Collapse to 0 and Collapse to small quantify underestimation.Collapse to small counts true large lags predicted as one to three sampling steps.Source: §4.2.1 · p. 14
4.3. Main Results
On LiquidSugar, DAMF achieves the best reported final-quality prediction performance across all three metrics, outperforming MultistageNet, the strongest baseline.
4.3. Main Results
- DAMF achieves the best LiquidSugar results across RMSE, MAE, and R2, improving on MultistageNet, the strongest baseline.Table 1 reports final quality prediction performance on LiquidSugar; its best result for each metric is highlighted.Source: Table 1 · p. 14, §4.3 · p. 15
- MultistageNet is the strongest baseline across all three metrics, while TimeXer and PEGNN-SS do not outperform the strongest traditional baselines.Generic LSTM, GRU, and Transformer models also do not consistently outperform traditional baselines.Source: §4.3 · p. 15
4.4. Lag recovery on controlled-delay benchmarks
On both controlled-delay benchmarks, DAMF improves lag detection and estimates lag magnitude more effectively overall than MNet and WCC. Decoded examples track injected trajectories, with differences near rapid changes and stronger agreement on CausalAI.
4.4.1. Quantitative comparison
- DAMF achieves higher precision and lower FAR than MNet and WCC on both controlled-delay benchmarks.On LiquidSugar it also reduces In-bump MAE, Outside mean delay, and Peak Value Error; on CausalAI its In-bump MAE is close to WCC, while its Outside mean delay and Peak Value Error are substantially lower.Source: §4.4.1 · p. 15
4.4.2. Representative lag recovery examples
- On LiquidSugar, decoded lag paths capture the main injected intervals and overall trajectory, with discrepancies mainly near transitions and rapidly changing lags.Both block-shaped and bump-shaped delay structures are retained.Source: §4.4.2 · p. 15
- On CausalAI, decoded paths show closer ground-truth agreement and recover the sustained block and smooth bump, including onset, peak, and decay.The passage relates this pattern to the benchmark’s cleaner source–target dependence.Source: §4.4.2 · p. 16
4.5. Ablation studies
Ablations find that the lag prior improves final quality prediction, while lag-predictor design, training losses, and structured decoding affect lag recovery.
4.5.1. Structural ablation of the lag predictor
- Removing source-target lag matching, lag-specific source value extraction, or the selector head worsens lag recovery; removing source-value extraction causes the largest degradation.Its largest effects include reduced large-lag recall and more collapse-to-zero errors.Source: §4.5.1 · pp. 16–17, Table 2 · p. 17
4.5.2. Loss ablation of the lag predictor
- The full lag-predictor objective performs best overall in Occ F1, Lag MAE, ±1 Acc, and Collapse to 0 while maintaining competitive large-lag recall.The simplified occurrence-plus-positive-lag-CE setting reduces large-lag recall and increases collapse-to-zero rate.Source: §4.5.2 · p. 18, Table 3 · p. 18
- Relaxed-TopN preserves plausible large-lag candidates, positive-lag CE estimates magnitude, and the large-occurrence recall loss protects large delays from being classified as no or small lag.The selector loss has a moderate, consistent effect and primarily regularizes candidate ranking.Source: §4.5.2 · p. 18
4.5.3. Effect of Viterbi decoding
- Viterbi decoding improves the main lag-recovery accuracy metrics and reduces collapse to small, producing less fragmented paths despite raw argmax’s slightly lower collapse-to-zero value.Figure 8 shows raw predictions with local spikes, short-lived drops, and seed variability; decoded paths better align with delay-region duration.Source: §4.5.3 · pp. 18–19, Table 4 · p. 19, Figure 8 · p. 19
4.5.4. Effect of the lag prior on final quality prediction
- Most lag-prior settings lower RMSE and MAE and raise R2 versus no prior; λ = 0.4 gives the best results across all three metrics.The variants share the forecasting architecture and training protocol, with only the prior weight varying.Source: §4.5.4 · pp. 19–20, Figure 9 · p. 20
5. Conclusion
DAMF integrates inter-stage delay estimation into quality forecasting and recovers interpretable lag patterns. The authors identify adjacent-stage modeling and discrete candidate lags as current scope boundaries.
5. Conclusion
- DAMF outperforms the compared baselines on final quality prediction and recovers more accurate and stable lag paths on controlled-delay benchmarks.The estimated delays also serve as temporal guidance for aligning upstream and downstream representations.Source: §5 · p. 20
- Ablations show that removing the lag prior weakens quality prediction, while simplifying lag prediction or replacing structured decoding degrades lag recovery or path stability.The findings support using estimated lag distributions both to guide cross-stage alignment and to recover coherent paths for interpretation.Source: §5 · p. 21
- The current framework models adjacent-stage delays with discrete candidate lags; non-adjacent interactions, branching, feedback, and continuous delays remain future extensions.Source: §5 · p. 21
Appendix C. Detailed lag-predictor objective
The appendix defines a lag-predictor objective for an adjacent-stage edge, combining occurrence, positive-lag magnitude, and candidate-ranking supervision.
Appendix C. Detailed lag-predictor objective
- The occurrence head estimates the probability that a lag is positive, while the positive-lag head estimates the conditional probability of each positive lag state.Source: Appendix C · p. 26
- The occurrence head uses a focal-style binary classification loss, and the positive-lag head uses conditional cross-entropy on positive-lag samples.The focal parameter and class-weighting term are specified in the objective.Source: Appendix C · p. 26
- The relaxed Top-N term encourages the ground-truth lag to remain among leading candidates without requiring it to have the highest matching score.The term uses a positive margin.Source: Appendix C · p. 26
- The selector supplies auxiliary ranking supervision only and is not used to construct the lag distribution at inference.The complete objective assigns auxiliary weights to the large-occurrence and selector terms.Source: Appendix C · p. 27