Source-linked AI summary
Fuzzy-MoE: Interpretable Regime-Conditioned Expert Routing for Non-Stationary Multivariate Time Series Forecasting
Lan Guo, Jie Xiao, Zhao Su, Jun Shen, Haoran Li, Weixia Ma, Qingguo Zhou, Binbin Yong
TL;DR
Non-stationary multivariate forecasting must handle heterogeneous regimes across variables and samples, while conventional unified models and MoE routers provide limited transparency. Fuzzy-MoE combines dual-view fuzzy state identification with interpretable expert routing, and benchmark results report superior forecasting accuracy with traceable rule diagnostics.
Problem
Heterogeneous temporal dynamics make unified forecasting mappings inadequate, while black-box MoE routing does not explain why particular experts are selected.
Method
Fuzzy-MoE uses dual-view local and global state cues, Gaussian fuzzy memberships, and explicit IF-THEN rules to route experts at the sample-channel level.
Results
Fuzzy-MoE achieves superior forecasting accuracy and intrinsic routing interpretability across benchmark evaluations.
Takeaways & Limitations
Traceable fuzzy memberships, expert weights, and rule activations expose the rationale behind variable-specific expert selection.
Abstract
from arXiv · showhide
In non-stationary multivariate time series, different variables and samples often exhibit heterogeneous latent dynamic states, while existing deep forecasting models usually compress them into a unified end-to-end mapping, leading to suboptimal modeling of time-varying dynamics and limited interpretability regarding which forecasting mechanism is activated under different latent states. To overcome these limitations, we reformulate time series forecasting as a unified framework of latent temporal state identification and interpretable expert routing, and propose Fuzzy-MoE, a fuzzy logic-based dynamic Mixture-of-Experts model. Fuzzy-MoE consists of multiple parallel expert mapping networks and a dual-view fuzzy router. By jointly exploiting local convolutional dynamics and global segmented statistics, the router infers latent temporal states and computes expert activation strengths through learnable Gaussian membership functions, enabling explicit IF-THEN rule-based expert selection. This fine-grained routing strategy allows different variables within the same sequence to activate different experts, effectively capturing heterogeneous temporal dynamics while improving model interpretability. Experimental results on multiple public time series benchmark datasets show that Fuzzy-MoE significantly outperforms mainstream forecasting methods in forecasting accuracy. Moreover, fuzzy memberships and rule activations provide interpretable routing diagnostics, demonstrating the effectiveness of the proposed framework in both forecasting performance and mechanism transparency. Unlike traditional MoE models that use black-box routing, Fuzzy-MoE`s routing is based on clear, interpretable fuzzy rules. This makes the expert selection transparent and traceable.
Introduction
Non-stationary multivariate series contain heterogeneous dynamics across segments and variables, limiting unified models. Fuzzy-MoE addresses this with interpretable, state-conditioned routing and variable-specific expert assignment.
- Heterogeneous dynamics across time segments and variables make a single unified mapping inadequate for non-stationary multivariate forecasting.
- Existing MoE predictors commonly use black-box Softmax gates that lack decision transparency and can collapse onto a few experts.
- Fuzzy-MoE reformulates forecasting as latent temporal state identification followed by conditional expert selection through inspectable IF-THEN rules.
- The dual-view router combines local convolutional dynamics and global segmented statistics, using Gaussian memberships to generate expert weights.
- The framework recasts forecasting as an interpretable state-conditioned expert-routing problem for heterogeneous data dynamics.
- Sample-channel routing lets variables within one sequence activate distinct experts and exposes memberships, weights, and rule activations as diagnostics.
Related Works
Time-series forecasting has progressed from classical and feature-dependent methods toward deep models, while interpretability and MoE routing remain unresolved challenges. Existing forecasting MoE systems generally retain black-box gating.
- ARIMA handles stationary linear data, whereas SVR adds nonlinear modeling but still depends on manually designed features.
- Transformer-based forecasters use self-attention or transformations such as Fourier-based reshaping to model global and multivariate temporal patterns.
- Deep models’ black-box behavior limits their use in high-risk domains, and post-hoc explanations may diverge from actual decisions.
- MoE architectures enlarge capacity by distributing subsets of data across multiple expert networks.
- Time-series MoE methods use varied expert architectures and sparse gating, but their linear-projection-plus-Softmax routers remain black-box.
Proposed Method
This section introduces the proposed Fuzzy-MoE framework.
- The section presents the proposed Fuzzy-MoE framework in detail.
Overall: From Sequence Mapping to Regime-Conditioned Routing
Fuzzy-MoE replaces one-size-fits-all sequence mapping with regime-conditioned routing for heterogeneous dynamics. It separates fuzzy regime identification from expert forecasting in two inspectable stages.
- Unified sequence-to-sequence forecasting assumes one parameter set can capture all regimes, but variables and segments may occupy heterogeneous dynamics.
- The model first identifies the current temporal regime and then selects the expert best suited to complete forecasting.
- The formulation uses multiple state views to generate fuzzy routing-based expert selections for historical input X ∈ R^L×C.
- The resulting pipeline separates regime identification through fuzzy routing from expert forecasting through expert networks.
Model Architecture Overview
Fuzzy-MoE normalizes and segments inputs, then combines local convolutional dynamics with global segment statistics for dual-view fuzzy routing.
- The architecture includes normalization and segmentation, dual-view state extraction and fuzzy routing, expert mapping, and weighted output recovery.
- Input Normalization and Segmentation: Reversible instance normalization mitigates distribution shift before the sequence is divided into non-overlapping segments.The normalized sequence is segmented into S = L/Tseg segments of length Tseg.
- Input Normalization and Segmentation: Segmentation reduces sequence length while preserving local temporal structures, enabling expert mapping at the segment level.
- Dual-View State Feature Extraction: The local pathway independently applies 1D convolutions to each variable channel to extract short-term patterns and abrupt changes.It uses two convolutional layers, with batch normalization and GELU activation between them.
- Dual-View State Feature Extraction: The global pathway computes segment means to capture long-term trends and periodic characteristics.The two pathways operate at different temporal scales and provide complementary state clues for fuzzy routing.
Dual-View Fuzzy Routing Mechanism
Fuzzy-MoE uses dual-view fuzzy gates to convert local and global temporal features into interpretable, sample-channel-level expert weights through Gaussian memberships and explicit rule activation.
- Dual-view fuzzy routing: The fuzzy router combines convolutional and global fuzzy gates, each performing fuzzy projection, Gaussian membership computation, rule activation, and gating-weight generation.The two pathway outputs are fused into final gating weights.
- Fuzzy variable projection: The projection network maps high-dimensional features into a low-dimensional fuzzy-variable space, encouraging structured latent representations for interpretable membership functions.The fuzzy space has M variables with M much smaller than the original feature dimension D.
- Membership computation: Gaussian membership functions measure how strongly each sample satisfies each expert’s fuzzy premise condition.Membership degrees lie in (0, 1] and use learnable centers and standard deviations.
- Rule activation: Rule firing strengths aggregate all premise memberships with a product T-norm, representing the confidence that the sample simultaneously satisfies an expert’s conditions.The product implements the fuzzy-logic AND operation.
- Interpretable gating: Temperature-scaled Softmax normalizes semantically meaningful rule firing strengths, making expert-weight generation more traceable than Softmax over arbitrary projections.Lower temperature sharpens the gating distribution, while higher temperature smooths it.
- Fine-grained routing: Fused gating weights are computed independently for each variable channel, allowing variables within one sequence to select different forecasting mechanisms.The router therefore produces expert assignments at sample-channel granularity.
Expert Networks and Output Fusion
Fuzzy-MoE uses adaptive residual expert networks to map segmented histories into forecasts, then combines expert outputs using the learned gating weights and restores the final output scale.
- Expert networks: Each expert uses a two-layer linear trunk and a single-layer shortcut branch to map input segments to output segments.The trunk and shortcut provide nonlinear and direct linear paths.
- Expert networks: Learnable Softmax fusion weights balance the trunk’s nonlinear fitting with the shortcut’s linear temporal fitting within each expert.The fusion weights are normalized over the two branches.
- Output fusion: Expert outputs are aggregated by weighted summation using the routing weights.The gating weights determine each expert’s contribution to the combined forecast.
- Output recovery: The combined forecast is flattened, denormalized using the original standard deviation and mean, and transposed into the final batch-by-horizon-by-channel output.The forecast shape is B×P×C after recovery.
Experiments
Across six benchmark datasets and multiple ablations, Fuzzy-MoE achieves strong forecasting performance while its fuzzy routing provides adaptive, interpretable expert allocation.
- Forecasting performance: On Electricity at horizon 720, Fuzzy-MoE reaches MSE 0.203 and MAE 0.294 versus iTransformer’s MSE 0.228 and MAE 0.313.These correspond to relative error reductions of 10.96% for MSE and 6.07% for MAE.
- Forecasting performance: Fuzzy-MoE consistently achieves the best forecasting performance across horizons 96, 192, 336, and 720 on the six benchmark datasets.
- Expert routing: Expert 1 receives higher weights across multiple ETTh2 channels, while other experts capture complementary latent patterns.The learned routing weights are described as balanced yet discriminative, mitigating expert collapse and encouraging specialization.
- Rule ablation: Removing highly activated rules causes larger MAE increases than removing the least activated rule, indicating differentiated rule contributions.On ETTh1, removing the top-1 rule yields ΔMAE 0.012, whereas removing the least activated rule yields ΔMAE 0.001.
- Ablation study: The best fuzzy temperature is τ=5, while larger temperatures smooth memberships and balance routing and smaller values produce concentrated assignments.
- Representation analysis: Fused local and global representations are more structured and discriminative than either individual branch, supporting more accurate latent-state routing.The fused representation promotes expert specialization and collaboration, which the authors connect to improved forecasting performance.
- Gating ablation: Replacing the MLP Gate with the Fuzzy Gate improves forecasting across datasets and horizons; on ETTh2 at horizon 96, MSE is 0.271 and MAE is 0.334.These values correspond to relative reductions of 10.56% in MSE and 7.73% in MAE versus the standard MLP gate.
Conclusion
Fuzzy-MoE combines dual-view Gaussian fuzzy routing with per-variable expert assignment for non-stationary multivariate forecasting. The authors report superior accuracy and interpretable routing, while identifying online rule adaptation and sparse activation as future directions.
- Fuzzy-MoE uses dual-view Gaussian fuzzy routing to generate traceable IF-THEN rules and assign experts per variable.
- Benchmarks confirm superior forecasting accuracy and intrinsic routing interpretability for non-stationary multivariate forecasting.
- Future work will extend fuzzy rule extraction to online adaptive forecasting and integrate sparse expert activation to reduce computation overhead.