Source-linked AI summary
HALO: Enhancing Time Series Generation via Hyperspherical Latents and Masked AutoregRessive Modeling
Chunyi Hou, Xiangfei Qiu, Hanyin Cheng, Yutong Li, Bin Yang
TL;DR
Existing two-stage time-series generators can lose fine-grained temporal information through discretization and accumulate errors during autoregressive generation. HALO combines continuous hyperspherical latents with masked autoregressive modeling, and experiments across 12 datasets show strong generation quality and improved inference efficiency.
Problem
Existing two-stage time-series generators can lose fine-grained temporal information during discretization and accumulate errors during autoregressive generation.
Source: §1 · p. 3Method
HALO combines a hyperspherical VAE for continuous latents with masked autoregressive modeling that balances parallel decoding and temporal correlation learning.
Source: §1 · pp. 2–3Results
Takeaways & Limitations
The results support continuous latent modeling as an effective approach to high-quality and efficient time-series generation.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Most existing time series generators rely on a two-stage modeling paradigm: the first stage learns discrete latent representations of time series; the second stage performs autoregressive modeling on these discrete latents through next token prediction. However, this paradigm suffers from two stage-specific limitations: the first stage can lead to information loss when discretizing continuous time series, while the second stage is prone to error accumulation during autoregressive generation. To address these limitations, our core idea is to perform generative modeling in a continuous latent space with a more efficient autoregressive framework. We propose HALO, which enhances time series generation via Hyperspherical Latents and Masked Autoregressive modeling to achieve this goal by tackling two key bottlenecks: (1) variance and scale heterogeneity of continuous latent representations; (2) the difficulty of balancing generation efficiency with temporal correlation modeling. HALO first introduces a hyperspherical VAE that constrains continuous latents to a fixed-radius hyperspherical shell, effectively stabilizing the numerical fluctuations of continuous latent representations. Secondly, we develop a masked autoregressive model that balances parallel decoding and temporal correlation learning, substantially reducing the number of inference steps required for generation and improving generation stability. Our extensive experiments demonstrate that HALO achieves state-of-the-art generation performance while offering significantly improved inference efficiency over existing advanced baselines.
1 INTRODUCTION
HALO addresses information loss from discrete time-series tokenization and error accumulation in sequential generation by modeling continuous latents with a more efficient autoregressive framework. Its hyperspherical VAE and masked autoregressive model target latent-scale heterogeneity and the trade-off between efficient decoding and temporal-correlation learning.
1 INTRODUCTION
- Existing two-stage generators can lose fine-grained temporal information through discrete tokenization, while sequential autoregression risks error accumulation and inefficient generation.The paper motivates continuous-latent modeling with a more efficient autoregressive framework as a response to these limitations.Source: §1 · p. 1
- Gaussian VAE latents show imbalanced directional variance and token-wise scale variation, which increase the difficulty of autoregressive distribution learning.Figure 1 compares relative variance spectra and token norms; S-VAE latents show more balanced directional variances and nearly constant norms.Source: §1 · p. 2, Figure 1 · p. 2
- HALO's von Mises–Fisher S-VAE constrains continuous latents to a fixed-radius hypersphere, alleviating scale heterogeneity and mitigating scale drift during generation.The fixed-radius constraint makes the latents scale-invariant for the autoregressive model.Source: §1 · pp. 2–3
- HALO predicts unknown tokens from known tokens, combining parallel decoding with temporal-correlation learning while reducing inference steps and improving generation stability and efficiency.This design targets the trade-off between sequential decoding's temporal modeling and parallel masked prediction's efficiency.Source: §1 · pp. 2–3, Figure 2 · p. 2
- HALO achieves state-of-the-art generation performance and superior inference efficiency compared with strong baselines.The introduction reports these outcomes across extensive experiments.Source: §1 · p. 3
2 RELATED WORKS
Prior time-series generation ranges from adversarial and interpretable VAE approaches to two-stage discrete-latent models. Autoregressive approaches trade off temporal-correlation learning against parallel decoding, motivating HALO's more general masked autoregressive design.
2.1 DEEP GENERATIVE MODELS FOR TIME SERIES
- TimeGAN combines supervised and adversarial objectives, TimeVAE models trends and seasonalities, and later two-stage methods use discrete codebooks that can limit fine-grained temporal fidelity.SDformer introduced discrete-latent generation to the field, while TimeMAR and MSDformer extended it with multi-scale architectures.Source: §2.1 · p. 3
2.2 AUTOREGRESSIVE MODELING FOR TIME SERIES GENERATION
- GPT-style unidirectional models use next-token prediction and learn temporal dependencies, but their sequential decoding poses stability and efficiency challenges.TimeMAR instead predicts the next scale over multi-scale latent representations.Source: §2.2 · pp. 3–4
- BERT-style bidirectional models enable parallel decoding but can weaken temporal-correlation modeling; HALO predicts unknown tokens from known ones to balance the two.The cited examples include SDformer-m and TimeVQVAE for cloze-style prediction.Source: §2.2 · p. 4
3 PRELIMINARIES
The preliminaries define multivariate time series and the VAE setup, including how Gaussian latent variances can vary across dimensions and samples. They frame generation as learning a distribution that approximates the data distribution while preserving temporal and statistical properties.
3.1 DEFINITION
- A multivariate time series has N variates over T time steps, and HALO uses a channel-independent strategy for modeling.The formulation omits the variate dimension for simplicity.Source: §3.1 · p. 4
- A VAE encodes inputs into a lower-dimensional latent distribution and reconstructs them with a decoder, with training formulated by maximizing the ELBO.The standard objective combines expected reconstruction log-likelihood and a KL-divergence penalty.Source: §3.1 · p. 4
- With diagonal Gaussian posteriors, data-dependent latent scales create variance heterogeneity that can amplify during autoregressive decoding as scale drift.The reparameterization trick expresses z as the encoder mean plus scaled Gaussian noise, enabling gradient backpropagation through z.Source: §3.1 · p. 4
3.2 PROBLEM STATEMENT
- The generation objective is to learn pθ(X) that approximates pdata(X), then sample synthetic time series that preserve real data's temporal dynamics and statistical properties.Source: §3.2 · p. 4
4 METHOD
HALO models time series in continuous latents through a two-stage pipeline: an S-VAE produces fixed-radius hyperspherical representations, then a masked autoregressive model predicts missing tokens from visible context. The design combines reconstruction and frequency-domain training with context-conditioned diffusion to address latent heterogeneity and balance generation efficiency with temporal correlation modeling.
4.1 STRUCTURE OVERVIEW
- HALO first trains an S-VAE to encode time series as fixed-radius hyperspherical latents, then trains a masked autoregressive model to learn their distribution.The second stage combines random masking, masked context modeling, and conditional diffusion modeling.Source: §4.1 · pp. 4–5
4.2 STAGE 1: HYPERSPHERICAL VAE (S-VAE)
- The encoder compresses the series into continuous latents, models them with a von Mises–Fisher posterior, and constrains their norms to a fixed radius.A uniform hyperspherical prior is used; fixing the radial component removes variance and scale heterogeneity and reduces the latent distribution's degrees of freedom.Source: §4.1 · p. 4, §4.2 · pp. 5–6
- The S-VAE reconstructs the input from its latents and adds a frequency-domain ℓ1 spectral loss to encourage preservation of periodic and seasonal components.Its encoder combines convolutional layers for local feature extraction with Transformer blocks for global modeling.Source: §4.2 · pp. 5–6
4.3 STAGE 2: MASK AUTOREGRESSIVE MODELING
- Random masking selects latent prediction targets while preserving their temporal positions, and varying the masked positions and ratios trains prediction from different visible contexts.The masked positions are prediction targets; visible tokens provide context.Source: §4.3 · p. 6
- A Transformer encoder-decoder builds temporal context for masked positions, and a conditional diffusion head predicts each missing continuous token from that context.The diffusion head and context network are jointly optimized while the S-VAE remains frozen.Source: §4.3 · pp. 6–7
5 EXPERIMENTS
Experiments evaluate HALO on in-domain and unseen-domain generation, then test component contributions and inference efficiency. HALO leads overall across 12 datasets and shows robust cross-domain performance.
5.1 EXPERIMENT SETTINGS
- The experiments use 12 datasets across four domains, compare seven selected deep generative baselines, and assess MMD, K-L divergence, and MDD, where lower values indicate better quality.The in-domain comparison reports mean and standard deviation across five random-seed runs for sequence length 168.Source: §5.1 · pp. 7–8, §5.2 · p. 8
5.2 RESULTS
- HALO achieves leading overall in-domain generation performance across 12 datasets.Table 1 reports MMD and K-L for sequence length 168.Source: §5.2 · p. 8, Table 1 · p. 8
- HALO achieves robust cross-domain performance on unseen Stock and Web domains, outperforming unconditional and fine-tuned methods.Figure 4 reports K-L divergence at generation length 168; HALO also maintains an advantage over TimeDP using unseen-domain prompts.Source: §5.2 · p. 8, Figure 4 · p. 9
5.3 ABLATION STUDIES
- Replacing the S-VAE, masked autoregressive backbone, or conditional diffusion head with a standard VAE, GPT-style Transformer, or MLP respectively makes each variant underperform the full model.The authors associate the VAE degradation with latent variance and scale heterogeneity, the AR variant with more sequential decoding steps, and the MLP result with conditional diffusion modeling.Source: §5.3 · p. 9, Table 2 · p. 9
5.4 EFFICIENCY ANALYSIS
- HALO substantially outperforms TimeMAR in inference efficiency for generation sequences of length 168.The authors attribute the advantage primarily to parallel decoding reducing sequential generation steps, and also to avoiding the computation from discrete quantization.Source: §5.4 · p. 9
6 CONCLUSIONS
HALO combines continuous latent representations with masked autoregressive modeling, and experiments across 12 datasets show strong generation quality, improved inference efficiency, and promising unseen-domain generalization. Its inference procedure progressively generates token groups in parallel from context, then decodes the completed latent sequence into a time series.
6 CONCLUSIONS
- Across 12 datasets, HALO shows strong generation quality and improved inference efficiency, with additional evaluations indicating promising generalization to unseen domains.The conclusion presents these results as support for continuous latent modeling as an effective approach to high-quality, efficient time series generation.Source: §6 · p. 9
A TOKEN PROCESSING PATH IN MASKED CONTEXT MODELING
- HALO starts from a fully masked latent sequence, progressively generates token groups in a random order using prior-token context, and decodes the completed sequence into a time series.The conditional diffusion head samples selected tokens in parallel, and completed tokens are projected onto the fixed-radius hypersphere before entering the visible context; unconditional generation uses a learned null condition.Source: A TOKEN PROCESSING PATH IN MASKED… · pp. 13–14
B DATASET DESCRIPTION
The dataset descriptions cover the 12 training datasets across energy, transport, nature, and economic time series, alongside Stock and Web datasets for unseen-domain generation.
Energy datasets
- The energy datasets are hourly electricity consumption, hourly Alabama solar production, and a single wind-power series measured every four seconds.Electricity covers 321 clients from 2012 to 2014; the wind series begins August 1, 2019.Source: B DATASET DESCRIPTION · p. 14
Transport datasets
- The transport datasets comprise freeway-lane occupancy rates, New York City taxi rides, and Melbourne pedestrian counts.They are recorded daily, every 30 minutes, and hourly, respectively.Source: B DATASET DESCRIPTION · p. 14
Nature datasets
- The nature datasets include Beijing and London air-quality measurements, Australian mean temperature, and rainfall extracted from the Temperature dataset.Air-quality data cover six pollutants across monitoring stations and include missing-value imputation; Temperature records span 2015–2017.Source: B DATASET DESCRIPTION · p. 14
Economic datasets
- The economic datasets are NN5 daily ATM withdrawals, FRED-MD monthly macroeconomic indicators, and daily exchange rates among eight currencies.FRED-MD series are differenced and log-transformed; NN5 missing values are imputed using weekday-specific medians.Source: B DATASET DESCRIPTION · p. 14
Unseen-domain datasets
- The unseen-domain datasets are daily Google stock prices and Wikipedia page-view series.The Web dataset contains 145,063 daily series from July 2015 to September 2017, with missing values replaced by zeros.Source: B DATASET DESCRIPTION · p. 15
C BASELINE DETAILS
The baselines span adversarial, variational, diffusion-based, and discrete-token generators. They use varied latent representations and generation strategies, including multi-scale tokenization, temporal objectives, and class- or domain-conditioned generation.
C BASELINE DETAILS
- The comparison covers adversarial, variational, diffusion-based, and discrete-token generation baselines; Table 4 lists their code repositories.TimeVQVAE-C shares TimeVQVAE’s implementation.Source: C BASELINE DETAILS · p. 15, Table 4 · p. 15
- TimeMAR uses multi-scale discrete trend and seasonal tokens with coarse-to-fine Transformer generation, adapting domain labels as prompts.Coarse seasonal representations guide reconstruction of fine-grained seasonal patterns.Source: C BASELINE DETAILS · p. 15
- TimeDP builds domain prompts from shared semantic prototypes and uses them to condition diffusion-based generation.Its prompts are constructed from a small set of examples from the target domain.Source: C BASELINE DETAILS · p. 15
- TimeGAN combines adversarial learning with a supervised temporal objective that encourages dependencies between consecutive latent states.Embedding and recovery networks provide the latent representations used for generation.Source: C BASELINE DETAILS · p. 15
- GT-GAN combines neural ordinary and controlled differential equations with continuous-time flow processes for regularly and irregularly sampled series.Source: C BASELINE DETAILS · p. 15
- TimeVAE uses continuous latents and an interpretable decoder that models level, trend, and seasonal components.Source: C BASELINE DETAILS · p. 15
- TimeVQVAE encodes low- and high-frequency components as discrete tokens and generates them through iterative masked decoding with bidirectional Transformer priors.Source: C BASELINE DETAILS · pp. 15–16
- TimeVQVAE-C conditions the bidirectional token prior on class labels while retaining TimeVQVAE’s time-frequency tokenization.Source: C BASELINE DETAILS · p. 16
D MORE IMPLEMENTATION DETAILS
HALO’s implementation uses a two-stage design: an S-VAE encodes sequences into constrained continuous latent tokens, followed by a masked autoregressive model with a diffusion head.
D MORE IMPLEMENTATION DETAILS
- The S-VAE uses a hybrid convolutional–Transformer encoder–decoder, with two residual downsampling blocks and three Transformer blocks in the encoder, and the reverse order in the decoder.Each Transformer block has hidden dimension 512 and eight attention heads.Source: D MORE IMPLEMENTATION DETAILS · p. 16
- The S-VAE encoder predicts a Power Spherical posterior over 16-dimensional latent tokens constrained to a hypersphere of radius 4, with temporal downsampling factor 4.Source: D MORE IMPLEMENTATION DETAILS · p. 16
- The masked autoregressive model combines random masking, a Transformer context network, and a conditional diffusion head; its minimum masking ratio is 0.5 and maximum sampling steps are 100.The context encoder and decoder each use two Transformer blocks; the diffusion head uses two residual MLP blocks.Source: D MORE IMPLEMENTATION DETAILS · p. 16
E COMPLETE EXPERIMENTAL RESULTS
The supplementary results provide additional metric coverage for in-domain generation at sequence lengths 96, 168, and 336, plus unseen-domain results at length 168.
E COMPLETE EXPERIMENTAL RESULTS
- Table 5 reports in-domain MDD for sequence length 168, supplementing the main-text results.The caption identifies best and second-best results but supplies no numerical values here.Source: Table 5 · p. 16, E COMPLETE EXPERIMENTAL RESULTS · p. 16
- Figure 6 presents MMD and MDD for unseen-domain Stock and Web generation at sequence length 168.Its labels distinguish unconditional generation, fine-tuning, and sample prompting.Source: E COMPLETE EXPERIMENTAL RESULTS · p. 16, Figure 6 · p. 17
- For in-domain generation at length 96, Table 6 reports MMD, K-L divergence, and MDD; Table 7 reports the same metrics at length 336.TimeMAR could not be evaluated at length 336 because of out-of-memory.Source: Table 6 · p. 17, Table 7 · p. 18
F ADDITIONAL EXPERIMENTS
Additional experiments qualitatively assess generated-series fidelity and examine how autoregressive generation rounds relate to quality. The accompanying related-work discussion situates HALO among broader time-series modeling directions.
F ADDITIONAL EXPERIMENTS
- t-SNE visualizations show strong distributional alignment between synthesized and real time series, supporting fidelity in statistical properties and temporal structures.Figure 7 visualizes synthesized series across sequence lengths.Source: Figure 7 · p. 18, F ADDITIONAL EXPERIMENTS · p. 19
- A more fine-grained autoregressive process may yield higher-quality series, potentially by better preserving temporal dependencies; the authors frame round count as a quality-efficiency trade-off.Table 8 reports generation quality for different inference-round counts K at sequence length 168, averaged over five seeds.Source: F ADDITIONAL EXPERIMENTS · p. 19, Table 8 · p. 19
G ADDITIONAL RELATED WORK
- HALO’s related-work framing combines hyperspherical constraints on continuous latents with masked autoregressive modeling to improve generation quality and efficiency.Source: G ADDITIONAL RELATED WORK · p. 19