Just for FUNS: LLM-Guided Spatio-Temporal Graph Node Generation for Forecasting Unobserved Node States

1 Introduction

• FUNS requires zero-shot forecasting for nodes with no historical observations, where autoregressive models lack input, proximity can miss functional differences, and point estimates may smooth uncertain trajectories.The introduction identifies absent temporal signals, semantic blindness, spatial heterogeneity, and the need for diverse, physically plausible generation.Source: §1 · pp. 1–2
• GenST recasts FUNS as semantic-guided spatio-temporal graph generation, using textual descriptions of functional zones and road hierarchies as conditions for missing histories.The proposed Semantic Alignment Module aligns urban-scenario semantics with the forecasting space.Source: §1 · p. 2
• A Spatio-Temporal VAE compresses dynamics into a latent space, and a Generative Transformer reconstructs unobserved-node trajectories from noise using semantic and neighborhood conditions.The introduction describes the framework as a multi-modal conditional generation pipeline.Source: §1 · p. 2
• The stated contributions include LoRA-fine-tuned LLM semantics and experiments across six traffic benchmarks and two non-traffic datasets for zero-shot forecasting.The paper also reports robust performance under sparse observations.Source: §1 · p. 2

2 Preliminaries

• The road network is represented as a graph whose nodes are road segments and whose edges and adjacency matrix encode physical connections, proximity, and spatial correlation.The node set is partitioned into observed sensor-equipped segments and unobserved segments without historical data during training or testing.Source: §2 · p. 2
• FUNS forecasting uses observed-node histories together with the global graph, geographic coordinates, and textual semantics to predict future states over the next h time steps.Textual semantics describe node attributes such as functional zone or road category; coordinates provide longitude and latitude.Source: §2 · p. 3
• Unlike traditional imputation of random, short-term sensor failures, FUNS is inductive zero-shot forecasting for target nodes entirely lacking historical records.The task leverages spatial correlations and semantic priors to synthesize future trajectories.Source: §2 · p. 3

3 Methodology

• GenST combines a latent ST-VAE with GenT to learn denoising and reconstruction from multimodal priors, then synthesize future states across the network.Its priors include LoRA-fine-tuned semantics, coordinates, and neighborhood topology.Source: §3 · p. 3

3.1.1 Semantic Alignment with LLMs.

• For each node, a LoRA-adapted LLM converts its text description into a continuous embedding of functional semantics for forecasting guidance.The adaptation freezes pretrained weights and adds low-rank trainable matrices to attention layers.Source: §3.1.1 · pp. 3–4

3.1.2 Spatial and Structural Encoding.

• Fourier coordinate features encode absolute location, while an inductive GNN aggregates neighbor histories and attributes, using a learnable mask for unobserved neighbors.This provides spatial and structural features under partial observability.Source: §3.1.2 · p. 4

3.1.3 Context Fusion.

• A Fusion MLP combines semantic, position, and structural features into a unified condition used as GenT's cross-attention Key and Value.The fused condition guides latent denoising.Source: §3.1.3 · p. 4

3.2 Spatio-Temporal VAE

• The ST-VAE compresses spatio-temporal dynamics into a low-dimensional latent target, reconstructs observed history, and trains with reconstruction and KL-regularization terms.The decoder maps the latent vector back to spatio-temporal dimensions; the regularization term pulls the posterior toward a standard normal distribution.Source: §3.2 · p. 4, §3.2.1 · p. 4, §3.2.2 · p. 4

3.3.1 Latent Diffusion Process.

• GenT models latent traffic states by adding Gaussian noise to VAE-derived future latents and learning a guided reverse process from noise.The VAE encoder acts as a temporal compressor, while forecasting is delegated to GenT.Source: §3.3.1 · pp. 4–5

3.3.2 Structure-Aware Generative Backbone.

• GenT alternates self-attention for global trajectory consistency with cross-attention that retrieves semantic and spatial priors; its weighted denoising loss predicts added latent noise.The objective restores the latent manifold under multimodal conditions.Source: §3.3.2 · p. 5, §3.3.3 · p. 5

3.4 Zero-Shot Inference

• At inference, GenST denoises global-network noise using multimodal conditions and a frozen VAE decoder, generating predictions for unobserved nodes without fine-tuning.Training uses only the observed subgraph; the model omits node-specific ID embeddings and is designed to generalize across graph scales while maintaining physical consistency across the network topology.Source: §3.4 · p. 5

4.1.1 Datasets Description.

• Evaluation uses six traffic and two non-traffic datasets, 16 baselines, and MAE, RMSE, and MAPE, with lower values indicating better accuracy.The baselines span statistical, end-to-end deep-learning, pre-trained, transfer-learning, and graph-node-generation categories.Source: §4.1.1 · p. 5, §4.1.2 · p. 5, §4.1.3 · pp. 5–6

4.1.4 Implementation Details.

• Experiments use chronological 7:1:2 splits, 12-step history and horizons, LLaMA3.1-8B by default, and results averaged across three random seeds.Training uses Adam for up to 100 epochs with early stopping; data are Z-Score normalized.Source: §4.1.4 · p. 6

4.2 Main Experimental Analysis

• GenST reduces Unobserved MAE versus runner-up ST-LLM+ by 56.0% on METR-LA, 33.4% on PeMS-Bay, and 67.1% on PeMS03, with Table 1 reporting the main comparison.It slightly trails ST-LLM+ on PeMS07 global forecasting but achieves the best performance on the corresponding unobserved task.Source: §4.2 · p. 6, Table 1 · p. 6

4.3 Impact of Unobserved Node Ratios

• As unobserved-node ratios rise from 10% to 60%, Unobserved MAE stays nearly flat on PeMS-Bay and PeMS04, while global error generally increases.On PeMS04, Unobserved MAE rises only from approximately 25 to 28; METR-LA fluctuates non-monotonically due to sample bias.Source: §4.3 · p. 7, Figure 4 · p. 7

4.4 Study of Different Backend LLMs

• Larger backend LLMs improve accuracy with diminishing returns, while Qwen3-32B trains 3.3× longer than 8B and reaches 129.6 GB peak VRAM.The comparison describes Llama3.1-8B as near-SOTA with low memory overhead and fast training.Source: §4.4 · p. 7, Figure 5 · p. 7

4.5 Ablation Study

• Removing Semantic Alignment causes the largest ablation degradation; on PeMS03, Unobserved MAE increases by 9.33 and MAPE by 17.54%.Removing ST-VAE causes the second-largest degradation; Spatial Context has the smallest contribution, with +0.11 MAE on PeMS03 within seed variance.Source: §4.5 · p. 7, Table 2 · p. 8

4.6 Efficiency Analysis

• DPM-Solver++ cuts PeMS04 sampling from 1000 to 20 steps and latency from 3.17s to 0.78s, a 4.1× speedup with a 12.6% Unobserved MAE cost.The efficiency analysis reports 36.98M trainable parameters regardless of node scale and 34.09 GB training VRAM on PeMS04 for the default setting.Source: §4.6 · p. 7, Metrics MAE RMSE MAPE MAE RMSE MAPE · p. 8, Table 3 · p. 8

4.7 Cross-Domain Generalization

• GenST achieves the best Unobserved performance on KnowAir and Electricity, and its Global–Unobserved gap remains small after domain shift.On Electricity, which lacks native coordinates and POI-style semantics, unobserved nodes receive generic semantic placeholders and structural context from adjacency.Source: §4.7 · p. 8, Table 4 · p. 8

5.1 Spatio-temporal Forecasting

• Pre-training and transfer-learning methods seek generalization, but the cited approaches still falter when sensor data are entirely missing or depend on historical sequences and node identity embeddings.The passage contrasts universal-pattern pre-training and adaptation with the cold-start setting of FUNS.Source: §5.1 · p. 8
• Many forecasting models remain transductive, relying on historical sequences or learned node embeddings that cannot supply patterns for unseen nodes without history.The related-work discussion describes a lack of systematic, inductive, semantic-driven FUNS methods validated against broad baselines.Source: §5.1 · p. 8

5.2 Graph-based Generation

• Most spatio-temporal generative work targets imputation, while RoadDiff addresses lane-node generation but depends on traffic theory and engineering features that limit transferability.The passage identifies a need for a general semantic-guided framework for inductive zero-shot inference.Source: §5.2 · p. 8

• GenST combines LLM-extracted semantics with ST-VAE and latent diffusion for fully inductive prediction at nodes without historical observations.The framework recasts spatio-temporal forecasting as conditional graph-node generation.Source: §6 · pp. 8–9
• Across six traffic benchmarks and two non-traffic datasets, GenST consistently outperforms state-of-the-art baselines in zero-shot unobserved-node forecasting.The authors also report robustness under sparse observations and accelerated sampling.Source: §6 · p. 9
• GenST has relatively high offline training costs from its LLM backbone and iterative diffusion sampling.Future work aims to improve efficiency through knowledge distillation and model quantization.Source: §6 · p. 9

A Robustness to Degraded Semantics

• Full semantics achieves the best performance, while errors generally increase as semantic degradation rises.The evaluation uses generic templates and shuffled descriptions at degradation ratios of 20%, 50%, and 80%, with the unobserved-node ratio fixed at 20%.Source: A Robustness to Degraded Semantics · p. 10, Table 5 · p. 10
• On METR-LA, MAE rises from 5.57 with full semantics to 7.24 with 80% generic templates and 7.72 with 80% shuffled descriptions.On PeMS04, the corresponding MAE rises from 27.40 to 35.14 and 36.91.Source: A Robustness to Degraded Semantics · p. 10
• Shuffled descriptions usually cause a larger performance drop than generic templates, suggesting incorrect semantic matching is more harmful than missing fine-grained detail.Coordinates, structural context, and the generative latent prior can compensate when semantic information is incomplete or noisy.Source: A Robustness to Degraded Semantics · p. 10

B Diffusion Step 𝐾Sensitivity Analysis

• GenST achieves the lowest mean MAE for both global and unobserved tasks at K=1000 on all six datasets.Increasing K from 200 to 1000 reduces MAE across benchmarks; further increasing it to 1500 or 2000 brings no additional gain, and some datasets rebound.Source: B Diffusion Step 𝐾Sensitivity Analysis · pp. 10–11
• At K=2000, PeMS04 Unobserved MAE rises from 27.40 at K=1000 to 29.86.PeMS08 Unobserved MAE falls from 23.70 at K=200 to 19.76 at K=1000, a relative reduction of 16.6%; the authors note that more iterations also increase computational overhead.Source: B Diffusion Step 𝐾Sensitivity Analysis · p. 11

C Stability under Different Random Seeds

• In a five-seed study with the unobserved-node ratio fixed at 20%, Unobserved MAE is 5.56 ± 0.11 on METR-LA, 21.30 ± 0.28 on PeMS03, 27.51 ± 0.34 on PeMS04, and 19.75 ± 0.25 on PeMS08.The study jointly varies the node split, training process, and stochastic inference factors.Source: C Stability under Different Random Seeds · p. 11, Table 6 · p. 11

D Horizon Analysis

• Across datasets, prediction errors rise gradually as the horizon increases, but GenST maintains low MAE without collapse at 18 steps, a 90-minute prediction.The evaluation varies horizon from 3 to 18 steps while holding the historical window at 12 steps.Source: D Horizon Analysis · p. 11, Figure 7 · p. 11
• On PeMS07, Unobserved MAE remains around 35 as the horizon extends from 12 to 18 steps.The error-growth curves for unobserved regions are nearly parallel to those for the global task.Source: D Horizon Analysis · p. 11
• Flow datasets have a slightly steeper error-growth slope than speed datasets, which the authors attribute to greater heterogeneity from sporadic events and signal control.The supplied robustness figure caption describes evaluation under varying unobserved-node ratios but provides no outcome.Source: D Horizon Analysis · p. 11, Figure 8 · p. 11

E.1 Additional Unobserved Ratios Analysis

• Across PeMS07 and PeMS08, GenST remains stable from 10% to 60% unobserved nodes, with flat linear Unobserved MAE growth on PeMS07.The authors attribute the reliable priors under extreme sparsity to multi-modal semantic alignment.Source: §E.1 · p. 11
• On PeMS08, errors at 30%–40% missingness slightly exceed those at 50%, consistent with the authors’ sample-bias hypothesis.They suggest lower-ratio random masking may select harder irregular nodes, while higher ratios may better approximate the global distribution.Source: §E.1 · p. 12

E.2 Additional Study of Backend LLMs

• Across PeMS-Bay, PeMS07, and PeMS08, semantic reasoning capability positively correlates with zero-shot accuracy; Qwen3-32B significantly outperforms Qwen2.5-1.5B on PeMS07.The expanded comparison also highlights computational bottlenecks when processing large graphs with massive backends.Source: §E.2 · p. 12, Figure 9 · p. 12

E.3 Ablation Study on Large-Scale Datasets

• Removing Semantic Alignment raises PeMS07 Unobserved MAE by 11.29 and MAPE by 17.74%, while on PeMS08 it increases MAE by 13.96 points.Table 7 reports that the Semantic Alignment Module is the most critical component on both large datasets.Source: §E.3 · p. 12, Table 7 · p. 12
• On PeMS08, removing ST-VAE raises Unobserved MAE by 5.57, versus 4.53 for Cross-Attention and 3.16 for Spatial Context; on PeMS07, Cross-Attention ranks second at +2.09.On PeMS07, removing ST-VAE (+1.00) or Spatial Context (+0.82) causes smaller degradations comparable to seed variance; the authors link ST-VAE to noise suppression on PeMS08.Source: §E.3 · p. 12, Table 7 · p. 12

F Dataset Statistics

• Evaluation uses six traffic datasets and two non-traffic datasets: METR-LA, PeMS-Bay, PeMS03/04/07/08, KnowAir, and Electricity.Traffic data are at 5-minute resolution, while KnowAir and Electricity are hourly; Electricity values are normalized and the other datasets use native units.Source: F Dataset Statistics · pp. 12–13, Table 8 · p. 13

F.1 Coordinate Acquisition.

• Coordinates and adjacency come from established repositories, with traffic coordinates cross-validated against the Caltrans station inventory; Electricity has no native coordinates.Unobserved nodes are randomly selected once per run and held constant across training, validation, and test partitions for inductive zero-shot evaluation.Source: §F.1 · p. 13

F.2 Semantic Description Generation.

• Traffic-node descriptions combine OpenStreetMap POIs within 500 m with coordinates and road attributes, then use DeepSeek to synthesize functional-context text.For KnowAir and Electricity, the structured inputs are adapted to available metadata or observed-node load statistics; unobserved nodes receive generic placeholders, and Electricity uses a shared <UNK_LOC> embedding.Source: §F.2 · p. 13

G.1 Adaptation Strategies and Fairness Check

• Each unseen node receives a pseudo-history formed by averaging first-order observed neighbors’ sequences; isolated nodes use the global average of observed sensors.This lets autoregressive baselines operate as neighborhood-mean regressors.Source: §G.1 · p. 13
• On METR-LA, pseudo-history versus native input adaptation changes Unobserved MAE by at most 1.3% for STEP and ST-LLM+.The authors conclude pseudo-history does not materially weaken these pretrained baselines; they identify missing node-specific histories as the main limitation under FUNS.Source: §G.1 · p. 14

G.2 Baseline Implementation Details

• Five baseline families cover statistical learning, end-to-end deep learning, pretraining and transfer, and graph node generation, with adaptations for missing histories and RoadDiff’s constraints.Statistical baselines include K-NN, Kriging, and HA; deep models use pseudo-history, pretrained models receive neighborhood context or native missing-input strategies, and RoadDiff drops physical-constraint losses while retaining its autoencoder and diffusion modules.Source: §G.2 · p. 14

H Accuracy Metric Formulas

• MAE, RMSE, and MAPE assess global and unobserved-node performance across the forecasting horizon; RMSE emphasizes large deviations, while MAPE measures relative error.The passages describe MAE as average linear residual magnitude, RMSE as applying a quadratic penalty to errors, and MAPE as normalizing deviations against ground truth.Source: H Accuracy Metric Formulas · p. 14