Source-linked AI summary
Agent-Based ML-LLM Fusion with Self-Optimizing Prompts for Plateau Weather Alerts
Shuai Yan, Yang Xu, Shan He
TL;DR
Tourism meteorological services require better contextualization, generalization, and scenario adaptation for complex highland conditions. SmartWeatherAgent integrates intent recognition, LightGBM hazard prediction, and reasoning-enhanced generation with self-optimizing prompts, achieving strong low-latency hazard recognition and progressively refined warning quality. The authors conclude that the system supports structured, scientifically grounded warnings, while noting geographic and evolutionary scope limitations.
Problem
Highland tourism weather services face insufficient contextualization, weak generalization, and poor scenario adaptation under dynamic and heterogeneous conditions.
Method
SmartWeatherAgent combines intent recognition, plateau-specific LightGBM hazard prediction, and LLM-based warning generation in a feedback-driven, self-adaptive architecture.
Results
LightGBM achieves F1-Macro = 0.61 with 1.60 ms inference latency while outperforming baselines in composite warning capability and key hazard recognition.
Takeaways & Limitations
The modular system generates timely warnings grounded in structured hazard features and supports professional, context-aware meteorological services.
Takeaways & Limitations
Training and validation data are limited to Lhasa, restricting geographic generalization, while prompt evolution remains constrained by predefined dimensions and a staged framework.
Abstract
from arXiv · showhide
To address insufficient contextualization, weak generalization, and poor scenario adaptation in tourism meteorological services, we propose SmartWeatherAgent--a unified three-stage architecture integrating intent recognition, hazard prediction, and reasoning-enhanced generation. The system fuses rule-based methods with large language models to parse queries at multiple granularities and employs a LightGBM model enriched with highland-specific features (e.g., wind speed abruptness rate), achieving an F1-Macro score of 0.605 with 1.60 ms latency on high-wind, precipitation, and low-temperature events. A 12-round micro-step prompt self-optimization loop boosts the composite warning quality score S_final from 4.2 (B01) to 8.9 (B12, +112%). Key improvements include a sharp rise in B08 from data source citation (6.5 -> 8.5), sustained high performance in B10 via physical mechanism explanation, and a peak scientific rigor score of 9.2 in B12 through explicit uncertainty statements. The system autonomously generates structured warnings that integrate causal mechanisms, spatiotemporal evolution, quantitative evidence, regulatory references, and confidence statements--enhancing professional depth, logical rigor, and scientific soundness, and advancing meteorological services toward proactive perception, explainable decision-making, and intelligent agency.
I. INTRODUCTION
SmartWeatherAgent addresses highland tourism meteorology’s dynamic, heterogeneous, and scenario-dependent conditions with a self-adaptive intelligent weather-service framework. It combines fine-grained intent analysis with hazard nowcasting and context-aware warning generation.
- Highland tourism meteorology is highly dynamic, spatially heterogeneous, and scenario-dependent, challenging existing service accuracy and timeliness.
- Existing static rule-based systems and generic large language models show high latency, weak context awareness, poor scenario adaptation, and no self-evolution mechanism.
- SmartWeatherAgent embeds prompt self-adaptation into an intelligent meteorological agent to address these service limitations.
- The framework supports fine-grained query-intent decomposition, short-range highland extreme-weather nowcasting, and context-aware warning generation with prompt self-evolution.
A. Intent Recognition Module
The intent-recognition module combines regular-expression filtering with Qwen3 to classify user queries across six intent categories while resolving contextual ambiguities.
- Six intent categories, including simple inquiry, hazard alert, and family travel, are identified through combined rule-based and large-language-model classification.
- Regular expressions perform initial filtering, while Qwen3 resolves contextual ambiguities to improve robustness.
B. Hazard Prediction Module
The hazard-prediction module uses LightGBM with plateau-specific, temporal, threshold, and extremeness features to support real-time prediction of high-impact weather events. These features also provide structured grounding for warning generation.
- The model targets strong winds, precipitation, and low temperatures using a short-term LightGBM nowcasting approach enhanced with plateau-specific features.
- Wind-related features include absolute wind-speed change, gust ratio, and temporal encoding, while precipitation uses a burst indicator.
- Additional inputs include rolling statistics, threshold-based binary features, and quantile-based extremeness markers.
- The resulting multidimensional representation supports real-time hazard prediction and structured grounding for the generation module.
C. Prompt Self-Adaptation Module
The prompt self-adaptation module uses a generation–evaluation–optimization loop to refine warning messages over 12 micro-step rounds. Its composite score weights semantic, logical, and scientific quality.
- A closed-loop generation–evaluation–optimization pipeline drives 12 rounds of micro-step prompt refinement for self-adaptation.
- S_final = 0.35 · S_semantic + 0.30 · S_logical + 0.35 · S_scientific weights semantic and scientific quality above logical quality.
- Semantic quality progresses from phenomenon description and single-cause attribution toward coupled mechanisms, regional differentiation, and physically linked defenses.
- Logical quality evaluates the chain from meteorological trigger through temporal evolution and impact propagation to targeted mitigation advice.
- Scientific quality sums quantitative clarity, data traceability, regulatory-citation accuracy, uncertainty completeness, and terminological rigor, while scientific inaccuracies cap the score at 3.
A. Model Selection and Hyperparameter Optimization
The study evaluates LightGBM against four baseline models using Lhasa hourly meteorological data, Bayesian hyperparameter optimization, and a composite hazard-recognition score. LightGBM achieves the highest composite score and combines strong class-balanced recognition with millisecond-level inference latency despite severe class imbalance.
- Dataset and feature engineering: Lhasa hourly observations are chronologically split 7:3, with normal weather dominating the training data and strong wind and precipitation forming minority classes.The dataset contains 12,168 records from January 2024 to May 2025, while another reported distribution gives 95.0% normal weather, 3.5% low-temperature, 1.2% strong-wind, and 0.3% precipitation events.
- Dataset and feature engineering: The models use rolling statistics, wind-change and gust-ratio features, temperature-range indicators, cyclical time encodings, and extreme-event markers to represent rapid, nonlinear hazard dynamics.The final engineered feature vector has 38 dimensions.
- Hyperparameter optimization: Bayesian optimization tunes each model with fivefold time-series cross-validation using F1-Macro as the objective, avoiding temporal leakage from random splitting.The compared models are LightGBM, Random Forest, Gradient Boosting, XGBoost, and CatBoost.
- Evaluation design: The composite score weights F1-Macro at 0.3, strong-wind and precipitation F1 at 0.2 each, and low-temperature F1 at 0.3.The weighting scheme assigns greater weight to low-temperature events because they pose significant hazards, while sparse strong-wind and precipitation events receive equal weights.
- Experimental results: LightGBM achieves the highest composite score, S = 0.55, while attaining F1 scores of 0.50 for precipitation, 0.77 for low temperature, and 0.17 for strong wind.The precipitation score has limited stability because of sparse samples, and Gradient Boosting reaches the best strong-wind value at 0.18.
- Experimental results: LightGBM delivers 1.60 ms inference latency and F1-Macro = 0.61 while handling datasets dominated by normal weather and sporadic hazardous events.The reported overall accuracy is 97%, and inference latency remains within 2 ms.
B. Self-Optimization Experiment of Meteorological Warning Prompts via Micro-Step Iteration
The micro-step experiment uses a generation–evaluation–optimization loop to refine meteorological warning prompts across staged semantic, logical, and scientific objectives. Across 12 rounds, prompt refinement substantially improves warning quality, with gains linked to data traceability, mechanism explanation, temporal reasoning, and uncertainty statements.
- The initial unstructured prompt omitted warning levels, quantified metrics, protective actions, and regulatory justification, deviating from operational norms.
- The adaptive design uses a three-layer loop in which generation produces warnings, evaluation scores semantic depth, logical coherence, and scientific rigor, and optimization refines prompts in stages.
- Semantic depth rose from 2.0 (B01) to 8.5 (B12), with a +1.5 jump at B10 after adding physical-mechanism explanations.
- Scientific rigor increased from 3.0 to 9.2, including a +2.2 gain at B08 after data-source citation and a B12 peak after uncertainty statements.
- Sfinal first exceeded 8.0 at B08, rising from 6.5 to 8.5 as enhanced data traceability was introduced.
- Sfinal improved from 4.2 to 8.9 (+112%), while late-stage optimization stabilized scores at 8.0–8.9 and B12 balanced semantic, logical, and scientific performance.
IV. DISCUSSION AND LIMITATIONS
The paper identifies geographic, evolutionary, and operational-data limitations that constrain SmartWeatherAgent’s generalization, sustained adaptability, and field validation.
- Training and validation data are limited to Lhasa, restricting geographic generalization to other highland cities.
- Prompt evolution remains constrained by predefined evaluation dimensions and a staged framework, limiting sustained adaptability in dynamic, complex scenarios.
- The system uses historical observations without real-time operational meteorological streams, so robustness under outages, sensor noise, and rare events requires field validation.
V. CONCLUSION
SmartWeatherAgent is presented as a modular, self-evolving framework for highland tourism nowcasting that combines intent recognition, hazard prediction, and LLM-based warning generation. Its design targets reliable, timely, scenario-adaptive services while identifying future extensions for broader operational use.
- SmartWeatherAgent integrates intent recognition, a lightweight high-impact weather predictor, and LLM-based generation for short-range highland-tourism nowcasting.
- Its three-stage feedback-driven architecture iteratively refines warning messages to improve professionalism, structural integrity, and alignment with meteorological standards.
- The highland-optimized predictor detects key hazards while remaining sensitive to sparse precipitation, supporting reliability and timeliness in complex terrain.
- The modular, self-evolving, scenario-adaptive design is described as applicable beyond tourism to public-safety and emergency-response contexts requiring real-time awareness.
- Future work includes real-time meteorological data, multimodal interaction, and a meta-prompt-driven self-reflective optimizer to improve generalization, interaction naturalness, and output credibility.