Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Training

1 Introduction

• SLMs use less memory and enable faster inference and deployment on limited resources, but may underperform larger models on complex or general-purpose generation.The paper motivates task-specific training to improve generation quality without sacrificing these computational advantages.Source: §1 · p. 1
• Power-outage report generation must preserve source information while producing XML organized according to the CIM IEC 61968-3 specification.The reports are part of ODIN, which standardizes outage-information exchange for response, restoration, and interoperability.Source: §1 · p. 1
• Oracle4CS uses Statistical Machine Translation to add a compact Abstract Syntax Tree representation to code-search pipelines, motivating investigation of classical techniques for domain-specific generation.The cited work is Phan and Jannesari [2024].Source: §1 · pp. 1–2
• PO-MRT fine-tunes Qwen2.5-7B-Instruct by sampling candidate reports, scoring sequence-level risk with sentence-level BLEU, and optimizing expected risk through parameter-efficient fine-tuning.The approach trains from complete generated reports rather than relying solely on token-level likelihood.Source: §1 · p. 2

2 Power Outage Report Generation

• ODIN’s Power Outage Tracker converts heterogeneous outage information—including incomplete or non-standard XML and JSON records and natural-language descriptions—into a unified CIM IEC 61968-3 representation.Generated reports must preserve source information while satisfying the target XML standard’s structural requirements.Source: §2 · p. 2

3 Motivation Example

• The example transforms a non-standard JSON outage report into a standardized XML report whose content and hierarchy conform to ODIN’s CIM IEC 61968-3 representation.The input may contain outage identifiers, customer counts, location, and report and restoration times.Source: §3 · p. 2
• Outage-report generation must preserve semantic content and produce XML with appropriate CIM tags, nesting, namespaces, and matching opening and closing elements.The passage illustrates that a correct customer count under the wrong element, or correct structure with hallucinated outage facts, fails one of these requirements.Source: §3 · p. 3
• Token-level likelihood can assign similar scores to reports with very different usefulness because it does not distinguish complete reports by both information accuracy and structural validity.This motivates evaluating multiple complete candidates by task-specific generation quality and optimizing toward lower-risk outputs.Source: §3 · p. 3

4.1 Why Minimum Risk Training

• MRT generates candidate sequences, evaluates each with a task-specific loss, and minimizes expected risk over the candidate set.Sampling approximates the output space, and the evaluation function need not be differentiable, allowing sequence-level metrics such as BLEU to be used directly.Source: §4.1 · p. 3

4.2 MRT Fine-tuning for XML Report Generation

• The adaptation loads Qwen2.5-7B-Instruct with 4-bit quantization and uses LoRA adapters for parameter-efficient fine-tuning.Each training example pairs a non-standard outage report prompt with a standardized XML reference.Source: §4.2 · p. 3
• For each prompt, the model samples K candidate XML reports, scores each against the reference with sentence-level BLEU, and defines risk as one minus BLEU.The objective minimizes expected risk across candidates, encouraging higher probability for outputs closer to the standardized reference.Source: §4.2 · p. 5
• MRT improves Qwen2.5-7B-Instruct fine-tuning by optimizing the quality of complete generated XML reports rather than relying only on token-level supervision.The paper adapts minimum-risk optimization from neural machine translation to fine-tuning a pretrained causal language model.Source: §4.2 · p. 5

5.1 Experimental Setup

• The experiments use 1,000 paired non-standard outage reports and standardized CIM IEC 61968-3 XML reports to fine-tune and evaluate Qwen2.5-7B-Instruct with and without MRT.Source: §5.1 · p. 5
• Overall Accuracy averages Text Accuracy, which measures textual similarity, and Tag Accuracy, which measures XML tag-sequence similarity.Source: §5.1 · p. 5

5.2 Results

• 68.95% Overall Accuracy versus 16.20% for baseline Qwen2.5-7B-Instruct (a 52.75-point gain), with Tag Accuracy rising from 3.56% to 87.17% and Text Accuracy from 28.85% to 50.73%.Source: §5.2 · p. 5

• MRT has been combined with maximum-likelihood training for referring-expression generation and applied to document-level biomedical translation and neural-metric analysis Panagiaris et al. [2020], Saunders and Byrne [2020], Yan et al. [2023].Source: §6 · pp. 5–6
• Recent language-model work explores sampling-based reinforcement learning, MBR decoding and preferences, multi-prompt MBR, and MBR-BoN for alignment Wang et al. [2023], Suzgun et al. [2022], Yang et al. [2024], Heineman et al. [2024], Jinnai et al. [2025].Source: §6 · p. 6
• Domain-specific data-science directions include privacy-preserving power forecasting, flexible Earth-observation models, graph-based electricity time-series geolocation, and paired Earth-science data Abebe et al. [2027], Dias et al. [2026], Tran et al. [2026], Potnis et al. [2026].Source: §6 · p. 6

• MRT improves a small language model's structured power outage report generation while requiring only modest hardware through parameter-efficient fine-tuning.Source: §7 · p. 6
• The XML-generation accuracy improvement suggests MRT may be practical for other data-science applications with structured or specialized outputs.Source: §7 · p. 6
• Future work will investigate domain-specific sequence-level risks based on structure, semantics, spatial relationships, or other task-specific constraints.Source: §7 · p. 6