PaperlayerView exact revision on arXiv

Source-linked AI summary

Tokamak-SI: Interpreting Tokamak Discharges with Labeled Catalogs

Nathaniel Chen, Alvin Garcia, Ricardo Shousha, Jalal Butt, Azarakhsh Jalalvand, Egemen Kolemen

arXiv:2610.32264v1physics.plasm-phcs.AI

TL;DR

Tokamak research often relies on task-specific labels, leaving datasets and analysis fragmented. Tokamak-SI combines established labeling methods into a time-aligned dataset across seven phenomena. Its models match or exceed earlier detectors on several supplied task results, though some categories remain under review.

  • Problem

    Tokamak datasets are often specialized to one purpose, and separate event labels and differing labeling methods make analysis across disciplines and algorithm evaluation difficult.

    Source: §1 · p. 1

  • Method

    Tokamak-SI combines established labeling methods into a shared, time-aligned dataset with categorical labels across seven phenomena and automated labels guiding expert review.

    Source: §1 · p. 1

  • Results

    Across tasks, ae-ours leads AE metrics (F1 0.944 against 0.774 and 0.715), confine-ours reaches macro-F1 0.909, and elm-ours reaches F1 0.845 against elm-elmo’s 0.842.

    Source: §5.1 · p. 5, §5.2 · p. 5, §5.3 · p. 5

  • Takeaways & Limitations

    The dataset makes archives of thousands of shots searchable by physics and provides labeled targets for tokamak foundation models and language-model agents.

    Source: §1 · p. 1

    AE labels use only present and absent, lie only on AE shots, and leave false alarms on AE-free shots and expert agreement unmeasured; several other categories remain under full review.

    Source: §6 · p. 5

Abstract

from arXiv · show

Finding which phenomena occur in a tokamak discharge, and when, is often a manual task. Previous efforts at labeling events have been task-specific, each with its own corpus of data. We introduce Tokamak-SI, which combines the labels and labeling methods of many earlier studies into a single scalable, time-aligned, and categorical dataset covering observations from confinement, detachment, alfven eigenmodes (AE), edge localized modes (ELM), sawtooth, tearing modes (TM), and resistive wall modes in DIII-D tokamak. Models based on this, which can extend the labels to new shots, match or beat earlier detectors on its labels, including for AE, ELM, and TM detection. Tokamak-SI is intended to enable task-specific benchmarking, physics-searchable archives, and direct identification of key events during an experiment.

1. Introduction

Tokamak discharge labels are fragmented across task-specific datasets, making cross-disciplinary analysis, event search, and algorithm evaluation difficult. Tokamak-SI combines established labeling methods into a time-aligned dataset and framework for seven phenomena, with benchmarks and diagnostic views.

1. Introduction

  • Task-specific fusion databases and differing diagnostics or heuristics can fragment analysis and produce different ground truths for the same shot.The resulting disjointed analysis makes event search and algorithm evaluation difficult.Source: §1 · p. 1
  • The dataset combines established labeling methods into a large, time-aligned corpus, with automated labels guiding expert review.It is intended to make archives of thousands of shots searchable by physics and provide labeled targets for foundation models and language-model agents.Source: §1 · p. 1
  • Tokamak-SI unifies labels for confinement, detachment, AEs, ELMs, sawtooth, TMs, and RWMs, with labels viewable on their diagnostics.TokEye assigns labels to coherent and transient modes in spectrograms.Source: §1 · p. 1
  • The framework provides aligned, categorical labels on 4,872 discharges, with 500 shots set aside for expert review across phenomena.Source: §1 · p. 1
  • Tokamak-SI benchmarks trivial baselines, earlier detectors, and new models against consistent labels and original labels where available.Known activity is also displayed directly on diagnostic views for automatic shot review.Source: §1 · pp. 1–2

2. Related Work

Related work spans categorical datasets, methods for recording uncertain or absent labels, and approaches to improving labels over time. Fusion databases and detectors have generally focused on particular entries, targets, or phenomena.

Large labeled datasets and label quality

  • Large image datasets show that common formats for training and evaluation can use categorical labels, including ImageNet (Deng et al., 2009) and COCO (Lin et al., 2015).Source: §2 · p. 2
  • LVIS (Gupta et al., 2019) and Open Images (Kuznetsova et al., 2020) record verified negatives, while CheXpert (Irvin et al., 2019), PTB-XL (Wagner et al., 2020), and GWTC (LIGO Scientific Collaboration et al., 2026) encode uncertainty or unobserved time.Source: §2 · p. 2
  • Repeated labeling passes can improve localization and align datasets, while temporal event metrics support forecasting and time-series classification tasks.AudioSet’s temporally strong relabeling of weak clip labels demonstrates recursive label improvement.Source: §2 · p. 2

Fusion data and event labels

  • Fusion databases and benchmarks often target specific entries or continuous values, while existing detectors address individual phenomena; this work re-scores most on DIII-D data.Prior work also proposes spectrogram segmentation and human-in-the-loop or software-accelerated workflows.Source: §2 · p. 2

3. The Tokamak-SI Dataset

Tokamak-SI brings seven phenomena into a shared DIII-D dataset, aligning categorical labels to a common time axis and recording their provenance. The label sets adapt earlier data or rules to cover time bins, event instances, and uncertain or unobserved observations.

3. The Tokamak-SI Dataset

  • Tokamak-SI covers AEs, ELMs, confinement, tearing modes, sawtooth, RWMs, and divertor detachment.Figure 1 shows labeled diagnostics and event states, while Figure 2 presents F1 on original and Tokamak-SI labels.Source: §3 · p. 2, Figure 1 · p. 3, Figure 2 · p. 4

3.1. Shots, tiers and alignment

  • The population comprises 4,872 DIII-D shots from 2021–2025 passing five rules; 500 review shots are split into 400 train, 50 validation, and 50 blind test shots before labeling.Source: §3.1 · p. 2
  • Labels record provenance in gold, silver, or bronze tiers, and standardized sets share one schema and time axis for bin-by-bin comparison.Bronze population labels are released only when the model meets stated precision and recall and paired-interval criteria on gold.Source: §3.1 · p. 2

3.2. Label sets

  • Each label set can mark uncertain or unobserved observations and distinguish a time-spanning label from an individual event instance.Source: §3.2 · p. 2
  • AE labels extend sparse expert annotations across full discharges at 10 ms resolution; ELM labels distinguish single events from unresolved ELMing periods, and confinement bins mark L-mode, H-mode, QH, or WPQH.The ELM set includes expert review of 119 cohort shots, and the AE labels cover 195 shots.Source: §3.2 · pp. 2–3
  • Tearing-mode labels attempt to distinguish 2/1 from 3/2 modes using poloidal phase, while sawtooth labels use ECE inversions and a crash-train criterion.The initial sawtooth set uses at least three crashes with inversions 20–250 ms apart; original RWM labels analyze onsets in 2014 and 2018.Source: §3.2 · p. 3
  • Detachment labels combine prior methods into time-bin categories of attached, detached, or MARFE.Source: §3.2 · p. 4

4. Benchmark Protocol and Baselines

The benchmark protocol compares trivial baselines, simpler signal-processing methods, prior learned detectors, and Tokamak-SI models. Most learned scores use cross-validation, but AE results are optimistic and all scores remain exploratory because gold labels are not yet scored across every category.

4. Benchmark Protocol and Baselines

  • Gold labels are not scored across all categories yet, so every reported score is exploratory and no model output is released as bronze.Source: §4 · p. 4
  • Comparisons include an always-present baseline and signal-processing methods such as an ELM peak finder, tm-rule, and saw-derivative.Source: §4 · p. 4
  • Earlier detectors are rerun, retrained, or reimplemented, and the paper’s models are identified by the suffix -ours.The prior models span AE, confinement, ELM, tearing-mode, sawtooth, and RWM tasks.Source: §4 · p. 4
  • AE scores are optimistic because ae-ours used validation shots for early stopping and run selection, and those shots have higher AE prevalence than its training shots.Other learned scores come from cross-validation on development shots; intervals are 95% shot-bootstrap percentiles and differences are paired.Source: §4 · p. 4

5. Benchmark Results

The benchmark scores earlier detectors and Tokamak-SI models against consistent labels, with selected original label sets also evaluated. Results vary by phenomenon, including strong performance by the new models on dense AE labels, confinement, and tearing-mode detection.

5. Benchmark Results

  • Table 1 scores earlier papers’ models and Tokamak-SI models, with Figure 2 showing F1 and Figure 5 showing AUROC and AUPRC.Each phenomenon is compared using trivial baselines, prior-study detectors, and the new models on consistent labels and, where available, original labels.Source: §5 · p. 5

5.1. Alfv´en eigenmodes

  • F1 0.944 against 0.774 and 0.715: ae-ours leads on every metric against dense AE labels.With 68% of frames present, neither older detector beats the always-present baseline at its own threshold.Source: §5.1 · p. 5

5.2. Confinement

  • Macro-F1 0.909 [0.79, 0.96]: confine-ours exceeds confine-cnn’s 0.703 [0.59, 0.79] on the same windows.The new model reaches 0.910 [0.882, 0.936] on 416 shots in run-day folds, but is experimental because its labels were set from the traces it reads.Source: §5.2 · p. 5

5.3. Edge localized modes

  • F1 0.845 against 0.842: elm-ours slightly exceeds elm-elmo on 73 reviewed shots with BES, with difference +0.002 [−0.052, +0.058].The hand-labeled windows of O’Shea et al. (2022) were selected around known ELMs, so their reported scores hold only on those windows; elm-ours reaches F1 0.830 on all 119 reviewed cohort shots.Source: §5.3 · p. 5

5.4. Other label sets

  • F1 0.960 and AUROC 0.999: tm-ours scores on 450 shots, above retrained tm-cnn and tm-dsm F1 scores of 0.844 and 0.799 on our labels.Against their original targets, tm-cnn and tm-dsm reach F1 0.611 and 0.177, respectively.Source: §5.4 · p. 5
  • Presence F1 0.958, AUROC 0.984, and crash F1 0.891: saw-ours leads saw-derivative on out-of-fold predictions for 386 training shots.Saw-hl3’s F1 0.483 is not clearly above always present (0.411), though it beats the majority class on its paper’s three-class window task (accuracy 0.715 against 0.522).Source: §5.4 · p. 5
  • F1 0.529 [0.432, 0.609] and AUROC 0.967: rwm-ours exceeds rwm-brf’s F1 0.034 and the always-present baseline’s 0.013.Both models classify 10 ms bins against mode durations derived from onsets; rwm-ours adds magnetic-probe, I-coil, and rotation inputs.Source: §5.4 · p. 5
  • Macro-F1 0.722 [0.55, 0.87] for detach-ours measures consistency rather than accuracy because no independent reference is available.The divertor tangential camera label ranks divertor Thomson Te with AUROC 0.95 [0.91, 0.98] on 23 shots, or 0.94 without the calibration shot.Source: §5.4 · p. 5

6. Limitations and Roadmap

Several label sets remain limited in scope or under review. The roadmap proposes further expert labeling and re-scoring.

6. Limitations and Roadmap

  • AE labels use only present and absent, cover AE shots only, and leave false alarms on AE-free shots and expert agreement unmeasured.Confinement, RWM, tearing-mode, and detachment shots are still undergoing full review.Source: §6 · p. 5
  • Planned work includes a second AE reader, gold labels on 50 blind shots across categories, and random-order cohort review to estimate population base rates.The authors also plan to re-score every method on gold labels, compare fully labeled with single-phenomenon training at equal expert cost, and release labels, splits, code, and weights.Source: §6 · p. 5

Impact Statement

Tokamak-SI combines selected DIII-D discharges and multiple label sources into a time-aligned catalog, while retaining label provenance and coverage. The authors identify over-trust in model outputs as a risk and present safeguards for how outputs are released and interpreted.

A. Catalog Details

  • The catalog selects 4,872 discharges using plasma, heating, duration, diagnostic, and flat-top criteria, with 500 shots set aside for expert review.Labels use a shared interval schema with four states; time without a row is not assessed.Source: A. Catalog Details · p. 11

B. How Each Label Is Made

  • Original tables become present labels where positive samples or intervals overlap bins, and absent only where the source provides zeros; reviewers leave unmarked time unassessed.Missing deciding diagnostics are marked not observable.Source: B. How Each Label Is Made · p. 11, Table 3 · p. 11

B.1. Alfv´en eigenmodes

  • The AE annotation marks 12% of each benchmark shot, while dense labels mark 90% and 51%; these sources therefore assess very different stretches.The annotation spreads samples across 0–2 s, whereas dense labels mark every 10 ms frame over that interval and often beyond.Source: Figure 3 · p. 12, §B.1 · p. 12

B.2. Edge localized modes

  • ELM labels begin from an automated peak finder and expert-marked spans, with a present span representing one ELM or a crowd of unseparated ELMs.About half of crowd edges remain within 1 ms of the finder’s edges, favoring models that read Dα.Source: §B.2 · p. 12

B.3. Confinement

  • Confinement labels combine two ELM-free H-mode databases where available, use a Dα H-mode detector elsewhere, and leave the remaining shots to confine-ours without review.Conflicting H–L calls are uncertain, and time not flagged by either database is unassessed.Source: §B.3 · p. 12

B.4. Tearing modes

  • The tearing-mode rule uses n=1 or n=2 magnetic RMS thresholds and a stable rotating line, while unsupported crossings and other ambiguous activity remain uncertain.Uncertain intervals comprise 34.0% of observable time; labels cover 450 development and 4,798 population shots in 10 ms bins.Source: §B.4 · p. 12

B.5. Sawtooth

  • Sawtooth presence requires at least three ECE crashes 20–250 ms apart, while absence is rare and also includes observable time without a q=1 surface.The rule labels 13,319 of 16,909 shots; only three shots were reviewed.Source: §B.5 · p. 13

B.6. Resistive wall modes

  • RWM durations are derived from onset lists and δBp signals; the remaining high-current window is treated as negative under the assumption that the lists are complete.The benchmark evaluates presence on 10 ms bins of listed shots.Source: §B.6 · p. 13, Table 5 · p. 13

B.7. Detachment

Detachment labels combine a TangTV front-height rule with strike-point and diagnostic checks. Models are evaluated on a small, shot-grouped set, and their reported macro-F1 measures consistency rather than accuracy.

B.7. Detachment

  • Detach-ours reaches macro-F1 0.722 [0.55, 0.87] and detach-frames reaches 0.54 [0.25, 0.75], against majority-class scores of 0.34 and 0.45, respectively.The results use five shot-grouped folds and measure consistency, not accuracy.Source: §B.7 · p. 13
  • The TangTV front-height rule calls attached below 0.35, detached from 0.5 to 1.2, and MARFE above that range.A bin is uncertain for an inward strike point, disagreement with target probes, or a detached call during an L-mode phase.Source: §B.7 · p. 13
  • The labels cover 821 bins on 23 shots with no expert review; all six calls on published points agree with those points.The figure also shows attached, detached, and attached intervals on shot 201081 alongside diagnostics and per-shot label shares.Source: §B.7 · p. 13, Figure 4 · p. 14

C. Benchmark Models

The benchmark models use different diagnostic inputs and architectures for each phenomenon, including reimplementations or retrained versions of earlier detectors. Their setup reflects task-specific data, labels, and evaluation constraints.

Alfv´en eigenmodes

  • AE models include ae-ours, a SELDnet-style detector on four CO2 chord spectrograms, and earlier LSTM and echo-state models using one chord or chord-pair inputs.The passage distinguishes scores over each chord or pair from scores averaged across a shot’s inputs.Source: C. Benchmark Models · p. 15

Confinement

  • The confinement benchmark rebuilds Gill et al.’s BES CNN, while experimental confine-ours uses a 1-D U-Net on slower signals and labels derived from the same traces it reads.The latter’s score is therefore circular.Source: C. Benchmark Models · p. 15

Edge localized modes

  • The ELM benchmark compares a 1-D U-Net on Dα and density signals with a refitted survival-model head and a reimplementation of ELM-O.ELM-O requires an interferometer chord, two filterscopes, and BES, so 46 reviewed shots without usable BES are not scored.Source: C. Benchmark Models · p. 15

Tearing modes

  • The tearing benchmark uses a dilated 1-D CNN on Mirnov features, retrained forecasters, and a rule based on n=1 and n=2 RMS thresholds.The CNN uses ±140 ms of centered context and does not read the RMS used to make the labels.Source: C. Benchmark Models · p. 15

Sawtooth

  • The sawtooth benchmark compares a PhaseNet-style picker using ECE channels with a refitted CNN–LSTM and a derivative crash picker.The CNN–LSTM uses an offline input set that includes soft X-ray, ECE, density, current, Mirnov, stored energy, and heating signals.Source: C. Benchmark Models · p. 15

Resistive wall modes

  • Both RWM models are balanced random forests; rwm-ours adds δBp, I-coil activity, and q=2 rotation to rwm-brf’s equilibrium, magnetic-RMS, and rotation inputs.The baseline inputs do not directly sense the RWM, and folds hold out one of three run days.Source: C. Benchmark Models · p. 15

D. Additional Results

The paper illustrates how its language interface distinguishes events from text mentions and supports evidence-ranked searches. Its AE benchmark table covers detector metrics on benchmark frames, while the reported best-threshold F1 is described as optimistic.

E. Language Interface Example

  • A text-mention row is a logbook lexicon hit, not an assertion that the named phenomenon occurred.The example’s caveat separates a written word from an event record.Source: E. Language Interface Example · p. 15
  • A query for “tearing” ranks results by evidence class, and each event records its source and coverage so a label source can be replaced as a unit.The example returns 302 events from seven sources, one text mention, and no forecasts.Source: E. Language Interface Example · p. 15

D. Additional Results

  • The AE table reports precision, recall, and F1 at each detector’s own threshold on 3,800 frames from 19 benchmark shots; best F1 over all thresholds is optimistic.The last row reports ae-ours across all 60 validation shots.Source: Table 6 · p. 16, E. Language Interface Example · p. 16

F. Datasheet Summary

The datasheet summarizes the dataset’s coverage, review, distribution, and benchmark setup. Its examples show how ELM and tearing-mode labels align with diagnostic signals.

F. Datasheet Summary

  • Four-state interval labels cover seven phenomena across 4,872 rule-selected DIII-D shots, alongside a 500-shot weighted cohort.The datasheet specifies expert review, label provenance, selection rules, scoring code, and numbered releases that re-score every model on gold; raw signals remain under the DIII-D data policy.Source: F. Datasheet Summary · p. 16
  • On confine-cnn’s test windows, confine-ours minus confine-cnn is +0.207 [+0.09, +0.31].Source: Table 7 · p. 17
  • On BES shots, elm-ours minus elm-elmo is AUROC +0.023 [−0.014, +0.067] and F1 +0.002 [−0.052, +0.058].Source: Table 8 · p. 17
  • The tearing-mode benchmark tests magnetic-rule recovery across original targets and uncertain-time scoring choices, comparing published forecasters with retrained detection models.F1 uses inner-validation thresholds except CNN rows marked 0.5; training overlap is unknown.Source: Table 9 · p. 17

G. Examples

  • Examples pair ELM reviewed spans with probability and detector outputs, and tearing-mode labels with Mirnov spectra and mode RMS; one ELM span is identity-ambiguous.The ambiguous span occurs during a high-recycling phase.Source: Figure 6 · p. 19, Figure 7 · p. 20

Related papers

Loading 2610.32264v1…