PaperlayerView exact revision on arXiv

Source-linked AI summary

GSO-Net: Visual State Machines for Hazardous Freight Transfer Compliance at Petrochemical Logistics Nodes

Yu Xie, Bangshu Xiong, Zhibo Rao, Rui Gan, Chongxuan Liu, Zechu Ouyang

arXiv:2609.12408v1cs.CV

TL;DR

Sparse infrastructure polling makes hazardous-freight SOP status difficult to infer from incomplete observations and localized evidence, while existing vision benchmarks rarely supervise procedural compliance. GSO-Net provides a hierarchical benchmark linking procedural steps and operational states; experiments show that current models still struggle to ground contact states, recognize transient phases, and maintain stage consistency.

  • Problem

    Existing vision benchmarks rarely address procedural compliance under realistic sparse-polling conditions, where transfer status must be inferred from incomplete observations and localized evidence.

    Source: §I · p. 2

  • Method

    GSO-Net is a benchmark that links 9 macroscopic procedural steps with 15 microscopic operational states and evaluates joint step-state detection alongside frame-level step classification.

    Source: §I · p. 2, §IV · p. 7

  • Results

    Current models recognize salient objects and dominant phases but remain weak on contact-level state grounding, transient phase recognition, and stage-consistent understanding under sparse, localized evidence.

    Source: E. Summary of Empirical Findings · p. 10, §VII · p. 12

  • Takeaways & Limitations

    GSO-Net provides a practical benchmark for fine-grained state perception and vision-based safety monitoring in hazardous freight transportation.

    Source: §VII · p. 12

    The benchmark is restricted to petrochemical unloading, independently sampled frames do not support richer temporal modeling or explicit association across sparse observations, and no explicit rule-based or consistency-aware SOP verification protocol is implemented.

    Source: §VI · p. 12

Abstract

from arXiv · show

Hazardous-freight operations at petrochemical logistics nodes are safety-critical for intelligent transportation systems, yet existing vision benchmarks rarely address procedural compliance under realistic deployment constraints. In large infrastructure networks, cameras often operate under sparse round-robin polling, so transfer status must be inferred from incomplete observations and localized evidence. We present GSO-Net, a large-scale benchmark for visual understanding of standard operating procedures (SOPs) in petrochemical unloading scenarios. To our knowledge, GSO-Net is the first public benchmark dataset dedicated to visual SOP understanding in petrochemical hazardous-freight transfer scenarios. It contains over 50,000 independently sampled frames from 64 real expressway petrochemical logistics nodes and adopts an SOP-derived hierarchy linking 9 macroscopic procedural steps with 15 microscopic operational states. Two tasks are defined: joint detection of microscopic states and macroscopic steps as the core benchmark, and frame-level step classification as a diagnostic reference. Experiments with lightweight, transformer-based, open-vocabulary, and holistic models reveal a clear gap between object perception and transfer-stage understanding. Current models remain weak on contact-level state grounding, transient step recognition, and stage consistency, especially under sparse polling, tiny critical targets, and long-tailed operational evidence. GSO-Net provides a practical benchmark for fine-grained state perception and vision-based safety monitoring in hazardous freight transportation. The dataset is publicly available at https://github.com/yuxieHarrison/GSO-Net

I. INTRODUCTION

Hazardous-freight monitoring must infer procedural status from sparse observations and localized visual cues, beyond recognizing objects alone. GSO-Net addresses a supervision gap in existing benchmarks by linking procedural steps with operational states.

I. INTRODUCTION

  • Sparse round-robin polling leaves each logistics node observed for only 3–5 s, making reliable transfer-stage inference from incomplete visual evidence the central deployment challenge.The relevant cues include small local changes that distinguish otherwise similar workflow stages.Source: §I · p. 1

A. Benchmarks for Safety Monitoring and Procedural Compliance

  • Existing anomaly, safety, and transportation benchmarks emphasize defects or object and scene-element presence rather than whether hazardous transfer is at the correct procedural stage.GSO-Net identifies the main gap as supervision structure, not simply scene domain.Source: §I · p. 1, A. Benchmarks for Safety Monitoring… · p. 2

I. INTRODUCTION

  • Monitoring must move beyond object detection to infer operational progress and correctness from distributed visual evidence.Critical contact states can occupy only a few pixels, while generic pre-training does not explicitly encode hazardous-freight procedure or stage logic.Source: §I · pp. 1–2
  • GSO-Net is a benchmark for visual SOP understanding that links macroscopic procedural steps with microscopic operational states under sparse polling.It is presented as the first publicly available benchmark for petrochemical hazardous-freight transfer SOP understanding.Source: §I · p. 2

B. Procedural Understanding Under Sparse Infrastructure Observation

Dense procedural-video benchmarks do not match infrastructure monitoring, where nodes are observed intermittently and transitions may appear in only a few frames. GSO-Net also makes measurable the gap between localized state recognition and global stage understanding.

B. Procedural Understanding Under Sparse Infrastructure Observation

  • Round-robin polling can expose critical procedural transitions in only one or two frames, unlike benchmarks that rely on dense temporal evidence.Stage recognition therefore cannot depend on dense motion cues, tracking, or long contiguous clips.Source: B. Procedural Understanding Under… · pp. 2–3
  • Table I contrasts benchmarks that typically emphasize one aspect with GSO-Net’s joint modeling of workflow order and localized operational states for hazardous-freight SOP understanding.The target setting is sparse visual polling at petrochemical logistics nodes.Source: Table I · p. 3

C. Fine-Grained State for Stage-Aware SOP Analysis

  • Stronger perception alone does not establish whether a local operational state is valid or consistent with the current workflow stage.GSO-Net’s unified annotations and evaluation make the relationship between microscopic cues and macroscopic stage inference measurable.Source: C. Fine-Grained State for Stage-Aware… · p. 3

III. GSO-NET CONSTRUCTION AND ANNOTATION PROTOCOL

GSO-Net translates petrochemical unloading procedures into a visual taxonomy that links chronological workflow phases to explicit operational evidence. Its hierarchy pairs nine macroscopic steps with fifteen microscopic states.

III. GSO-NET CONSTRUCTION AND ANNOTATION PROTOCOL

A. SOP Taxonomy and Hierarchical Semantic Mapping

  • The macroscopic level partitions unloading into nine chronologically dependent phases, S0 to S8, each tied to an operational stage and safety requirement.The phases range from idle and tanker entry through unloading and supervision to receipt acceptance and post-processing.Source: A. SOP Taxonomy and Hierarchical… · p. 4
  • The microscopic level defines 15 fine-grained operational states as bounding-box annotations that provide visual evidence for stage understanding.Examples include clamp connected and discharge disconnected.Source: A. SOP Taxonomy and Hierarchical… · p. 4

B. Step-State Correspondence Derived from SOPs

GSO-Net relates step labels to localized state annotations using SOP-derived operational meaning, rather than a formal rule engine. Independent frame sampling and a staged quality process reflect sparse deployment observations and visible operational evidence.

B. Step-State Correspondence Derived from SOPs

  • Macroscopic steps and microscopic states are linked by operational meaning derived from SOPs and practice, not by a formally defined rule engine.For example, continuous supervision is associated with connected clamp and discharge states and on-site personnel.Source: B. Step-State Correspondence Derived… · pp. 4–5

C. Independent Frame Sampling and Environmental Complexity

  • The dataset samples surveillance frames independently from 64 active expressway petrochemical logistics nodes over eight months rather than extracting continuous high-frame-rate clips.More than 50,000 frames were curated from unconstrained real-world settings with diurnal, seasonal, and adverse-weather variation.Source: C. Independent Frame Sampling and… · p. 5

D. Annotation Procedure and Quality Assurance

  • A three-stage quality protocol combines state-aware box annotation, frontline expert verification, and multi-round semantic and procedural review.Clearly visible unsafe or non-compliant operations were retained rather than forcing every sample to represent an ideal SOP-compliant state.Source: D. Annotation Procedure and Quality… · pp. 5–6

E. Dataset Characteristics and Structural Difficulty

GSO-Net combines a structured SOP lifecycle with microscopic states whose frequencies are strongly imbalanced. Its data also spans varied environmental conditions and viewpoints, while neighboring procedural stages can require attention to localized evidence.

E. Dataset Characteristics and Structural Difficulty

  • Long-duration standby and supervision stages dominate the workflow timeline.Source: E. Dataset Characteristics and… · p. 6
  • Frequent microscopic states dominate, while safety-critical interactions such as clamp floating and clamp disconnected are rarer.Step S6 contributes the most samples because fuel transfer lasts longer than preparatory and post-processing stages.Source: E. Dataset Characteristics and… · p. 6
  • Microscopic states cluster around specific procedural stages, showing empirical correspondence between macroscopic steps and microscopic states.Source: E. Dataset Characteristics and… · p. 6
  • Adjacent SOP phases can share similar global views while differing in localized operational evidence, requiring dynamic semantic focus rather than global scene matching.Source: E. Dataset Characteristics and… · p. 6

F. Dataset Statistics

  • Head microscopic classes have tens of thousands of instances, whereas tail classes such as clamp floating and staff bending have fewer than 1,100.Source: F. Dataset Statistics · p. 6
  • 50,325 curated frames come from 64 logistics nodes, and clear daylight accounts for 42.2% of the data.The remaining frames cover night scenes, adverse weather, viewpoint variation, and heavy occlusion.Source: F. Dataset Statistics · p. 6

G. Ethical Compliance and Data Integrity

The dataset pipeline follows project-level ethical constraints, and its state distribution is long-tailed. Experiments assess hierarchical state-and-step detection and holistic step classification, using a split designed to test generalization to unseen logistics nodes.

G. Ethical Compliance and Data Integrity

  • Data acquisition used workplace safety-monitoring feeds within an authorized project scope, and acquisition, annotation, and release followed local data-governance requirements and project-level ethical constraints.Source: G. Ethical Compliance and Data Integrity · pp. 6–7
  • Microscopic state counts are long-tailed across the 15 states, while Table IV covers frame distributions across environmental conditions, viewpoints, and visual challenges.Table III reports bounding-box instance counts across training and validation sets; frequent states dominate, and safety-critical interactions appear less often.Source: Figure 4 · p. 7, Table III · p. 7, Table IV · p. 7

IV. EXPERIMENTS AND ANALYSIS

  • The experiments benchmark whether models recover procedural meaning from localized evidence under sparse polling, using joint state-and-step detection as the core task and frame-level step classification as a diagnostic reference.Source: §IV · p. 7

A. Benchmark Protocol and Experimental Setup

  • A strict cross-node split assigns 40,976 images from 51 nodes to training and 9,349 images from 13 unseen nodes to validation, avoiding scene leakage from random frame-level splitting.The protocol measures generalization to new nodes rather than memorization within the same node.Source: A. Benchmark Protocol and Experimental… · p. 8
  • Task 1 predicts 24 categories jointly: 15 microscopic states use bounding boxes, while 9 macroscopic steps use full-frame targets.Microscopic-state AP measures localized detection, whereas macroscopic-step AP measures image-level step recognition.Source: A. Benchmark Protocol and Experimental… · p. 8
  • Task 2 assigns each full image one of nine SOP steps from global visual evidence, without explicit modeling of localized states.It is evaluated with Top-1 Accuracy, Top-5 Accuracy, and Macro F1 as a diagnostic reference.Source: A. Benchmark Protocol and Experimental… · p. 8
  • YOLO-based detectors were trained for 100 epochs at 640×640 resolution, while other methods generally used official default configurations.All baselines were trained solely on GSO-Net without supplementary industrial data.Source: A. Benchmark Protocol and Experimental… · p. 8

B. Task 1: Joint Detection of Microscopic States and Macroscopic Steps

Task 1 jointly evaluates microscopic state detection and macroscopic step recognition, where stage inference depends on tiny local cues in high-resolution scenes. Stronger detectors improve overall performance, but contact-level evidence remains difficult to recover.

B. Task 1: Joint Detection of Microscopic States and Macroscopic Steps

  • Task 1 tests joint recovery of microscopic states and macroscopic steps, requiring fine-grained evidence grounding and scene-level stage discrimination from tiny cues in diverse, high-resolution frames.Clamp and hose-connection indicators can be much smaller than foreground actors, making the task a multi-scale state-grounding problem under practical resizing constraints.Source: B. Task 1: Joint Detection of… · p. 8, Figure 5 · p. 8

1) Lightweight Detection Baselines:

  • YOLOv9-t offers the strongest lightweight trade-off at 44.0% AP50 and 36.5% AP50-95 with 1.98M parameters, while APS stays between 3.0% and 4.9% across all five models.YOLO26-n leads lightweight APM and APL at 13.9% and 42.2%, respectively; the large APS–APL gap exposes difficulty recovering small state-sensitive evidence.Source: Table V · p. 8, 1) Lightweight Detection Baselines · p. 8

2) Comparison with Stronger Detectors:

  • Relation-DETR-SwinL leads all reported metrics, reaching 56.4% AP50, 46.3% AP50-95, 13.3% APS, 23.4% APM, and 55.1% APL.Even its APS remains far below its APL, so stronger global attention does not resolve the small-object problem.Source: Table VI · p. 9, 2) Comparison with Stronger Detectors · p. 9
  • YOLO-World reaches 45.6% AP50 and 6.8% APS, exceeding every lightweight closed-set YOLO baseline on APS, but mechanically defined small categories remain difficult.The passage attributes its broader semantic transfer to language-aligned pre-training while emphasizing that precise state grounding requires more than semantic name transfer.Source: 2) Comparison with Stronger Detectors · p. 9

3) Category-wise Diagnosis:

Category-level findings distinguish frequent, visually salient states from mechanically decisive contact states and transient workflow phases. Frame-level classification captures some coarse procedural context, but precise stage discrimination remains difficult.

3) Category-wise Diagnosis:

  • Frequent, visually salient states such as tanker stopped true, staff standing, staff operating, and extinguisher deployed achieve relatively high scores.These classes are more frequent and have larger visual extent.Source: 3) Category-wise Diagnosis · p. 9, Table VII · p. 10
  • Mechanically decisive contact states remain weak: clamp connected is at or below 5.3% across models, clamp floating at or below 6.0%, and clamp disconnected is essentially unresolved.Discharge disconnected reaches 15.7% only for Relation-DETR-SwinL and stays near zero for most other methods.Source: 3) Category-wise Diagnosis · p. 9, Table VII · p. 10
  • Stable Step0 Idle and Step6 Supervision are detected more reliably than transitional Steps 2, 3, 4, 5, and 7, whose cues are localized rather than global.The benchmark’s frame distribution is dominated by long-duration phases, while several transitional steps are comparatively rare.Source: 3) Category-wise Diagnosis · p. 9, Figure 6 · p. 9, Table VII · p. 10
  • Detecting the tanker, worker, and extinguisher does not guarantee the decisive connection state or correct procedural phase, because unresolved categories include rare transitions and small contact points.Strong object detection alone does not ensure SOP understanding.Source: 3) Category-wise Diagnosis · p. 9

C. Task 2: Frame-Level SOP Step Classification

  • Task 2 uses full-frame classifiers to predict one of nine SOP steps and tests scene-level recognition before localized state evidence is introduced.Macro F1 accompanies Top-1 and Top-5 because long-duration phases dominate while short transitional stages are sparse.Source: C. Task 2: Frame-Level SOP Step… · p. 9, Figure 6 · p. 9
  • Without localized state supervision, holistic appearance remains under-constrained: functional state transitions define many step boundaries, and sparse local cues limit fine-grained stage discrimination.GSO-Net therefore treats frame-level classification as a diagnostic reference and links macroscopic steps with microscopic state evidence in Task 1.Source: C. Task 2: Frame-Level SOP Step… · pp. 9–10
  • YOLO26-cls reaches 44.3% Top-1, 86.2% Top-5, and 16.1% Macro F1, while all models remain within a narrow 41.7%–44.3% Top-1 range.The large Top-1/Top-5 gap indicates coarse procedural context is captured, but precise stage boundaries remain unresolved.Source: C. Task 2: Frame-Level SOP Step… · p. 9, Table VIII · p. 10

D. Qualitative Analysis and Error Statistics

The qualitative analysis and benchmark discussion show that accurate SOP understanding requires more than detecting salient objects or dominant phases. Models still struggle to ground contact-level states and short transitions in sparse visual evidence.

D. Qualitative Analysis and Error Statistics

  • Qualitative predictions show adjacent-phase ambiguity, omitted decisive micro-state evidence, and stage predictions that contradict visible cues.Relation-DETR-SwinL usually provides the most complete visual evidence and strongest phase sensitivity, but retains ranking ambiguity and incomplete evidence support.Source: D. Qualitative Analysis and Error… · p. 10, Figure 7 · p. 11

E. Summary of Empirical Findings

  • Cross-node generalization is difficult, and current detectors still miss microscopic contact states, transient phases, and stage-consistent understanding; scene-level recognition alone is insufficient.Fine-grained state grounding is essential to the core Task 1 evaluation.Source: E. Summary of Empirical Findings · pp. 10–11

V. DATASET ADVANTAGES

  • Performance is stronger on salient objects and stable phases than on tiny contact states, rare cases, and short transitions, so correct detection does not guarantee correct SOP understanding.Sparse polling can expose a transition in only one frame, while frame-level classification remains weak on short transitions.Source: §V · pp. 11–12
  • The proposed direction combines micro-perception, sparse temporal memory, and stage-aware reasoning to address contact-state failures, discontinuous observations, and ambiguous stage predictions.The discussion argues that stronger backbones alone are unlikely to resolve the remaining gap.Source: §V · p. 12

VI. DISCUSSION

The discussion identifies hierarchical state-stage coupling, sparse localized evidence, and real-world variation as benchmark challenges, while noting limits in domain scope, temporal supervision, and explicit SOP reasoning. The conclusion frames GSO-Net as a benchmark for stage-aware state grounding under sparse polling.

VI. DISCUSSION

  • The benchmark couples macroscopic SOP stages with microscopic states, requiring inference from isolated frames and subtle cues amid long-tail imbalance, occlusion, low illumination, adverse weather, and viewpoint variation.Examples of localized cues include clamp status and hose-port connection.Source: §VI · p. 12
  • GSO-Net’s current scope is petrochemical unloading at logistics nodes, with independently sampled frames, no explicit sparse temporal association, and no rule-based or consistency-aware SOP verification.The paper identifies these as limitations in domain scope, temporal supervision, and reasoning scope.Source: §VI · p. 12

VII. CONCLUSION

  • GSO-Net links 9 procedural steps with 15 operational states across 64 real nodes, and experiments show persistent difficulty with contact-level grounding, transient phases, and stage consistency.Future work includes broader scenarios, sparse temporal annotations and association signals, stage-aware models, and cross-node continual evaluation.Source: §VII · p. 12, §VI · p. 12

APPENDIX A HYPERPARAMETER CONFIGURATIONS

The appendix specifies training and comparison protocols for the YOLO and non-YOLO baselines, then describes visibility-first annotation rules for mechanically sensitive states. Ambiguous visual evidence was not assigned speculatively.

APPENDIX A HYPERPARAMETER CONFIGURATIONS

  • YOLO baselines used 100 epochs, batch size 16, 640 × 640 inputs, and shared training settings; non-YOLO baselines followed their official implementations and default protocols.YOLO training used SGD, standard augmentation, and early stopping; Mosaic augmentation was disabled for the final 10 epochs.Source: Appendix A · pp. 12–13

APPENDIX B ANNOTATION GUIDELINES AND TOOLING

  • Annotators assigned operational states only when visually supported, labeling an occluded grounding clamp as floating rather than connected and tightly boxing engaged contact points.Weak evidence from occlusion, low illumination, motion blur, or viewpoint limits was retained for review rather than assigned speculatively.Source: Appendix B · p. 13
Loading 2609.12408v1…