PaperlayerView exact revision on arXiv

Source-linked AI summary

From Signals to Evidence: Interpretable Underwater Acoustic Target Sensing with Large Language Models - Part I: Framework and Recognition

Yuge Zhang, Jie Chen, Zhengqiao Zhao, Xiling Yao, Lingji Xu, Jingdong Chen, Yixin Yang

arXiv:2610.22064v1cs.SDeess.SP

TL;DR

Underwater sensing systems often return labels or confidence scores without showing the physical evidence behind their decisions, while general audio encoders and scarce physical annotations pose challenges for this domain. This paper combines physics-aware acoustic encoding with a pretrained LLM in a feature–evidence–decision framework and tests its recognition instantiation on simulated recordings of four vessels. The experiments demonstrate accurate vessel recognition and reliable estimation of several physical attributes.

  • Problem

    Underwater recognition systems typically provide labels or confidence scores with limited insight into the physical acoustic evidence supporting their decisions.

    Source: §1 · pp. 1–2

  • Method

    The framework couples a physics-aware acoustic encoder with a pretrained LLM so judgments are produced through an evidence-then-decision chain.

    Source: §1 · p. 2, §3 · p. 4, §3.3 · p. 5

  • Results

    Experiments on simulated recordings of four vessels demonstrate accurate recognition and reliable estimation of shaft-frequency, blade-passing, and machinery-related attributes.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results validate the feasibility of evidence-grounded underwater acoustic recognition within the tested simulated-vessel setting.

    Source: §1 · p. 3, §7.2 · p. 14

    The causal role of cited acoustic evidence remains unverified; testing whether decisions change when those structures are removed requires intervention-based analysis.

    Source: §7.1.2 · p. 12

Abstract

from arXiv · show

Underwater acoustic target detection and recognition are essential to marine surveillance and intelligent ocean sensing. Despite substantial advances in deep learning, existing systems typically produce binary detection decisions, categorical labels, or scalar confidence scores, offering limited insight into the physical evidence underlying their outputs. Large language models (LLMs) offer an opportunity to move underwater acoustic sensing toward interpretable, evidence-grounded understanding; however, generic audio encoders struggle to capture the distinctive physical characteristics of underwater acoustics, and physically annotated data are scarce. In this paper, the first of a two-part series, we propose a from-signals-to-evidence framework that integrates physics-aware acoustic encoding with a pretrained LLM, organized around a unified feature-evidence-decision chain with a shared reasoning core. An open input layer accommodates heterogeneous information-raw waveforms, time-frequency representations, environmental and geographic data, and textual physical priors-unifying detection and recognition within a common formulation. In Part I, we instantiate this framework for target recognition through an Underwater Acoustic Physics-aware Transformer (UPT), which tokenizes complementary acoustic representations and learns task-disentangled physical representations through four physics-oriented learning tasks. We develop a physics-parameterized data engine that provides traceable supervision at two levels of granularity and introduce an attribute-level evaluation protocol to assess the physical fidelity of generated descriptions. Experiments on simulated recordings of four vessels demonstrate accurate vessel recognition and reliable estimation of shaft frequency, blade-passing frequency, and machinery-related attributes.

1 INTRODUCTION

Underwater recognition systems often provide labels or confidence scores without explaining which physical acoustic evidence supports them. This paper proposes an evidence-grounded framework and instantiates it for recognition with physics-aware representations and simulated-data supervision.

1 INTRODUCTION

  • Existing systems offer limited insight into decisions, while confidence scores may be poorly calibrated and do not show how acoustic cues support a conclusion.The motivation is to move beyond labels and scalar scores toward decisions grounded in identifiable physical signatures.Source: §1 · pp. 1–2
  • UPD organizes sensing as a feature–evidence–decision chain, with a physics-aware encoder and pretrained LLM producing judgments conditioned on stated acoustic evidence.Part I applies the shared framework to recognition, using complementary time–frequency representations, physical priors, and structured descriptions of vessel attributes and supporting evidence.Source: §1 · p. 2
  • The framework couples a physics-oriented acoustic encoder with an LLM, while UPT learns task-disentangled physical representations from complementary acoustic representations.A physics-based simulation pipeline supplies discrete attribute labels and structured descriptions for physically grounded training and field-wise evaluation.Source: §1 · p. 3
  • Experiments on simulated recordings of four vessels demonstrate accurate recognition and reliable estimation of shaft-frequency, blade-passing, and machinery-related attributes.The framework is presented as a feasibility demonstration of evidence-grounded underwater acoustic recognition.Source: §1 · p. 3

2 RELATED WORK

Underwater recognition research has moved from handcrafted features to learned representations and language-guided methods, but outputs generally remain labels or scores rather than evidence-bearing descriptions. General audio-language models also show limited transfer to fine-grained underwater recognition, motivating physics-aware adaptation.

2.1 Underwater Acoustic Target Recognition

  • Earlier recognition systems used LOFAR, DEMON, MFCCs, and statistical classifiers to extract physically meaningful cues, but weak tones, changing signals, and expert-defined features constrain robustness and adaptability.Deep learning improved recognition accuracy, while the decision output remained largely opaque.Source: §2.1 · p. 3
  • Modern recognition methods improve classification with learned representations, attention, transformers, augmentation, transfer learning, and few-shot adaptation, but usually return labels or confidence scores.These outputs do not identify supporting acoustic structures or explain how environmental factors shaped the observation.Source: §2.1 · p. 3
  • Physics- and language-informed methods incorporate physical or semantic information, yet mainly use it to improve category prediction rather than generate evidence-bearing descriptions.The review identifies structured accounts of acoustic evidence as a remaining need for human assessment and machine-to-machine verification.Source: §2.1 · p. 3

2.2 Large Audio-Language Models and Cross-Modal Alignment

  • General audio-language models can generate sound descriptions and support multiple tasks, but underwater signals require grounding descriptions in verifiable acoustic evidence.The reviewed systems use pretrained audio encoders aligned to LLMs; the cited discussion notes uncertainty about their recognition of physically structured underwater signals.Source: §2.2 · pp. 3–4
  • UA-Bench reports 13.1% highest leaf-level accuracy against a 7.7% random reference for nine unadapted open audio-language models on synthesized underwater signals.The benchmark findings motivate physics-aware adaptation for physically structured underwater recognition.Source: §2.2 · p. 4

3 FROM-SIGNALS-TO-EVIDENCE FRAMEWORK

The framework makes intermediate physical reasoning explicit in a shared feature–evidence–decision chain, with decisions conditioned on stated evidence. It applies this chain to heterogeneous acoustic information and shares a reasoning core across detection and recognition.

3 FROM-SIGNALS-TO-EVIDENCE FRAMEWORK

  • The framework turns observations into physically meaningful features, identifies relevant evidence using physical priors, and conditions final decisions on that evidence.This evidence-conditioned chain distinguishes the framework from post-hoc explanations generated independently of the decision process.Source: §3 · p. 4

3.1 Multimodal Input Layer

  • Acoustic signals, time–frequency representations, and textual physical priors are mapped into a shared semantic space for the reasoning core.The framework is designed to accommodate heterogeneous information without prescribing a closed set of input types.Source: §3 · p. 4

3.2 Physics-Aware Acoustic Encoding

  • The acoustic encoder preserves complementary time–frequency representations in separate pathways and learns tokens through physics-oriented tasks rather than only final-decision optimization.Parameterized physical processing, including filtering, normalization, beamforming, and Doppler compensation, is retained where it incorporates established physical knowledge.Source: §3.2 · p. 5

3.3 LLM Reasoning Core and Cross-Modal Alignment

  • A shared LLM reasoning core aggregates acoustic tokens, environmental imagery, and textual context, then generates evidence-then-decision responses.Pretrained parameters remain frozen and are adapted through low-rank modules; training supervises answer tokens while masking losses over queries, instructions, and inputs.Source: §3.3 · pp. 5–6

3.4 Evidence-Grounded Output

  • Responses pair a decision with evidence linked to verifiable acoustic features, using fixed field order to support parsing and independent checks against the input.Evidence can identify acoustic structures or distinguish propagation-related missing components from target absence.Source: §3.4 · p. 6

3.5 A Unified View of Detection and Recognition

  • Recognition uses target-bearing segments with relatively favorable SNR, while detection handles continuous observations, multichannel signals, and spatial outputs; detection also relies more on parameterized physical processing.Recognition emphasizes spectral and temporal structure, whereas detection may need to identify targets buried in noise.Source: §3.5 · p. 6
  • Environmental priors explain propagation effects in recognition and also support noise-field and distortion prediction for detection; Part I validates recognition, while Part II targets low-SNR detection.Part II emphasizes multichannel processing and environmental prediction.Source: §3.5 · p. 6

4 RECOGNITION INSTANTIATION: PHYSICS-AWARE ENCODING

The recognition instantiation couples UPT with a pretrained LLM, using complementary acoustic representations and physics-oriented tasks to produce task-disentangled physical features. It aligns these features with the LLM using answer-only adaptation.

4.1.1 Complementary acoustic representations.

  • Three representations expose broadband energy, tonal-line structures, and propeller-related modulations at complementary physical meanings and time–frequency resolutions.They are the Log-Mel spectrogram, time-varying line spectrum, and time-varying DEMON spectrogram.Source: §4.1.1 · p. 6

4.1.2 Independent tokenization.

  • Each representation is tokenized independently with representation-specific patching, positional embeddings, and feature-type embeddings that preserve token identity and link tokens to physical regions.For example, the line spectrum uses frequency-wise patches.Source: §4.1.2 · p. 7

4.2 Task-Specific Feature Attention

  • Task-specific attention weights the three representations for four physics tasks; each stream retains a minimum contribution, and the learned coefficients indicate its contribution to each task.The line spectrum informs SFE and MLD, while DEMON is essential for STE and BPFE.Source: §4.2 · p. 7

4.3 Task-Disentangled Branches

  • Four task-specific branches project representations into dedicated subspaces while sharing a Transformer backbone, jointly learning task-specialized representations from acoustic information.SFE, STE, and BPFE use single-label classification; MLD uses multi-label classification because multiple machinery components may coexist in a recording.Source: §4.3 · pp. 7–8

4.4 Alignment and Answer-Only Adaptation

  • UPT is coupled to a pretrained LLM; only the answer tokens of structured descriptions contribute to the alignment loss, while query and input tokens are masked.UPT is first optimized on its physics-oriented tasks, then the complete framework is optimized with the answer-only objective.Source: §4.4 · p. 8

5 PHYSICS-PARAMETERIZED DATA ENGINE

The physics-parameterized data engine generates traceable recordings and supervision from explicitly sampled physical parameters, addressing the scarcity of physically annotated underwater acoustic data.

5 PHYSICS-PARAMETERIZED DATA ENGINE

  • The data engine synthesizes each recording from sampled vessel, operating-state, source-composition, propagation, and background-noise parameters, making its generating conditions traceable.That traceability supplies ground truth for UPT’s physics-oriented tasks and evidence chains for UPD’s structured descriptions; the eight stages cover source synthesis, propagation and noise, and dual-granularity supervision.Source: §5 · p. 8

5.1 Source Signal Synthesis

Source synthesis combines interpretable vessel identity and operating-state parameters with modeled acoustic components. The resulting signals represent different propulsion states and vessel-specific spectral characteristics.

5.1.1 Vessel identity and operating states.

  • Vessel identities use interpretable parameters with deliberate overlap across vessels, while recordings receive one of four operating states: steady low speed, steady high speed, accelerating, or decelerating.Transition excerpts are eight seconds long and retain realistic frequency drift rather than an idealized full sweep.Source: §5.1.1 · p. 8

5.1.2 Source composition.

  • Each source is a weighted sum of tonal, continuous-spectrum, and modulation-spectrum components, with mixing weights drawn from vessel-specific distributions.The components are normalized before mixing.Source: §5.1.2 · p. 8
  • The tonal component models shaft-frequency, blade-passing-frequency, and machinery harmonics, using distinct harmonic decay exponents to capture their spectral characteristics.Shaft and blade-passing contributions each have one fundamental frequency; machinery may have several.Source: §5.1.2 · p. 9
  • The continuous-spectrum component uses Gaussian white noise and an autoregressive model whose coefficients are obtained from a prescribed autocorrelation sequence via Yule–Walker equations.The prescribed spectral shape is controlled by its center and bandwidth.Source: §5.1.2 · p. 9
  • The modulation-spectrum component models propeller-related periodic amplitude fluctuations with an envelope of Gaussian pulses, whose widths vary with the instantaneous blade-passing frequency.Operating-state frequency changes affect shaft and blade-passing harmonics, blade-passage timing, and the modulation envelope.Source: §5.1.2 · pp. 9–10

5.2 Propagation and Background Noise

Received signals are generated by propagating source signals through multipath channels and combining them with recorded ocean noise. Varying channel realizations, noise segments, and SNR produces diverse observations of the same source.

5.2 Propagation and Background Noise

  • BELLHOP-based multipath propagation and real ocean-noise segments are combined to generate received signals, with the channel response set to identity for clean recordings.Noise scaling is determined by the target SNR.Source: §5.2 · p. 10
  • Independent variation of noise segments, SNR levels, and channel realizations produces diverse observations of the same source and supports a clean-to-hard curriculum.The illustrative propagation setup covers shallow, sloping, and deep-water Korean coastal regions, with channels modeled from BELLHOP3D arrivals, bathymetry, and sound-speed profiles.Source: §5.2 · p. 10

5.3 Dual-Granularity Supervision

The data engine derives discrete physical labels for UPT and precise structured responses for UPD from the same recordings, providing supervision at two levels of granularity.

5.3 Dual-Granularity Supervision

  • Known synthesis parameters yield two complementary supervision signals from each recording.The signals support UPT's physical-analysis tasks and UPD's structured acoustic descriptions.Source: §5.3 · p. 10

5.3.1 Physics-aware acoustic representation labels (PAR).

  • PAR quantizes shaft frequency, BPF, and shaft-frequency trend into labels, and represents machinery fundamentals as multi-label targets.The bins are 0.25 Hz for mean shaft frequency, 0.5 Hz for mean BPF, three trend classes, and a 1 Hz grid for machinery frequencies.Source: §5.3.1 · p. 10

5.3.2 Structured acoustic descriptions (SAD).

  • SAD pairs each recording with an audio-question-response triplet covering vessel, operating, acoustic, and SNR fields in a fixed order.Numerical frequency values use 0.01 Hz precision and SNR uses 0.1 dB precision; fixed templates limit phrasing variability.Source: §5.3.2 · p. 10

5.4 Dataset Configuration and Analysis

The dataset combines simulated recordings of four vessels with varied channel and noise realizations across Korean coastal environments. Its design uses overlapping physical attributes so vessel identity depends on their joint configuration.

5.4.1 Configuration.

  • The engine generates 36,000 recordings from 1,200 source signals across four vessels and 30 channel-noise realizations per source.Signals are 32 kHz and 8 seconds long, with two SNR ranges: [−5, 5] dB and [−20, −5] dB.Source: §5.4.1 · pp. 10–11
  • Figure 5 presents selected Korean sea regions—shallow, slope, and deep—with bathymetry and locations, alongside representative channel impulse responses at a range of 15 km and azimuth 195°.Source: Figure 5 · p. 11

5.4.2 Analysis.

  • Vessel identity depends on joint physical-attribute configurations because shaft-frequency ranges overlap and machinery sets share components.Mixing weights vary by vessel, and the analysis reports that attributes provide complementary evidence rather than allowing trivial parameter lookups.Source: §5.4.2 · p. 11

6 EXPERIMENTS

The experiments evaluate UPT's physical tasks and UPD's structured descriptions on simulated vessel recordings. They report accurate recognition and physical-attribute estimates, while identifying greater noise sensitivity in machinery-frequency and SNR estimates.

6.1.1 Implementation details.

  • The three inputs use Log-Mel, line-spectrum, and DEMON representations, with pretrained AudioMAE ViT-Base weights initializing the encoder.The LLM is Qwen3-4B-Instruct, loaded in 4-bit precision and adapted with LoRA rank 16.Source: §6.1.1 · p. 11

6.1.2 Training protocol.

  • Training first optimizes UPT for 50 epochs, then the complete UPD framework for 3 epochs while keeping pretrained LLM parameters frozen.The backbone is frozen for UPT's first three epochs; both stages use AdamW and cosine learning-rate schedules with 0.05 warmup ratio.Source: §6.1.2 · p. 11

6.2 Evaluation Protocol

  • UPT tasks are assessed with accuracy for SFE, STE, and BPFE and Micro-F1 for MLD; UPD descriptions are evaluated field by field.Categorical fields use accuracy, numerical fields use MAE and RMSE, and machinery frequencies use permutation-invariant matching with a 1 Hz tolerance and precision, recall, and F1.Source: §6.2 · pp. 11–12

6.3 Physics-Aware Representation Performance

  • Across four physical tasks and SNR conditions, combining all three acoustic representations consistently outperformed single-representation variants.The comparison is reported in Table 2.Source: §6.3 · p. 12
  • Under −5 to 5 dB, feature attention raised AccSFE from 0.984 to 0.995, AccSTE from 0.951 to 0.967, AccBPFE from 0.963 to 0.969, and Micro-F1MLD from 0.882 to 0.892.At −20 to −5 dB, the corresponding values rose from 0.979, 0.947, 0.928, and 0.558 to 0.988, 0.967, 0.933, and 0.639.Source: §6.3 · p. 12
  • The line spectrum received the highest average attention weight across task branches, while MLD showed comparable weights for DEMON and line-spectrum representations.Training and validation losses decreased rapidly early on; SFE, STE, and BPFE reached high validation accuracy sooner than MLD.Source: §6.3 · p. 12

6.4 End-to-End Recognition and Description Performance

  • Under −5 to 5 dB, UPD reached 0.992 vessel-identity accuracy; at −20 to −5 dB, vessel-identity accuracy remained 0.977.At −5 to 5 dB, operating-state and shaft-frequency-trend accuracies were 0.967 and 0.970; at −20 to −5 dB, they were 0.955 and 0.955.Source: §6.4 · p. 12
  • At −20 to −5 dB, machinery-frequency MAE rose from 0.911 to 0.930 Hz and SNR MAE from 1.35 to 1.86 dB, while BPF errors remained relatively stable.UPD continued to produce structured descriptions containing vessel identity, operating conditions, and numerical physical attributes.Source: §6.4 · p. 12

7 DISCUSSION AND CONCLUSION

The discussion presents structured physical descriptions as inspectable evidence and identifies causal testing and field-recording transfer as future work. The conclusion summarizes the recognition framework and its simulated-vessel results.

7.1.1 From description to evidence.

  • The framework's descriptions place physical attributes before and alongside decisions, with field-by-field verification and task-specific attention as additional evidence checks.The discussion distinguishes verifiable attribute correctness from the still-unestablished causal role of evidence in decisions.Source: §7.1.1 · p. 12

7.1.2 Perspectives.

  • Future work includes intervention tests of whether decisions change when cited acoustic structures are removed, and transfer to field recordings.The discussion also proposes calibrated confidence estimates and richer physical priors, including sound-speed profiles and bathymetry.Source: §7.1.2 · pp. 12, 14

7.2 Conclusion

  • Part I couples physics-aware acoustic encoding with a pretrained LLM and uses UPT to learn task-disentangled physical representations from complementary acoustic inputs.The paper reports traceable dual-granularity supervision, attribute-level evaluation, and accurate recognition and physical-attribute estimation on simulated recordings of four vessels.Source: §7.2 · p. 14

Related papers

Loading 2610.22064v1…