Source-linked AI summary
Bridging Neural Representations and Symbolic Semantics in Audio Classification
Avishek Das, Pascal Hitzler
TL;DR
Deep audio classifiers are difficult to interpret internally, and common audio explanations focus on input regions or features rather than hidden-neuron representations. This paper combines symbolic acoustic tags with hidden-layer activations for concept induction and validation. Across genre and instrument classification, several neurons are associated with meaningful induced concepts.
Problem
Audio explanations often emphasize input-level attribution, offering limited insight into what hidden neurons or latent representations encode in human-understandable terms.
Source: §1 · p. 1Method
The pipeline maps acoustic features to symbolic tags, then combines those tags with hidden-neuron activation sets for concept induction and held-out audio validation.
Source: §1 · p. 2, §3.2.3 · p. 5, §3.2.4 · p. 5Results
Takeaways & Limitations
Symbolic concept induction offers a direction for analyzing latent audio representations beyond input-level attribution.
Source: §5 · p. 7Concept quality depends on thresholded feature-to-tag mappings, whose tags are proxy descriptors rather than exact psychoacoustic measurements.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Deep audio classifiers can achieve strong predictive performance, but their hidden representations remain difficult to interpret. Existing audio explainability methods often focus on input-level attribution, such as waveform segments, spectrogram regions, or acoustic features, and provide limited insight into what hidden neurons encode. Interpreting hidden neurons at the concept level can provide a more meaningful view of what an audio model learns internally. This paper investigates whether hidden neurons in audio classifiers can be associated with human-understandable symbolic acoustic concepts. A model-agnostic neurosymbolic framework is presented for analyzing the final hidden layer of audio classifiers, containing 32 neurons, through symbolic concept induction. The framework is evaluated on two audio classification settings: music genre classification using GTZAN dataset and musical instrument classification using IRMAS dataset. Results show that several hidden neurons can be associated with meaningful induced concepts, and most confirmed neuron-concept pairs are statistically supported using the Mann-Whitney U test. These findings suggest that symbolic concept induction is a promising direction for explaining latent audio representations beyond input-level attribution.
1 Introduction
Audio classifiers are difficult to interpret internally, while common audio explanations emphasize influential input regions or features rather than what hidden units represent. This paper proposes symbolic concept induction for audio neurons and evaluates the pipeline across two classification settings, with independent-data validation.
1 Introduction
- Audio predictions can depend on subtle patterns across time, frequency, and latent channels, making deep classifiers difficult to understand.The paper notes this need for explanations in settings including healthcare, wildlife monitoring, accessibility, and urban safety.Source: §1 · p. 1
- Input-level attribution can locate influential audio regions but does not explain what hidden units or latent representations encode in human-understandable terms.Audio concepts may include temporal, spectral, rhythmic, and timbral patterns, not only identifiable sound sources.Source: §1 · p. 1
- The proposed neuron-level pipeline combines hidden-layer activation analysis with symbolic concept induction and maps numerical acoustic descriptors to human-understandable tags.The tags provide the symbolic input for inducing concepts associated with individual neurons.Source: §1 · p. 2
- The pipeline is evaluated on music genre and instrument classification, and induced neuron concepts are checked using independent audio data and statistical testing.The stated aim is to assess whether hidden neurons can be associated with interpretable acoustic concept expressions.Source: §1 · p. 2
2 Related Work
Audio XAI has largely explained predictions through input or feature attribution, while concept-based and neuron-level methods have mainly been developed for vision and, to a lesser extent, text. This paper applies neurosymbolic concept induction to audio classifiers to investigate interpretable symbolic descriptions of hidden neurons.
2.1 Explainable AI for Audio
- Many audio XAI methods explain predictions through waveform, spectrogram, MFCC, or feature attributions rather than describing what hidden neurons represent.Prior work includes LIME, SHAP, Grad-CAM, DeepLIFT, and LRP, while prototype-based sound classification offers a different direction.Source: §2.1 · p. 2
2.2 Concept-Based and Neuron-Level Explainability
- Concept-based and neuron-level explanation methods have focused mainly on vision, with related hidden-neuron approaches also applied to text, leaving audio comparatively unexplored.Existing approaches may depend on predefined concepts, manual annotation, visual datasets, or vision-language representations; this paper applies neurosymbolic induction to audio neurons.Source: §2.2 · p. 2
3 Methodology
The methodology has two stages: first, raw audio is converted into human-understandable symbolic tags; second, those tags are combined with hidden-layer activations to induce and validate neuron-level explanations.
3 Methodology
- The methodology is organized into two major stages.The first constructs symbolic tags from raw audio, and the second uses those tags with hidden-layer activation patterns.Source: §3 · p. 3
- Stage one converts raw audio samples into human-understandable symbolic tags.Source: §3 · p. 3
- Stage two uses symbolic tags and hidden-layer activation patterns to induce and validate concept-level explanations for individual neurons.Source: §3 · p. 3
3.1 Stage 1: Audio-to-Symbolic Tag Construction
The first methodology stage bridges raw audio and concept induction by turning audio segments and numerical acoustic descriptors into symbolic tags. The study uses two classification settings and threshold-based mappings to construct those tags.
3.1 Stage 1: Audio-to-Symbolic Tag Construction
- The tag-construction stage bridges low-level audio signals and high-level acoustic descriptions because concept induction does not operate directly on waveforms or numerical features.The illustrated sequence segments recordings, extracts acoustic features, and applies statistical thresholding to map feature values into symbolic tags.Source: §3.1 · p. 3, Figure 1 · p. 3
3.1.1 Dataset
- Genre analysis trains and induces concepts on GTZAN, then validates on FMA Large; instrument analysis uses IRMAS for training, induction, and validation.Both settings select five classes and use consistent 3-second audio chunks.Source: §3.1.1 · p. 3
3.1.2 Feature Extraction
- Each 3-second segment is represented by 57 numerical acoustic features, including spectral, energy, harmonic, tempo, and MFCC descriptors.For most descriptors, both mean and variance capture average behavior and its variation over time.Source: §3.1.2 · p. 3
3.1.3 Statistical Thresholding and Symbolic Tag Mapping
- Feature thresholds are estimated on training data at the 25th and 75th percentiles; values beyond those cutoffs receive low- or high-level tags, while middle values remain untagged.Thresholds are fixed when applied to test data; the resulting tag names are proxy labels for relative acoustic behavior, not exact psychoacoustic measurements.Source: §3.1.3 · pp. 3–4, Table 2 · p. 4
3.2 Stage 2: Concept Induction Pipeline
The pipeline analyzes the classifier’s final hidden layer by separating neuron activations, inducing symbolic concepts from acoustic tags, and validating the resulting explanations on held-out audio.
3.2.1 Model Training
- A three-layer MLP trained on 57 acoustic features uses its 32-neuron final hidden layer as the focus for neuron-level explanation.The network has 256, 128, and 32 neurons across its hidden layers and is trained for 100 epochs.Source: §3.2.1 · p. 4
3.2.2 Neuron Analysis
- Neuron-specific positive and negative sets use thresholds of 0.60 and 0.20 times each neuron’s maximum test activation, while intermediate examples are excluded as neutral.The cutoffs follow prior activation-discretization work and are described as heuristic choices for obtaining strong neuron-concept associations [25].Source: §3.2.2 · pp. 4–5
3.2.3 Concept Induction
- DL-Learner searches a flat OWL knowledge base for tag-based class expressions that cover positive examples while excluding negative examples.Each audio segment is an OWL individual, each symbolic tag an OWL class, and the resulting expression is passed to validation.Source: §3.2.3 · p. 5
3.2.4 Concept validation
- An induced expression is confirmed when at least 50% of its target-group validation samples trigger the corresponding neuron, after which target and non-target activations are compared using the Mann-Whitney U test.Genre validation uses FMA-large; instrument validation uses a separate IRMAS subset not used in training or concept induction.Source: §3.2.4 · p. 5
4 Results
Across genre and instrument classification, several final-layer neurons were associated with symbolic acoustic concepts, and most confirmed pairs showed statistically supported activation differences. The analysis covers 32-neuron hidden layers and notes limits in the feature-to-tag mapping, ontology, thresholds, and model scope.
4 Results
- The MLP achieved 0.81 accuracy, precision, recall, and F1 on instrument classification and 0.94 for each metric on genre classification.The paper treats these scores as evidence of useful audio representations, while focusing subsequent analysis on neuron-concept associations rather than classifier architecture optimization.Source: §4 · p. 5
- At p < 0.05, the activation-difference null was rejected for 7 of 9 confirmed genre neurons and 9 of 10 confirmed instrument neurons.The exceptions were genre neurons 21 and 12 (p = 0.0636 and p = 0.5642) and instrument neuron 17 (p = 0.1354).Source: §4 · p. 6, Table 5 · p. 7, Table 6 · p. 7
- Confirmed neurons in both tasks had positive trigger percentages of 50%–80%, with generally lower negative trigger percentages.The final hidden layers each contained 32 neurons; four instrument-task neurons were excluded for zero activation, leaving 10 confirmed among 28 remaining neurons.Source: §4 · p. 6, Table 3 · p. 6, Table 4 · p. 6
- Effectively labeled and validated neurons were reported to fall in a range comparable to related hidden-neuron studies, particularly for instrument classification.The authors note that audio concepts are less directly grounded than visual objects or text terms.Source: §4 · p. 7, Table 7 · p. 7
- The study analyzes only an MLP’s final hidden layer; concept quality depends on thresholded acoustic features and their feature-to-tag mapping.The authors also identify the flat tag-based ontology and heuristic activation thresholds as limitations; changing thresholds may produce different expressions.Source: §4 · p. 7
5 Conclusion and Future Work
The paper presents a pipeline that maps numerical acoustic features to symbolic tags and uses hidden-layer activations for neuron-level concept induction. Experiments associate several neurons with meaningful concepts, motivating broader evaluation and richer semantic resources.
5 Conclusion and Future Work
- The pipeline maps numerical acoustic features to human-understandable tags, then applies concept induction to hidden-layer activation patterns.It is presented as a neuron-level symbolic explanation approach for audio classifiers.Source: §5 · p. 7
- Experiments on genre and instrument classification associate several hidden neurons with meaningful induced concepts.Source: §5 · p. 7
- Future work should test robustness across more datasets, architectures, and audio representations, and assess concept usefulness with larger validation sets and audio experts.The paper also proposes richer semantic relations and alternative activation thresholds.Source: §5 · p. 7
A Confirmed Neurons Based on Positive/Negative Trigger Percentage
Appendix Tables 8 and 9 present neuron trigger counts and best-matching expressions for the genre and instrument classification tasks.
A Confirmed Neurons Based on Positive/Negative Trigger Percentage
- Appendix Table 8 contains neuron trigger counts and best-matching expressions.Source: Table 8 · p. 11
- Appendix Table 9 contains neuron trigger counts and best-matching expressions.Source: Table 9 · p. 12