PaperlayerView exact revision on arXiv

Source-linked AI summary

HARMONY: Hierarchical Anchor Retrieval on Manifold for Oblivious-source Machinery Acoustic Anomaly Detection

Yuhang Zhang, Yang Li, Qiang Wu, Mingyuan Li

arXiv:2610.31093v1cs.SD

TL;DR

Metadata-free machinery ASD must monitor mixed machine sounds without reliable machine identifiers, but heterogeneous normal modes overlap in frozen feature spaces. HARMONY repartitions those features into compact anchor-defined regions and reuses the geometry for hierarchical retrieval. On DCASE 2020, it improves mean AUC by 14.60 percentage points over the strongest unified baseline, while hierarchical retrieval achieves a 7.8× CPU speedup with a 0.20-point AUC drop.

  • Problem

    Metadata-free machinery ASD asks one model to detect anomalies across mixed machine sounds without machine identifiers, while heterogeneous normal modes overlap in frozen feature spaces.

    Source: §1 · pp. 1–2

  • Method

    HARMONY learns an anchor-guided projection that reorganizes mixed-source features into compact Voronoi regions and reuses the learned geometry for hierarchical retrieval.

    Source: §1 · p. 2, §4.1 · p. 3, §4.2 · p. 4

  • Results

    HARMONY improves mean AUC by 14.60 percentage points over the strongest unified baseline and achieves 7.8× CPU speedup over exhaustive search with only a 0.20-point AUC drop.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The DCASE 2020 results support explicit geometric partitioning as an approach to improving detection accuracy and retrieval efficiency for machinery ASD.

    Source: §1 · p. 2, §6 · p. 10

    Performance depends on the initial pre-trained features, and a global temperature can be suboptimal for strongly imbalanced modes.

    Source: §6 · p. 10

Abstract

from arXiv · show

In standard acoustic anomaly detection (ASD), models are usually trained separately for each machine or domain using explicit metadata. However, in realistic deployments, machine identifiers are incomplete, unreliable, or simply unavailable. Thus, metadata-free machinery ASD asks a single model to monitor mixed machine populations without machine IDs. In this paper, we argue that the main failure mode is not a weak detector family, but representation entanglement. The heterogeneous normal modes overlap in frozen pre-trained feature spaces, making anomaly scoring and approximate retrieval unreliable. We study this problem directly and propose HARMONY (Hierarchical Anchor Retrieval on Manifold for Oblivious-source acoustic aNomalY detection), an anchor-guided geometric repartitioning framework that reorganizes mixed-source features into compact Voronoi regions and reuses this structure for hierarchical retrieval. On DCASE 2020, HARMONY improves mean AUC from 75.40% to 90.00% (14.60 percentage points) over the strongest unified baseline and from 83.90% to 90.00% over a strong foundation-feature baseline. On a single CPU, its hierarchical retrieval achieves a 7.8x speedup over exhaustive search with only a 0.20 percentage-point AUC drop. These results suggest that incorporating explicit geometric partitioning can improve both detection accuracy and retrieval efficiency for ASD. Code is available at https://github.com/Zzyh0208/HARMONY.

1 Introduction

Metadata-free machinery ASD requires one model to monitor mixed machine sounds when machine identifiers are unavailable or unreliable. HARMONY addresses entangled normal features by repartitioning them into compact regions and reusing that geometry for retrieval.

Introduction

  • Machine identifiers are often incomplete, unreliable, or unavailable, motivating a single model for mixed machine sounds without machine IDs.Conventional ASD commonly trains separately for each machine or domain using explicit metadata [17].Source: §1 · p. 1
  • Pre-trained feature spaces entangle heterogeneous normal modes, leaving nearest-neighbor, density-estimation, and regression detectors to operate on ambiguous geometry.Approximate retrieval such as HNSW speeds access to those same entangled neighborhoods rather than changing their structure.Source: §1 · pp. 1–2
  • HARMONY learns a projection into compact Voronoi regions and reuses the learned topology as an index, coupling partitioning with retrieval.The paper frames the missing ingredient as an explicit geometry-aware partition, rather than another detector head.Source: §1 · p. 2

Contributions

  • HARMONY improves mean AUC by 14.60 percentage points over the strongest unified baseline and by 6.10 percentage points over a foundation-feature baseline, with 7.8× CPU speedup and only a 0.20-point AUC drop.The retrieval comparison is against exhaustive search.Source: §1 · p. 2

2 Related Work

Prior ASD methods use feature-space geometry for scoring, while clustering and unified detection address latent structure or mixed domains. The paper identifies unresolved feature entanglement and retrieval trade-offs in metadata-free mixtures.

Feature-Based ASD

  • Feature-based ASD methods rely on normal-feature geometry, but overlapping distributions from diverse machine types degrade distance-based discriminability in metadata-free mixtures.The described pipeline combines representation learning and density estimation; related approaches include BYOL-A [21], PatchCore [23], SimpleNet [18], and EfficientAD [2].Source: §2 · p. 2

Deep Clustering and Theoretical Foundations

  • Clustering methods organize latent space with prototypes, but general representation learning promotes uniformity while anomaly detection requires compactness.The cited work connects representation learning, clustering, and anomaly scoring [8], while class-aware contrastive learning aims to reduce multi-class confusion [7].Source: §2 · p. 2

Unified Anomaly Detection and Indexing

  • Unified anomaly detection methods address multiple categories or domains, but some rely on computationally costly heavy backbones or flow architectures.Examples include Dinomaly [13], CADA [12], RLR [14], CFLOW-AD [11], UniFlow, and VQ-Flow [29].Source: §2 · p. 2
  • Standard approximate indexing such as HNSW operates passively over frozen features, leaving manifold entanglement unresolved and trading retrieval precision against inference efficiency.This is the related-work contrast motivating geometry-aware indexing.Source: §2 · pp. 2–3

3 Preliminaries

Unsupervised ASD scores deviations from normal training sounds, typically through pretrained embeddings and density-based methods. The metadata-free setting pools heterogeneous acoustic modes without revealing a sample’s mode during training.

Acoustic Anomaly Detection

  • ASD learns a score for deviation from normal machine sounds and flags a query as anomalous when its score exceeds a threshold.Industrial anomalies can manifest as broken periodicity, impulsive noise, or spectral-energy shifts.Source: §3 · p. 3

Feature-Based Unsupervised Pipelines

  • Feature-based pipelines map spectrograms into pretrained embeddings and score anomalies using nearest-normal-neighbor distance or reconstruction error.These methods assume normal samples occupy a compact feature-space region while anomalies lie in low-density areas.Source: §3 · p. 3

Metadata-free Machinery ASD

  • Metadata-free machinery ASD uses one model for a diverse machine fleet, modeling normal training data as a mixture of heterogeneous modes whose identities are unknown during training.The latent mode count is unknown and need not equal HARMONY’s anchor count; machine metadata is used only to group test scores for evaluation.Source: §3 · p. 3

4 Method

HARMONY reconstructs frozen mixed-source features into compact, separated anchor regions, then derives a hierarchical index from that geometry. Its method combines spectral anchor mining, anchor-guided alignment, and coarse-to-fine retrieval, with separation and retrieval guarantees conditioned on stated assumptions.

4 Method

  • HARMONY combines intrinsic subspace estimation, anchor-guided disentanglement, and hierarchical retrieval to address ambiguities in frozen features.The same learned topology supports both repartitioning and retrieval.Source: §4 · p. 3, Figure 1 · p. 4

4.1 Geometric Problem Analysis

  • Overlapping supports of heterogeneous normal modes in frozen feature space break the compactness assumption of distance-based detectors, motivating projection into locally compact, globally separated Voronoi regions.The target projection maps features onto a hypersphere.Source: §4.1 · p. 3

4.2 Spectral Anchor Mining

  • An anchor is a unit-norm prototype for a local acoustic region, not a machine label; spectral reduction filters orthogonal noise before diverse anchors are initialized and mapped back to the input dimension.The retained subspace is selected using covariance eigenvalues and a retained-variance threshold.Source: §4.2 · p. 4

4.3 Anchor-Guided Manifold Disentanglement

  • Anchor-guided alignment uses a temperature-scaled affinity distribution and pseudo-label objective to compact assigned regions through competing attraction and repulsion forces.The separation result is conditional on bounded within-anchor residuals; normal-only alignment does not guarantee anomaly separation, which is evaluated empirically.Source: §4.3 · pp. 4–6

4.4 Hierarchical Expert Indexing

  • Hierarchical retrieval routes queries through selected anchor-defined regions and local buckets, restricting approximate nearest-neighbor scoring to a candidate set.The retrieval-error bound requires candidate coverage of a point in the same radius-ε bucket as a true nearest neighbor; expert selection alone does not ensure coverage.Source: §4.4 · p. 6

5 Experiments

The experiments test whether geometric repartitioning addresses mixed-source entanglement, improves metadata-free anomaly detection, and supports efficient retrieval. Across DCASE 2020 and additional robustness and cross-dataset evaluations, HARMONY improves detection while hierarchical retrieval reduces latency with a small AUC drop.

5.1 Experimental Setup

  • The evaluation pools normal samples from all 23 DCASE 2020 machine IDs without metadata during training and also tests MIMII and DCASE 2022.It uses frozen BYOL-A features; anchor count and temperature are selected with Silhouette Score on pooled normal data.Source: §5.1 · pp. 6–7

5.2 RQ1: Is Representation Entanglement the Main Obstacle?

  • Repartitioning improves AUC for every strong baseline, including gains of 6.30 for Audio-MAE, 9.20 for Audio-PC, and 29.70 points for PatchCore.The findings support mixed-source geometry, rather than detector family alone, as the primary bottleneck.Source: Table 1 · p. 7, §5.2 · p. 7

5.3 RQ2: Does Active Repartitioning Improve Metadata-free machinery ASD Accuracy?

  • 90.00% mean AUC on DCASE 2020 is 14.60 percentage points above UniFlow (75.40%) and 6.10 points above Audio-PC (83.90%).All methods use the same pre-trained features.Source: §5.3 · p. 7, Table 2 · p. 8
  • HARMONY improves over Audio-PC by 4.84 AUC points on MIMII and 4.52 overall points on DCASE 2022, including 4.27 points on its target domain.DCASE 2022 withholds machine metadata from the model.Source: Table 3 · p. 8, §5.3 · p. 8

5.4 RQ3: Are the Gains Attributable and Robust?

  • Removing the projection lowers mean AUC from 90.00% to 62.40%; replacing spectral initialization or soft assignment costs 8.60 and 2.40 AUC points, respectively.Projection also lowers the quantization-error proxy from 0.842 to 0.181 and raises the margin proxy from 0.418 to 0.852.Source: §5.4 · pp. 8–9, Table 4 · p. 9
  • Under dynamic MIMII noise, HARMONY has a flatter performance curve than baselines, while Silhouette Score tracks AUC; normal-only selection stays within 0.19–0.37 AUC points of the retrospective oracle.Across mixtures of two to six machine types, the selected count changes from 4 to 8.Source: Figure 2 · p. 9, §5.4 · p. 9, Table 6 · p. 10
  • At −6 dB DEMAND noise HARMONY reaches 79.42% AUC versus Audio-PC’s 66.11%; at 5% training contamination it reaches 83.31%, down from 89.65% at 0%.On DCASE 2020, anchor assignments have NMI 0.78 and purity 0.89, versus 0.42 and 0.63 for raw-feature KMeans; labels are used only for analysis.Source: §5.4 · p. 10

5.5 RQ4: Does the Learned Partition Also Improve Retrieval Efficiency?

  • On the same CPU, hierarchical retrieval reduces latency from 42.3 ms to 5.4 ms, a 7.8× speedup with a 0.20-point AUC drop.The comparison holds the learned representation and backbone constant.Source: Table 7 · p. 10, §5.5 · p. 10

6 Conclusion

HARMONY reorganizes mixed normal features into anchor-defined regions and reuses that geometry for hierarchical retrieval. On DCASE 2020, hierarchical retrieval reduces CPU latency with a small AUC reduction, while the authors note limits tied to features, temperature, and assumptions.

6 Conclusion

  • On DCASE 2020, flat retrieval reaches 90.00% mean AUC, while hierarchical retrieval reduces CPU latency from 42.3 to 5.4 ms with a 0.20-point AUC reduction.Comparisons on MIMII and DCASE 2022 further support the approach across industrial acoustic conditions.Source: §6 · p. 10
  • HARMONY reorganizes mixed normal features into anchor-defined regions and reuses the geometry for hierarchical retrieval.The conclusion presents the framework as addressing metadata-free machinery ASD.Source: §6 · p. 10
  • Performance depends on initial pre-trained features, and a global temperature may be suboptimal for strongly imbalanced modes.The separation and retrieval bounds also require bounded residuals and candidate coverage, respectively; validation beyond industrial machinery remains future work.Source: §6 · p. 10

NeurIPS Paper Checklist

This section documents empirical-claim validation, limitations, reproducibility materials, and experimental details, and provides notation and mathematical analysis of the method. The analysis derives an attraction–repulsion gradient and a separation bound under an explicit residual condition, without requiring anchors to correspond one-to-one with machine identities.

NeurIPS Paper Checklist

  • The checklist says the paper’s stated accuracy and retrieval-efficiency claims are empirically validated and that limitations include extreme domain shifts and uniform temperature scaling.The limitations are discussed in Section M.Source: NeurIPS Paper Checklist · p. 13
  • The authors report detailed experimental setups, architectures, baseline adaptations, training configurations, and public code, with means and standard deviations over 5 random seeds.The checklist points to Sections 5, J, and F for these materials and results.Source: NeurIPS Paper Checklist · pp. 14–15
  • The checklist says the paper discusses positive impacts on industrial safety and potential privacy risks from acoustic monitoring, and does not involve human subjects.It also states that standard, publicly available acoustic anomaly detection datasets are used without privacy violations.Source: NeurIPS Paper Checklist · pp. 16, 18

A Comprehensive Notation Table

B Theoretical Proofs and Analysis

  • The theoretical appendix provides derivations for the geometric properties and retrieval bounds in Section 4, plus asymptotic complexity analysis for the efficiency gains reported in Section 5.Source: B Theoretical Proofs and Analysis · p. 20

B.1 Derivation of the gradient decomposition

  • The gradient derivation begins from an anchor-alignment loss, using hypersphere-projected features and temperature-scaled similarities to anchors.For each sample, the alignment loss is defined using the pseudo-labeled anchor.Source: §B.1 · p. 20
  • The negative loss gradient decomposes into attraction toward the assigned anchor and repulsion away from competing anchors.Source: §B.1 · p. 21

B.2 Optimization Assumptions and Residual Bound

  • Smoothness with step size η ≤ 1/L guarantees monotonic loss descent, but does not establish a prescribed loss level or uniform sample-to-anchor residual.The local loss–distance relation needed to infer residuals from loss requires a separate error-bound condition.Source: §B.2 · p. 22

B.3 Proof of the manifold separation bound

  • For samples assigned to distinct anchors, the separation bound assumes each sample lies within δtrain + ηgen of its assigned anchor.The test deviation ηgen is an assumption for the samples covered by the bound, not a guarantee derived from Lipschitz continuity alone.Source: §B.3 · p. 22, §2 · p. 22

2. Bounding the Terms.

  • The separation lower bound is positive when ∆min > 2(δtrain + ηgen), and does not require a one-to-one correspondence between anchors and machine identities.Mean cosine-distance proxies in Table 5 are not estimates of the theorem’s uniform Euclidean residual or minimum anchor distance.Source: §2 · p. 22

B.4 Proof of the retrieval error bound

The retrieval analysis bounds score distortion under a same-bucket coverage condition, describes the routing costs, and reports broad anchor use in evaluated configurations. The index remains asymptotically linear while reducing the constant factor of search.

B.4 Proof of the retrieval error bound

  • Under same-bucket coverage and radius-ε quantization, retrieved distance satisfies s(q) ≤ ŝ(q) ≤ s(q) + 2ε.Top-K1 expert selection alone does not establish the coverage condition.Source: §B.4 · p. 23

B.5 Complexity Analysis of Hierarchical Retrieval

  • Hierarchical routing computes distances to K anchors, then to B centroids in selected experts, before scoring candidates in retrieved buckets.The candidate count is approximated under a uniform-distribution assumption.Source: §B.5 · p. 23
  • With small fixed routing parameters and fixed dimension, the index remains O(N) but reduces the constant factor relative to raw-dataset search.The N-dependent candidate term dominates scaling.Source: §B.5 · p. 23

C Empirical Analysis of Learned Geometry

C.1 Anchor Occupancy and Stability

  • Across evaluated configurations, no anchors are empty and Shannon entropy remains near its theoretical maximum; Figure 3 shows entropy ratio near 1.0 and maximum occupancy tracking the ideal uniform boundary.These statistics indicate broad anchor utilization in the evaluated mixtures.Source: §C.1 · p. 24, Figure 3 · p. 24

C.2 Quantization Error and Geometric Margin

The section evaluates HARMONY’s geometric separation and its consequences for anomaly scoring, then tests the method’s visual organization, anchor meaning, component robustness, and retrieval efficiency.

C.2 Quantization Error and Geometric Margin

  • HARMONY improves the Margin/QE Ratio by nearly 10× over raw features, alongside lower quantization error, a wider margin proxy, and higher detection performance.QE is mean distance to assigned anchors; the Margin proxy is the mean distance difference between the first and second nearest anchors.Source: §C.2 · p. 24

C.3 Optimization Smoothness; C.4 Anomaly Score Distribution

  • Training loss decreases smoothly over 20 epochs, while anomalous test queries have median scores approximately 15× larger than normal queries.Figure 4 shows the observed alignment-loss trajectory; Table 9 reports the normal-versus-anomalous score distributions.Source: §C.3 · p. 24, §C.4 · p. 24, Figure 4 · p. 25

D Visualization of Manifold Disentanglement

  • The projection reorganizes entangled raw features into compact, semantically consistent Voronoi regions in the qualitative visualization.The t-SNE figure contrasts raw features on the left with separated Voronoi regions on the right.Source: D Visualization of Manifold… · p. 25, Figure 5 · p. 25

D.1 Acoustic Meaning of the Anchors

  • Post-hoc machine-type alignment measures improve, while multiple anchors can describe different operating sounds from one machine type.Machine labels are used only for this post-hoc analysis.Source: §D.1 · p. 26, Table 10 · p. 26

E.1 Universality Across Backbones

  • HARMONY improves spatial decoupling and detection performance across pre-trained feature extractors, including Masked SSL and Contrastive models.Table 11 reports the backbone comparison.Source: §E.1 · p. 26, Table 11 · p. 26

E.2 Robustness of Topological Initialization; E.3 Necessity of Push-Pull Assignment

  • Alternative parametric clustering methods retain stable performance, whereas replacing soft assignment with hard labels causes a significant performance drop.The initialization comparison and the soft-versus-hard assignment ablation are reported in Tables 12 and 13.Source: §E.2 · p. 26, Table 12 · p. 26, §E.3 · p. 26

E.4 Efficiency vs. Accuracy Trade-off

  • On a standard CPU, hierarchical IVF retrieval achieves a 7.8× speedup over exhaustive search with a 0.20-point AUC drop.The comparison evaluates retrieval strategies’ efficiency and accuracy trade-off.Source: §E.4 · p. 27, Table 14 · p. 27

F Multi-Seed Stability and Detailed Metrics

The authors assess DCASE 2020 performance across five random seeds and report both AUC and pAUC, with low variability across most machine types.

F Multi-Seed Stability and Detailed Metrics

G Direct Control Experiments

The experiments probe whether HARMONY’s gains come from geometric repartitioning, test robustness under challenging conditions, and evaluate unsupervised configuration selection and deployment efficiency.

G Direct Control Experiments

  • Applying IVF or HNSW directly to raw features yields no performance benefit, while projecting into unbounded Euclidean space substantially degrades results relative to the spherical constraint.These direct controls test approximate indexing on raw features and the unit-hypersphere design choice.Source: G Direct Control Experiments · p. 29, Table 17 · p. 29

G.1 Matched Prototype, Clustering, and Index Controls

  • Shared routing raises nearest-neighbor recall from 97.26% to 98.49%; the matched controls distinguish representation gains from the routing effect.The controls report routed AUC, nearest-neighbor recall, and CPU latency over five seeds.Source: §G.1 · p. 29, Table 18 · p. 29

H Stress Tests under Extreme Conditions; H.1 Recorded Environmental Noise

I Hyperparameter Sensitivity and Unsupervised Selection

  • Performance has broad plateaus across anchor count K, intrinsic dimensionality d′, and temperature τ; the Silhouette Score correlates with test AUC at Pearson ρ = 0.973 across variations and stress tests.The figures report sensitivity and the global correlation across hyperparameter variations and extreme stress tests.Source: I Hyperparameter Sensitivity and… · p. 30, Figure 7 · p. 30, Figure 8 · p. 31

I.1 Selection Across Machine Mixtures

  • Silhouette-based selection uses pooled normal data without anomaly labels and adapts anchor count to mixture composition; on MIMII it selects 12 rather than DCASE’s 8, reducing the oracle gap from 0.63 to 0.27 AUC points.Test-oracle AUC is used only for retrospective analysis.Source: §I.1 · p. 31, Table 20 · p. 31, Table 21 · p. 31

I.2 Long-Tailed Mixtures and Temperature

  • Under evaluated imbalance, increasing imbalance reduces tail performance and occupancy entropy, while a smaller temperature is more effective than on balanced data.The long-tailed analysis selects more anchors and a smaller temperature at IR ≥10; the temperature comparison fixes K = 8.Source: §I.2 · p. 32, Table 22 · p. 32, Table 23 · p. 32

I.3 Rare-Type Absorption

  • At IR = 20, changing K from 8 to the selected 16 reduces rare-type absorption from 0.39 to 0.29 and raises rare-type AUC from 75.91% to 79.36%.Rare-type separation deteriorates with imbalance, although HARMONY retains more rare-type structure than the compared alternatives at IR = 20.Source: §I.3 · p. 32, Table 24 · p. 32

J.1 Network Architecture; J.2 Principled Baseline Adaptation Setup

  • Experiments use frozen BYOL-A features and a projection from 3072 to 128 dimensions; baseline methods use identical features and equivalent hyperparameter budgets.The projection output is L2-normalized, and the baseline adaptations are described separately.Source: §J.1 · pp. 32–33, §J.2 · p. 33

J.3 Training Configuration

The appendices specify training and evaluation procedures, report per-ID and cross-dataset comparisons, and state limitations and broader impacts.

J.3 Training Configuration; J.4 Environment

  • Training uses Adam with weight decay 10^-4 and cosine annealing from 10^-3 to 10^-5; CPU latency is measured single-threaded and includes the frozen BYOL-A feature-extraction pass.Latency is measured on an independent server to avoid resource contention.Source: §J.3 · p. 33, §J.4 · p. 33, Table 25 · p. 34

K.1 Dataset Details; K.2 Universal Unsupervised Protocol Definition

  • The evaluation covers DCASE 2020 Task 2 and MIMII’s −6 dB subset, with a single model trained on pooled normal data without machine labels or IDs.At test time, ground-truth machine labels group scores for macro-average AUC reporting.Source: §K.1 · p. 34, Table 26 · p. 34, Table 27 · p. 34

L Detailed Experimental Results; L.1 DCASE 2020 Per-ID Performance; L.2 MIMII -6dB Per-ID Performance

M Limitations and Broader Impacts

  • HARMONY depends on pre-trained features with meaningful initial local distances, so extreme target-domain shifts beyond pre-training data may make its initial topology suboptimal; uniform τ may also limit margins when mode densities differ greatly.The authors identify density-aware temperature as a potential improvement for highly imbalanced mixtures.Source: M Limitations and Broader Impacts · p. 36
  • The paper identifies potential industrial benefits in safety and predictive maintenance, while noting possible privacy risks if acoustic features capture sensitive background information.It recommends following workplace privacy regulations and not permanently storing raw audio streams.Source: M Limitations and Broader Impacts · p. 36

N.1 MIMII at −6 dB; N.2 DCASE 2022 Domain Shifts

  • On MIMII, HARMONY improves over Audio-PC by 4.84 Macro AUC, 1.90 pAUC, and 4.12 Worst-ID AUC points; on DCASE 2022, gains are 4.52 overall, 4.27 on the target domain, and 4.15 in minimum section AUC.DCASE 2022 withholds machine-type and section identifiers from training and routing.Source: §N.1 · p. 36, Table 30 · p. 36, §N.2 · p. 37

O Robustness to Training Contamination

HARMONY retains an advantage over Audio-PC under training contamination, but contamination still reduces its AUC, leaving the clean-training assumption relevant.

O Robustness to Training Contamination

Related papers

Loading 2610.31093v1…