Mitigating Convergence Collapse in Fixed-Target Anomaly Detectors via Kernel-Anchored Locality Regularization

1 Introduction

• At convergence, fixed-target networks can match their target off-distribution, shrinking anomaly residuals as squared-error loss falls and AUROC degrades.The paper reports this guarantee for backbones whose training covers the off-data region and empirical evidence for clean-input backbones.Source: §1 · pp. 1–2
• Reconstruction, contraction-matching, and flow-matching detectors show the same early-peak decline toward convergence, making their signal rely on implicit non-convergence.The implicit controls include early stopping and capacity caps.Source: §1 · p. 2, Figure 1 · p. 2
• Locality is the missing constraint: classical detectors use bounded neighborhoods, whereas fixed-target neural detectors can extrapolate freely off the training data.The paper describes collapse as the absence of a locality constraint.Source: §1 · p. 2
• KCM replaces the learned contraction map with a kernel-weighted average of training targets, giving a closed-form, training-free CPU method that matches established neural baselines on ADBench.Its local influence prevents collapse: predictions level off as test points leave the data, so residuals grow on far anomalies.Source: §1 · p. 2
• KAR penalizes neural predictions that deviate from KCM across the input domain, keeping scores near the collapse-immune anchor during prolonged training.Its tube width is the only knob; the paper reports mitigation across collapse-prone ADBench datasets and three backbones.Source: §1 · p. 2
• Across all 47 ADBench datasets and three backbones, KAR removes collapse at no accuracy cost against its anchor and improves on it on collapse-prone datasets.The paper distinguishes this benchmarkwide contribution from an accuracy gain over KCM.Source: §1 · pp. 2–3

2 Related Work

• Deep tabular detectors use reconstruction error, distance to a compact centre, pretext consistency, denoising signals, or fixed-target flow residuals to score anomalies.The paper studies a failure shared by the fixed-target template rather than proposing another detector family.Source: §2 · p. 3
• Classical detectors remain strong tabular baselines, with several deriving scores from nearby samples or kernels rather than a parametric map free to extrapolate.The paper contrasts neighborhood methods with whole-sample rankings from isolation forests and COPOD.Source: §2 · p. 3
• Prior work documents training-related failures in contaminated-data detectors, reconstruction autoencoders, and Deep SVDD, while this paper attributes fixed-target collapse to missing locality.Existing responses include stopping criteria, autoencoder-specific mechanisms, and architectural restrictions or regularizers.Source: §2 · p. 3
• Table 1 organizes detector families by what constrains off-data scores, identifying fixed-target methods as the family that can collapse.Source: Table 1 · p. 3
• The paper uses the lazy-training link between wide networks and kernel regression to identify a fixed-target smoother that retains locality, applying kernel smoothing to the contraction target rather than data density.This reverses the usual use of neural tangent kernel theory, which is commonly used to study generalization.Source: §2 · p. 3

3.1 The fixed-target template

• Fixed-target detectors lack an off-data score floor because their targets do not encode normal-data density, unlike bounded-neighbourhood or density-coupled approaches.The section identifies explicit locality and density-coupled targets as contrasting families.Source: §3.1 · pp. 3–4

3.2 The collapse, formally

• At a zero-loss optimum capable of realizing the fixed target, continuity forces the residual to vanish across the training distribution’s support, giving AUROC 1/2 there.Normal and anomalous points in that support receive the same score.Source: §3.2 · p. 4
• The formal guarantee covers contraction and rectified-flow detectors when their noised training distribution’s support includes off-manifold anomalies; the autoencoder and Deep SVDD claim there is empirical.For the latter methods, collapse off the clean normal cloud is not proved without an additional regularity assumption.Source: §3.2 · p. 4
• Across six ADBench datasets, residual-score detectors peak early and then degrade; the gap is largest on mid-difficulty datasets and spans backbones and datasets.On 7_Cardiotocography, nine detectors across three seeds show falling AUROC despite decreasing training loss.Source: §3.2 · p. 4, Table 2 · p. 4
• The best stopping epoch varies by more than two orders of magnitude and depends on the seed, while monotonically decreasing training loss can push training toward collapse.The section therefore argues that training loss is a misleading stopping signal for these detectors.Source: §3.2 · p. 4
• Convergence collapse is a population-objective failure, not ordinary overfitting: the optimum itself can have no anomaly signal, independently of train–test generalization.Along the contraction trajectory, held-out-normal and training residuals differ by a median of 0.01.Source: §3.2 · p. 4

4 Kernel Contraction Matching

• KCM replaces a trained fixed-target map with a kernel-regression estimator of the same target, producing a closed-form detector with no optimization trajectory to degrade along.For contraction matching, the estimator uses an RBF kernel and predicts a kernel-weighted average of training normals.Source: §4 · pp. 4–6
• Early stopping acts as implicit regularization in the lazy limit: more training weakens the spectral shrinkage filter, while KCM regularizes through finite kernel bandwidth.The per-dataset stopping epoch is therefore an implicit regularization choice.Source: §4 · p. 5
• The trained detector and KCM are both linear smoothers of fixed targets, but use different weights: tangent-kernel weights for the former and Nadaraya–Watson weights for KCM.KCM is not the lazy limit; its finite-bandwidth kernel supplies a distinct form of regularization.Source: §4 · p. 5
• KCM’s kernel anchor is a constraint set, not a claim that the lazy limit describes trained networks; how much feature learning survives within the regularizer’s tube is unmeasured.The paper identifies empirical-tangent-kernel drift along training as a way to quantify that missing measurement.Source: §4 · p. 5
• KCM selects its bandwidth by trimmed leave-one-out reconstruction error on training normals, with the largest 10% of errors discarded to limit contamination’s influence.The procedure uses no anomaly labels, validation split, or gradient steps.Source: §4 · pp. 5–6
• For every positive bandwidth, KCM’s reconstruction lies in the convex hull of training normals, so its anomaly score is at least the query’s distance to that hull and is positive outside it.The bound grows without bound as a query leaves the bounded hull.Source: §4 · p. 6

• The semi-supervised evaluation fits KCM on normal samples and scores held-out normal-anomaly mixtures across 47 datasets from varied domains and scales.Features are standardized per coordinate; means are averaged over three seeds, and baseline results come from the TCCM benchmark.Source: §5 · p. 6
• KCM reaches this benchmark position with a closed-form fit and no gradient steps, epochs, GPU, early-stopping search, or labels.For N training normals and M test points, bandwidth selection costs O(N^2D) and scoring costs O(NMD); a reference subsample bounds large-N fitting.Source: §5 · p. 6
• 0.871 mean AUROC puts KCM first among 46 label-free detectors across 47 ADBench datasets.DTE-NP scores 0.865 and LUNAR 0.860; KCM ranks third on mean AUPR at 0.644.Source: §5 · p. 6, Figure 3 · p. 7
• KCM shares the leading statistical group with DTE-NP and LUNAR, and is clear of the other compared methods.The Nemenyi diagram uses CD = 2.43; the reported margins over KDE and TCCM are +0.026 and +0.028.Source: §5 · p. 6, Figure 4 · p. 7

6 Kernel-Anchored Regularization

• KAR retains a trainable neural map while using KCM as its anchor, addressing the fact that KCM itself has no trainable parameters.For contraction matching, the anchor is the kernel-weighted average itself; the bounded correction allows limited movement from that prediction.Source: §6 · p. 7, §6.2 · p. 8, §6.1 · p. 8

6.1 Definition

• KAR keeps the neural map within a radius-ρ tube around KCM, with the penalty equivalent to optimizing under that hard constraint.The constraint applies across the whole input space; at ρ=0 KAR reproduces KCM, while as ρ grows it approaches the vanilla detector.Source: §6.1 · pp. 7–8, §6.2 · p. 8

6.2 Propositions

• For the contraction target, KAR’s score exceeds distance to the training-normal convex hull minus ρ, so it cannot vanish sufficiently far outside that hull.The collapsed zero-score solution is outside the hypothesis class for every finite ρ.Source: §6.2 · p. 8

6.3 Experiments

• On six collapse-prone datasets, ρ=0.5 limits the gap to +0.004, while removing the regularizer restores collapse and gaps reach +0.044.At the best radius, KAR improves over KCM on three datasets, including +0.019 on annthyroid and +0.028 on Pima.Source: §6.3 · p. 8, Table 4 · p. 9
• The unchanged default ρ=0.5 costs a mean 0.0049 AUROC and median 0.0007 versus the oracle radius over 47 datasets.It is within 0.001 of the oracle on five of six collapse-prone datasets; eight of 47 datasets lose more than 0.005.Source: §6.3 · pp. 8–9
• At ρ=0.5, KAR keeps the peak-to-final gap below 0.02 on 42 of 47 datasets; without the regularizer, 14 datasets exceed that threshold.Mean final AUROC at ρ=0.5 is within 0.001 of the ρ=0 smoother; the removed-regularizer limit falls more than 0.01 below the anchor on 17 datasets.Source: §6.3 · p. 9, Figure 5 · p. 9
• Across TCCM, autoencoder, and Deep SVDD backbones, final AUROCs have a cross-backbone spread at most 2.2 × 10^-2 on five competitive datasets.The median spread is 2.4 × 10^-3, consistent with the proposition that KAR’s loss and score do not depend on the fixed target.Source: §6.2 · p. 8, §6.3 · p. 9
• Through 500 epochs, the anchored detector’s gap stays at most +0.0022 on four datasets, while the unconstrained gap reaches +0.018 to +0.069.The anchored final AUROC remains above both KCM and the unconstrained limit.Source: §6.3 · p. 9

7 Discussion

• KCM matches the strongest label-free detectors on ADBench, with mean AUROC about 0.88 for D≤10, 0.87 in the middle range, and 0.85 for D>50.Its mean-AUROC lead over the next methods is a tie within run-to-run noise.Source: §7 · p. 9
• KCM’s O(N^2D) fit is the main limitation; capping N0 at 2,000 makes fitting constant in N at a measured cost within ±0.01 AUROC.Source: §7 · p. 9
• KAR matches the anchor across the benchmark and cannot sharpen near-chance anchors; clustered, conditional, and in-hull anomalies were not separately evaluated.The necessity of locality is also unshown, and whether other locality constraints work equally well is untested.Source: §7 · p. 9

8 Conclusion

• KAR holds accuracy across all 47 ADBench datasets and three backbones under prolonged training, and improves on the kernel anchor where data are collapse-prone.Source: §8 · p. 9
• A backbone can swing by 0.1 AUROC with stopping epoch; where KCM matches the best neural detector, deep methods pay training cost without accuracy gain.Source: §8 · pp. 9–10
• Over 500 epochs, KAR at ρ=0.5 stays flat across four of six datasets, while removing the regularizer (ρ=100) lets the backbone drift.The figure reports means of three seeds with shading of ±1 standard deviation.Source: Figure 6 · p. 10

A Proofs

• In the infinite-width limit, constant tangent kernels make training evolve as a linear model; the proof propagates the training-input flow to a query and takes the time limit.For vector targets, the flow decouples into scalar copies under the standard NTK parameterization.Source: A Proofs · p. 10
• Because R_ρ is zero on C and infinite off C, adding it changes neither L’s value nor its infima on C.Source: A Proofs · p. 10
• The Proposition 5 derivation yields squared and norm expressions for the deviation that depend on g_θ, r_kcm, and the input distribution, but not T_B.Source: A Proofs · p. 10

B Backbone independence

• Table 5 reports KAR through three fixed-target backbones at ρ=1 and 100 epochs, using three-seed means and a spread defined by the maximum pairwise gap across seeds.Source: B Backbone independence · p. 10, Table 5 · p. 10
• TCCM and Deep SVDD share the same loss and score in exact arithmetic, while the autoencoder route uses the sign reparameterization g_θ→−g_θ.The passage identifies vertebral as the Scope case.Source: B Backbone independence · p. 10

C Reproduction

• 47 ADBench datasets [12] are evaluated using standardized semi-supervised splits with 50% of normals reserved for training.Features are z-scored.Source: C Reproduction · p. 10
• The KAR experiments use a three-layer, width-256 ReLU MLP trained with Adam at 10^-3 and batch size 256.The KCM fit and KAR setup follow Sections 4 and 6.3, respectively.Source: C Reproduction · p. 10
• Full configurations, run logs, and per-run results behind every table and figure are available at the stated GitHub repository.The passage provides the repository address: https://github. com/jose-melo/kernel-contraction-matching.Source: C Reproduction · p. 10

C Disclosure of Generative AI Use

• For any finite ρ bounding the perturbation gθ, the anomaly score remains bounded below by distance to the convex hull minus ρ, excluding s≡0.The proof combines the reverse triangle inequality with the KCM score bound.Source: Disclosure of Generative AI Use · p. 10
• The proof expresses the detector residual as the KCM residual plus the perturbation gθ, with their norms differing by less than ρ..Source: Disclosure of Generative AI Use · p. 10
• Claude assisted with grammar checking, writing, and figure-generation code; the authors say they produced and verified the theoretical statements, proofs, experimental designs, and results.The authors take full responsibility, and no generative tool is listed as an author.Source: Disclosure of Generative AI Use · p. 10