PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data

Lamine Diop, Marc Plantevit

arXiv:2609.03878v1cs.LGcs.AI

TL;DR

Reconstruction-based anomaly detectors are accurate but often opaque about which feature ranges drive an alert. DIFFINT replaces their entangled bottleneck with differentiable numerical intervals, and across 48 ADBench datasets it achieves the best overall mean rank on both ROC-AUC and AUPR while exposing auditable constraints.

  • Problem

    Accurate reconstruction-based anomaly detectors provide limited feature-range accountability because their latent codes are entangled and do not explain which values caused an alert.

  • Method

    DIFFINT trains an autoencoder on raw numerical data with a soft interval bottleneck, using differentiable hyper-rectangles, reconstruction scoring, clipping-margin analysis, and label-free interval importance.

  • Results

    DIFFINT achieves mean ranks of 4.10 on ROC-AUC and 4.16 on AUPR across 48 ADBench datasets, while remaining competitive across training regimes.

  • Takeaways & Limitations

    The structured bottleneck turns learned intervals into auditable candidate constraints while retaining the reconstruction-based detection paradigm.

  • Takeaways & Limitations

    The method assumes finite feature extrema, stationarity, and complete features; heavy tails, concept drift, and missing-value handling were not evaluated.

Abstract

from arXiv · show

Reconstruction-based anomaly detectors are accurate but opaque: a deep autoencoder flags a sample without telling a practitioner which feature ranges made it anomalous. We propose DIFFINT, an autoencoder whose latent bottleneck is structured as a set of soft, axis-aligned interval memberships learned end-to-end directly from raw numerical data, without any discretization or binarization. Each latent unit corresponds to a human-readable hyper-rectangle in feature space; an instance is encoded by how strongly it falls inside each interval relative to the other units, and its reconstruction error is the anomaly score. This keeps the power of differentiable representation learning while exposing an inspectable internal structure. We make the inductive bias precise: a certified reconstruction-error lower bound for points that fall outside every active coordinate of the learned support (with a Lipschitz-enforced decoder), and a graded, empirically verified suppression mechanism for the usual case in which only a few features are abnormal; and we provide a closed-form, label-free importance that ranks each (unit, feature) pair from quantities the model already maintains, turning trained intervals into auditable candidate constraints without ever seeing an anomaly label. On 48 ADBench benchmarks against 22 baselines under a common [-1, 1]-normalized protocol, DIFFINT attains the best mean rank overall on both metrics (4.10 on ROC-AUC, 4.16 on AUPR); among inlier-only detectors it leads its regime clearly, and it is competitive with the strongest contaminated-data detectors (see the stratified and complete-case analyses). It is the only interpretable detector in the statistically-tied leading cluster of seven methods.

I. INTRODUCTION

DIFFINT addresses the accountability gap in accurate but opaque reconstruction-based anomaly detectors by replacing entangled latent codes with differentiable numerical intervals. It trains on raw features without discretization and evaluates the resulting interpretable detector across broad benchmark and related-method settings.

  • Motivation: Reconstruction-based deep detectors provide scalar anomaly errors but do not identify the feature ranges responsible for a flagged record.Their latent representations are described as entangled and rotationally ambiguous, limiting feature-level accountability.
  • Method: DIFFINT uses soft interval memberships as an autoencoder bottleneck, with each latent unit representing a learnable axis-aligned hyper-rectangle.Centers, softplus half-widths, and sigmoid boundaries make the interval representation differentiable.
  • Method: The model trains on normal data with reconstruction loss and uses reconstruction error as the anomaly score.Its fixed interval-unit bottleneck is a capacity choice rather than a claim of a minimal pattern set.
  • Scope: DIFFINT limits its claim to interpretable anomaly detection rather than minimality, optimal pattern count, or description-length pattern mining.The interval structure is presented as a detector mechanism, not as a general pattern-set mining objective.
  • Method: DIFFINT operates on raw numerical features without discretization and converts interval structure into auditable candidate constraints.This distinguishes the method from classical interval mining and prior differentiable pattern-mining approaches that require discretized or binary inputs.

B. Input normalization to [−1, 1]

Min–max scaling to [−1, 1] is treated as part of DIFFINT’s method because its initialization, temperature, geometric bounds, and two-sided interval behavior are calibrated to that domain.

  • Design rationale: Min–max scaling to [−1, 1]^d is an integral method component rather than interchangeable preprocessing.The bounded symmetric domain supports a single configuration across datasets.
  • Initialization: Scaling aligns random interval initialization with the data support, reducing the need to translate initially off-center or out-of-domain intervals.The default half-widths concentrate around log 2 ≈0.69, covering approximately [−0.7, 0.7] per feature.
  • Temperature: A fixed dimensionless temperature τ yields a consistent sigmoid transition zone because every scaled feature spans length 2.Non-uniform scaling would require feature-specific temperatures and would remove the single-parameter clipping-margin bound.
  • Theory: The bounded domain fixes geometric diameter and supports dataset-independent Lipschitz slack and encoder constants.These properties are used in transferring the method’s theoretical quantities across datasets.
  • Interval geometry: Symmetric scaling balances the two boundary gradients, favoring two-sided interval rules rather than collapsed half-rectangles.The resulting rules are readable as “x_j ∈ [a, b]”.

E. Decoder, training, and EMA support

DIFFINT reconstructs inputs from its interval bottleneck, trains on inliers with RMSE, and maintains an EMA support statistic while scoring test instances with MAE.

  • Decoder and training: A decoder maps the bottleneck representation back to input space, with training minimizing per-sample RMSE on inliers only.The default decoder uses a LayerNorm hidden layer and a linear read-out.
  • Decoder and training: The Certified-Lipschitz decoder spectrally normalizes every linear map and removes non-1-Lipschitz layers to expose an explicit Lipschitz bound.This regularity converts the clipping result into a deterministic certificate.
  • EMA support: DIFFINT keeps an exponential moving average of soft support to obtain a stable dataset-level activation statistic for interval scoring and interpretability.The EMA is a diagnostic statistic and is not part of the forward pass.
  • Anomaly scoring: At test time, MAE(x) is the mean absolute reconstruction error over d scaled coordinates and lies in [0, 2].RMSE is used for optimization while MAE is used as the readable anomaly score.

IV. WHAT THE INTERVAL STRUCTURE PROVIDES

DIFFINT’s interval bottleneck suppresses units whose active coordinates a point violates, with full code collapse and a certified error lower bound in the strongest out-of-support case. The theory distinguishes this guarantee from the broader empirical mechanism used for partial violations.

  • Clipping mechanism: The clipping-margin result is the core mechanism linking out-of-support behavior to reconstruction-based anomaly scoring.Out-of-support points are clipped toward the inlier image, producing separation in reconstruction error.
  • Definitions: The active-coordinate definition identifies interval dimensions that constrain inliers, while inactive coordinates span nearly the full feature range.This distinction determines which coordinates contribute to the clipping and suppression behavior.
  • Theorem 1: A point violating every active coordinate outside the learned support collapses toward the decoder’s fixed empty-code reconstruction.Theorem 1 lower-bounds its error by distance to g_θ(f0) minus a temperature-controlled slack.
  • Corollary 1: Partial violations geometrically suppress a unit according to the number of violated active coordinates, without guaranteeing a fixed reconstruction.The unnormalized weight is bounded by β^s_k, and full collapse occurs only when all active coordinates are violated.
  • Theory versus experiments: The certified lower bound requires a Lipschitz-enforced decoder, whereas the default decoder provides an empirical structural mechanism rather than a guarantee.The certified variant had 0 violations on six datasets but reduced mean AUROC from 0.908 to 0.851.
  • Capacity: The interval representation can approximate compact inlier supports with finite soft box unions, with residual error confined to an arbitrarily thin boundary shell.This is a capacity statement; the operative detection result remains Theorem 1.

V. TURNING INTERVALS INTO AUDITABLE CANDIDATE CONSTRAINTS

DIFFINT turns its inspectable interval bottleneck into label-free, global rankings of which unit–feature pairs form useful candidate constraints. Its explanations remain tied to the detector’s own suppression mechanism rather than a separate attribution model.

  • Label-free importance: LFI scores each (unit, feature) pair globally, while per-instance abnormal features are identified by the active coordinates that suppress matching units.LFI ranks candidate constraints across the model; Corollary 1 links instance-level violations directly to unit suppression and reconstruction error.
  • Label-free importance: LFI combines inlier firing frequency, interval narrowness relative to the feature range, and cross-run repeatability without anomaly labels.The three factors are the EMA soft support, learned half-width, and stability across reruns.
  • Candidate constraints: Units are displayed by summing their top-N feature importances, then retaining the top-N features within the selected unit.This selection rule underlies the interval displays in Fig. 6.
  • Faithfulness: DIFFINT’s violated constraints are exactly the suppressed units that alter reconstruction, avoiding the alignment gap of post-hoc SHAP or residual explanations.The explanation and anomaly score therefore share one mechanism, without requiring a second model to align them.
  • Scope: LFI is a minor, auxiliary, global constraint-ranking tool for unsupervised settings, not a claim of superiority over supervised per-instance attributors.Its formula uses quantities already maintained by the trained model, and its intended use is when anomaly labels are unavailable.

VI. EXPERIMENTS

DIFFINT is evaluated on 48 native-numerical ADBench datasets under a shared normalized protocol, with stratified and complete-case analyses addressing training-regime and missing-run concerns. It leads or co-leads across regimes and metrics, while reconstruction and runtime checks support its numerical interval design and practical cost.

  • Protocol: 48 native-numerical ADBench datasets and 22 baselines are evaluated with shared [−1, 1] normalization, 10 seeds, and one configuration fixed across datasets.DIFFINT and three baselines train on inliers only; the headline claims are restricted to inductive methods in this clean-training setup.
  • Anomaly detection accuracy: DIFFINT leads the inlier-only pool by mean rank 1.69/1.71 of 4 and is first or tied with LUNAR on the contaminated complete-case subset.The complete-case comparison reports an AUPR gap of 0.11 and removes rank imputation.
  • Anomaly detection accuracy: 4.10 is DIFFINT’s best overall ROC-AUC mean rank across 48 datasets, ahead of LUNAR at 6.00, while remaining statistically tied with the seven-method leading cluster.The pooled analysis uses worst-rank imputation for incomplete runs and reports a significant separation between the cluster and approximately 16 remaining detectors.
  • Anomaly detection accuracy: DIFFINT attains the lowest mean rank in 7 of 8 regime-by-metric cells and leads or co-leads every cell after per-scale statistical ties are considered.In the remaining high-dimensional AUPR cell, LUNAR is slightly ahead but within the per-scale Nemenyi critical difference.
  • Reconstruction and cost: DIFFINT trains faster than BinaPs by 1–2 orders of magnitude and faster than LUNAR on most datasets, including shuttle at 194s versus 658s.The BinaPs comparison is explicitly a reconstruction sanity check because BinaPs targets binary data, not numerical reconstruction or pattern-set quality.
  • Bottleneck width: As K grows, inlier reconstruction sharpens while test-outlier MAE remains high; accuracy saturates below K=200, supporting K=200 as a generous default.The paper does not interpret K=200 as a minimal pattern set and reports that halving K can preserve accuracy under tight compute budgets.

D. Ablations: normalization and contamination

The ablations show that bounded symmetric normalization is central to DIFFINT’s performance and transfer, while training contamination causes graceful rather than catastrophic degradation. The certified decoder verifies the clipping bound but trades detection accuracy for guarantees, and learned intervals provide label-free auditable constraints.

  • Normalization: Dropping bounded normalization costs approximately 3 AUROC points, while [−1, 1] achieves 0.833 versus 0.818 for StandardScaler and 0.824 for [0, 1].The bounded symmetric domain supports the theory-driven transfer of one configuration and the clipping bound.
  • Training contamination: AUROC declines from 0.833 to 0.751, 0.735, and 0.720 as contamination increases from 0% to 5%, 10%, and 20%, respectively.The degradation remains graceful because EMA support averages across batches, limiting the influence of a minority of anomalies.
  • Certification: The certified decoder respects the reconstruction-error bound for every tested out-of-support point, while certification reduces mean AUROC from 0.908 to 0.851.The unconstrained decoder remains the default; the certified variant is used when a guarantee is required.
  • Interpretability: LFI-selected intervals form explicit candidate constraints that concentrate inliers and suppress outliers through the clipping margin without anomaly labels.The intervals are intended as auditable constraints rather than minimal or optimal pattern sets.
  • Scale dependence: DIFFINT’s gains are largest on small and medium data, while it remains competitive on large and high-dimensional datasets, including rank 3.18 on AUC and 2.73 on AUPR for large data.On high-dimensional ROC–AUC, DIFFINT ranks 5.25 versus 5.75 for LUNAR; the high-dimensional AUPR difference is statistically tied.
  • Scope: Heavy-tailed features, concept drift, and missing values remain outside the evaluated assumptions, and robust normalization, drift handling, and missing-aware membership are unevaluated.The current implementation assumes complete features and stationary inlier distributions.

APPENDIX A PROOFS

The appendix proofs establish encoder sensitivity, EMA-support behavior, and the clipping-margin mechanism. Full violations make the bottleneck approach an input-independent empty code, while partial violations geometrically suppress affected units.

  • Clipping-margin proof: For a point violating all active coordinates, the shared input-dependent terms cancel across units, forcing the softmax code toward the fixed empty-activation code.The deviation is bounded as O(|V|β), and decoder Lipschitzness transfers code collapse into a reconstruction-error lower bound.
  • Partial violations: With s_k violated active coordinates, unit k’s unnormalized weight is bounded by β^s_k, producing geometric suppression under partial violations.This graded result covers the realistic case in which only a few features are abnormal.
  • EMA support: The EMA support recursion has a unique stationary moving-average limit whose expectation equals the stationary support mean and whose transient bias decays as ρ^t.The effective sample size is (1+ρ)/(1−ρ) under the stated independent-noise calculation.
  • Sensitivity and decoder regularity: The encoder is globally Lipschitz on [−1, 1]^d with constant K√d/τ, expressed directly through the number of units, dimension, and sigmoid temperature.Spectral normalization of the decoder then supplies the regularity used by the clipping theorem.

C. Proof of Proposition 1 (capacity)

The capacity argument supports approximating compact measurable regions with differentiable interval units, while the benchmark protocol evaluates DIFFINT across broad numerical-data scales under one shared configuration and common normalization.

  • Capacity proof: Any compact Jordan-measurable region can be approximated in L1 by a finite union of axis-aligned dyadic boxes.The approximation error is controlled by selecting boxes whose symmetric difference with the target region has arbitrarily small measure.
  • Capacity proof: As τ approaches zero, sigmoid interval memberships converge almost everywhere to box indicators, yielding uniform convergence away from box boundaries.Dominated convergence and boundary refinement provide the final approximation guarantee.
  • Evaluation scope: The evaluation spans 48 ADBench datasets grouped into 12 small, 15 medium, 11 large, and 10 high-dimensional datasets.The suite ranges from 80 to 6.2×10^5 samples and from 3 to 1,555 features.
  • Protocol: DIFFINT uses one [−1, 1]-normalized configuration shared across all datasets, with inlier-only training and no per-dataset hyperparameter tuning.The protocol uses a 40% test split and 10 random seeds.
  • Baselines and metrics: The comparison contains 22 baselines plus DIFFINT, evaluated with ROC–AUC and AUPR under common [−1, 1] normalization.The baselines include classical, deep, inductive, and flow-matching detectors.

APPENDIX D ROBUSTNESS ABLATIONS

The robustness studies support the shared temperature and ranking conclusions while clarifying the limits of the strict certification hypothesis. They also assess decoder Lipschitz constants, inference clamping, and stability of label-free feature importance.

  • Missing-run handling: DIFFINT retains the lowest mean rank under the paper’s missing-run rule, complete-case analysis, and median-rank imputation.The smallest gap is 0.05 above LUNAR for AUPR under scheme B, and the top-cluster conclusion is unchanged.
  • Temperature: The default τ=0.1 remains within 0.06 AUC of the best temperature on every tested dataset, while τ=0.05 substantially hurts GLASS.Performance is robust across τ∈[0.1, 0.5], supporting one shared temperature.
  • Certification coverage: The strict full-violation hypothesis is dataset-dependent: it covers 100% of GLASS anomalies at θ=0.9 but 0% of CARDIO anomalies at θ=0.9 and 16% at θ=0.7.Remaining anomalies receive graded suppression without the full reconstruction-error bound.
  • Certified decoder: Certified decoder layer norms remain near 1, with L between 1.03 and 1.08, keeping the clipping-bound slack small.The ReLU contributes exactly 1-Lipschitz behavior.
  • Inference clamping: Inference clamping changes AUC by at most 0.012 and never reorders the leading score on the full test set.The comparison isolates clipping from the scaling transformation itself.
  • LFI stability: LFI feature rankings show mean cross-seed Spearman correlations of 0.74 on GLASS, 0.88 on IONOSPHERE, and 0.92 on CARDIO.The rankings are computed across ten seed pairs after aggregating per-feature LFI over units.

APPENDIX F INTERPRETABILITY AS EVIDENCE

The appendix tests whether DIFFINT’s label-free global feature ranking is faithful, stable, concise, and comparable to post-hoc explanations within its axis-aligned regime. Results indicate that LFI recovers relevant coordinates and closely matches expensive attribution rankings without model queries.

  • Faithfulness and readability: Top-LFI boundary perturbations raise anomaly scores 1.5× more than random-feature perturbations on 69% of inliers.The perturbation test evaluates whether the highlighted constraints drive the score rather than merely correlate with it.
  • Faithfulness and readability: Approximately half the features, 10.8 of 20, account for 80% of a flagged point’s anomaly score, while intervals remain candidate constraints rather than crisp rules.Mean interval width is 0.70 of the feature range, and units softly constrain most coordinates.
  • Scope: LFI is scoped to marginal, axis-aligned anomalies and does not diagnose anomalies visible only through feature interactions.The appendix evaluates strictly inside DIFFINT’s documented design regime.
  • Quantitative evidence: 1.00 sep-AUC shows that LFI and every compared ranking separate relevant coordinates from irrelevant ones in the controlled axis-aligned regime.The study uses three controlled families with seven retrainings, where anomalies violate known tightly constrained coordinates.
  • Quantitative evidence: Spearman ρ = 0.69–0.74 shows that LFI reproduces the same ordering as aggregated KernelSHAP, integrated gradients, and gradient rankings.LFI reads the model parameters directly, whereas gradient- and sampling-based methods use up to 120 model passes per pooled anomaly.
  • Quantitative evidence: Cross-seed rank stability is ρ = 0.71 for LFI, comparable to KernelSHAP at 0.67 and gradient methods at 0.66–0.69.A randomized-model sanity check gives ρ ≈0.0, indicating the ranking reflects learned structure rather than architecture alone.

APPENDIX H COMPLEXITY AND SCALABILITY

DIFFINT has MLP-class resource demands with interval parameters scaling linearly in feature count and unit count, while avoiding pairwise, kernel, or neighbor computations. Its evaluation remains strongest among inlier-only detectors, but benchmark coverage excludes several newer detector families and robust normalization.

  • Parameterization and complexity: 2Kd interval-bottleneck parameters are linear in K and d and independent of the training-set size n.The decoder adds a compact MLP, keeping total model size in the MLP class even at d=1,555.
  • Parameterization and complexity: O(E nKd) total training complexity uses one decoder pass and no pairwise, kernel, or neighbor computation.A minibatch performs soft-membership computation, per-unit aggregation, and decoder forward/backward operations.
  • Runtime and deployment: DIFFINT trains one to two orders of magnitude faster than BinaPs, faster than LUNAR on most datasets, and scales to n≈6.2×10^5.It uses one forward pass per sample, while memory-based detectors require data-dependent neighbor queries.
  • Runtime and deployment: Reducing K below the default can preserve accuracy under tight budgets because ROC–AUC saturates after a few interval units.The paper retains K=200 as one untuned default across all 48 datasets.
  • Evaluation scope: On complete cases, DIFFINT and LUNAR are effectively tied at 3.59 versus 4.03 on AUC and 3.74 versus 3.85 on AUPR.This analysis removes worst-rank imputation for unfinished runs and preserves the conclusion that DIFFINT is competitive with contaminated-data detectors.
  • Limitations: The benchmark excludes diffusion reconstruction, retrieval or memory-based, and tabular foundation-model detectors, while robust or quantile normalization is not evaluated.The paper identifies controlled comparisons and robust mapping as future work.

B. Average precision (AUPR)

Across AUPR evaluations, DIFFINT remains in the leading group across dataset scales and is usually first on small, medium, and large data. High-dimensional results remain competitive, with per-scale diagrams treating the slight LUNAR advantage as a statistical tie.

  • Metric consistency: DIFFINT’s ranking is consistent across ROC–AUC and AUPR, so the reported gains are not an artifact of the chosen metric.AUPR weights the rare anomaly class more heavily than ROC–AUC.
  • Evaluation protocol: The AUPR tables use the same per-dataset highlighting and protocol as the ROC–AUC tables, enabling scale-level critical-difference comparisons.Figures 8–9 support the regime-level claims summarized in Table II.
  • Per-scale results: DIFFINT leads strictly on small, medium, and large data in the AUPR critical-difference diagrams.The diagrams use missing-run worst-rank flooring and bars for non-significant differences at α=0.05.
  • Per-scale results: On high-dimensional data, LUNAR ranks 5.45 versus DIFFINT at 6.40 in AUPR, but the gap lies within the per-scale Nemenyi CD of approximately 11.The result is therefore a statistical tie rather than a separated lead.
Loading 2609.03878v1…