PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Energy-based Tabular Anomaly Detection: Energy and Reconstruction are Complementary

Junichiro Niimi

arXiv:2608.14186v1cs.LGstat.ML

TL;DR

Tabular anomaly detection mainly uses indirect density and reconstruction proxies, while explicit energy-based models remain underexplored. The paper evaluates a two-hidden-layer DBM across Bank Marketing and NSL-KDD, finding that its energy complements Autoencoder reconstruction and improves their combination. The findings support DBMs as a complementary option, but the evidence remains bounded by two datasets and an incomplete baseline panel.

  • Problem

    Tabular anomaly detectors largely approximate the inlier distribution indirectly, leaving explicit joint energy modeling insufficiently evaluated.

  • Method

    The paper evaluates a two-hidden-layer DBM and its mean-field energy against eight baselines on Bank Marketing and NSL-KDD across twenty random seeds, including fusion with Autoencoder reconstruction.

  • Results

    DBM-derived signals are the only partners that productively augment Autoencoder ensembles across both datasets, with DBM energy significantly improving fusion results.

  • Takeaways & Limitations

    Classical DBMs provide a non-redundant energy-based perspective that complements reconstruction-based anomaly scores in the evaluated tabular settings.

  • Takeaways & Limitations

    Evidence covers two tabular benchmarks and omits several newer tabular-specific, diffusion-based, and transformer-based detectors; generalisation is dataset-dependent.

Abstract

from arXiv · show

Tabular anomaly detection is dominated by classical density-proxy methods (Isolation Forest, OCSVM, LOF), reconstruction-based detectors (Autoencoders, VAEs), and modern non-parametric scorers (COPOD, ECOD, Deep SVDD), all of which approximate the inlier distribution only indirectly; explicit energy-based models are largely absent. Motivated by the recent revival of EBMs in deep learning (e.g., Energy-Based Transformers, JEPA), we revisit the classical Deep Boltzmann Machine (DBM) for this task and hypothesize that its mean-field energy combines more effectively with a reconstruction-based score than same-lineage pairs do. We evaluate a two-hidden-layer DBM on two tabular benchmarks spanning distinct domains (UCI Bank Marketing and NSL-KDD) against eight classical and modern baselines across twenty random seeds. The DBM mean-field energy matches the strongest baseline (the Autoencoder) on Bank Marketing and statistically beats it on NSL-KDD, while significantly outperforming the remaining seven on both datasets. When fused with the Autoencoder via rank fusion, the DBM energy yields a statistically significant improvement on both datasets (AUROC=+0.014, p<0.01 on Bank Marketing; +0.002, p<0.001 on NSL-KDD); every non-DBM-derived base model instead fails to improve or significantly degrades the AE-paired ensemble. Our position is that classical EBMs, exemplified by the DBM, deserve a place in the tabular anomaly detection toolbox as a non-redundant complementary view to the reconstruction-based scores that dominate current practice.

1 Introduction

Tabular anomaly detection relies mainly on indirect density proxies and reconstruction scores, leaving explicit joint energy modeling underexplored. This paper evaluates a DBM as a complementary energy-based view and finds that DBM–AE combinations consistently improve detection across two domains.

  • Tabular detectors predominantly use density proxies, reconstruction error, or non-parametric scores to approximate the inlier distribution indirectly.
  • A DBM supplies a directly evaluable mean-field energy over visible samples, providing an explicit scalar anomaly score based on joint feature compatibility.
  • The study evaluates a two-hidden-layer DBM on Bank Marketing and NSL-KDD against eight baselines using twenty random seeds.
  • DBM mean-field energy ties the Autoencoder on Bank Marketing, beats it on NSL-KDD, and significantly outperforms the other seven baselines on both datasets.
  • DBM energy fused with Autoencoder reconstruction significantly outperforms either single method on both datasets, whereas non-DBM partners do not improve the AE ensemble.

2 Related Work

Related work spans geometric density proxies, reconstruction-based detectors, and renewed energy-based modeling. The paper positions DBMs as a structurally distinct, explicitly energy-based lineage whose anomaly-detection role has lacked systematic tabular evaluation.

  • Isolation Forest, OCSVM, and LOF represent anomalies through isolation, support boundaries, or local-density comparisons rather than explicit likelihoods.
  • Autoencoders and VAEs use bottleneck reconstruction error, with Autoencoders identified as particularly strong anomaly-detection baselines.
  • Existing density-proxy and reconstruction approaches do not provide an explicit differentiable scalar energy over the joint configuration of all input features.
  • Modern EBM work has renewed interest in scalar energy formulations, although training remains challenging because of the intractable partition function.
  • EBM anomaly scores should be treated as one informative view because deep generative models can assign higher likelihood to out-of-distribution samples.
  • DBMs are bidirectional Boltzmann-machine models whose tractable mean-field energy scores visible samples by compatibility with the learned joint distribution.
  • Systematic multi-seed evaluation of DBM energy as a tabular anomaly detector against strong unsupervised baselines remains largely unreported.

3 Experiments

The experiments evaluate a two-hidden-layer DBM and eight established detectors on Bank Marketing and NSL-KDD using consistent multi-seed comparisons. The DBM mean-field energy is competitive with the Autoencoder and improves anomaly detection across the reported benchmarks.

  • Datasets: Bank Marketing contains 45,211 customers with 16 input attributes, while NSL-KDD contains 74,826 connections with 41 input attributes.The datasets are converted to fixed-width binary vectors, yielding D = 69 and D = 144 visible dimensions, respectively.
  • DBM design: The DBM uses one visible layer and two hidden layers with widths H1 = 48 and H2 = 24.This moderate-bottleneck architecture was selected through a preliminary single-seed Bank Marketing ablation.
  • Scoring: The primary anomaly score is Fbefore(v) = E(v, µ), computed after 10 mean-field iterations, with higher values indicating lower model likelihood.The score omits the entropy term because subtracting H(µ) degraded anomaly-detection performance.
  • Evaluation: The DBM is compared with eight detectors spanning density-proxy, reconstruction-based, and modern non-parametric approaches, using the same inlier-only training split.Neural detectors use the full split, while five non-neural methods use a fixed 8,000-sample subsample because OCSVM is O(n^2).
  • Results: Across twenty random seeds, DBM Fbefore ties the Autoencoder on Bank Marketing and beats it on NSL-KDD while outperforming the other seven baselines.On NSL-KDD, the gains over Autoencoder are ∆AUROC = +0.0016, ∆PR-AUC = +0.022, and ∆F1 = +0.010.

3.3 Hybrid scoring: energy and reconstruction are complementary

The DBM mean-field energy and Autoencoder reconstruction error provide complementary anomaly signals: fusing them outperforms either alone, while non-DBM partners do not reliably improve the Autoencoder ensemble.

  • Hybrid scoring: α⋆=0.4 occurs in 18/20 Bank Marketing seeds, while α⋆=0.6 occurs in 17/19 NSL-KDD seeds.The tuned convex combination uses z-standardised scores and leave-one-seed-out tuning.
  • Hybrid scoring: Both fusion variants significantly outperform either single method on Bank Marketing and NSL-KDD.Rank fusion achieves the highest mean AUROC: 0.745 on Bank Marketing and 0.9953 on NSL-KDD.
  • Hybrid scoring: The convex AUROC sweep peaks above both AE-only and DBM-only endpoints on Bank Marketing and NSL-KDD.The peak forms a smooth concave curve, while the shaded band shows ±1 SE and the dashed line marks parameter-free rank fusion.
  • AE-paired comparison: Only DBM-derived partners significantly improve AE-paired rank fusion across both datasets.The tested DBM-derived partners are Fbefore and DBM-reconstruction; every non-DBM partner either fails to improve or significantly degrades the ensemble.
  • AE-paired comparison: Fbefore outperforms DBM-reconstruction on Bank Marketing but matches its boost on NSL-KDD.The Bank Marketing improvements are +0.010 versus DBM-reconstruction, while NSL-KDD reaches the same boost for both DBM scores.
  • Interpretation: DBM-derived scores aggregate joint hidden-unit configuration evidence, whereas AE reconstruction is computed point-wise through a feed-forward bottleneck.This asymmetry allows rank fusion to combine globally plausible but locally unusual samples with locally plausible but globally unusual samples.

3.5 Mean-field energy vs. proper variational free energy

For anomaly ranking, the paper favors mean-field energy over proper variational free energy because entropy subtraction removes discriminative signal, although the effect’s magnitude depends on dataset and architecture.

  • Energy definition: Fproper degrades performance on every metric relative to mean-field energy on Bank Marketing.The gap is borderline-significant for AUROC and significant for PR-AUC and F1.
  • Mechanism: Hanomaly−Hinlier=+0.43±0.02 nats, with p≈3×10^-14 and positive differences in 20/20 seeds.Anomalous inputs elicit less-confident mean-field posteriors, so entropy subtraction removes more from anomaly scores than from inlier scores.
  • Generalisability: The entropy asymmetry reproduces in the depth-1 RBM ablation across both datasets and is therefore not specific to the two-hidden-layer DBM.The asymmetry is +0.42±0.04 nats on Bank Marketing and +1.27±0.11 on NSL-KDD, positive in 20/20 seeds with p<10^-8.
  • Dataset dependence: On NSL-KDD, the penalty is small but significant for both architectures, with ΔAUROC=−0.0003 and p≤10^-3.On Bank Marketing, the penalty is borderline for the DBM and zero within noise for the RBM.
  • Dataset dependence: The magnitude of the energy-versus-free-energy difference remains a per-dataset empirical question.Preliminary runs on further tabular benchmarks observed both signs of the difference.
  • Computational cost: DBM training costs 176 s on Bank Marketing and 468 s on NSL-KDD, roughly 11× and 17× Autoencoder training.Classical density proxies fit in well under a second, while DBM scoring is 2–3× cheaper than Autoencoder scoring.

4 Conclusion

The DBM mean-field energy is a strong standalone anomaly score and a complementary partner to Autoencoder reconstruction, but the evidence remains bounded by the evaluated datasets and baseline panel.

  • Findings: The DBM mean-field energy matches the Autoencoder on Bank Marketing and statistically beats it on NSL-KDD while outperforming the other seven baselines on both datasets.On NSL-KDD, gains over the Autoencoder were ΔAUROC = +0.002, ΔPR-AUC = +0.022, and ΔF1 = +0.010, all with p ≤0.002.
  • Findings: Rank fusion of DBM energy and Autoencoder reconstruction significantly improves anomaly detection on both datasets.The AUROC improvement was +0.014 on Bank Marketing and +0.0021 on NSL-KDD.
  • Complementarity: Only DBM-derived scores significantly improve the Autoencoder-paired ensemble; every non-DBM partner fails to improve or degrades it on both datasets.Among DBM-derived partners, Fbefore is the cleaner joint-configuration view and outperforms DBM-reconstruction by an order of magnitude on Bank Marketing.
  • Conclusion: The results support using energy-based scoring as a complementary, non-redundant view rather than a replacement for reconstruction-based detection.The energy-based view concerns the joint configuration, whereas reconstruction provides a coordinate-wise view.
  • Limitations: The study covers two domains but does not establish that the complementarity claim holds universally across tabular anomaly-detection settings.The authors characterize the findings as evidence of cross-domain generalisation and note that results on other datasets are dataset-dependent.

Disclosure of Interests

The author reports a technical-advisor role in manufacturing that is unrelated to the study’s models, data, methodology, and results.

  • Disclosure of Interests: The author serves as a technical advisor to a manufacturing-sector company unrelated to the reported study.The disclosure states that the role is unrelated to the models, data, methodology, and results.

A Architecture ablation

A moderate-bottleneck two-hidden-layer DBM was retained because it outperformed the expansion configuration and avoided the training stagnation observed with more aggressive bottlenecks.

  • Architecture selection: The selected architecture uses hidden-layer widths H1 = 48 and H2 = 24, chosen through a preliminary single-seed ablation on Bank Marketing.The comparison included expansion, strong bottleneck, moderate bottleneck, and three-layer alternatives.
  • Architecture selection: AUROC was 0.737 for moderate compression versus 0.663 for expansion, with expansion underperforming on every metric.The result suggests that wider top layers used in image-domain DBMs are poorly matched to this tabular anomaly-detection setting.
  • Architecture selection: Aggressive bottlenecks with H2 = 16 or H3 = 12 stopped improving at epoch 3 and reached full-dataset AUROCs of 0.677 and 0.608.Neither configuration recovered during joint fine-tuning.
  • Architecture selection: The single-RBM variant nearly matched the two-layer model in aggregate AUROC, but the two-layer DBM widened the ΔF-sum gap and exposed the non-additive structure analysed later.The single-RBM difference was ΔAUROC = 0.005 in favour of the DBM, while the ΔF-sum gap was +0.020.

B Mean-field energy vs proper variational free energy

The paper uses mean-field energy rather than proper variational free energy because the entropy term systematically reduces anomaly discrimination on Bank Marketing.

  • Score definitions: Proper variational free energy is defined as Fproper = E(v, µ) − H(µ), where H(µ) is the Bernoulli entropy of the mean-field posterior.The hidden units are replaced by mean-field expectations when evaluating E(v, µ).
  • Score definitions: The main score Fbefore(v) = E(v, µ) omits the entropy term and is therefore not the proper variational free energy.This convention was inherited from the authors’ prior workshop paper and was evaluated empirically in the appendix.
  • Evaluation: Across 20 already-trained DBMs, the appendix recomputed both scores without retraining and used 10 mean-field iterations as in the main experiments.The recomputation included a numerical clamp for mean-field probabilities before evaluating logarithms.
  • Why energy is preferred: Anomaly-side posterior entropy was 6.48 ± 0.15 nats versus 6.04 ± 0.15 for inliers, a positive difference in every seed.The difference was +0.43 ± 0.02 nats, with t = 19.95 and p ≈3 × 10^-14.
  • Why energy is preferred: Because the larger anomaly entropy is subtracted in Fproper, it partially cancels the discriminative signal carried by E(v, µ).Although Fproper remains the correct upper bound on −log p(v), the authors prefer Fbefore for Bank Marketing anomaly ranking.

Architecture and dataset scope

The entropy-term comparison is scoped across two architectures and two datasets, with consistent posterior entropy asymmetry but dataset- and architecture-dependent ranking effects.

  • The comparison covers DBM and single-RBM architectures on Bank Marketing and NSL-KDD, using 20 seeds per setting.
  • Hanomaly > Hinlier holds in 20/20 seeds across all four architecture–dataset settings.This consistency indicates the entropy asymmetry is a property of the mean-field posterior rather than network depth.
  • Entropy-term ranking effects vary by setting: significant in favor of FE on NSL-KDD for both architectures, borderline for the DBM on Bank Marketing, and indistinguishable from zero for the RBM there.

C Pairwise column interactions in the DBM

The DBM’s pairwise interventions expose non-additive relationships among columns, concentrating interpretable anomaly-versus-inlier differences in a subset of plausible feature pairs rather than providing a competing detector.

  • Pairwise intervention: Pairwise interventions replace two columns with their inlier modes to measure the DBM’s joint response.The construction extends the single-column intervention to expose non-additive joint structure.
  • Pairwise intervention: Ic,c′(v) is identically zero for column-additive energies, so its magnitude measures the DBM’s non-additive use of column pairs.
  • Results: 31/91 pairs remain significant after Bonferroni correction, while the remaining 60 pairs have small median |¯Ic,c′| of 0.02 nats and often flip sign across seeds.
  • Results: The strongest pairs capture plausible occupational, demographic, and temporal–channel relationships, including job × education and month × contact.
  • Scope: AUROC reaches at most 0.71 for an individual pairwise score, so ¯Ic,c′ is used as an interpretability diagnostic rather than a competing predictor.
Loading 2608.14186v1…