Source-linked AI summary
ECoLAD: Selecting Anomaly Detectors for Automotive Deployment via Compute-Reduction Evaluation
Kadir-Kaan Özer, René Ebeling, Markus Enzweiler
TL;DR
Automotive TSAD selection often relies on accuracy-only workstation benchmarks despite deployment constraints on latency and CPU parallelism. ECoLAD addresses this mismatch with an auditable compute-reduction and throughput-feasibility protocol, showing that accuracy stability can diverge from deployment feasibility and that classical detectors may retain high scoring rates.
Problem
Existing TSAD benchmarks report accuracy under unconstrained workstation execution, providing limited evidence about feasibility under reduced compute and CPU parallelism.
Method
ECoLAD evaluates detectors with a monotone compute-reduction ladder, explicit thread caps, uniform integer-only scaling, separated timing measures, and target-specific feasibility reporting.
Results
Accuracy stability and deployment feasibility diverge: TimesNet retains nearly flat AUC-PR while SMD throughput falls from 9,569 to 1,483 wps at CPU-1T, whereas HBOS and COPOD retain high throughput and near-flat AUC-PR.
Takeaways & Limitations
Per-tier, per-dataset feasibility screening can expose backend sensitivity and exclude throughput-infeasible detectors before metric-based selection.
Takeaways & Limitations
Absolute throughput numbers are specific to the M3 Max, and quantitative transfer to ECU-class hardware requires target-platform validation.
Abstract
from arXiv · showhide
Automotive anomaly detectors are often selected from accuracy only benchmarks on workstation class hardware, whereas in-vehicle monitoring requires predictable scoring latency under limited CPU parallelism. This mismatch can make methods that appear competitive offline infeasible for deployment. We present ECoLAD (Efficiency Compute Ladder for Anomaly Detection), a deployment-oriented evaluation protocol for automotive time-series anomaly detection (TSAD). ECoLAD defines a monotone compute reduction ladder with explicit CPU thread caps, mechanical integer only hyperparameter scaling, inference/full run timing separation, and auditable run logs. Applied to proprietary in-vehicle telemetry and two public benchmarks, it shows that accuracy stability and deployment feasibility can diverge: some deep detectors retain AUC-PR while losing feasible throughput, whereas lightweight classical detectors sustain high scoring rates with positive lift above the random baseline, providing a practical screening procedure for detector selection under deployment relevant constraints.
I. INTRODUCTION
ECoLAD targets the gap between accuracy-only workstation benchmarks and automotive deployment constraints. It introduces an auditable compute-and-throughput evaluation protocol and applies it to automotive telemetry and public benchmarks.
- In-vehicle telemetry anomaly detection requires predictable latency under limited CPU parallelism and restricted memory bandwidth.
- Accuracy-only workstation benchmarks can misrepresent deployment feasibility because constrained execution can change rankings without degrading accuracy.
- ECoLAD combines a four-tier monotone compute ladder, uniform integer-only scaling, and throughput-target reporting of coverage and best achievable AUC-PR.
- The protocol is instantiated on proprietary automotive telemetry and public benchmarks as a reproducible feasibility gate before detector calibration and vehicle integration.
- Unlike prior evaluations, ECoLAD treats model capacity reduction, thread caps, and throughput feasibility as first-class protocol variables.
II. ECOLAD PROTOCOL
ECoLAD defines a four-tier compute reduction ladder with explicit execution, thread, and scaling controls. Configuration changes and caps are logged for each run on the experimental platform.
- The four tiers apply compute reduction factors s ∈{1.0, 0.75, 0.50, 0.25} while fixing the execution backend and CPU thread cap.
- CPU thread limits are enforced through PyTorch, OpenMP, and BLAS settings before each run, with all configuration changes logged.
- Experiments use an Apple MacBook Pro with an M3 Max, 14 performance cores, and 32 GB unified memory.
- For the five classical detectors, the GPU tier reduces to the reference CPU configuration, differing only in thread caps and scale semantics.
B. Mechanical Hyperparameter Scaling
Mechanical scaling transforms baseline configurations with role-specific integer-only rules and no per-tier retuning. Deep models undergo broader capacity reduction than classical detectors because their architectural parameters scale simultaneously.
- Each baseline configuration is transformed using role-specific integer-only rules determined by the tier reduction factor s, without per-tier retuning.
- Width and attention heads scale with √s to approximate proportional capacity reduction when capacity scales quadratically with width.
- Depth scales more slowly as s^1/4 to avoid collapsing shallow models, while decision-semantic parameters such as contamination thresholds remain unchanged.
- Classical detectors apply only the work rule, whereas deep models reduce width, depth, and heads simultaneously at low s.
- Constraint repairs occurred in fewer than 4% of runs and affected only TranAD and GDN.
C. Throughput and Feasibility
ECoLAD separates inference timing from full-run timing and evaluates throughput feasibility through target-specific coverage and achievable AUC-PR. Throughput is measured per entity and configuration under explicit scoring-rate thresholds.
- Inference-only time tinf is preferred over full-run time te2e because online scoring capacity should not be conflated with offline training cost.
- Throughput is defined as wps = N/tinf, and an entity is feasible at target τ when wps ≥τ.
- The reference target τ = 500 wps represents scoring at 500 Hz, or 2 ms per sample, under unit-stride windowing.
- For each method, dataset, and target, achievable AUC-PR is the best measured value among configurations meeting the throughput target.
- Coverage is the fraction of entities with at least one qualifying configuration.
III. EXPERIMENTAL SETUP
The evaluation spans ten unsupervised or self-supervised detectors across proprietary automotive telemetry and public benchmarks, with labels used only for evaluation. Three research questions examine cross-tier quality, degradation modes, and throughput-constrained behavior.
- Ten detectors cover classical, deep, attention-based, and graph-based families, evaluated with unsupervised or self-supervised training.Labels are used only for evaluation.
- Datasets: The proprietary Telemetry dataset contains 80,000 datapoints, 19 synchronized features, and a 0.022 random-scorer AUC-PR baseline.The data are split evenly into 40,000 training and 40,000 test points.
- Research questions: RQ1 measures how AUC-PR and rankings change across compute tiers.
- Research questions: RQ2 compares throughput bottlenecks with accuracy drift as degradation modes.
- Research questions: RQ3 evaluates coverage and achievable AUC-PR under throughput targets at CPU-1T.
A. RQ1: Cross-Tier Detection Quality
AUC-PR changes across tiers in method- and domain-dependent ways, while constrained execution can alter rankings even when absolute accuracy changes are small. The Telemetry results include a 0.022 random-scorer baseline, and pooled single-tier reporting misses these effects.
- AUC-PR drift exists across tiers but is strongly method and domain dependent.
- SMD: On SMD, OmniAnomaly remains near 0.51 AUC-PR and PCA remains essentially constant at 0.448 across tiers.
- SMD: LOF declines from 0.145 to 0.073 at CPU-1T on SMD, while modest GDN and TimesNet drift can change relative ordering.
- Telemetry: On Telemetry, HBOS reaches approximately 2.9× the 0.022 random baseline with minor tier-to-tier drift.
- Cross-domain comparison: Top methods differ between SMD and Telemetry, and tier-sensitive methods such as LOF shift substantially under constrained execution.Both effects are invisible in a single-tier leaderboard.
B. RQ2: Degradation Modes
The evaluation identifies three degradation modes: backend-limited feasibility, quality-limited accuracy, and graceful degradation. These modes separate throughput failure from accuracy loss under compute reduction.
- Backend limited: TimesNet’s AUC-PR remains nearly flat while SMD inference throughput falls from 9,569 to 1,483 wps at CPU-1T.Telemetry throughput falls from 11,164 to 1,751 wps, making feasibility loss throughput driven.
- Quality drift limited: LOF sustains over 76,000 wps on Telemetry and 193,000 wps median on SMD at CPU-1T while showing the largest negative AUC-PR change.Runtime is not the bottleneck for this quality-drift-limited mode.
- Graceful degraders: HBOS and COPOD retain high throughput and near-flat AUC-PR across all tiers.HBOS exceeds 2,000,000 wps on Telemetry at s=0.25 after reducing histogram bins per scoring call.
C. RQ3: Throughput-Constrained Behavior
Throughput-constrained evaluation reveals that classical detectors maintain broader feasible coverage, while deep methods can exhaust feasible configurations as targets rise. The protocol therefore supports feasibility-first screening, subject to platform-specific validation.
- IForest reaches 4,199 wps and PCA 1,752 wps, whereas TimesNet reaches 1,483 wps on SMD before feasible configurations quickly exhaust as τ rises.
- HBOS sustains positive AUC-PR at the highest feasible throughput target, while early-coverage methods provide no above-random operating point at high targets.
- ECoLAD recommends screening detectors for deployment-relevant scoring rates before metric-based selection.Throughput feasibility is made explicit and auditable as a protocol variable.
- Platform boundary: Absolute throughput values are specific to the M3 Max and require platform-specific correction before mapping to ECU-class targets.Validation on ECU-class hardware remains future work.
- Reproducibility boundary: The telemetry dataset and pipeline code are proprietary, although the protocol is specified for independent re-implementation.Mechanical scaling provides a reproducible no-retuning baseline.
VI. CONCLUSION
ECoLAD makes compute reduction, CPU parallelism caps, throughput feasibility, and auditability explicit in automotive TSAD evaluation. Across automotive telemetry and public benchmarks, constrained execution shifts accuracy rankings and can exclude otherwise competitive methods from feasible operating points.
- ECoLAD explicitly evaluates compute reduction, CPU parallelism caps, throughput feasibility, and auditability.
- Accuracy rankings shift under constrained execution across automotive telemetry and public benchmarks.
- Throughput-feasible operating points can exclude otherwise competitive anomaly detectors.