PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Medical Anomaly Detection -- A Survey

Tharindu Fernando, Harshala Gammulle, Simon Denman, Sridha Sridharan, Clinton Fookes

arXiv:2012.02364v2cs.LGcs.CVeess.IVstat.ML

TL;DR

Medical anomaly detection is challenging because anomalies vary by context and existing methods lack structured comparison across applications. This survey systematically reviews deep learning architectures, training strategies, and interpretation methods, while identifying limitations and research directions. It concludes that generalisation, causal identification, and trustworthy interpretation remain important challenges.

  • Problem

    Medical anomaly detection lacks a structured organisation of diverse applications that would enable comparison of methods, advantages, and limitations.

  • Method

    The survey systematically compares deep learning architectures and training strategies across medical applications and reviews model interpretation methods.

  • Results

    The survey identifies key limitations of existing methods, including weak generalisation across operating conditions, limited causal identification, and challenges in interpreting model decisions.

  • Takeaways & Limitations

    Deep medical anomaly detection research should address generalisation, causal estimation, and model interpretation to support reliable medical applications.

  • Takeaways & Limitations

    Existing approaches can be highly specific to their training conditions, with one abnormal heart sound model falling from more than 99% accuracy to 52.27% on an unseen dataset.

Abstract

from arXiv · show

Machine learning-based medical anomaly detection is an important problem that has been extensively studied. Numerous approaches have been proposed across various medical application domains and we observe several similarities across these distinct applications. Despite this comparability, we observe a lack of structured organisation of these diverse research applications such that their advantages and limitations can be studied. The principal aim of this survey is to provide a thorough theoretical analysis of popular deep learning techniques in medical anomaly detection. In particular, we contribute a coherent and systematic review of state-of-the-art techniques, comparing and contrasting their architectural differences as well as training algorithms. Furthermore, we provide a comprehensive overview of deep model interpretation strategies that can be used to interpret model decisions. In addition, we outline the key limitations of existing deep medical anomaly detection techniques and propose key research directions for further investigation.

I. INTRODUCTION

Medical anomaly detection is important but difficult because anomalies and normal data can vary across contexts, patients, modalities, and capture conditions. This survey structures deep learning approaches, interpretation strategies, limitations, and future directions for the field.

  • Anomaly detection identifies samples that do not fit the overall data distribution and is crucial for medical signal analysis.
  • Medical data processing uses anomaly detection across both prediction and diagnosis tasks.
  • Strong visual similarity can occur between normal and abnormal endoscopy images, while normal heart sounds may vary because of capture noise.
  • Medical applications require highly accurate abnormality detection because test sensitivity is decisive for diagnosis.
  • Deep learning can model nonlinear patterns, automatically learn features, and capture long-term temporal relationships through recurrent architectures.
  • The survey addresses the lack of systematic comparison by analysing architectures, training strategies, interpretation methods, limitations, and research directions.

II. DETECTING MEDICAL ANOMALIES WITH DEEP LEARNING

Deep medical anomaly detection spans diverse biomedical data types, including images, electrical signals, and miscellaneous sources. The survey organises these modalities and discusses how deep learning methods address their differing data characteristics.

  • Biomedical data are broadly grouped into images, electrical signals, and other sources such as laboratory results, audio, and wearable devices.
  • Biomedical Imaging: X-rays support diagnosis across conditions including fractures, dental problems, pneumonia, and some tumors, while CT supports detecting tumors, lesions, injuries, and clots.
  • Biomedical Imaging: Medical imaging applications include X-ray, CT, MRI, PET, ultrasound, and medical optical imaging.CT produces stacked cross-sectional slices, while MRI is typically used for soft-tissue imaging.
  • Electrical Biomedical Signals: Electrical biomedical signals include ECG, EEG, MEG, and EMG, which capture cardiac, brain, or muscle-related activity for medical analysis.ECG analysis can use timing and electrical-strength information; EEG records amplified electrical activity through scalp electrodes.
  • The survey classifies deep learning algorithms by training objectives, label availability, recurrent architectures, and how dimensionality differences across data types are handled.

1) Unsupervised Anomaly Detection:

Unsupervised anomaly detection trains without normal/abnormal labels, commonly using autoencoders or GANs to model data and identify deviations. The survey reviews architectural variants and medical applications while noting reconstruction limitations for high-dimensional distributions.

  • Unsupervised algorithms receive no normal/abnormal supervision during training, making them applicable without labelled datasets.
  • Autoencoders: Autoencoders learn compressed representations and reconstruct their inputs, with reconstruction loss commonly based on Mean Squared Error.
  • Autoencoders: Sparse, denoising, contractive, and variational autoencoders modify representation learning through sparsity, noise removal, sensitivity regularisation, or latent-distribution modelling.The sparsity strength is controlled by λ, while denoising autoencoders reconstruct clean signals from corrupted inputs.
  • Autoencoder Applications: Medical autoencoder applications include respiratory-disease detection, EEG anomaly detection, volumetric CT analysis, skin-image anomalies, and MRI anomaly detection.
  • Limitations: Autoencoders can model high-dimensional data distributions inaccurately, producing erroneous reconstructions and motivating the use of GANs.
  • Generative Adversarial Networks: GANs train a generator and discriminator adversarially: the generator produces data from noise, while the discriminator distinguishes synthesised from real examples.Although the discriminator receives real/fake labels, the GAN is unsupervised for anomaly identification because anomalous examples are not provided.
  • GAN Variants: Conditional GANs condition generator and discriminator outputs on additional data, while Cycle-GANs add a constraint requiring recovery of the original input from generated output.
  • GAN Applications: A medical GAN application generated normal OCT scans and used an encoder-generator mapping to support anomaly detection.

2) Supervised Anomaly Detection:

Supervised anomaly detection uses labels distinguishing normal from anomalous examples, while recurrent architectures address temporal medical data and memory limitations through gated or external-memory designs.

  • Supervised anomaly detection provides labels identifying which examples are normal and which are anomalous.
  • Recurrent neural networks model sequential medical signals by feeding each time step’s output into the next time step.
  • Simple RNNs use backpropagation through time, but vanishing gradients limit their ability to model long-term dependencies.
  • LSTMs address long-term dependencies with a memory cell and gated operations that control information stored and updated over time.
  • GRUs simplify LSTMs by combining the forget and input gates into a single update gate.
  • Neural Memory Networks use external memory stacks and controllers to read, write, and output information across multiple memory blocks.

C. Background and Related Applications

This subsection surveys application domains in deep medical anomaly detection and shows how architectural variants are used across those domains.

  • The subsection discusses popular application domains within deep medical anomaly detection.
  • It illustrates how previously discussed architectural variants are leveraged across these application domains.
  • The discussion connects domain applications with the deep learning architectures introduced earlier.

1) MRI based Anomaly Detection:

MRI anomaly detection predominantly uses supervised deep CNNs, alongside modality fusion, multi-task learning, auto-encoders, and neural memory approaches for varied diagnostic tasks.

  • MRI studies fuse modalities including T1-weighted MRI with myelin water imaging and MRI with FDG-PET samples.
  • Some approaches combine ADC and T2-weighted MRI streams using similarity loss to produce consistent segmentation maps.
  • Decision-level fusion ensembles three separately trained CNN classifiers using majority voting across four Alzheimer’s disease classes.
  • Supervised deep CNN models dominate MRI anomaly detection because they effectively extract task-specific spatial information from two-dimensional inputs.
  • Auto-encoder approaches may begin with unsupervised feature extraction but add supervised classification layers for anomaly detection.
  • A neural memory approach uses supervised recurrence to compare samples across the dataset and store characteristics separating normal and anomalous samples.

2) Detecting abnormalities in Endoscopy Data:

Endoscopy anomaly detection is dominated by CNN-based image analysis, often using pretrained natural-image features, while newer systems combine multi-scale relationships with recurrent processing.

  • CNNs pretrained on large-scale object-detection benchmarks such as ImageNet are extensively applied to RGB endoscopy data.
  • A two-stage system uses RetinaNet for classification and bounding boxes, then ResNet-18 on 160 × 160 patches for binary ulcer classification.
  • A two-stream framework combines ResNet-50 features from two levels through a relational network before LSTM processing and classification.
  • CNN-based endoscopy analysis must capture both distinctive patterns and their locations for diagnosis.
  • Most endoscopy methods use supervised DCNNs and increasingly include multi-class classification rather than only binary classification.
  • Heart-sound anomaly detection differs from image applications by operating on one-dimensional audio, using 1D CNNs and RNNs or transformed image representations.

3) Heart Sound Anomaly Detection:

Medical anomaly detection spans heart-sound and seizure applications, using architectures suited to temporal signals while facing substantial variation across subjects and preprocessing choices.

  • Heart Sound Anomaly Detection: Segmentation of heart sounds is debated: it may isolate informative S1 and S2 regions, but preprocessing errors can propagate into detection.
  • Heart Sound Anomaly Detection: WaveNet classifies heart sounds into five categories using residual blocks with 1D dilated convolutions over raw audio.Features from six residual blocks are aggregated before convolutional and fully connected classification layers.
  • Seizure Prediction: Seizure prediction detects transitions from the normal interictal state to the pre-ictal state before seizure onset.The pre-ictal duration varies from minutes to hours across subjects, complicating generalized prediction.
  • Seizure Prediction: Reported seizure-prediction performance varies substantially: sensitivity falls to 33.3% for some subjects, while another system reaches 69% mean sensitivity.A separate LSTM-based system reaches 99.28% sensitivity for a 15 min pre-ictal period, illustrating dependence on setting and evaluation conditions.
  • Seizure Prediction: GAN-based seizure prediction is semi-supervised because its discriminator is fine-tuned with labelled examples for normal/abnormal classification.

III. MODEL INTERPRETATION

Model interpretation methods explain deep medical anomaly-detection decisions through architecture-specific or model-agnostic approaches, including attribution, visualization, surrogate, and local/global strategies.

  • Motivation and Taxonomy: Interpretability is important because deep diagnostic models rely on many parameters whose decision processes are not human interpretable.
  • Motivation and Taxonomy: Interpretation methods vary by architecture dependence, locality, and whether they use surrogate models or visualizations.Model-agnostic methods are not restricted to one architecture, while local methods explain individual predictions and global methods characterize overall behavior.
  • Visualization Methods: CAM generates activation maps by weighting final convolutional feature maps with classification-layer weights and aggregating them.CAM requires global average pooling over convolutional maps immediately before prediction.
  • Attribution Methods: Grad-CAM uses gradients entering the final convolutional layer to estimate pixel contributions, then applies ReLU to remove negative values.It can fail to localize multiple occurrences of the same object.
  • Attribution Methods: LIME explains local model behavior by perturbing interpretable input components and observing prediction changes, but sampled points can be unrealistic and explanations unstable.
  • Attribution Methods: SHAP measures feature or feature-group contributions through combinations of included and excluded features, offering a theoretically grounded full interpretation at high computational cost.In a hypoxaemia example, purple features increase risk and green features reduce it.

IV. CHALLENGES AND OPEN RESEARCH QUESTIONS

The survey identifies clinician usefulness, multimodal complexity, and the need for more interpretable decision processes as open challenges in medical deep anomaly detection.

  • Lack of Interpretability: Interpretation methods should be selected according to whether explanations are global or local, users’ expertise, and real-time constraints.
  • Lack of Interpretability: GradCAM, LIME, and GBP may inform machine-learning practitioners yet be less useful to clinicians because they were not developed specifically for medical explainability.The survey calls for expert-clinician evaluation across application domains.
  • Multimodal Interpretation: Model-agnostic interpretation remains underdeveloped for multimodal models whose decisions depend on multiple input streams.
  • Future Directions: Reinforcement learning is proposed as a future direction because an agent’s exploration process may provide intuition about its anomaly-detection behavior.

B. Causality and Uncertainty

The survey identifies causality, uncertainty estimation, cross-condition generalisation, data imbalance, and limited labelled data as important unresolved issues in medical anomaly detection. It reviews possible directions including causal methods, Bayesian uncertainty, domain adaptation, meta-learning, semi-supervised learning, and self-supervised learning.

  • Causality: Existing deep medical anomaly detection methods often identify associations rather than the causes of symptoms.The survey distinguishes causal identification from merely associating symptoms with disease categories.
  • Uncertainty: Most current anomaly detection algorithms lack uncertainty estimation that quantifies how input changes affect predictions and model confidence.Bayesian deep learning can generate probabilistic scores and uncertainty information about model weights and outputs.
  • Generalisation: More than 99 % accuracy fell to 52.27 % on an unseen dataset, illustrating poor generalisation across operating conditions.The survey attributes this instability partly to scarce, highly curated datasets that may not capture the full population distribution.
  • Future directions: Domain adaptation and meta-learning are proposed to improve generalisation when medical datasets contain few labelled examples.Meta-learning adapts knowledge across tasks, whereas domain adaptation transfers a same-task model across sub-domains using few labelled examples.
  • Data imbalance: GAN-based augmentation can balance minority classes, but generating realistic abnormal samples remains an open research problem.Further work is needed to improve synthetic-sample quality and adapt GAN training to novel abnormalities.
  • Unlabelled data: Semi-supervised and self-supervised learning could exploit cheaper unlabelled data, although self-supervised learning has not yet become established in medical anomaly detection.Self-supervised methods use pretext tasks such as context prediction, colorization, and jigsaw puzzles without labelled data.
  • Survey scope: The survey compares existing techniques, reviews interpretation strategies, and identifies limitations and future research directions.Its structured analysis covers diverse deep-learning approaches despite differences among medical applications.

APPENDIX DEEP LEARNING FOR MEDICAL ANOMALY DETECTION -

Medical anomaly detection research spans imaging modalities including X-ray, CT, MRI, PET, ultrasound, and optical imaging. Deep learning studies use CNNs, RNNs, hybrid models, pretrained features, and multimodal systems across these applications.

  • X-ray radiography: Deep learning has been applied to X-ray images for chest-abnormality detection using CNNs, pretrained architectures, and two-branch diagnosis models.Reported examples include VGG-16, GoogLeNet, and ChestNet.
  • Computed Tomography: CT studies use deep learning to detect COVID-19 pneumonia, kidney tumors, and lung cancers.The surveyed applications include multiple chest-CT studies and distinct tumor-detection tasks.
  • Magnetic Resonance Imaging: MRI research applies CNNs and CNN–RNN hybrids to detect abnormalities, including brain tumors.Studies also examine transferring CNN activations from natural-image pretraining to MRI data.
  • Positron Emission Tomography: PET anomaly detection includes CNN-based lymphoma and whole-body scan classification, plus PET–CT systems combining learned features with rule-based and SVM classifiers.PET uses radioactive tracers whose emitted energy is detected to reveal metabolically active disease regions.
  • Ultrasound: Ultrasound research includes unsupervised and supervised deep learning for breast-tumor classification and abnormality detection.Some approaches fuse pretrained-model features with handcrafted features.
  • Medical optical imaging: Optical imaging, especially endoscopy, supports abnormality diagnosis in digestive diseases, while pretrained CNN features are transferred from large datasets to small endoscopy collections.Applications include bowel disease, gastrointestinal bleeding, and cancer detection.

B. Electrical Biomedical Signals

Electrical biomedical signals provide temporal information that anomaly-detection models must represent over time. The literature applies CNNs, RNNs, and hybrids across ECG, EEG, MEG, and EMG data, with EMG abnormality detection comparatively under-studied.

  • ECG: ECG analysis supports screening for irregular electrical activity through time intervals and indicates stressed heart regions through signal strength.Deep-learning studies use CNNs for arrhythmia detection and RNNs to model temporal evolution.
  • EEG: EEG records brain electrical activity through scalp electrodes, and deep models detect schizophrenia, epilepsy, sleep abnormalities, and related conditions.CNNs, RNNs, and CNN–LSTM hybrids are reported across these applications.
  • MEG: MEG captures magnetic fields produced by brain electrical activity and is used to identify abnormal brain regions and activities.Reported models include spatio-temporal neural networks and 1D and 2D CNNs for Alzheimer’s detection.
  • EMG: EMG records muscle-cell electrical potentials for diagnosing muscles and motor neurons, but deep-learning-based EMG abnormality detection is not widely studied.Existing work includes muscle-activation detection, movement-information extraction, and movement-intent prediction.
  • Summary: Temporal modelling is vital for electrical biomedical signals because their informative characteristics change over time.The survey also lists publicly available electrical-signal datasets for machine-learning research.

C. Miscellaneous data types

Miscellaneous medical data types include phonocardiography and wearable-device signals, which are temporal despite differing sensory sources. Deep-learning applications include abnormal heart-sound, gait, and fall detection, supported by public datasets.

  • Phonocardiography: Phonocardiography records cardiac sounds and is used to screen for arrhythmia, valve disease, and heart failure.Electronic stethoscopes and public datasets have encouraged deep-learning research on abnormal heart sounds.
  • Phonocardiography: Deep-learning heart-sound studies combine temporal quasi-periodic features with LSTMs, CNNs with MFCCs, or CNNs with handcrafted acoustic features.The surveyed approaches differ in how they represent and process cardiac audio recordings.
  • Wearable medical devices: Wearable devices measure health parameters such as heart rate, temperature, muscle activity, and blood or tissue oxygenation.Examples include smartwatches, smart clothing, smart footwear, and fitness trackers.
  • Wearable medical devices: Wearable anomaly-detection studies use insole pressure data with LSTMs for gait abnormalities and smartwatch accelerometers for fall detection.These applications analyse temporal measurements collected during walking, standing, or movement.
  • Datasets: Public datasets cover audio, vital-sign, and wearable-device data for machine-learning research.The survey also lists publicly available electrical-signal datasets separately.
Loading 2012.02364v2…