Quasi-Binarized Autoencoders: An Architecture-Independent Information Bottleneck for Medical Image Anomaly Detection

1. Introduction

• Unsupervised anomaly detection learns from normal images to flag deviations, a useful medical-imaging setting because abnormal studies are diverse and costly to annotate.The full range of pathologies cannot be listed in advance.Source: §1 · p. 1
• An autoencoder must reconstruct normal images well and abnormal images poorly, which requires limiting encoder-to-decoder information to prevent near-identity reconstruction of lesions.Current practice uses dataset-tuned latent sizes, depth, and skip-connection choices; latent dimension does not quantify transmitted information in bits.Source: §1 · p. 2
• Prior approaches only partly resolve the bottleneck problem: variational autoencoders tie it to a prior, codebooks remain architectural and can be bypassed by skips, and input corruption does not measure transmitted information.Denoising autoencoders have strong image-level brain-MRI results in MedIAnomaly, but corruption shapes learning rather than directly bounding information.Source: §1 · p. 2
• A QB layer squashes latent elements into [0, 1], adds Laplace noise of scale 1/ε, and gives each element ε-local differential privacy with closed-form information bounds.The supplied bounds are at most min{ε, ε^2} nats and an amplitude-based bound that grows logarithmically with ε, reaching about 2 bits per element at ε = 10.Source: §1 · pp. 2–3
• QBAE puts QB layers on every encoder-decoder path, including skips, so its total information budget depends on ε and the number of QB elements rather than network depth, width, or connectivity.The noise remains at test time, ensuring every detection reconstruction is produced under the budget.Source: §1 · p. 3
• With 32,768 QB elements, the seven-level QBAE carries at most 6.5 × 10^4 bits at ε = 10, versus 131,072 bits in a raw 8-bit 128 × 128 image.Because the information limit is architecture-independent, one expressive network can be adapted across datasets by changing ε rather than redesigning latent spaces or skips.Source: §1 · p. 3
• 0.828 mean AUROC across seven MedIAnomaly datasets made QBAE the highest-scoring method that does not adapt to each dataset, with best reported BraTS2021 AUROC 0.911 and pixel-level AP 0.838.The evaluation used one network and one configuration for all datasets.Source: §1 · p. 3

Reconstruction-based anomaly detection in medical imaging

• MedIAnomaly found well-tuned autoencoders competitive across seven datasets and five modalities, while its companion analysis identified latent size and decoder information as key factors in detection.Other reconstruction approaches include adversarial, context-encoding, uncertainty-estimating, perceptual, and denoising autoencoders; diffusion methods “heal” anomalies at higher inference cost.Source: §2 · p. 4
• MSDE uses pretrained features and mean-shift refinement for density scoring, but on BraTS2021 its AUROC is 0.736 versus 0.859 for DAE.MSDE performs strongly on chest X-ray and brain-tumor MRI image-level AUROC but falls behind reconstruction methods on BraTS2021.Source: §2 · p. 4

Limiting and measuring information in autoencoders

• Earlier autoencoder bottlenecks use prior-relative KL rates, codebook capacity, or activation sparsity, while skip connections can bypass some bottlenecks and sparse autoencoders bound no information-theoretic quantity.These limitations motivate QBAE’s explicit budget on every encoder-decoder path.Source: §2 · p. 4
• QB differs from hashing, compression, and noisy-channel coding by adding noise after bounded squashing, placing layers on every path including skips, and withholding information rather than transmitting it efficiently.Training pushes outputs toward 0 or 1, making the noisy outputs nearly binary; the bounded post-squashing noise also supplies differential-privacy guarantees and closed-form information bounds.Source: §2 · pp. 4–5

Differential privacy and information

• In QBAE, ε-DP links noise to a mutual-information bound, and ε directly sets how much of each image the decoder may see.Prior deep-learning DP work mostly perturbs training gradients; intermediate-activation noise has also been studied for inference-time privacy.Source: §2 · p. 5

Architectural components and pretrained features

• QBAE’s attention-gated U-Net and residual autoencoding components can be used without restoring an identity shortcut because every route to the decoder crosses a QB layer.Its perceptual loss and anomaly score use ImageNet-pretrained VGG19 features, so the method relies on external pretrained weights, while the bottleneck representation is learned from normal medical images.Source: §2 · p. 5

3.1. Problem setting and overview

• QBAE trains on normal images to reconstruct them, then derives pixel- and image-level anomaly scores from discrepancies between each image and its reconstruction.The training set contains only normal images; at test time, images may contain unseen abnormalities.Source: §3.1 · p. 5

3.2.1. Definition

• A QB layer with ε > 0 acts element-wise on pre-activations, squashing each into the unit interval; it adds independent Laplace noise during training and retains noise at test time.As ε → ∞, noise vanishes and the layer reduces to a sigmoid; experiments approximate this with ε = 108 as the no-bottleneck control. As ε → 0, noise swamps the signal and no information is transmitted.Source: §3.2.1 · p. 6

3.2.2. Information budget

• The mutual information between input and reconstruction is bounded by a quantity depending only on ε and the number of QB elements, for any encoder and decoder.The bound applies to normal and abnormal images, with or without input corruption; for large ε, the per-element bound grows roughly as log2 ε bits.Source: §3.2.2 · pp. 6, 8

3.3. Network architecture

• The seven-level U-Net routes all six skip connections and its fully connected bottom through QB layers, so no input-to-reconstruction path avoids the information bottleneck.The network uses 32,768 QB elements; its encoder-decoder components include residual autoencoding blocks, cross-attention skip gates, and decoder attention blocks.Source: §3.3 · pp. 9–10

3.4. Training

• Training corrupts the input with localized smooth noise while reconstructing the clean image, a process that shapes learning separately from the QB layers’ information limit.The corruption consists of localized blobs formed from Gaussian-smoothed noise fields; the paper evaluates configurations with and without corruption.Source: §3.4 · p. 10

3.2.3. Practical remarks

• Optional sparsity regularization encourages each code element to activate in only a fraction ρ of images, but the information bounds do not depend on this penalty.The loss also uses a perceptual L1 distance between VGG-19 features of the clean image and its reconstruction.Source: §3.2.3 · p. 9, §3.4 · p. 10

3.5. Anomaly scores

• At test time, input corruption is disabled, QB noise remains, and the pixel anomaly map combines squared reconstruction error with upsampled perceptual error.The image-level score is the spatial mean of the perceptual term alone; scores are averaged over eight independent noise draws.Source: §3.5 · pp. 10–11

4.1. Datasets

• The evaluation covers all seven MedIAnomaly datasets, spanning five imaging modalities and abnormalities that vary in size and visibility.All images are resized to 128 × 128; BraTS2021 is the only dataset with pixel-level annotations.Source: §4.1 · p. 11

4.2. Implementation details

• The same 32,768-element network is used throughout, with ε, input corruption, and sparsity penalty as the only varied hyperparameters.The ε sweep uses {0.3, 1, 3, 10, 30, 100}; ε = 10^8 serves as the no-bottleneck control.Source: §4.2 · pp. 11–12

4.3. Evaluation protocol

• Image-level AUROC is the main metric, and each configuration is trained with five random seeds; results are reported as mean and standard deviation.BraTS2021 also uses pixel-level AP and best achievable Dice; scores are averaged over eight noise draws.Source: §4.3 · p. 12
• Because MedIAnomaly has no validation set, the main comparison uses ε = 10 with corruption and sparsity enabled for every dataset, a choice that retains some test-selection optimism.The shared configuration gave the best mean AUROC across the seven datasets; dataset-specific best-ε results are reported separately as an optimistic upper bound, and mean AUROC changes by at most 0.012 for ε between 3 and 30.Source: §4.3 · p. 12

5.1. Comparison with existing methods

• 0.828 mean AUROC is the highest in MedIAnomaly; MSDE’s 0.826 is on par but uses per-dataset backbones and test-image refinement.The difference is smaller than QBAE’s mean seed-to-seed variation (SD ≈0.004).Source: §5.1 · p. 12, Table 4 · p. 13
• On BraTS2021, image-level AUROC is 0.911; pixel-level AP is 0.838 and best Dice 0.773, versus DAE’s 0.755 and 0.711 for the pixel metrics.Source: §5.1 · p. 12, Table 5 · p. 13
• Fig. 2 examples show tumors removed in reconstructions and lesion-following maps, while small multifocal lesions receive only weak responses.Source: §5.1 · pp. 12–13, Figure 2 · p. 14

5.2. Effect of the information budget

• 0.816–0.828 mean AUROC holds across ε=3–30 with corruption; without corruption, AUROC peaks at 0.805 (ε=1) and falls to 0.590 without QB.Source: §5.2 · p. 13
• Choosing ε per dataset raises mean AUROC to 0.847, an optimistic upper bound because ε was selected on the test set.The best ε ranges from 30 for BraTS2021, Brain Tumor, and VinDr-CXR to 1 or 3 for RSNA and Camelyon16.Source: §5.2 · p. 14, Table 6 · p. 14

6. Discussion

• 0.828 mean image-level AUROC is the highest reported among methods that do not adapt to each dataset; on BraTS2021, the best reported results are AUROC 0.911 and pixel-level AP 0.838.The same network and configuration are used across datasets, with noise retained at test time.Source: §6 · pp. 14–15, §6.6 · p. 17

6.1. One parameter for the information budget

• Mean AUROC varies by at most 0.012 for ε between 3 and 30, although the best budget differs by dataset.The selected budget is large for BraTS2021, Brain Tumor, and VinDr-CXR, and small for RSNA and Camelyon16; the proposed explanation is untested.Source: §6.1 · p. 15

6.2. Bottleneck and input corruption are complementary

• Without input corruption, the bottleneck raises mean AUROC from 0.590 to 0.805; with corruption, its average gain is smaller but reaches 0.04–0.13 on Camelyon16, RSNA, and LAG.It adds little on BraTS2021 and Brain Tumor, where focal lesions resemble the blob-shaped corruption.Source: §6.2 · pp. 15–16

6.3. A provable budget at test time

• Keeping noise at test time ensures each detection reconstruction is produced under the stated information budget.The discussion also notes that a bit budget bounds how much image information could be recovered across sites.Source: §6.3 · p. 16

6.4. Relation to feature-density methods

• QBAE and MSDE have the same mean AUROC but different dataset strengths, and the authors describe the approaches as complementary.QBAE is better on BraTS2021, ISIC2018, and LAG; MSDE is better on VinDr-CXR, Camelyon16, and Brain Tumor.Source: §6.4 · p. 16

6.5. Limitations

• The blob-shaped corruption encodes a local, smooth anomaly prior; it does not help on RSNA and slightly lowers AUROC on Camelyon16.The authors note that its fit to brain tumors may contribute to QBAE's advantage on BraTS2021.Source: §6.5 · p. 16
• The common setting was selected for best mean AUROC on test sets, so reported per-dataset results are upper bounds; the information bound applies per reconstruction and requires test-time noise.The averaged score combines eight independent passes and can carry up to eight times the per-reconstruction budget.Source: §6.5 · p. 16
• Evaluation uses downsampled 128×128 two-dimensional images, not full-resolution images or three-dimensional volumes.The authors also note dependence on ImageNet-pretrained VGG-19 features and that pixel-level evaluation is available only on BraTS2021.Source: §6.5 · p. 17

• Shouhei Hanaoka is credited with conceptualization, methodology, software, formal analysis, investigation, visualization, original-draft writing, and funding acquisition.Source: CRediT authorship contribution statement · p. 17
• Takahiro Nakao and Atsushi Takamatsu are credited with validation and review-and-editing contributions.Source: CRediT authorship contribution statement · p. 17
• Takeharu Yoshikawa is credited with resources and review-and-editing contributions; Osamu Abe with supervision, resources, and review-and-editing contributions.Source: CRediT authorship contribution statement · p. 17

• HIMEDIC Inc. and Siemens Healthcare K.K. sponsored the University of Tokyo Hospital department, but had no role in the study or publication decision; the authors declare no other competing interests.Source: Declaration of competing interest · p. 17

• No additional ethical approval was required because the study used publicly available, de-identified datasets and collected no new data.Source: Ethics statement · p. 17

Appendix A. Information-theoretic analysis of the QB layer

• Each QB output is modeled as a [0,1]-valued latent plus independent Laplace noise of scale 1/ε; information is measured in nats and converted to bits by dividing by ln 2.The analysis treats the encoder input as including any encoder randomness and, during training, input corruption.Source: Appendix A · p. 19

Appendix A.2. Proof of Proposition 1

• Each noisy QB element is ε-locally differentially private, yielding a per-element information bound of min{ε, ε²} nats.For ε < 1, the ε² bound follows from the stated privacy-to-mutual-information result.Source: Appendix A.1 · p. 19, Appendix A.2 · p. 19

Appendix A.1. Formal statements

• Theorem 1 bounds image–reconstruction mutual information by N times the smaller per-element privacy and amplitude bounds, regardless of the encoder-decoder architecture.The per-element bound is expressed in bits in the numerical analysis, and totals multiply it by N.Source: Appendix A.1 · p. 19, Appendix A.4 · p. 20, Appendix A.5 · p. 20

Appendix A.3. Proof of Proposition 2

• Because the pre-noise value lies in [0,1], an amplitude-constrained channel analysis supplies an explicit, generally loose upper bound C(ε).The proof bounds output entropy using the variance constraint and the maximum-entropy property of the Gaussian.Source: Appendix A.3 · p. 20

Appendix A.4. Proof of Proposition 3

• Independent noise across QB elements and data processing let the per-element bounds add while preserving the bound for any reconstruction derived from the noisy vector.The proof uses conditional independence of the noisy outputs and subadditivity of differential entropy.Source: Appendix A.4 · p. 20

Appendix A.5. Numerical values

• At ε = 0.3 the two bounds are close (0.090 vs. 0.078 nats); privacy is tighter for ε ≲ 0.28, while amplitude is tighter above that, including settings used.The reported per-element values are converted to bits, and totals multiply by N = 32,768.Source: Appendix A.1 · p. 19, Appendix A.5 · pp. 20–21

Appendix B. Network details

• The 128 × 128 network has 60.7 M parameters and costs 4.2 GFLOPs per image; most parameters are in the deepest decoder levels, which add little computation.The deepest levels operate on 4 × 4 and 8 × 8 feature maps.Source: Appendix B · p. 21
• The network uses residual convolution and attention blocks, with cross-attention gates weighting decoder features against post-QB skip features.The bottom QB path includes fully connected layers around the QB layer.Source: Appendix B · p. 21

Appendix C. Input corruption

• Training corruption combines smoothed texture noise with a smooth blob mask shared across channels, with σ = 1 in all experiments.The corruption is scaled by the image standard deviation; its smoothing scales, threshold, and edge exponent vary by draw.Source: Appendix C · p. 23
• Unlike coarse noise that perturbs the whole image, this corruption alters only selected regions at image-varying scales and leaves the rest unchanged.The comparison is with the coarse noise of Kascenas et al. (2022).Source: Appendix C · p. 23

Appendix D. Other test-time readouts

• Mean AUROC over seven datasets is 0.828 with noise kept and averaged over K = 8 draws, versus 0.817 without noise and 0.812 with binarization.Binarization costs little except on BraTS2021, where AUROC is 0.852 versus 0.911 with noise kept and averaged over K = 8 draws.Source: Appendix D · p. 23
• Binarization limits the code to N bits, and its estimated information averages 14,800 bits per image with the sparsity penalty versus 26,300 bits without it.The estimate sums binary entropy from test-set firing frequencies.Source: Appendix D · p. 23