PaperlayerView exact revision on arXiv

Source-linked AI summary

An Attention-Guided Global and Local Fusion Framework for Lesion-Focused Image Classification

Mst Shafia Tasnima, Md Samaun Elaheea, Tanjim Taharat Aurpab, Md Musfique Anwar

arXiv:2609.04791v1cs.CV

TL;DR

Lesion-focused classification must identify sparse, spatially dispersed cues while preserving useful global context. The paper proposes a three-branch DenseNet-121 framework that uses Grad-CAM and CBAM for attention-guided local learning and adaptively fuses global and local features. Across four datasets, the fusion branch consistently produced more stable and accurate predictions, while increased background complexity remained a limitation.

  • Problem

    Sparse, localized lesion cues can be obscured in full-image representations, while lesion-only approaches may discard contextual information.

  • Method

    A three-branch DenseNet-121 framework uses full-image features, Grad-CAM-generated masked inputs, CBAM-enhanced local features, and sample-specific adaptive fusion weights.

  • Results

    Across four datasets, adaptive fusion consistently combined complementary global and local features more effectively, producing more stable and accurate predictions.

  • Takeaways & Limitations

    The findings support jointly modeling global context and localized lesion regions for lesion- or spot-focused classification.

  • Takeaways & Limitations

    Performance degrades as background complexity increases, limiting effectiveness in cluttered scenes.

Abstract

from arXiv · show

Lesion-focused image classification presents a core analytical challenge, as discriminative signals are often sparse, spatially dispersed, and easily obscured by background noise, while conventional convolutional neural networks (CNNs) process entire images uniformly and may dilute signal relevance. This study hypothesizes that adaptive fusion of global contextual information and lesion-focused local information can improve classification performance compared with using either representation independently. We propose a three-branch, attention-guided deep learning framework built on Densely Connected Convolutional Network-121 (DenseNet-121) to improve feature attribution, interpretability, and classification reliability. The architecture consists of a global branch that learns representations from full images, followed by Gradient-weighted Class Activation Mapping (Grad-CAM) to generate attention maps that highlight prediction-relevant regions and produce masked inputs, and a local branch enhanced with a Convolutional Block Attention Module (CBAM) to extract refined spatial and channel-wise features from these focused regions. An adaptive fusion branch integrates global and local representations by learning instance-specific weights, allowing dynamic prioritization between contextual and localized information. The framework is evaluated on a synthetic Spot Pattern Dataset (SSPD) and three benchmark datasets, including skin lesion, guava leaf, and grape leaf image datasets, where the fusion branch outperformed the individual global and local branches, reaching 97.75% accuracy on the skin lesion dataset and 99.64% on the guava leaf dataset. The results highlight the value of attention-guided architectures in healthcare analytics by improving model transparency, strengthening feature relevance, and supporting more reliable data-driven decision-making in medical image analysis.

1. Introduction

Lesion-focused classification must capture sparse, localized cues without discarding useful global context. The proposed framework addresses this through attention-guided localization and adaptive fusion of global and lesion-focused representations.

  • Small, scattered lesion cues may be insufficiently captured by full-image representations, especially when lesion size, shape, and location vary.
  • ROI-focused methods can discard contextual information and remain sensitive to localization or segmentation errors.
  • The framework extracts global features, uses Grad-CAM heatmaps to create masked images, and learns lesion-focused local features before fusion.
  • Adaptive weighting dynamically balances global context and localized cues according to each input image.
  • The study evaluates one synthetic and three real-world datasets spanning medical and agricultural domains, including analysis of background complexity.
  • Related global–local systems commonly concatenate or fuse branch features, whereas this framework emphasizes adaptive integration of branch contributions.

Lesion-focused classification

Lesion-focused classification targets classes defined primarily by lesion characteristics, motivating approaches that localize lesions while preserving informative image context. The proposed method combines full-image and localized representations instead of relying exclusively on either.

  • Lesion-centric classes are primarily defined by lesion characteristics rather than global image appearance.
  • Reported related methods include threshold-based lesion extraction, optimized sparse autoencoders, and lightweight CNN architectures for lesion classification.
  • Segmentation-based classification can be sensitive to the correctness of lesion localization or extracted lesion regions.
  • The proposed method combines global contextual information with localized lesion-specific features while retaining complementary background cues.

Attention/Heatmap Guided Learning

Attention and heatmap methods support lesion localization and interpretability, but many prior approaches apply attention internally or rely on region-level supervision. This framework uses Grad-CAM for masked local learning and CBAM for local feature refinement.

  • Prior approaches use Grad-CAM-guided ROI cropping, feature masks, multi-scale attention, or region-level supervision to emphasize informative regions.
  • Some attention mechanisms operate mainly within intermediate feature representations without explicitly providing region-level visual localization.
  • The framework generates Grad-CAM heatmaps and masked images to guide lesion-focused learning without requiring region-level supervision.
  • CBAM refines spatial and channel-wise features in the local branch while supporting visual interpretability through attention-guided inputs.

3. Methodology

The methodology combines full-image context with lesion-focused representations in a unified, attention-guided architecture. It uses Grad-CAM masking, CBAM refinement, and adaptive fusion to produce sample-specific global–local features for classification.

  • Framework overview: The framework processes full images globally, generates Grad-CAM-guided masked inputs, extracts CBAM-refined local features, and adaptively fuses both representations.The architecture is organized around global, attention, local, and fusion stages.
  • Global branch: DenseNet-121 extracts global features from resized, normalized images, with the global branch fine-tuned by unfreezing its last 50 layers.The global feature vector is obtained after global average pooling.
  • Attention module: Grad-CAM identifies prediction-relevant regions, and thresholding at τ = 0.5 creates binary masks that suppress background while preserving discriminative cues.The mask is applied pixel-wise to generate the local branch input.
  • Local branch with CBAM: The local branch applies CBAM sequentially through channel and spatial attention before global average pooling to refine features from masked images.CBAM is inserted after DenseNet-121 convolutional feature extraction and before pooling.
  • Adaptive weighted fusion: A gating network maps concatenated global and local features to normalized sample-specific weights, then combines them into a fused representation for classification.The fusion module uses a lightweight network with fully connected layers of 256 and 128 units followed by a two-unit softmax.
  • Training strategy: The proposed system is trained in three stages, including separate global and local model training before adaptive fusion.The supplied methodology identifies a three-stage training strategy and describes local training on Grad-CAM-masked images.

4. Results

The study evaluates the model across four distinct datasets spanning synthetic, medical, and agricultural images, with dataset splits and class distributions summarized in Fig. 2.

  • Four datasets span synthetic patterns and real-world medical and agricultural images.The setup is intended to assess adaptability across domains and scales.
  • Testing results and ablation analyses use the corresponding held-out test subsets for all datasets.

(i) Dataset 1: Synthetic Spot Pattern Dataset (SSPD)

The Synthetic Spot Pattern Dataset contains four classes with distinct spatial structures and randomly placed bulbs that simulate scattered, localized visual cues.

  • Four classes represent filled circular bulbs, hollow ring bulbs, elliptical bulbs with filaments, and string-like bulbs on diagonal curves.
  • Random bulb placement simulates scattered, small, localized visual cues.
  • The dataset is designed to demonstrate the fusion branch’s advantage over the individual global and local branches.

(iii) Dataset 3: Guava Leaf Disease Dataset

The augmented guava leaf disease dataset contains 10,000 images across five classes and targets diseases expressed as visible spots or localized leaf regions.

  • 10,000 augmented images are distributed across five classes: canker, dot, mummification, rust, and healthy.Augmentation increased sample size and balanced class distributions.
  • Visible spot patterns and localized leaf regions make the dataset suitable for evaluating the model.

(iv) Dataset 4: Grape Leaf Disease Dataset

The grape leaf dataset contains 3,600 augmented images across four disease classes, and the fusion branch consistently outperformed the global and local branches. Results indicate that combining global context with lesion-focused features improves classification and reduces class-specific errors.

  • Dataset: The grape leaf dataset contains 3,600 augmented images across four classes: bacterial leaf spot, downy mildew, powdery mildew, and healthy.The images were obtained from the publicly available NGLD dataset and augmented for class balance.
  • Model configuration: DenseNet-121 was selected because it achieved the highest overall fusion accuracy across the evaluated datasets.The global branch uses the fine-tuned backbone for full-image contextual feature extraction.
  • Model configuration: The local model was trained independently on masked images, and independent training produced better fusion performance than global-weight initialization.Global initialization slightly improved local accuracy for Dataset 3, but fusion performance benefited more from independent branches.
  • Overall performance: The fusion branch achieved the best performance across all four datasets, integrating global contextual information with lesion-focused local representations.The reported accuracy gains over the global and local branches were 2.36% and 7.08%, respectively, for Dataset 4.
  • Limitations: A limitation is that Grad-CAM masking may suppress useful global cues, causing the Local branch to underperform the Global branch when class information lies outside highlighted regions.The fusion strategy addresses this limitation by combining complementary global and local representations.
  • Class-wise performance: The fusion configuration consistently achieved the highest class-wise F1-scores and restored balanced performance where the local branch degraded across multiple classes.This pattern was reported for all four datasets, including Dataset 4.
  • Error analysis: The Fusion model reduced cross-class errors in Dataset 2 and substantially reduced rust-related misclassification in Dataset 3.For Dataset 3, correct predictions increased to 498 canker, 499 dot, 499 healthy, 497 mummification, and 498 rust samples.
  • Statistical comparison: On the Guava dataset, the Fusion model misclassified 9 samples versus 31 for the Global model, with a significant McNemar test result of p = 5.95 × 10^-5.The Fusion model corrected 26 errors made by the Global model, while four Global-correct samples became incorrect.

5. Discussion

Across four datasets, adaptive fusion consistently combined complementary global and local representations more effectively than either branch alone. The discussion attributes this advantage to sample-specific weighting, attention-guided localization, and complementary branch design.

  • The framework uses Grad-CAM within the learning pipeline to guide local feature extraction rather than only applying it after prediction for explainability.
  • The global branch captured contextual and structural cues but could confuse classes with similar overall appearance and different lesion details.
  • The local branch emphasized lesion-specific details but could lose contextual structure, increasing interclass confusion when background information was removed.In source-held-out experiments, the local branch performed better than the global branch, suggesting lesion-centered features may transfer better across sources.
  • Adaptive fusion consistently achieved the highest accuracy by dynamically balancing complementary global and local representations.The weighting mechanism emphasizes the more reliable information source for each image, particularly when lesion visibility, size, and distribution vary.
  • Grad-CAM generally aligned with discriminative regions, but imperfect localization could omit contextual cues needed for classification.This limitation supports fusion instead of relying exclusively on ROI-based learning.
  • CBAM refined channel and spatial attention in masked regions, while independently trained branches produced better fusion performance than weight sharing.These ablations indicate that representation diversity strengthens complementary learning.

Sensitivity to Background Complexity

The framework is most effective with clear subjects and limited background clutter, but performance declines as background complexity increases. Background variation can impair Grad-CAM masks and subsequently affect local-branch learning, while fusion retains the value of contextual information.

  • Performance of the global, local, and fusion branches decreases under increased Gaussian background noise while the lesion region is preserved.The experiment adds noise to non-ROI regions while preserving the lesion area and a 10-pixel safety margin.
  • Background variation can reduce global-branch attribution quality, affecting Grad-CAM masks and subsequently influencing the local branch.
  • The local branch may suppress useful contextual information, including lesion shape, spatial distribution, surrounding texture, and background patterns.Fusion mitigates this limitation by combining global and local features.
  • The findings support adaptively balancing global contextual and local lesion-specific features because neither source alone is consistently optimal.
  • Prediction-relevant highlighted regions can provide clinicians with additional visual support during medical image analysis.The framework is also described as applicable to agricultural disease detection and other localized visual inspection tasks.

6. Conclusion

The study presents a three-branch attention-driven architecture for lesion- and spot-focused classification. Across four datasets, it reports consistent improvements over baseline models while preserving global context and selecting informative representations per sample.

  • The architecture captures lesion-relevant cues, preserves global contextual information, and adaptively selects the most informative representation for each sample.
  • Across four datasets, the proposed model demonstrated consistent performance improvements over baseline models.The study also used ablation studies to assess model components and hyperparameters.
  • Future work will explore advanced preprocessing, noise-aware attention mechanisms, and validation on larger independent datasets.

Funding sources

The study reports no specific grant funding from public, commercial, or not-for-profit funding agencies.

  • The research did not receive any specific grant from public, commercial, or not-for-profit funding agencies.
Loading 2609.04791v1…