Source-linked AI summary
DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion
Jinyuan Liu, Bowei Zhang, Qingyun Mei, Xingyuan Li, Yang Zou, Zhiying Jiang, Long Ma, Risheng Liu, Xin Fan
TL;DR
Existing IVIF methods often separate fusion from high-level vision, limiting task gains and feedback for fusion optimization. DCEvo jointly addresses visual quality and task accuracy through evolutionary multi-objective optimization, discriminative enhancement, and cross-dimensional feature embedding. Experiments across benchmark tasks report superiority over state-of-the-art methods, while the framework assumes temperature-highlighted objects for its discriminative enhancement.
Problem
Existing IVIF approaches commonly optimize visual fusion separately from high-level tasks, providing only marginal detection gains and limited feedback for fusion optimization.
Method
DCEvo uses an Evolutionary Algorithm to balance cross-task objectives, with discriminative enhancers and cross-dimensional embedding connecting modality, fusion, and task features.
Results
DCEvo surpasses state-of-the-art methods across three task benchmarks while enhancing both visual quality and task accuracy.
Takeaways & Limitations
DCEvo presents evolutionary learning as an integrated direction for improving visual fusion and perception in intelligent systems.
Takeaways & Limitations
The discriminative enhancer assumes that objects are temperature-highlighted in infrared and that µo > µ > µb.
Abstract
from arXiv · showhide
Infrared and visible image fusion integrates information from distinct spectral bands to enhance image quality by leveraging the strengths and mitigating the limitations of each modality. Existing approaches typically treat image fusion and subsequent high-level tasks as separate processes, resulting in fused images that offer only marginal gains in task performance and fail to provide constructive feedback for optimizing the fusion process. To overcome these limitations, we propose a Discriminative Cross-Dimension Evolutionary Learning Framework, termed DCEvo, which simultaneously enhances visual quality and perception accuracy. Leveraging the robust search capabilities of Evolutionary Learning, our approach formulates the optimization of dual tasks as a multi-objective problem by employing an Evolutionary Algorithm (EA) to dynamically balance loss function parameters. Inspired by visual neuroscience, we integrate a Discriminative Enhancer (DE) within both the encoder and decoder, enabling the effective learning of complementary features from different modalities. Additionally, our Cross-Dimensional Embedding (CDE) block facilitates mutual enhancement between high-dimensional task features and low-dimensional fusion features, ensuring a cohesive and efficient feature integration process. Experimental results on three benchmarks demonstrate that our method significantly outperforms state-of-the-art approaches, achieving an average improvement of 9.32% in visual quality while also enhancing subsequent high-level tasks. The code is available at https://github.com/Beate-Suy-Zhang/DCEvo.
1. Introduction
Infrared and visible images provide complementary strengths, but existing fusion methods often separate visual enhancement from downstream perception. DCEvo addresses this gap through evolutionary multi-objective optimization and cross-dimensional feature learning.
- Infrared images work in darkness and adverse weather but have low resolution and limited textures, whereas visible images provide detail but struggle in low light.
- IVIF seeks both clearer fused images for human observers and improved downstream performance in tasks such as object detection and scene analysis.
- Existing methods commonly separate fusion and detection, yielding only marginal detection gains and little feedback for optimizing fusion.
- Joint optimization is difficult because fusion and high-level tasks require different features, while manual loss weighting can improve one objective at the expense of the other.
- DCEvo uses an Evolutionary Algorithm to update cross-task loss coefficients within a multi-objective formulation.
- Its Discriminative Enhancer learns complementary modality features in encoder and decoder components, while Cross-Dimensional Embedding transfers task features into fusion features.
2. Related work
Related work spans deep learning architectures for IVIF and increasingly task-oriented methods that connect fusion with detection or segmentation. Evolutionary learning contributes global, population-based search for complex multi-objective optimization.
- IVIF research uses autoencoders, CNNs, GANs, Transformers, and diffusion models to improve multimodal feature processing and fusion.
- Task-oriented methods integrate detection or segmentation networks to guide fusion toward downstream perception objectives.
- TarDAL jointly optimizes fusion and detection, MetaFusion uses detection features to enhance fusion, and SegMiF connects fusion with segmentation.
- DCEvo’s architecture couples pixel-level fusion features with task-level features and uses evolutionary learning to search optimization-objective coefficients.
- Evolutionary learning iteratively refines candidate solutions through mechanisms including selection, crossover, and mutation.
- Multi-objective optimization represents trade-offs among conflicting objectives through Pareto-optimal solutions, which evolutionary learning searches without gradient flow.
3. Motivation
Collaboratively optimizing image fusion and downstream perception is challenging because the problem is typically discrete and existing methods can become trapped in local optima. Evolutionary learning is motivated as a way to explore these complex optimization spaces.
- Image fusion and downstream perception involve severe collaborative-optimization challenges because their problems are typically discrete and existing methods may fall into local optima.
- Evolutionary learning is suited to this setting because it simulates natural selection and genetic mechanisms to explore candidate solutions.
4. Methodology
DCEvo jointly optimizes image fusion and downstream perception using evolutionary hyperparameter learning, discriminative feature enhancement, and cross-dimensional embedding. Its modules balance competing objectives, emphasize prominent objects, and integrate detection features into fusion.
- 4.1. Learning evolutionary hyperparameters: Figure 3 contrasts low-level-only fusion, cascaded detection-guided fusion, and DCEvo’s evolutionary cooperative optimization of both tasks.The comparison highlights DCEvo’s training strategy rather than a new network workflow for inference.
- 4.1. Learning evolutionary hyperparameters: DCEvo frames fusion and downstream perception as a multi-objective problem and uses an evolutionary algorithm to optimize loss-function coefficients.The genetic algorithm evaluates fitness from loss values, then applies selection, crossover, mutation, and roulette-wheel selection across generations.
- 4.1. Learning evolutionary hyperparameters: The fusion objective combines structural similarity, decomposition, gradient, and intensity constraints, while detection uses classification, bounding-box, and distribution-focal losses.The detection loss is defined as LDet = Lcls + Lbox + Ldfl; the fusion loss includes LSSIM, Ldeco, Lgrad, and Lint.
- 4.2. Discriminative enhancer: The Discriminative Enhancer models object and background feature differences and is inserted into both the fusion encoder and decoder.It uses sigmoid-mapped pixel values to assign importance weights and generate enhanced feature maps with prominent-object information.
- 4.3. Cross-dimensional feature embedding: Cross-Dimensional Embedding integrates high-dimensional detection features with low-dimensional infrared and visible fusion features.Patch Align extracts the corresponding detection patch before separate CNN blocks and cross-dimensional embedding combine the representations.
5. Experiments
DCEvo is evaluated for image fusion quality, downstream detection and segmentation, and component-level effectiveness. Across these experiments, it achieves strong fusion metrics and the best reported downstream and ablation results.
- Experimental Setup: Experiments cover IVIF on M3FD, RoadScene, TNO, and FMB, plus object detection on M3FD and segmentation on FMB.The fusion model is pretrained on MSRS and then trained on M3FD with evolutionary learning.
- Infrared and Visible Image Fusion: DCEvo ranks best on all four fusion metrics for RoadScene, TNO, and FMB, while achieving the highest MI and SSIM on M3FD.The results indicate higher fidelity, more edge information, and less distortion relative to the inputs.
- Task-driven Downstream IVIF Applications: DCEvo attains the highest mAP for object detection on M3FD, and its fused images detect all objects in the qualitative comparison.Most fusion methods outperform visible-only or infrared-only inputs on mAP.
- Task-driven Downstream IVIF Applications: DCEvo improves semantic-segmentation mIoU by 1.4–7.2% over compared approaches and achieves the best qualitative segmentation results.The evaluation uses Segformer-b1 on the FMB dataset.
- Ablation Study: The evolutionary algorithm outperforms experience-based, uniform, and separate-optimization training strategies in quantitative metrics and qualitative precision.The comparison includes cooperative training for jointly optimizing fusion and detection networks.
- Ablation Study: Ablations show that the complete DCEvo architecture achieves the highest downstream-task performance, while DE activations focus on object regions near the ground truth.The architecture study evaluates discriminative-enhancer variants and cross-dimensional feature-embedding strategies.
6. Concluding Remarks
DCEvo is presented as an evolutionary-learning framework that jointly improves visual quality and task accuracy in infrared and visible image fusion. Its reported results surpass state-of-the-art methods across three task benchmarks, with broader implications for integrated intelligent-vision applications.
- Concluding Remarks: DCEvo combines discriminative enhancers, cross-dimensional feature embedding, and evolutionary learning to balance visual-fusion and perception objectives.The framework optimizes feature interactions and dual-task objectives.
- Concluding Remarks: DCEvo surpasses state-of-the-art methods across benchmarks covering three tasks and is reported to support complex environments and intelligent systems.The conclusion characterizes the approach as a robust solution within the evaluated scope.
- Broader Impacts: The framework is positioned as an evolutionary-learning direction for perception in intelligent systems, using adaptive hyperparameters for multi-dimensional optimization.The stated broader impact concerns integrated vision applications.