Source-linked AI summary
Bayesian Fusion for Infrared and Visible Images
Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, Jiangshe Zhang
TL;DR
Infrared–visible fusion seeks to combine visible-image texture and detail with infrared thermal and target information for improved fused imagery. The paper proposes a hierarchical Bayesian regression model with a TV penalty and EM inference, reporting better target highlighting and texture details on TNO and NIR datasets than previous methods. The model relies on an exponential prior for aij, yielding an inverse-gamma distribution for ãij.
Problem
Infrared–visible fusion seeks one image containing visible-image gradient and texture information together with infrared thermal radiation and highlighted targets.
Method
The paper casts fusion as regression in a hierarchical Bayesian model, adds a total-variation penalty, and performs inference with the EM algorithm.
Results
Compared with previous methods on TNO and NIR datasets, the method generates fused images with highlighted thermal targets and abundant texture details that facilitate automatic detection and accurate target positioning.
Takeaways & Limitations
The reported fused-image improvements can facilitate automatic detection and accurate positioning of targets.
Takeaways & Limitations
The inference assumes an exponential prior for aij, resulting in an inverse-gamma distribution for ãij.
Abstract
from arXiv · showhide
Infrared and visible image fusion has been a hot issue in image fusion. In this task, a fused image containing both the gradient and detailed texture information of visible images as well as the thermal radiation and highlighting targets of infrared images is expected to be obtained. In this paper, a novel Bayesian fusion model is established for infrared and visible images. In our model, the image fusion task is cast into a regression problem. To measure the variable uncertainty, we formulate the model in a hierarchical Bayesian manner. Aiming at making the fused image satisfy human visual system, the model incorporates the total-variation(TV) penalty. Subsequently, the model is efficiently inferred by the expectation-maximization(EM) algorithm. We test our algorithm on TNO and NIR image fusion datasets with several state-of-the-art approaches. Compared with the previous methods, the novel model can generate better fused images with high-light targets and rich texture details, which can improve the reliability of the target automatic detection and recognition system.
1. Introduction
Infrared–visible image fusion combines complementary thermal, target, appearance, texture, and detail information, but existing approaches span several method categories. The paper introduces a Bayesian fusion model that treats fusion as regression, models uncertainty hierarchically, uses a TV penalty, and is inferred with EM; experiments report improved fused-image targets and texture.
- Fusing visible and infrared images improves human visual perception for target detection and recognition.
- Infrared images provide thermal radiation and are less affected by illumination changes, while visible images offer richer appearance, texture, and detail information.
- Existing infrared–visible fusion methods are broadly grouped into multi-scale transformation, sparse representation, subspace learning, and saliency-based approaches.
- The paper addresses the stated Bayesian-model gap by presenting a novel Bayesian fusion model for infrared and visible images.
- The model casts fusion as regression, uses hierarchical Bayesian modeling for variable uncertainty, incorporates a TV penalty, and applies EM for inference.
- On TNO and NIR datasets, the method is reported to produce fused images with highlighted targets and rich texture details, improving target detection and recognition reliability.
2. Bayesian fusion model
The paper formulates infrared–visible image fusion as a hierarchical Bayesian regression model that combines source-image profiles with visible-image gradients. EM inference and alternating optimization estimate the fused image while incorporating uncertainty and regularization for texture preservation.
- Infrared–visible fusion is posed as obtaining an informative image from registered infrared and visible inputs.
- The model minimizes differences between the fused image and both source images to preserve their general profiles.The formulation uses loss functions, typically based on the L1 norm.
- The formulation represents the task as a linear regression model with Laplacian noise and a Laplacian prior, then rewrites it hierarchically using Gaussian scale mixtures.The mixture construction uses Gaussian distributions and exponential priors for latent variances.
- A gradient-sparsity regularizer preserves visible-image edge information and gives the fused image textures similar to the visible image.The regularization strength is controlled by λg.
- The hierarchical Bayesian graph contains latent variables and observed variables across two levels, with λ and τ acting as hyper-parameters.
- EM alternates expectation and maximization steps, while half-quadratic splitting and coordinate descent update X, F, and H.The updates include least-squares, L1-penalized regression, and deconvolution subproblems; FFT and inverse FFT solve the deconvolution step.
3. Experiments
Experiments evaluate the Bayesian fusion model on TNO and RGB-NIR datasets through qualitative comparisons and objective metrics. The method produces fused images with highlighted targets and rich texture details, and performs strongly across reported measures.
- Experimental setup: The experiments use 20 TNO image pairs and 52 RGB-NIR country-scene pairs to evaluate the fusion algorithm.TNO images were captured at night, whereas the NIR dataset was obtained in daylight.
- Subjective visual evaluation: The comparison includes TSIFVS, CSR, ADF, FPDE, and TVADMM methods across TNO and NIR examples.Figure 2 presents qualitative results for selected TNO and NIR image pairs.
- Subjective visual evaluation: Qualitative comparisons show the proposed method preserves highlighted targets and rich texture details better than the compared methods.Competing methods exhibit missing face, house, ground, window, tree, or cloud details, as well as low target brightness or visual blur.
- Objective quantitative evaluation: The evaluation uses EN, MI, QAB/F, SD, and SSIM, which quantify information, edge preservation, source agreement, and structural similarity.Larger metric values indicate better fused-image performance.
- Objective quantitative evaluation: In TNO, the proposed method ranks first in MI, QAB/F, and SD and second in EN and SSIM.Table 1 reports quantitative results for the proposed model and other popular fusion methods.
- Objective quantitative evaluation: In the RGB-NIR dataset, the method ranks first in MI and SD, and second in EN, QAB/F, and SSIM.The reported ranking supports the method’s strong performance relative to the compared fusion approaches.
4. Conclusion
The paper concludes with a Bayesian fusion model that formulates image fusion as hierarchical Bayesian regression, adds a TV penalty, and uses EM-based inference. Compared with previous methods, it generates images with highlighted thermal targets and abundant texture details that can facilitate target detection and positioning.
- Model: The proposed model formulates infrared-visible image fusion as a regression problem in a hierarchical Bayesian framework.The hierarchical formulation is used to model variable uncertainty.
- Model: A total-variation penalty is incorporated to make the fused image similar to the human visual system.The penalty is part of the model design rather than a post-processing step.
- Inference: The model is efficiently inferred using the expectation-maximization algorithm with half-quadratic splitting.The half-quadratic splitting algorithm is used within the EM-based inference procedure.
- Results: Compared with previous methods on TNO and NIR datasets, the method produces fused images with highlighted thermal targets and abundant texture details.The conclusion summarizes the method’s reported qualitative advantage over previous approaches.
- Results: These fused images can facilitate automatic target detection and accurate target positioning.The stated consequence concerns the use of fused images for target-analysis systems.