基于N-RGAN模型的红外与可见光图像融合

Infrared and Visible Image Fusion Based on N-RGAN Model

  • 摘要: 目前,红外与可见光图像融合算法依然存在着对复杂场景适用性低、融合图像细节纹理信息大量丢失、对比度与清晰度不高等问题,针对上述存在的问题,本文结合非下采样剪切波变换(Non-Subsampled Shearlet Transform, NSST)、残差网络(Residual Network, ResNet)与生成对抗网络(Generative Adversarial Network, GAN)提出一种N-RGAN模型。通过NSST变换将红外与可见光图像分解为高频子带和低频子带;对高频子带进行拼接并输入由残差模块改进过的生成器,并将源红外图像作为判决标准,以此提升网络融合性能与融合图像细节刻画以及目标凸显能力;对红外图像与可见光图像进行显著性特征提取,通过自适应加权对低频子带进行融合,提升图像对比度与清晰度;对高频子带的融合结果与低频子带的融合结果进行NSST逆变换,从而得到红外与可见光图像的融合结果。通过与各类算法的融合结果进行对比,本文所提方法在峰值信噪比(Peak Signal to Noise Ratio, PSNR)、平均梯度(Average Gradient, AVG)、图像熵(Image Entropy, IE)、空间频率(Spatial Frequency, SF)、边缘强度(Edge Strength, ES)、图像清晰度(Image Clarity, IC)等多个客观指标上均有提高,可提升复杂场景下的红外与可见光图像融合效果,改善图像细节纹理信息损失严重的问题,同时提升图像对比度与清晰度。

     

    Abstract: At present, infrared and visible image fusion algorithms still have problems such as low applicability to complex scenes, large loss of detail and texture information in fusion images, and low contrast and sharpness of fusion images. In view of the above problems, this study proposes an N-RGAN model that combines a non-subsampled shearlet transform (NSST) and a residual network (ResNet). Infrared and visible images are decomposed into high- and low-frequency sub-bands using NSST. The high-frequency sub-bands are spliced and input into the generator improved by the residual module, and the source infrared image is taken as the decision standard to improve network fusion performance, fusion image detail description, and target-highlighting ability. The salient features of infrared and visible images are extracted, and the low-frequency sub-bands are fused by adaptive weighting to improve image contrast and sharpness. The fusion results of the high- and low-frequency sub-bands are obtained by the NSST inverse transformation. Based on a comparison of various fusion algorithms, the proposed method improves peak signal-to-noise ratio (PSNR), average gradient (AVG), image entropy (IE), spatial frequency (SF), edge strength (ES), and image clarity (IC), thereby improving infrared and visible light image fusion effects in complex scenes, alleviating information loss in image detail texture, and enhancing image contrast and resolution.

     

/

返回文章
返回