+高级检索
基于TransNeXt的红外与可见光图像融合
DOI:
作者:
作者单位:

作者简介:

通讯作者:

基金项目:


Fusion of Infrared and Visible Based on TransNeXt
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    针对红外与可见光图像融合过程中出现的细节丢失和易产生伪影等问题,本文提出了一种基于TransNeXt的融合算法.首先,通过卷积神经网络与TransNeXt对源图像进行浅层与深层特征提取,并通过信息补偿模块对红外浅层特征进行信息补偿,使其具有更多的语义信息.然后,通过基于交叉注意力的融合模块进行特征融合,它能够根据源图像不同区域的重要性调整权重以适应场景变化,提高融合结果的鲁棒性和准确性.最后,通过基于Transformer的模块进行图像重建以得到最终融合图像.此外,本文通过基于VGG19显著区域掩膜的损失函数约束融合过程,使融合结果在重要区域保留更丰富的信息.实验结果表明,与其他7种对比方法相比,本文方法的客观评价指标信息熵、标准差、差异相关性总和、峰值信噪比和像素特征互信息分别平均提高了10.92%、14.85%、24.80%、2.26%、1.30%,并且能够在保留丰富的纹理信息的同时伪影较少,具有优异的夜间灯光融合效果,在目标检测上相较对比方法也取得了更好的效果.

    Abstract:

    This paper proposes a fusion algorithm utilizing TransNeXt to address detail loss and artifact generation issues in the fusion of infrared and visible images. Firstly, shallow and deep features are extracted from the source images using convolutional neural networks and TransNeXt. An information compensation module is employed to enhance the semantic information of the infrared shallow features. Secondly, a cross-attention-based fusion module integrates these features, and dynamically adjusts weights based on the importance of different regions in the source images to adapt to scene variations, thereby improving fusion robustness and accuracy. The final fused image is obtained through Transformer-based image reconstruction. In addition,the proposed method constrains the fusion process through a VGG19-based saliency mask loss function, preserving richer information in key regions of the fused results. The experimental results indicate that, compared with the other seven methods, this approach has improved the objective evaluation metrics: namely information entropy, standard deviation, sum of correlation differ-ences, peak signal-to-noise ratio, and pixel feature mutual information, by an average of 10.92%, 14.85%, 24.80%, 2.26%, and 1.30%, respectively. Furthermore, it effectively preserves rich texture information while minimizing artifacts, demonstrating outstanding performance in night light fusion. Additionally, it has achieved superior results in object detection relative to the comparison methods.

    参考文献
    相似文献
    引证文献
文章指标
  • PDF下载次数:
  • HTML阅读次数:
  • 摘要点击次数:
  • 引用次数:
引用本文

杨艳春 ?,杨万轩 ,雷慧云.基于TransNeXt的红外与可见光图像融合[J].湖南大学学报:自然科学版,2025,52(8):69~79

复制
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2025-08-29
  • 出版日期:
作者稿件一经被我刊录用,如无特别声明,即视作同意授予我刊论文整体的全部复制传播的权利,包括但不限于复制权、发行权、信息网络传播权、广播权、表演权、翻译权、汇编权、改编权等著作使用权转让给我刊,我刊有权根据工作需要,允许合作的数据库、新媒体平台及其他数字平台进行数字传播和国际传播等。特此声明。
关闭