(1.School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070,China; 2.Gansu Provincial Engineering Research Center for Artificial Intelligence and Graphics & Image Processing, Lanzhou730070, China) 在知网中查找 在百度中查找 在本站中查找
Affiliation:
Fund Project:
摘要
|
图/表
|
访问统计
|
参考文献
|
相似文献
|
引证文献
|
资源附件
摘要:
针对现有深度学习算法在壁画修复时,存在全局语义一致性约束不足及局部特征提取不充分,导致修复后的壁画易出现边界效应和细节模糊等问题,提出一种双向自回归Transformer与快速傅里叶卷积增强的壁画修复方法.首先,设计基于Transformer结构的全局语义特征修复模块,利用双向自回归机制与掩码语言模型(masked language modeling, MLM),提出改进的多头注意力全局语义壁画修复模块,提高对全局语义特征的修复能力.然后,构建了由门控卷积和残差模块组成的全局语义增强模块,增强全局语义特征一致性约束.最后,设计局部细节修复模块,采用大核注意力机制(large kernel attention, LKA)与快速傅里叶卷积提高细节特征的捕获能力,同时减少局部细节信息的丢失,提升修复壁画局部和整体特征的一致性.通过对敦煌壁画数字化修复实验,结果表明,所提算法修复性能更优,客观评价指标均优于比较算法.
Aiming at the lack of global semantic consistency constraints and insufficient acquisition of local features of the current deep learning algorithms in the process of image restoration of broken murals, resulting in the restored murals being prone to boundary effects and blurring of details, this paper proposes a bidirectional autoregressive Transformer with fast Fourier convolutional enhancement of murals restoration method. First, a global semantic feature repair module based on the Transformer structure is designed, and an improved multi-head attention global semantic mural repair module is proposed using the bidirectional autoregressive mechanism with masked language modeling (MLM) to improve the repair capability of global semantic features. Then, a global semantic enhancement module consisting of gated convolution and a residual module is constructed to enhance the global semantic consistency constraint. Finally, the local detail repair module is designed, which adopts large kernel attention (LKA) and fast Fourier convolution (FFC) to improve the ability of capturing detailed features while reducing the loss of local detail information, so as to enhance the consistency of the local and overall features of the repaired murals. The experimental results of the digital restoration of real Dunhuang murals show that the proposed algorithm can effectively restore the structure and texture of the murals, and the subjective visual effect and objective evaluation indexes are better than the comparative algorithms.