焊接缺陷识别中的多模态注意力方法

赵新玉; 马小创; 李正光; 张佳莹

doi:10.12073/j.hj.20220126003

焊接缺陷识别中的多模态注意力方法

计量
- 文章访问数: 127
- HTML全文浏览量: 27
- PDF下载量: 45
出版历程
- 收稿日期: 2022-01-25
- 修回日期: 2022-03-28
- 发布日期: 2022-12-24

Multi-modal attention method in welding defect recognition

摘要

摘要: 提出了一种多模态焊接缺陷识别方法，构建了包含3个分支的卷积神经网络,以分别对焊接熔池图片、电弧声、焊接电流和电弧电压进行处理。并在图像分支网络中加入了通道注意力模块和空间注意力模块，以聚焦焊接熔池图片的重要区域。为了验证文中模型的稳定性和可靠性，在自构建的包含10种焊接缺陷的数据集上进行了试验。试验结果表明，双通道注意力机制嵌入到卷积神经网络的浅层效果优于深层。同时，相比于不加注意力机制，双通道注意力机制识别结果的F值得到了明显的提升，为焊接实时分类识别提供参考，有助于焊接质量评定。创新点： (1)提出了多模态卷积神经网络自动提取焊接熔池图像、焊接电流、电弧电压、电弧声的显著特征。(2)在图像网络分支加入了注意力机制，帮助模型捕获缺陷显著区域，并验证了在卷积浅层引入注意力效果优于深层。
- 焊接缺陷 /
- 卷积神经网络 /
- 交叉验证 /
- 注意力机制
Abstract: A multi-modal welding defect identification method was proposed. A convolutional neural network with three branches was constructed to process welding pool pictures, arc sound, welding current and arc voltage, respectively. A channel attention module and a spatial attention module were added to image branch network to focus on important areas of weld pool image. In order to verify stability and reliability of the model in this paper, experiments were carried out on a self-constructed dataset containing 10 welding defects. The experimental results demonstrated that performance of introducing attention mechanism into front layers was better than that of back layers. Furthermore, attention mechanism contributed to promoting F value, which provided a reference for real-time welding classification and identification and was helpful for welding quality assessment.Highlights: (1) A multimodal convolutional neural network was proposed to automatically extract salient features of welding pool image, welding current, arc voltage and arc sound.(2) An attention mechanism was fused into image network branch to help model capture significant defect regions, and it was verified that effect of introducing attention in shallow convolution layer was better than that in deep layer.
- welding defect /
- convolutional neural network /
- cross-validation /
- attention mechanism

HTML全文

参考文献(17)

[1]	Bo C, Chen R. Multi-sensor information fusion in pulsed GTAW based on fuzzy measure and fuzzy integral[J]. Assembly Automation, 2010, 30(3):276-285.
[2]	Zhang Z, Chen H, Xu Y, et al. Multisensor-based real-time quality monitoring by means of feature extraction, selection and modeling for Al alloy in arc welding[J]. Mechanical Systems & Signal Processing, 2015, 60-61(8):151-165.
[3]	Zhang Z, Wen G, Chen S. Audible sound-based intelligent evaluation for aluminum alloy in robotic pulsed gtaw: mechanism, feature selection and defect detection[J]. IEEE Transactions on Industrial Informatics,2017,12:2973-2983.
[4]	Zhang B, Shi Y, Gu S. Narrow-seam identification and deviation detection in keyhole deep-penetration TIG welding[J]. International Journal of Advanced Manufacturing Technology, 2019，101：2051-2064.
[5]	You D, Gao X, Katayama S. Multisensor fusion system for monitoring high-power disk laser welding using support vector machine[J]. IEEE Transactions on Industrial Informatics, 2014, 10(2):1285-1295.
[6]	Cui Y, Shi Y, Zhu T, et al. Welding penetration recognition based on arc sound and electrical signals in K-TIG welding[J]. Measurement, 2020,163:107966.
[7]	Sumesh A, Nair B B, Rameshkumar K, et al. Decision tree based weld defect classification using current and voltage signatures in GMAW process[J]. Materials Today: proceedings, 2018, 5(2):8354-8363.
[8]	Sumesh A, Rameshkumar K, Mohandas K,et al. Use of machine learning algorithms for weld quality monitoring using acoustic signature[J].Procedia Computer Science,2015,50:316-322.
[9]	孙明健，刘婷，程星振，等. 基于多模态信号的金属材料缺陷无损检测方法[J]. 物理学报,2016,65(16):227-240.
[10]	胡文刚，刚铁. 基于超声信号和图像融合的焊缝缺陷识别[J]. 焊接学报,2013,34(4):53-56.
[11]	Du Bolun,He Yigang,He Yunze,et al.Intelligent classification of silicon photovoltaic cell defects based on eddy current thermography and convolution neural network[J]. IEEE Transactions on Industrial Informatics,2020,16(10):6242-6251．
[12]	张龙飞，高炜鑫，冯小星．基于郑积神经网络X射线球焊缺陷检测[J]. 焊接，2022(3): 26-34．
[13]	Bacioiu D, Melton G, Papaelias M, et al. Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning[J]. NDT & E international, 2019, 107(10):102139.1-102139.9.
[14]	尚远航，余游江，吴刚.基于混合注意力机制的植物病害识别[J].塔里木大学学报,2021,33(2):94-103.
[15]	Woo S, Park J, Lee J Y, et al. Cbam: convolutional block attention module[J]. Proceedings of the European Conference on Computer Vision， 2018：1-17．
[16]	周志华.机器学习[M].北京: 清华大学出版社, 2016.
[17]	Geoffrey E Hinton. To recognize shapes, first learn to generate images[J]. Progress in Brain Research,2007,165:535.

施引文献(3)

期刊类型引用(2)

1.	李国栋，吴志生，彭甫镕，郝康将，昝晓亮，郭威. 基于有监督对比学习的焊缝缺陷X射线检测方法. 焊接. 2024(07): 7-14 . 本站查看
2.	彭晓风，徐宏亮. 基于音视频特征的多模态英语发音纠错模型研究. 皖西学院学报. 2023(03): 123-129 . 百度学术

其他类型引用(1)

资源附件(0)

计量

文章访问数: 127
HTML全文浏览量: 27
PDF下载量: 45
被引次数: 3

焊接缺陷识别中的多模态注意力方法

计量

出版历程

Multi-modal attention method in welding defect recognition

期刊类型引用(2)

其他类型引用(1)

计量

出版历程

目录