Skip to main content
SpringerLink
Log in

Research on recognition methods of spot-welding surface appearances based on transfer learning and a lightweight high-precision convolutional neural network

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Resistance spot welding is the most commonly used welding method in the welding process of automotive body-in-white manufacturing, but the appearance quality of the welding spot still relies on manual inspection, which is inefficient and error-prone. To this end, two methods based on deep learning are proposed to recognize welding spot appearances in this paper. In the first method, a practical convolutional neural network (CNN) model is quickly obtained by fine-tuning the VGG net. In the second method, the Release-Compression (RC) block is designed to fully utilize the power of convolution operation and greatly reduce the parameter number, and the information retention strategies are proposed to optimize the bottom and top of the network, so an ad-hoc CNN model named RswNet is obtained by combining RC block and information retention strategies. Experiment results show that the accuracies of the proposed two models are both higher than existing models, and RswNet has the higher accuracy and its parameters are reduced by more than 56% compared with existing models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Ali, H., Rada, L., & Badshah, N. (2018). Image segmentation for intensity inhomogeneity in presence of high noise. IEEE Transactions on Image Processing, 27(8), 3729–3738.

    Article  Google Scholar 

  • Amiri, N., Farrahi, G. H., Kashyzadeh, K. R., & Chizari, M. (2020). Applications of ultrasonic testing and machine learning methods to predict the static and fatigue behavior of spot-welded joints. Journal of Manufacturing Processes, 52, 26–34.

    Article  Google Scholar 

  • Bacioiu, D., Melton, G., Papaelias, M., Shaw, R. (2019). Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning. NDT and E International 107, 102139.1–102139.9.

  • Chen, H., Pang, Y., Hu, Q., & Liu, K. (2020). Solar Cell Surface Defect Inspection Based on Multispectral Convolutional Neural Network. Journal of Intelligent Manufacturing, 31(2), 453–468.

    Article  Google Scholar 

  • Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848.

    Article  Google Scholar 

  • Dai, W., Li, D., Tang, D., Jiang, Q., Wang, D., Wang, H., & Peng, Y. (2021). Deep learning assisted vision inspection of resistance spot welds. Journal of Manufacturing Processes, 62, 262–274.

    Article  Google Scholar 

  • Espinosa, A. R., Bressan, M., & Giraldo, L. F. (2020). Failure signature classification in solar photovoltaic plants using RGB images and convolutional neural networks - ScienceDirect. Renewable Energy, 162, 249–256.

    Article  Google Scholar 

  • Fan, J., & Wang, J. (2018). A two-phase fuzzy clustering algorithm based on neurodynamic optimization with its application for PolSAR image segmentation. IEEE Transactions on Fuzzy Systems, 26(1), 72–83.

    Article  Google Scholar 

  • Gavidel, S. Z., Lu, S., & Rickli, J. L. (2019). Performance analysis and comparison of machine learning algorithms for predicting nugget width of resistance spot welding joints. International Journal of Advanced Manufacturing Technology, 105(9), 3779–3796.

    Article  Google Scholar 

  • Guo, Z., Ye, S., Wang, Y., Lin, C. (2017). Resistance welding spot defect detection with convolutional neural networks. International Conference on Computer Vision Systems. Springer, Cham, 10528, 169–174.

  • He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 770–778.

  • Hu, Y., Soltoggio, A., Lock, R., & Carter, S. (2019). A fully convolutional two-stream fusion network for interactive image segmentation. Neural Networks, 109, 31–42.

    Article  Google Scholar 

  • Huang, G., Liu, Z., & Maaten, L. V. D. (2017). Densely connected convolutional networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2261–2269

  • Jiang, H., Hu, Q., Zhi, Z., Gao, J., Gao, Z., Wang, R., He, S., & Li, H. Convolution neural network model with improved pooling strategy and feature selection for weld defect recognition. Welding in the World, 2020(1).

  • Kim, Y., Kim, T., Youn, B. D., Ahn, S. H. (2021). Machining quality monitoring (MQM) in laser-assisted micro-milling of glass using cutting force signals: an image-based deep transfer learning. J Intel Manuf

  • Kim, M., Lee, M., An, M., & Lee, H. (2020). Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel. Journal of Intelligent Manufacturing, 31(5), 1165–1174.

    Article  Google Scholar 

  • Krizhevsky, A., Sutskever, I., Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), 2012: 1097–1105.

  • Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Article  Google Scholar 

  • Li, Y., Cao, G., Yu, Q., & Li, X. (2018). Active contours driven by non-local Gaussian distribution fitting energy for image segmentation. Applied Intelligence, 48(12), 4855–4870.

    Article  Google Scholar 

  • Lin, M., Chen, Q., Yan, S. (2014). Network in network. In: International Conference on Learning Representations (ICLR).

  • Lu, L., Shin, Y., Su, Y., & Karniadakis, G. E. (2020). Dying ReLU and Initialization: Theory and Numerical Examples. Communications in Computational Physics, 28(5), 1671–1706.

    Article  Google Scholar 

  • Martín, Ó., López, M., & Martín, F. (2007). Artificial neural networks for quality control by ultrasonic testing in resistance spot welding. Journal of Materials Processing Tech, 183(2–3), 226–233.

    Article  Google Scholar 

  • Maskey, A., Cho, M. (2020). CubeSatNet: Ultralight Convolutional Neural Network designed for on-orbit binary image classification on a 1U CubeSat. Engineering Applications of Artificial Intelligence 96.

  • Miao, J., Huang, T., Zhou, X., Wang, Y., & Liu, J. (2018). Image segmentation based on an active contour model of partial image restoration with local cosine fitting energy. Information Sciences: An International Journal, 447, 52–71.

    Article  Google Scholar 

  • Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J. (2017). Pruning convolutional neural networks for resource efficient inference. In: International Conference on Learning Representations (ICLR),.

  • Pereda, M., Santos, J. I., Martín, Ó., & Galán, J. M. (2015). Direct quality prediction in resistance spot welding process Sensitivity, specificity and predictive accuracy comparative analysis. Science and Technology of Welding and Joining, 20(8), 479–685.

    Article  Google Scholar 

  • Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014). CNN Features off-the-shelf: An Astounding Baseline for Recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2014, 512–519.

    Article  Google Scholar 

  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 779–788.

    Google Scholar 

  • Shang, J., An, W., Liu, Y., Han, B., & Guo, Y. (2020). Oil Pipeline Weld Defect Identification System Based on Convolutional Neural Network. Ksii Transactions on Internet and Information Systems, 14(3), 1086–1103.

    Google Scholar 

  • Shelhamer, E., Long, J., & Darrell, T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.

    Article  Google Scholar 

  • Simonyan, K. & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR).

  • Spampinato, C., Palazzo, S., Giordano, D., Aldinucci, M., & Leonardi, R. (2017). Deep learning for automated skeletal bone age assessment in X-ray images. Medical Image Analysis, 36, 41–51.

    Article  Google Scholar 

  • Sun, H., Yang, J., & Wang, L. (2016). Resistance spot welding quality identification with particle swarm optimization and a kernel extreme learning machine model. The International Journal of Advanced Manufacturing Technology, 91(5–8), 1879–1887.

    Google Scholar 

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1–9.

  • Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z., (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2818–2826.

  • Wang, X., Guan, S., Lin, H., Wang, B., & He, X. (2019). Classification of spot-welded joint strength using ultrasonic signal time-frequency features and PSO-SVM method. Ultrasonics, 91, 161–169.

    Article  Google Scholar 

  • Xia, Y., Zhou, L., Shen, Y., Wegner, D. M., Haselhuhn, A. S., Li, Y., & Carlson, B. E. (2020). Online measurement of weld penetration in robotic resistance spot welding using electrode displacement signals. Measurement, 168, 108397.

    Article  Google Scholar 

  • Yan, Y., Liu, D., Gao, B., Tian, G. Y., & Cai, Z. C. (2020). A Deep Learning-Based Ultrasonic Pattern Recognition Method for Inspecting Girth Weld Cracking of Gas Pipeline. IEEE Sensors Journal, 20(14), 7997–8006.

    Article  Google Scholar 

  • Yang, Y., Pan, L., Ma, J., Yang, R., Zhu, Y., Yang, Y., & Zhang, L. (2020). A High-Performance Deep Learning Algorithm for the Automated Optical Inspection of Laser Welding. Applied Sciences, 10(3), 933.

    Article  Google Scholar 

  • Ye, S., Guo, Z., Zheng, P., Wang, L., Lin, C. (2017). A vision inspection system for the defects of resistance spot welding based on neural network. In: International Conference on Computer Vision Systems. Springer, Cham 10528, 161–168.

  • Yu, F., Koltun, V. (2016). Multi-scale context aggregation by dilated convolutions. In: International Conference on Learning Representations (ICLR).

  • Zaharuddin, M. F. A., Kim, D., & Rhee, S. (2017). An ANFIS based approach for predicting the weld strength of resistance spot welding in artificial intelligence development. Journal of Mechanical Science and Technology, 31(11), 5467–5476.

    Article  Google Scholar 

  • Zhang, J., Zhang, P. (2012). A SVM regression predicting model for indentation depth of welding spot based on digital image processing. In: Proceedings of SPIE—The International Society for Optical Engineering 8334

Download references

Acknowledgements

The presented work was supported by the National Key Research and Development Project of China (no. 2020YFB1713300), the Technological Innovation and Application Development Project of Chongqing (no. cstc2019jscx-mbdxX0056), the National Natural Science Foundation of Chongqing (no. cstc2021jcyj-msxmX0732), and the Fundamental Research Funds for the Central Universities (no. 2021CDJKYJH021).

Author information

Authors and Affiliations

  1. State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing, 400044, China

    Meng Xiao, Bo Yang & Shilong Wang

  2. Chongqing Changan Automobile Co. Ltd, Chongqing, 401147, China

    Yongsheng Chang, Song Li & Gang Yi

Authors
  1. Meng Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shilong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongsheng Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Song Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gang Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Bo Yang or Shilong Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, M., Yang, B., Wang, S. et al. Research on recognition methods of spot-welding surface appearances based on transfer learning and a lightweight high-precision convolutional neural network. J Intell Manuf 34, 2153–2170 (2023). https://doi.org/10.1007/s10845-022-01909-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01909-0

Keywords

Search

Navigation