Skip to main content
SpringerLink
Log in

RSOD: Real-time small object detection algorithm in UAV-based traffic monitoring

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The prevailing applications of Unmanned Aerial Vehicles (UAVs) in transportation systems promote the development of object detection methods to collect real-time traffic information through UAVs. However, due to the small size and high density of objects from the aerial perspective, most existing algorithms are difficult to accurately process and extract informative features from the traffic images collected by UAVs. To address the challenges, this paper proposes a new real-time small object detection (RSOD) algorithm based on YOLOv3, which improves the small object detection accuracy by (i) using feature maps of a shallower layer containing more fine-grained information for location prediction; (ii) fusing local and global features of shallow and deep feature maps in Feature Pyramid Network(FPN) to enhance the ability to extract more representative features; (iii)assigning weights to output features of FPN and fusing them adaptively; and(iv) improving the excitation layer in Squeeze-and-Excitation attention mechanism to adjust the feature responses of each channel more precisely. Experimental results show that, when the input size is 608 × 608 × 3, the precision of the proposed RSOD algorithm measured by mAP@0.5 is 43.3% and 52.7% on the Visdrone-DET2018 and UAVDT datasets, which is 3.4% and 5.1% higher than those of YOLOv3, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Zhao P, Hu H (2019) Geographical patterns of traffic congestion in growing megacities: Big data analytics from Beijing [J]. Cities 92:164–174

    Article  Google Scholar 

  2. El-Sayed H, Chaqfa M, Zeadally S et al (2019) A traffic-aware approach for enabling unmanned aerial vehicles (UAVs) in smart city scenarios [J]. IEEE Access 7:86297–86305

    Article  Google Scholar 

  3. Kong Z, Zhang N, Guan X et al (2021) Detecting slender objects with uncertainty based on keypoint-displacement representation [J]. Neural Networks 139:246–254

    Article  Google Scholar 

  4. Ren J, Guo Y, Zhang D et al (2018) Distributed and efficient object detection in edge computing: challenges and solutions [J]. IEEE Network 32(6):137–143

    Article  Google Scholar 

  5. Sun W, Zhang X, He X (2020) A Two-stage vehicle type recognition method combining the most effective gabor features [J]. Cmc-Computers Materials And Continua 65(3):2489–2510

    Article  Google Scholar 

  6. Fan H, Wen L, Du D et al (2020) VisDrone-SOT2020: The Vision Meets Drone Single Object Tracking Challenge Results [C]. In: 2020 European conference on computer vision (ECCV), pp 728–749

  7. Du D, Qi Y, Yu H et al (2018) The unmanned aerial vehicle benchmark: Object detection and tracking [C]. In: 2018 European conference on computer vision (ECCV), pp 370–386

  8. Zhu P, Wen L, Du D et al (2018) Visdrone-det2018: The vision meets drone object detection in image challenge results [C]. In: 2018 European conference on computer vision workshops

  9. Liu L, Ouyang W, Wang X et al (2020) Deep learning for generic object detection: A survey [J]. International Journal of Computer Vision 128(2):261–318

    Article  Google Scholar 

  10. Haas T, Schubert C, Eickhoff M et al (2020) BubCNN: Bubble detection using Faster RCNN and shape regression network [J]. Chemical Engineering Science 216:115467

    Article  Google Scholar 

  11. Li Z, Li Y, Yang Y et al (2021) A high-precision detection method of hydroponic lettuce seedlings status based on improved Faster RCNN [J]. Computers and Electronics in Agriculture 182:106054

    Article  Google Scholar 

  12. Zhang Q, Chang X, Bian S B (2020) Vehicle-damage-detection segmentation algorithm based on improved mask RCNN [J]. IEEE Access 8:6997–7004

    Article  Google Scholar 

  13. Sun X, Wu P, Hoi S C H (2018) Face detection using deep learning: An improved faster RCNN approach [J]. Neurocomputing 299:42–50

    Article  Google Scholar 

  14. Zhai S, Shang D, Wang S et al (2020) DF-SSD: an improved ssd object detection algorithm based on DenseNet and feature fusion [J]. IEEE Access 8:24344–24357

    Article  Google Scholar 

  15. Fu K, Zhang T, Zhang Y et al (2019) Meta-SSD: Towards fast adaptation for few-shot object detection with meta-learning [J]. IEEE Access 7:77597–77606

    Article  Google Scholar 

  16. Huang Z, Wang J, Fu X et al (2020) DC-SPP-YOLO: Dense connection and spatial pyramid pooling based YOLO for object detection [J]. Information Sciences 522:241–258

    Article  MathSciNet  Google Scholar 

  17. Liu Y, Sun P, Wergeles N et al (2021) A survey and performance evaluation of deep learning methods for small object detection [J]. Expert Systems with Applications 2021:114602

    Article  Google Scholar 

  18. Tong K, Wu Y, Zhou F (2020) Recent advances in small object detection based on deep learning: A review [J]. Image and Vision Computing 97:103910

    Article  Google Scholar 

  19. Pérez-Hernández F, Tabik S, Lamas A et al (2020) Object detection binary classifiers methodology based on deep learning to identify small objects handled similarly: application in video surveillance [J]. Knowledge-Based Systems 194:105590

    Article  Google Scholar 

  20. Singh B, Davis L S (2018) An analysis of scale invariance in object detection snip [C]. In: 2018 IEEE conference on computer vision and pattern recognition (CVPR), pp 3578–3587

  21. Lin T Y, Girshick R (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition(CVPR), pp 2117–2125

  22. Liu Y, Yang F, Hu P (2020) Small-object detection in UAV-captured images via multi-branch parallel feature pyramid networks [J]. IEEE Access 8:145740–145750

    Article  Google Scholar 

  23. Zhang X, Izquierdo E, Chandramouli K (2019) Dense and small object detection in uav vision based on cascade network [C]. In: 2019 the IEEE/CVF international conference on computer vision workshops

  24. Bai Y, Zhang Y, Ding M et al (2018) Sod-mtgan: Small object detection via multi-task generative adversarial network [C]. In: The European conference on computer vision (ECCV), pp 206–221

  25. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv:1804.02767

  26. Sun W, Zhang X, Shi S et al (2019) Vehicle classification approach based on the combined texture and shape features with a compressive DL [J]. IET Intell Transp Syst 13(7):1069– 1077

    Article  Google Scholar 

  27. Tian F, Gao Y, Fang Z et al (2021) Depth estimation using a self-supervised network based on cross-layer feature fusion and the quadtree constraint [J]. In: IEEE transactions on circuits and systems for video technology

  28. Liu S, Qi L, Qin H et al (2018) Path aggregation network for instance segmentation [C]. In: IEEE conference on computer vision and pattern recognition(CVPR), vol 2018, pp 8759–8768

  29. Ghiasi G, Lin T Y, Le Q V (2019) Nas-fpn: Learning scalable feature pyramid architecture for object detection [C]. In: The IEEE/ CVF conference on computer vision and pattern recognition (CVPR), pp 7036–7045

  30. Xu H, Yao L, Zhang W et al (2019) Auto-fpn: Automatic network architecture adaptation for object detection beyond classification [C]. In: The IEEE/CVF international conference on computer vision (ICCV), vol 2019, pp 6649–6658

  31. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks [C]. In: The IEEE conference on computer vision and pattern recognition (CVPR), vol 2018, pp 7132–7141

  32. Woo S, Park J, Lee J Y et al (2018) Cbam: convolutional block attention module [C]. In: The European conference on computer vision (ECCV), vol 2018, pp 3–19

  33. Gao P, Yuan R, Wang F et al (2020) Siamese attentional keypoint network for high performance visual tracking [J]. Knowledge-based systems 193:105448

    Article  Google Scholar 

  34. Gao P, Zhang Q, Wang F et al (2020) Learning reinforced attentional representation for end-to-end visual tracking [J]. Information Sciences 517:52–67

    Article  Google Scholar 

  35. Gómez-Ríos A, Tabik S, Luengo J et al (2019) Towards highly accurate coral texture images classification using deep convolutional neural networks and data augmentation [J]. Expert Systems with Applications 118:315–328

    Article  Google Scholar 

  36. Li C, Yang T, Zhu S et al (2020) Density map guided object detection in aerial images [C]. In: The IEEE/CVF conference on computer vision and pattern recognition workshops, vol 2020, pp 190–191

  37. Yang F, Fan H, Chu P et al (2019) Clustered object detection in aerial images [C]. In: The IEEE/CVF international conference on computer vision (ICCV), vol 2019, pp 8311–8320

  38. Liu Y, Gu Y C, Zhang XY et al (2020) Lightweight salient object detection via hierarchical visual perception learning [J]. IEEE Transactions on Cybernetics 2020:1–11

    Google Scholar 

  39. Sun W, Zhang X, He X. (2019) Lightweight image classifier using dilated and depthwise separable convolutions [J]. Journal of Cloud Computing 2020 9(1):1–12

    Google Scholar 

  40. Choudhary T, Mishra V, Goswami A et al (2020) A comprehensive survey on model compression and acceleration [J]. Artif Intell Rev 2020 53(7):5113–5155

    Article  Google Scholar 

  41. Gou J, Yu B, Maybank SJ et al (2021) Knowledge distillation: a survey [J]. Int J Comput Vis 2021 129(6):1789–1819

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported in part by National Natural Science Foundation of China (No.61304205, 61502240), the Natural Science Foundation of Jiangsu Province (BK20191401, BK20201136) and the Innovation and Entrepreneurship Training Project of College Students(202010300290, 202010300211, 202010300116E).

Author information

Authors and Affiliations

  1. School of Automation, Nanjing University of Information Science and Technology, 210044, Nanjing, China

    Wei Sun, Liang Dai & Pengshuai Chang

  2. Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, 210044, Nanjing, China

    Wei Sun & Xiaorui Zhang

  3. Jiangsu Engineering Center of Network Monitoring, 210044, Nanjing, China

    Xiaorui Zhang

  4. Wuxi Research Institute, Nanjing University of Information Science and Technology, Wuxi, 214100, China

    Xiaorui Zhang

  5. Department of Civil and Environmental Engineering, Rensselaer Polytechnic Institute, 12180, Troy, MI, USA

    Xiaozheng He

Authors
  1. Wei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaorui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pengshuai Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaozheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, W., Dai, L., Zhang, X. et al. RSOD: Real-time small object detection algorithm in UAV-based traffic monitoring. Appl Intell 52, 8448–8463 (2022). https://doi.org/10.1007/s10489-021-02893-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02893-3

Keywords

Search

Navigation