Muhammad K. Kabir; Anika N. Binte Kabir; Jahid H. Rony; Jia Uddin

Drone Detection from Video Streams Using Image Processing Techniques and YOLOv7

PDF (1010KB), PP.83-95

Views: 0 Downloads: 0

Author(s)

Muhammad K. Kabir ¹ Anika N. Binte Kabir ¹ Jahid H. Rony ² Jia Uddin ^3,*

1. Department of Computer Science and Engineering, Brac University, Dhaka 1212, Bangladesh

2. Department of Computer Science and Engineering, Dhaka University of Engineering and Technology, Gazipur, Bangladesh

3. Artificial Intelligence and Big Data Department, Woosong University, Daejeon, South Korea

* Corresponding author.

DOI: https://doi.org/10.5815/ijigsp.2024.02.07

Received: 14 Apr. 2023 / Revised: 6 Jul. 2023 / Accepted: 23 Oct. 2023 / Published: 8 Apr. 2024

Index Terms

Unmanned aerial vehicles (UAV), automatic drone detection, image annotation, You Only Look Once (YOLO)

Abstract

For ensuring the safety issues, a country should establish a secure monitoring system around the most important places. Due to the huge development in unmanned aerial vehicles (UAV), drone detection is a vital part of the safety monitoring system for reducing threats from neighboring countries or terrorist groups. This paper presents a deep learning-based drone detection method. A You Only Look Once (YOLO) v7 architecture is used to train on the dataset. The training dataset consists of drone images in various environments. The trained model was tested on multiple videos of drones from YouTube. Experimental results demonstrate that the model exhibited a recall of 0.9656 and a precision of 0.9509. In addition, the performance of the model compares with the state-of-art models with YOLOv8, YOLO-NAS, Faster-RCNN architectures and it outperforms the other models by maintaining a more stable precision and recall curve.

Cite This Paper

Muhammad K. Kabir, Anika N. Binte Kabir, Jahid H. Rony, Jia Uddin, "Drone Detection from Video Streams Using Image Processing Techniques and YOLOv7", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.16, No.2, pp. 83-95, 2024. DOI:10.5815/ijigsp.2024.02.07

Reference

[1]M. Ayamga, S. Akaba, and A. A. Nyaaba, “Multifaceted applicability of drones: A review,” Technological Forecasting and Social Change, vol. 167, p. 120677, 2021. https://doi.org/10.1016/j.techfore.2021.120677
[2]B. Mishra, D. Garg, P. Narang, and V. Mishra, “Drone-surveillance for search and rescue in natural disaster,” Computer Communications, vol. 156, pp. 1–10, 2020. https://doi.org/10.1016/j.comcom.2020.03.012
[3]G. Ding, Q. Wu, L. Zhang, Y. Lin, T. A. Tsiftsis, and Y.-D. Yao, “An amateur drone surveillance system based on the cognitive internet of things,” IEEE Communications Magazine, vol. 56, no. 1, pp. 29–35, 2018. https://doi.org/10.1109/MCOM.2017.1700452
[4]A. Gynnild, “The robot eye witness,” Digital Journalism, vol. 2, no. 3, pp. 334–343, 2014. https://doi.org/10.1080/21670811.2014.883184
[5]A. Singh, D. Patil and S. N. Omkar, "Eye in the Sky: Real-Time Drone Surveillance System (DSS) for Violent Individuals Identification Using ScatterNet Hybrid Deep Learning Network," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 2018, pp. 1710-17108, doi: 10.1109/CVPRW.2018.00214.
[6]M. Nalamati, A. Kapoor, M. Saqib, N. Sharma and M. Blumenstein, "Drone Detection in Long-Range Surveillance Videos," 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan, 2019, pp. 1-6, doi: 10.1109/AVSS.2019.8909830.
[7]A. N. O. U. K. S. RIGTERINK, “The Wane of Command: Evidence on Drone Strikes and Control within Terrorist Organizations,” American Political Science Review, vol. 115, no. 1, pp. 31–50, 2021. https://doi.org/10.1017/S0003055420000908.
[8]R. Luppicini and A. So, “A technoethical review of commercial drone use in the context of governance, ethics, and privacy,” Technology in Society, vol. 46, pp. 109–119, 2016. https://doi.org/10.1016/j.techsoc.2016.03.003.
[9]Y. Yao, H. Xia, Y. Huang, and Y. Wang, “Free to fly in public spaces: Drone controllers’ privacy perceptions and practices,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, (New York, NY, USA), p. 6789–6793, Association for Computing Machinery, 2017. https://doi.org/10.1145/3025453.3026049
[10]B. Kotkova, "Airport defense systems against drones attacks," 2022 26th International Conference on Circuits, Systems, Communications and Computers (CSCC), Crete, Greece, 2022, pp. 85-90, doi: 10.1109/CSCC55931.2022.00025.
[11]N. P. Walsh, N. Gallón, E. Perez, D. Castrillon, B. Arvanitidis, C. Hu, “Inside the August plot to kill Maduro with drones,” CNN, Mar. 14, 2019. https://edition.cnn.com/2019/03/14/americas/venezuela-drone-maduro-intl/index.html (accessed Feb. 21, 2023).
[12]J. P. Daniels, “Venezuela’s Nicolás Maduro survives apparent assassination attempt,” The Guardian, Aug. 05, 2018. Available: https://www.theguardian.com/world/2018/aug/04/nicolas-maduros-speech-cut-short-while-soldiers-scatter (accessed Feb. 21, 2023).
[13]K. Taneja, “Saudi Aramco attack shakes up fragile global ecosystem,” Policycommons.net, Sep. 18, 2019. https://policycommons.net/artifacts/1352659/saudi-aramco-attack-shakes-up-fragile-global-ecosystem/1964818/ (accessed Feb. 21, 2023).
[14]“Man fined after flying drones over Premier League stadiums,” BBC News, Sep. 15, 2015. Accessed: Apr. 12, 2023. [Online]. Available: https://www.bbc.com/news/uk-england-nottinghamshire-34256680 (accessed Feb. 21, 2023).
[15]D.-H. Shin, D.-H. Jung, D.-C. Kim, J.-W. Ham, and S.-O. Park, “A Distributed FMCW Radar System Based on Fiber-Optic Links for Small Drone Detection,” IEEE Transactions on Instrumentation and Measurement, vol. 66, no. 2, pp. 340–347, Feb. 2017, doi: https://doi.org/10.1109/tim.2016.2626038.
[16]B. Nuss, L. Sit, M. Fennel, J. Mayer, T. Mahler and T. Zwick, "MIMO OFDM radar system for drone detection," 2017 18th International Radar Symposium (IRS), Prague, Czech Republic, 2017, pp. 1-9, doi: 10.23919/IRS.2017.8008141.
[17]Á. D. de Quevedo, F. I. Urzaiz, J. G. Menoyo and A. A. López, "Drone Detection With X-Band Ubiquitous Radar," 2018 19th International Radar Symposium (IRS), Bonn, Germany, 2018, pp. 1-10, doi: 10.23919/IRS.2018.8447942.
[18]J. Ochodnický, Z. Matousek, M. Babjak and J. Kurty, "Drone detection by Ku-band battlefield radar," 2017 International Conference on Military Technologies (ICMT), Brno, Czech Republic, 2017, pp. 613-616, doi: 10.1109/MILTECHS.2017.7988830.
[19]S. Al-Emadi, A. Al-Ali, A. Mohammad and A. Al-Ali, "Audio Based Drone Detection and Identification using Deep Learning," 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco, 2019, pp. 459-464, doi: 10.1109/IWCMC.2019.8766732.
[20]S. Jamil et al., “Malicious UAV Detection Using Integrated Audio and Visual Features for Public Safety Applications,” Sensors, vol. 20, no. 14, p. 3923, Jul. 2020, doi: 10.3390/s20143923. [Online]. Available: http://dx.doi.org/10.3390/s20143923.
[21]S. Mandal, L. Chen, V. Alaparthy, and M. L. Cummings, “Acoustic Detection of Drones through Real-time Audio Attribute Prediction,” AIAA Scitech 2020 Forum, Jan. 2020, doi: https://doi.org/10.2514/6.2020-0491.
[22]F. Svanstrom, C. Englund and F. Alonso-Fernandez, "Real-Time Drone Detection and Tracking with Visible, Thermal and Acoustic Sensors," in 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 2021 pp. 7265-7272. doi: 10.1109/ICPR48806.2021.9413241
[23]S. Salman, J. Mir, M. T. Farooq, A. N. Malik and R. Haleemdeen, "Machine Learning Inspired Efficient Audio Drone Detection using Acoustic Features," 2021 International Bhurban Conference on Applied Sciences and Technologies (IBCAST), Islamabad, Pakistan, 2021, pp. 335-339, doi: 10.1109/IBCAST51254.2021.9393232.
[24]H. Kolamunna et al., “DronePrint,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 5, no. 1, pp. 1–31, Mar. 2021, doi: 10.1145/3448115.
[25]S. Al-Emadi, A. Al-Ali, and A. Al-Ali, “Audio-Based Drone Detection and Identification Using Deep Learning Techniques with Dataset Enhancement through Generative Adversarial Networks,” Sensors, vol. 21, no. 15, p. 4953, Jul. 2021, doi: 10.3390/s21154953. [Online]. Available: http://dx.doi.org/10.3390/s21154953
[26]G. Lacava, F. Mercaldo, F. Martinelli, A. Santone, and M. Pizzi, “Drone Audio recognition based on Machine Learning Techniques,” Procedia Computer Science, vol. 207, pp. 848–857, 2022, doi: 10.1016/j.procs.2022.09.140.
[27]C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” arXiv:2207.02696 [cs], Jul. 2022, Available: https://arxiv.org/abs/2207.02696
[28]C.-Y. Wang, H.-Y. M. Liao, and I.-H. Yeh, “Designing Network Design Strategies Through Gradient Path Analysis,” arXiv:2211.04800 [cs], Nov. 2022, Accessed: Apr. 12, 2023. [Online]. Available: https://arxiv.org/abs/2211.04800
[29]G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics,” GitHub, Jan. 01, 2023. https://github.com/ultralytics/ultralytics (accessed Feb. 21, 2023)
[30]Mehdi (2023) drone-dataset. In: GitHub. https://github.com/dasmehdix/drone-dataset. (accessed Feb. 21, 2023)
[31]Birds vs Drone Dataset. https://www.kaggle.com/datasets/harshwalia/birds-vs-drone-dataset. (accessed Feb. 21, 2023)
[32]C. Aker and S. Kalkan, "Using deep networks for drone detection," 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy, 2017, pp. 1-6, doi: 10.1109/AVSS.2017.8078539.
[33]M. Saqib, S. Daud Khan, N. Sharma and M. Blumenstein, "A study on detecting drones using deep convolutional neural networks," 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy, 2017, pp. 1-5, doi: 10.1109/AVSS.2017.8078541.
[34]J. Peng et al, “Using Images Rendered by PBRT to Train Faster R-CNN for UAV Detection,” Václav Skala-UNION Agency: 2018.
[35]D.-H. Lee, “CNN-based single object detection and tracking in videos and its application to drone detection,” Multimedia Tools and Applications, Oct. 2020, doi: 10.1007/s11042-020-09924-0.
[36]U. Seidaliyeva, D. Akhmetov, L. Ilipbayeva, and E. T. Matson, “Real-Time and Accurate Drone Detection in a Video with a Static Background,” Sensors, vol. 20, no. 14, p. 3856, Jul. 2020, doi: 10.3390/s20143856. [Online]. Available: http://dx.doi.org/10.3390/s20143856
[37]O. O. Medaiyese, A. Syed and A. P. Lauf, "Machine Learning Framework for RF-Based Drone Detection and Identification System," 2021 2nd International Conference on Smart Cities, Automation & Intelligent Computing Systems (ICON-SONICS), Tangerang, Indonesia, 2021, pp. 58-64, doi: 10.1109/ICON-SONICS53103.2021.9617168.
[38]D. Lee, W. Gyu La and H. Kim, "Drone Detection and Identification System using Artificial Intelligence," 2018 International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Korea (South), 2018, pp. 1131-1133, doi: 10.1109/ICTC.2018.8539442.
[39]M. Z. Anwar, Z. Kaleem and A. Jamalipour, "Machine Learning Inspired Sound-Based Amateur Drone Detection for Public Safety Applications," in IEEE Transactions on Vehicular Technology, vol. 68, no. 3, pp. 2526-2534, March 2019, doi: 10.1109/TVT.2019.2893615.
[40]D. K. Behera and A. Bazil Raj, "Drone Detection and Classification using Deep Learning," 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 2020, pp. 1012-1016, doi: 10.1109/ICICCS48265.2020.9121150.
[41]H. R. Alsanad, A. Z. Sadik, O. N. Ucan, M. Ilyas, and O. Bayat, “YOLO-V3 based real-time drone detection algorithm,” Multimedia Tools and Applications, Mar. 2022, doi: 10.1007/s11042-022-12939-4.
[42]S. Singha, B. Aydin, “Automated Drone Detection Using YOLOv4,” Drones, vol. 5, no. 95, 2021, pp. 1-20, https://doi.org/10.3390/drones5030095
[43]B. Aydin, S. Singha, “Drone Detection Using YOLOv5,” Eng, vol. 4, pp. 416–433, 2023. https://doi.org/10.3390/eng4010025.
[44]V. T. Hai, V. N. Le, D. Q. Khanh, N. P. Nam, and D. V. Sang, “Multi-size drone detection using YOLOv5 network,” Journal of Military Science and Technology, no. 80, pp. 142–148, Jun. 2022, doi: 10.54939/1859-1043.j.mst.80.2022.142-148.
[45]Q Cheng, Li X, Zhu B, et al. “Drone Detection Method Based on MobileViT and CA-PANet,” Electronics, vol. 12, no. 223, pp. 1-16, 2023. https://doi.org/10.3390/electronics12010223
[46]Valaboju R, Vaishnavi, Harshitha C, et al. “Drone Detection and Classification using Computer Vision,” 7th International Conference on Trends in Electronics and Informatics (ICOEI), 2023, pp 1320–1328
[47]Lindenheim-Locher W, Świtoński A, Krzeszowski T, et al., “YOLOv5 Drone Detection Using Multimodal Data Registered by the Vicon System,” Sensors, vol. 23, no. 6396, pp. 1-16, 2023, https://doi.org/10.3390/s23146396
[48]J. -H. Kim, N. Kim and C. S. Won, "High-Speed Drone Detection Based on Yolo-V8," ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023, pp. 1-2, doi: 10.1109/ICASSP49357.2023.10095516.
[49]S. Rahman, J. H. Rony, J. Uddin, and M. A. Samad, “Real-Time Obstacle Detection with YOLOv8 in a WSN Using UAV Aerial Photography,” J. Imaging, vol. 9, no. 10, Art. no. 10, Oct. 2023, doi: 10.3390/jimaging9100216
[50]A. Munir, A. J. Siddiqui, “Vision-based UAV Detection in Complex Backgrounds and Rainy Conditions,” 2023, arXiv preprint arXiv:2305.16450.
[51]B. Benjdira, T. Khursheed, A. Koubaa, A. Ammar, K. Ouni, “Car detection using unmanned aerial vehicles: Comparison between faster r-cnn and yolov3,” 1st International Conference on Unmanned Vehicle Systems-Oman (UVS), 2019, pp. 1-6.

International Journal of Image, Graphics and Signal Processing (IJIGSP)