Drone Detection and Surveillance

Along with being used for entertainment, drones are becoming more and more popular across a range of industries, including engineering, logistics, crisis management, airport security, and many more. Along with their beneficial uses, due to the potential for criminal use, the physical infrastructure at airports is becoming a key source of worry for security, safety, and monitoring. Numerous incidents of unlicensed usage of different drone types near airports and interference with airline operations have been reported in recent years. For the effective detection and identification of drones and birds, a deep learning-based solution is applied to solve these issues. During the day, predictions are highly likely to come true. In addition, due to their resemblance to birds in terms of appearance and activity, drones are frequently mistaken for them. The suggested method is able to identify and separate drones from birds as well as detect if they are present or absent in a given location. A9G-based GPS tracker modules are used to correctly determine the drone's present location, track it constantly in real time, and differentiate between authorised and unauthorised drones. The authorised drone's location can be sent through SMS. The user will take the necessary measures if the drone is discovered to be unlicensed. The proposed model has an F-1 score of 93.63%, precision of 96.2%, mAp of 95%, recall of 91.2%, and accuracy of 95.5%.


Introduction
More people are using drones greater than ever as their popularity grows and their availability for public is expanding. As a result, there is a rise in misusing of drones. To prevent unauthorised and undesired drone interventions, drone detection that is automated is necessary. There were 368508 commercial and 500601 recreational unmanned aircraft systems (UAS) or drones registered in the United States as of April 13, 2021, according to the Federal Aviation Administration. The drone industry is quickly growing. They are getting easier to access to the general public and more affordable. Drones can be used for a range of applications like functions, including inspection, delivery, monitoring, photography, and many others, depending on their payload capability. Drones, however, can be abused, particularly by hobbyists, as well as for illicit purposes like drug smuggling, terrorist attacks, or even interfering with emergency services such as fire prevention and disaster response. Drones can also be turned into lethal weapons by equipping them with explosives. Controlling the unauthorised use of drones is essential for maintaining public safety and preventing security breaches. They are, however, difficult to detect in the air. Small drones transmit relatively limited electromagnetic signals, making them difficult to detect with conventional radar. Acoustic and radio frequency detectors are expensive and have difficulty dealing with the Doppler effect. On the other hand, deep learning for object recognition has achieved substantial success because of its high accuracy. In fact, owing to its very precise real-time detection capabilities, the "You Only Look Once" (YOLO) method has surpassed other object identification algorithms such as the region-based convolutional neural network (R-CNN) and the single-shot multi-box detector (SSD). YOLO outperforms in terms of both accuracy and quickness. Though it has several versions, YOLOv5 is the latest, attaining 10% more average precision (AP) on the Microsoft Common Objects in Context (MS COCO) dataset than the previous version, YOLOv4. Both YOLOv4 and YOLOv5 have a similar architecture. However, YOLOv5 had advantages in engineering. YOLOv5 is developed in Python rather than C, as in prior versions, which simplifies installation and integration on IoT devices. Training the model with a greater number of photos strengthens it. Predictions can be made with assurance. Approaches utilising YOLOv5 outperform all previous proposed detectors in relation to efficiency when compared to existing detection systems. This paper describes an automated drone detection method that makes use of YOLOv5 (You only look once version 5) has been proposed. Datasets of drones and birds were used to train the model. The testing dataset is subsequently used to evaluate the trained model. After the drone is detected, using the A9G-based GPS tracker, we can find where the allied drones are located and can distinguish between authorized and unauthorized drones.

Motivation
Drones' growing popularity has raised worries about their possible misuse, including espionage, smuggling, and unauthorised surveillance. As a result, dependable and effective drone detection systems are required to maintain public safety and security. Drone misuse can be dangerous to important infrastructure, private property, and public events. The incident at Gatwick airport in December 2018, when drones were flown near the airport, causing the runway to close and all flights to be cancelled. This event created havoc and inconvenience for thousands of people, and it shows the potential dangers of uncontrolled drone activities near airports. A drone detection project that might identify and monitor drones near airports and alert airport security staff in advance could be highly beneficial. As a result, they may take the proper measures to stop flight disruptions and guarantee the security of both passengers and aircraft. By implementing this, airport security and airspace safety would be increased, and incidents like this one might be prevented in coming years. As a result, there is an increasing demand for drone detection systems that can accurately and quickly identify illegal drone activity in a range of scenarios. The creation of such systems may aid in reducing the dangers connected with drone abuse and enhance protection for people and property.

Related Work
Using a computer vision technique, [1] seeks to find drone detection solutions. Today, it is crucial to consider the risk posed by drones operating close to airports and other key locations. The suggested methods provide pictures with a high rate of detection and resolution. Information regarding the drones' flight paths is used in the technique. The authors of [1] updated the deep learning model by taking into account the object's relative speed and trajectory path in order to more effectively differentiate between drones and birds. Up to a distance of 200 metres, the system can detect drones. An automated drone detection system has been developed utilising YOLOv4 [2]. Datasets from drones and birds were used to train the model. On the trained model, performance parameters including mean average precision (mAP), frames per second (FPS), precision, recall, and F1-score have been computed. With a mAP of 74.36%, precision of 0.95, recall of 0.68, and F1-score of 0.79, this study outperformed earlier, comparable studies in terms of performance. [3] have unveiled a thorough machine-learning-based drone detection solution. This system utilises drones with cameras and uses OpenCV image processing to determine position and categorise the vendor model. [4,30] explores the process of designing an automatic multi-sensor drone detection system. A fish-eye camera is also incorporated in [5] system to watch a larger area of the sky and direct the other cameras towards items of interest. An ADS-B receiver, a GPS receiver, and a radar module are added to the sensing solutions. But these multisensory systems are expensive for low profile usage. [7] method is able to identify two different types of drones, separate them from birds, and determine whether drones are present or absent in a given location. They have used only 10,000 images for training and gained an accuracy of 83%. [8] the drones could be detected and tracked all the way even in the cluttered back ground, as long as they appeared in the camera scene. [13] The multi sensor system which uses visible cameras, acoustic sensors, tilt camera, radar for counter UAV applications. [15] presented multiple surveillances technologies with an anti-drone system. [23] explored an audio-based drone detection system with deep learning. This system is not so accurate in case of crowded background. [9] Explained the architecture and working of all the 5 versions of YOLO (V1, V2, V3, V4, V5). object detection experiment is performed on the Global Wheat dataset contains 3432 wheat images using YOLOv5 model. [9,10] The performance of the YOLOv5 is higher than the YOLOv4 in terms of both accuracy and speed. model is highly reliable and effective in identifying drones from birds and tracking permitted drones There are many existing systems for detection of the drones, but many of these system uses multi-sensor systems in which more than one sensor are integrated to get the accurate results. These types of systems are expensive and more complex and requires an operator to take care of the device. The systems which multi-sensor are used in defense applications and are not suitable for low profile usage because of their cost and complexity. In this paper the advanced object identification algorithm YOLOv5 is used and combined with an A9G-based GPS tracker and ESP32C3 board for real-time tracking. The research suggests that the presented. It also assists users in identifying authorised drones from unauthorised drones. The suggested solution is affordable, simple to set up, and more focused in its use for low-profile applications.

Proposed System
Block Diagram for the proposed system A total of 36,000 files (including images and annotation files) were split into 3 categories of which 80% of files were used for training, 10% for validation and remaining 10% for testing the trained model. A total of 12,700 drone images and 5,300 bird images were collected and annotation files for the same have been prepared using "makesense.ai" tool. Format of the annotation file for YOLOv5 is: Object_class X Y Width Height Where Object_class: 0 -Bird 1 -Drone X and Y are bottom left co-ordinates of the annotation box Width and Height represents the width and height of the annotation box Model was trained using "Google Colaboratory" with the training dataset uploaded to Google drive prior to training. After the model is trained, the weights are obtained in the best.pt file, which will be saved in the google drive. As soon as the model is trained, we obtain the performance metrics of the trained model. Now, the model is validated using the validation dataset and tested using testing dataset. After validation and testing, we obtain the optimal weights in the best.pt file. Download the best.pt file. For the Software setup, Install python version 3.9.x by going to official pyton.org and download the latest version. Install the Tensor flow, PyTorch libraries. Download the CUDA from NVIDIA official website (version 11.6), used for enabling GPU for object detection. Download YOLOv5 repository from GitHub and extract the downloaded Zip file. Camera setup is done by installing "IP Webcam" app in mobile. Click on start server. Copy the RTSP link with IP address and port generated by the server. This link is given as the source camera link for running the real time detection. Run the detect.py file in the command prompt, which is located in the previously extracted YOLOv5 folder. The detection results will be stored in the following path: "YOLOv5/Runs/detect/exp" The results will have the detected object labels along with the video stream. Now, the path of the labels folder is given to the Python program for sending the Telegram message to the user if the detected object label is 1 (i.e., drone). Flow chart for python program for sending telegram message is as shown below. The required libraries are imported first and the path to the label folder which stores the text files containing the information about the detected drone or bird is specified. The API token, Chat ID and API URL of the telegram group is provided in the program. The text file is read in a loop until it finds the label==1. Whoever are present in the telegram, everyone will receive an alert message automatically if drone is detected stating that "DRONE DETECTED". To get the location of the drone A9G based GPS tracker is integrated with the ESP32C3 board as shown.
IO 25 GND GND 5v-Battery Pin connections This A9G based GPS tracker integrated with ESP32 C3 board is provided with power supply through USB cable and the code to send the GPS location is dumped using the Arduino IDE. The flowchart for the program is shown above. The required pins are defined and A9G board is set up for sending the GPS location using the Arduino IDE serial monitor. If user want the location of the detected drone (drone is a allied drone) then the "SEND LOCATION" message is sent and the A9G GPS tracker present on the drone will send the current location of the drone by GPS Co-ordinates with Google Map link to the registered user mobile. If the tracker is not able to get the location then "Unable to fetch the location, Please try again" message is sent. If the location matches it is an authorized drone, if the location does not match then it is an unauthorized drone. After the system verifies the status of the UAV, the information is sent to the user and necessary action is taken.

Results and Discussion
The project aimed to develop a deep learning model that could detect and track drones in an area with high accuracy. The model had to distinguish between drones and birds and also differentiate between authorized A confusion matrix represents the prediction summary in matrix form. It shows how many prediction are correct and incorrect per class. It helps in understanding the classes that are being confused by model as other class. The confusion matrix of the proposed model is given as follow:

Confusion matrix
The formulas used to calculate the performance metrics of the trained model are: The proposed model has the precision of 96.2% , mAp of 95%, Recall of 91.2% and F-1 score of 93.63%. Model accuracy is 95.5%. The trained model detects the drones presence in an area with high accuracy and distinguish them from birds.

Drone detection
Bird Detection After detecting the object, If the object_class is 1 (i.e. drone) then an alert message is sent to the user as "DRONE DETECTED" on telegram. The location of the drones is sent successfully to the telegram chat, on the request of the user "SEND LOCATION" using A9G GPS module the GMAP link with coordinates is sent and the drone is also identified whether it is authorized or unauthorized. Alert telegram message GMAP link to the user mobile YOLOv5 can be easily deployed on a variety of hardware platforms, including GPUs, CPUs, and even mobile devices, making it a flexible and accessible solution for drone detection and surveillance. It is a light weight algorithm that can be deployed on low-power devices. It provides warning / alert message of the object detection directly to the user through telegram, which helps in early detection of the occurrence of any threats. The proposed A9G GPS tracker is efficient, compact in size and can be easily deployed on any drone. The drone location can be directly sent to the registered mobile number of the user. The proposed system is cost effective, so it can be deployed in areas where there is a potential threat of privacy for civilians and monuments and it can be a great gap filler in the areas that are outside the human line of sight. But the system cannot detect the objects during night times as IP Webcam is used. Since the camera is statically fixed, we cannot get the 360 ° view for the surveillance. When the drone is in the areas where the signal strength for the SIM is too low, it is difficult to get the location.

Conclusion
In conclusion, the drone detection project using YOLOv5 has successfully developed a robust and efficient system for detecting drones in real-time. YOLOv5 is a state-of-the-art object detection algorithm with great accuracy and fast processing speeds, making it a good fit for the task of drone detection. The system was trained on a large dataset of drone images, using transfer learning to fine-tune the pre-trained YOLOv5 model. The resulting system achieves high accuracy in detecting and classifying drones, even in challenging lighting and weather conditions. The system was also able to detect multiple drones simultaneously, making it suitable for securing large areas. The project has demonstrated the potential of using YOLOv5 and deep learning techniques for addressing emerging security threats and protecting critical infrastructure. The proposed model has the precision of 96.2%, mAp of 95%, Recall of 91.2% and F-1 score of 93.63%. By using the A9G GPS tracker integrated with ESP32C3 the accurate location of the drone is sent directly to the user registered mobile number. Using of YOLOv5 for training and A9G module for location tracking make the system more reliable and robust compared to other existing systems. Integration of the drone detection system with other security systems could be a future work, such as alarms or cameras, to provide a comprehensive security solution. Overall, the proposed system has yielded promising outcomes and is a significant step towards enhancing security and safety in various applications.

Future scope
Drone detection and Surveillance is one of the applications where it is used to detect the objects during night times along with day time. IR cameras can be used in place of IP Web cam in order to detect the drone irrespective of the time of the day. Training the model with Infrared images can make the model more robust and reliable. Cameras with high resolution and range can be deployed so that the proposed system can detect the objects from far distances. The rotatable cameras can be used to get the 360° view of the surrounding area under detection. The proposed model can be trained with a larger dataset to improve the accuracy. After detecting and tracking the location of the drone, a jammer can be used so that the unauthorized drone will lose the communication with its operator. However, jammer should be used carefully as it can interrupt the surrounding networks. There are other counter actions along with jamming signal such as destroying the drone by shooting, depending on the situation. In future, as there will be a scope of increasing in drone usage drastically, the detection and surveillance systems will have a major role in security, safety and privacy.