AI-powered Surveillance of Drones and Vehicles: A Step Towards Smart Cities

Unmanned Aerial Vehicles (UAVs) have been increasingly popular for a diverse range of applications in recent years thanks to their versatile design. The exploitation of UAVs has indeed raised severe privacy, safety, and security concerns for both individuals and corporations. This has resulted in an unprecedented need for drone surveillance systems to keep a closer eye on any unidentified UAV engaging in potentially illegal or malicious behaviour. Detecting authorized/unauthorized drones, categorizing different drones, localizing and tracking intruders in different fly/no-fly zones and so on are diverse capabilities implemented by drone surveillance systems. The goal of the first part of this thesis (Chapter 2) is to present a thorough overview of the research conducted in the development of a system to detect, localize, and track drones using Radio Frequency (RF)- and Wireless Fidelity (WiFi)-based approaches. In the domain of smart city development, particular interest in the research and development of traffic monitoring systems, under the umbrella of transportation, has been carried out extensively due to the boom in the Artificial Intelligence (AI) field, and the second part of this thesis is an attempt to move forward in that direction. In this perspective, the purpose of the second part of this thesis work (Chapters 3-6) is to design and develop the drone-based road traffic monitoring system from the Deep Learning (DL) perspective, which is responsible for reliably performing the task of Region of Interest (RoI) extraction, vehicles detection, vehicles tracking, vehicles counting, direction finding of vehicles, energy consumption analysis and transmission of traffic analytics to the concerned personals. An experimental analysis and performance evaluation of vehicle detection using a one-stage object detection framework on the VisDrone-DETection (VisDrone-DET)-benchmark dataset is carried out. To deal with the problem of monitoring traffic on RoI from drone images and videos, especially when the surveillance drone is in a moving state, a DL algorithm is developed to predict the RoI and then vehicle detection is done on the predicted RoI. Two bespoke aerial datasets are built for RoI extraction and detection tasks by collecting aerial sequences from flying UAVs and transferring them to the base station leveraging 5G technology. In addition, a drone energy consumption profile is developed and examined, and a drone flight strategy under a surveillance scenario is proposed. Further, the relationship between the video processing task and the drone energy profile is investigated. Concerning the Multi-Object Tracking (MOT) task, the performance is improved by pairing the spatial and visual cues of incoming detections and existing trajectories individually and then determining the optimal pair. The results of the MOT task are utilized to determine the number of vehicles in an aerial scene, by adopting the counting-by-tracking approach. A centre-point subtraction approach is used to find the direction finding of vehicles. Finally, the base station uses the OneM2M standard to provide road traffic statistics to traffic personnel and stack-holders for analysis and recording. The outcome of this research produced multiple contributions to surveillance and monitoring applications and highlighted the challenges, issues, and future directions for both drone surveillance and road traffic monitoring systems. Extensive simulation-based experiments are carried out to validate the outcome of the proposed research work, which yielded desired results, thus establishing a baseline and a test-case scenario to fly a drone for traffic surveillance and monitoring applications to attain optimum results.