Significance  Small object detection holds a crucial position in various industrial fields and everyday life. In remote sensing, it is used to identify and track small objects, providing key support for military reconnaissance and national defense security. For instance, infrared small object detection can detect invasive targets and take subsequent interception measures against them. In autonomous driving, the system automatically detects objects such as traffic signs, vehicles, pedestrians, and obstacles, to help to deeply analyze the meaning of the driving scene and to predict the behavior of surrounding objects to ultimately make appropriate decisions. In public safety and surveillance, small object detection systems can accurately identify and track small objects hidden in the distance or complex backgrounds, enabling functionalities such as pedestrian face recognition, vehicle identification, and detection of illegal crowd lingering, counting of individuals, and estimation of crowd density. For industrial automation, small object detection is also necessary to locate visible small defects on the surface of materials. Overall, small object detection technology significantly enhances the work efficiency across various sectors, demonstrating its broad application prospects and profound research value.

Progress  This paper provides a comprehensive review of current deep learning-based small object detection techniques and systematically categorizes, analyzes, and compares existing algorithms. We initially outline the definition of small object detection, the challenges faced, and its application areas. The definition of small objects is elaborated from two perspectives: based on absolute and relative scales. We also summarize the main challenges encountered in small object detection, including information compression loss, low signal-to-noise ratio and detectability in complex backgrounds, high sensitivity to minor deviations in bounding boxes, complexity in network structure and optimization, and the scarcity of large-scale small object datasets.

Following, we delve into several key optimization approaches. These include the enhancement of model robustness through data augmentation, the improvement of small object visibility via super-resolution methods, the augmentation of detection accuracy through the application of multi-scale information fusion, and the refinement of feature representation using contextual information and large-kernel convolution techniques. Moreover, the discussion extends to anchor-free detection frameworks, DETR technology, and dual-modal strategies for small object detection tailored to particular contexts, offering an exhaustive evaluation of their benefits and drawbacks.

We ultimately provide a comprehensive introduction to currently available small object datasets, encompassing twelve major datasets: DOTA, AI_TOD, DIOR, VisDrone2019, TT100K, BSTID, TinyPerson, CityPerson, WiderPerson, WIDER FACE, BIRDSAI, and MS COCO. These datasets offer a rich resource for research purpose and performance evaluation of small object detection. Further, we also conduct a detailed performance evaluation of existing small object detection algorithms on several widely-used public datasets, such as MS COCO, DOTA, AI_TOD, TinyPerson, and TT100K. Additionally, we forecast future research directions in this field, proposing four main potential challenges: feature fusion, contextual learning, optimization of large kernel convolution, and improvements in DETR technology. These directions not only illustrate the developmental trends of small object detection, but also highlight technical challenges that current research needs to overcome, providing guidance and inspiration for future studies.

Conclusions and Prospects  Small object detection is one of the most critical and fundamental tasks in the field of computer vision, with broad application demands in the real world, such as military reconnaissance, autonomous driving, public safety and surveillance, and robotic vision. Although substantial algorithms have shown relatively satisfactory performance in specific applications and scenarios, overall, their effectiveness, robustness as well as speed still need improvement. This paper aims to provide references and bases for the development of algorithms through in-depth research and analysis of small object detection technology.