Source-linked AI summary
DeepSOCIAL: Social Distancing Monitoring and Infection Risk Assessment in COVID-19 Pandemic
Mahdi Rezaei, Mohsen Azarmi
TL;DR
The paper addresses limited automated evidence for accurate social-distancing monitoring and infection-risk analysis in crowded public environments. It develops DeepSOCIAL, a YOLOv4-based people detector combined with perspective mapping, tracking, and spatio-temporal risk analysis. On Oxford Town Centre, it reports 99.8% accuracy and 24.1 fps under challenging conditions, while identifying remaining scope limitations for extreme lighting, lens distortion, and risk-map weighting.
Problem
Social-distancing research lacked accurate crowd distance estimation, challenging-dataset evaluation, common comparisons, and post-detection infection-risk analysis.
Method
DeepSOCIAL combines a YOLOv4-based DNN detector with adapted inverse perspective mapping, SORT tracking, and statistical analysis of movement trajectories and violations.
Results
99.8% accuracy and 24.1 fps were reported on Oxford Town Centre under occlusion, lighting variation, shades, and partial visibility.
Takeaways & Limitations
The viewpoint-independent detector is intended for social-distancing monitoring and broader human-detection applications including autonomous vehicles, anomaly detection, and security systems.
Takeaways & Limitations
Extreme lighting, lens distortion, and the weighting and temporal normalisation of risk maps require further investigation.
Abstract
from arXiv · showhide
Social distancing is a recommended solution by the World Health Organisation (WHO) to minimise the spread of COVID-19 in public places. The majority of governments and national health authorities have set the 2-meter physical distancing as a mandatory safety measure in shopping centres, schools and other covered areas. In this research, we develop a hybrid Computer Vision and YOLOv4-based Deep Neural Network model for automated people detection in the crowd in indoor and outdoor environments using common CCTV security cameras. The proposed DNN model in combination with an adapted inverse perspective mapping (IPM) technique and SORT tracking algorithm leads to a robust people detection and social distancing monitoring. The model has been trained against two most comprehensive datasets by the time of the research the Microsoft Common Objects in Context (MS COCO) and Google Open Image datasets. The system has been evaluated against the Oxford Town Centre dataset with superior performance compared to three state-of-the-art methods. The evaluation has been conducted in challenging conditions, including occlusion, partial visibility, and under lighting variations with the mean average precision of 99.8% and the real-time speed of 24.1 fps. We also provide an online infection risk assessment scheme by statistical analysis of the Spatio-temporal data from people's moving trajectories and the rate of social distancing violations. The developed model is a generic and accurate people detection and tracking solution that can be applied in many other fields such as autonomous vehicles, human action recognition, anomaly detection, sports, crowd analysis, or any other research areas where the human detection is in the centre of attention.
1. Introduction
The paper addresses the need for automated social-distancing monitoring in public places during COVID-19, where maintaining distance is important but difficult to assess continuously. It introduces DeepSOCIAL, an AI-based system for detecting people, tracking movement, estimating distances, and assessing infection risk.
- Motivation: Social distancing limits close physical contact in crowded or covered public places to reduce infection risk.The paper discusses settings including schools, workplaces, gyms, and lecture theatres.
- Motivation: Governments introduced distancing practices, but monitoring compliance and evaluating their effectiveness remained difficult while people continued essential activities outside the home.The paper describes required distances of 1.6–2 m and the practical difficulty of monitoring spread and compliance.
- Research challenge: 14 AI-based challenges include high accuracy, varying lighting, occlusion, and real-time performance.The paper identifies these challenges as targets for its proposed solution.
- Contribution: DeepSOCIAL combines people detection, tracking, distance estimation, and automated monitoring of social-distancing violations.The model is presented as a robust DNN system for these tasks.
- Contribution: The system performs statistical risk assessment from spatio-temporal movement data, including violation ratios and high-risk-zone detection.The analysis considers trajectories, behaviours, and short- and long-term risk zones.
- Broader scope: The detector is positioned as a generic human-tracking system applicable beyond social distancing, including autonomous vehicles, action recognition, anomaly detection, and security.The paper explicitly extends the potential application scope beyond pandemic monitoring.
2. Related Works
Prior work established social distancing as an important disease-control measure and explored epidemiological, location-based, surveillance, and computer-vision approaches. The paper identifies limited common-dataset evaluation, challenging-condition testing, accurate distance estimation, and post-detection risk analysis as gaps addressed by DeepSOCIAL.
- Medical and epidemiological research: SIR-based studies model infections over time, with extensions incorporating social-distancing parameters to estimate infected and recovered populations.The SIR framework is described as an epidemiological model for contagious disease spread.
- Medical and epidemiological research: R₀ > 1 indicates increasing infection, whereas R₀ < 1 indicates a declining disease rate in the target population.The paper presents R₀ as a criterion for evaluating social-distancing practices.
- Tracking technologies: Technology-based approaches include GPS and Bluetooth exposure monitoring, drones, surveillance cameras, and wireless-signal human detection.These approaches differ in sensing modality and deployment requirements.
- AI-based research: Computer vision and AI can analyse spatio-temporal visual information for pedestrian behaviour, sports, medical imaging, and anomaly detection.The paper connects visual feature analysis with statistical analysis of image sequences.
- AI-based research: Existing people-detection research is often restricted to indoor settings or loses accuracy under challenging outdoor lighting, while manual tuning limits functionality.These limitations motivate more robust automated detection.
- Research gaps: The field lacked dedicated evaluation of crowd distance estimation, challenging-dataset experiments, common benchmarks, and post-detection infection-risk analysis.The paper identifies these as gaps in social-distancing research.
- Research gaps: DeepSOCIAL is proposed to improve speed and accuracy through training and testing on comprehensive datasets and challenging environments, followed by analytical post-processing.The proposed scope includes varied lighting and real-world covered environments.
3. Methodology
DeepSOCIAL combines people detection, tracking, and inter-distance estimation for social-distancing monitoring using CCTV video. Its YOLOv4-based design uses multi-scale detection, selected backbone and neck components, training techniques, and real-time tracking.
- The proposed three-stage model performs people detection, tracking, and inter-distance estimation for social-distancing monitoring and zone-based infection-risk analysis.It is intended for CCTV surveillance cameras with varied resolutions and real-time operation.
- The detector processes CCTV video to produce people detections with localization bounding boxes under varied distances, occlusion, postures, clothing, and lighting.
- The model design evaluates input augmentation, activation, backbone, neck, and head options, with Mosaic augmentation integrating four images without increasing batch size.Cross-Iteration Batch Normalisation uses normalized values from previous iterations to address noisy batch statistics after batch-size reduction.
- Mish converged faster than Swish and ReLU with higher accuracy in the reported human-detection evaluations.The result remained consistent across parameter initializers, regularization methods, and lower learning rates.
- CSPDarknet53 was selected as the backbone, while the neck combines parameters from different network layers to improve feature representation.The authors report that its higher parameter count supported multiple-object detection while maintaining real-time performance.
- YOLO-style fully convolutional processing supports varying image sizes, while pyramid features and multi-scale detection address small-object detection.The design avoids fixed fully connected input dimensions and extracts features at different image scales.
- SORT tracking combines a Kalman filter with Hungarian optimization to assign identities and track people across frames.The resulting temporal information is used to analyze social-distancing violations and high-risk scene zones.
4. Model Training and Experimental Results
The study trains DeepSOCIAL on large annotated datasets and evaluates its detection, distancing, and risk-assessment capabilities under challenging conditions. DeepSOCIAL-DS outperforms assessed models in speed and accuracy while supporting trajectory-based risk heat maps.
- Model Training: 3,762,615 samples from COCO and Google Open Images were used to train the human-detection model.The training set included 257,253 COCO samples and 3,505,362 Google Open Images samples.
- Model Training: Annotated body-part categories, including legs, were included to improve detection under occlusion and partial visibility.The training design targeted cases where only part of a person is visible or the full body reaches an image border.
- Model Training: Transfer learning from Microsoft COCO was followed by fine-tuning and optimization of the YOLO-based model.Training also used stochastic gradient descent with warm restarts and cosine-annealed learning rates.
- Performance Evaluation: DeepSOCIAL-DS outperformed all assessed models in both speed and accuracy, while Faster-RCNN was limited by the computational cost of region proposals.The comparison covered three challenging indoor and outdoor datasets and required real-time performance above 10 fps.
- Performance Evaluation: In partial-visibility footage, DeepSOCIAL produced 10 more true-positive detections than the second-best approach.The comparison involved YOLOv3, standard YOLOv4, and ground-truth annotations, including unannotated partially visible people.
- Social Distancing Evaluation: 98.7% accuracy and 23.9 fps were achieved for coupled-group detection, slightly below normal human detection because tracking side-by-side people is more complex.The authors associate the lower result with added tracking complexity and potentially more difficult occlusion scenarios.
- Zone-Based Risk Assessment: The risk-assessment pipeline combines movement trajectories, zone density, distancing violations, violation duration, and steady-state people into risk heat maps.The heat maps identify potentially risky zones; steady-state people receive greater contamination weight, and redder grids indicate potentially riskier locations.
- Zone-Based Risk Assessment: The weighting and temporal normalization of the long-term risk maps remain outside the study’s scope and require further investigation.The authors present the method as a feasibility demonstration rather than resolving the appropriate averaging coefficients.
5. Conclusions
DeepSOCIAL combines deep neural network detection with geometric mapping and tracking to monitor people and social distancing across varied environments. It achieved high accuracy and real-time performance while supporting broader human-detection applications and reproducibility.
- DeepSOCIAL detects and tracks static and dynamic people to monitor social distancing in public places.The model is designed for use with CCTV surveillance cameras in varied environments.
- 99.8% accuracy and 24.1 fps were achieved on the Oxford Town Centre dataset despite occlusion, lighting variation, shades, and partial visibility.The evaluation covered 7530 frames and approximately 150,000 people detections and distance estimations.
- Adapted inverse perspective mapping and SORT tracking estimate inter-person distances, track moving trajectories, and support infection-risk assessment.
- The viewpoint-independent human classification algorithm supports applications including autonomous vehicles, anomaly detection, sports action recognition, and crowd surveillance.
- DeepSOCIAL materials and sample videos are publicly shared through a GitHub repository and YouTube channel to support research reproducibility.