Source-linked AI summary
A Vision-based Social Distancing and Critical Density Detection System for COVID-19
Dongfang Yang, Ekim Yurtsever, Vishnu Renganathan, Keith A. Redmill, Ümit Özgüner
TL;DR
The paper addresses how to support social distancing while avoiding privacy violations from recording or storing surveillance data. It proposes a real-time monocular-camera system that detects breaches, warns crowds non-intrusively, and modulates ROI inflow at critical density; the method was evaluated on pedestrian crowd datasets.
Problem
An automatic system is needed to help people maintain social distance, but storing surveillance data can violate privacy, requiring real-time operation without data storage.
Method
A fixed monocular camera and pretrained CNN detect pedestrians, transform image detections into world coordinates, issue non-intrusive audio-visual cues for breaches, and control ROI entry above critical social density.
Results
The method measured social distancing and critical-density statistics across three pedestrian crowd datasets, with real-time performance reported for both detectors.
Takeaways & Limitations
Critical social density can be used to modulate ROI inflow and help keep the probability of social-distancing violations below an empirically selected threshold.
Takeaways & Limitations
The system has missing detections in extremely dense, occluded areas of the Train Station Dataset, and it does not model pedestrians belonging to groups as groups.
Abstract
from arXiv · showhide
Social distancing has been proven as an effective measure against the spread of the infectious COronaVIrus Disease 2019 (COVID-19). However, individuals are not used to tracking the required 6-feet (2-meters) distance between themselves and their surroundings. An active surveillance system capable of detecting distances between individuals and warning them can slow down the spread of the deadly disease. Furthermore, measuring social density in a region of interest (ROI) and modulating inflow can decrease social distancing violation occurrence chance. On the other hand, recording data and labeling individuals who do not follow the measures will breach individuals' rights in free-societies. Here we propose an Artificial Intelligence (AI) based real-time social distancing detection and warning system considering four important ethical factors: (1) the system should never record/cache data, (2) the warnings should not target the individuals, (3) no human supervisor should be in the detection/warning loop, and (4) the code should be open-source and accessible to the public. Against this backdrop, we propose using a monocular camera and deep learning-based real-time object detectors to measure social distancing. If a violation is detected, a non-intrusive audio-visual warning signal is emitted without targeting the individual who breached the social distancing measure. Also, if the social density is over a critical value, the system sends a control signal to modulate inflow into the ROI. We tested the proposed method across real-world datasets to measure its generality and performance. The proposed method is ready for deployment, and our code is open-sourced.
1. Introduction
The paper proposes a real-time, privacy-conscious AI system that detects social-distancing breaches, issues non-intrusive warnings, and uses critical density to help regulate ROI entry.
- Motivation: Social distancing supports COVID-19 control, but people may not reliably maintain the required safety distance without automated assistance.The paper frames automatic warning as a way to augment individuals’ perceptive capabilities.
- Ethical design: The system is designed to avoid recording data, discrimination, targeted warnings, human involvement in detection, and closed implementation.Its ethical design emphasizes real-time processing, AI-based detection, non-intrusive cues, and public accessibility.
- Proposed system: A pre-trained deep CNN detects people in monocular camera frames, transforms image detections into bird’s-eye coordinates, and measures interpersonal distances.When distance falls below the threshold, the system emits an omnidirectional audio-visual cue.
- Proposed system: The proposed system defines critical social density to measure overcrowding and modulate inflow into a region of interest.The paper presents critical density as a novel contribution alongside social-distancing measurement.
- Evaluation: Measurements cover social distancing and critical-density statistics in New York Central Station, an indoor mall, and a busy Oxford town center.These common crowded places are used to assess the proposed system’s measurements.
2. Related Work
Prior work uses cameras and alternative sensors for social-distancing monitoring, while this paper emphasizes softer warnings and critical-density-based inflow modulation without recording data.
- Social distancing monitoring: Social distancing is defined as maintaining at least 2 meters (6 feet) between people to avoid possible contact during COVID-19.The paper also notes reported economic benefits of social distancing.
- Pedestrian detection: Object detectors include two-stage R-CNN models, one-stage models such as YOLO and SSD, and anchor-free approaches such as CornerNet and CenterNet.These detectors are commonly evaluated on Pascal VOC and MS COCO datasets.
- Social distancing monitoring: Existing prototypes monitor social distancing with surveillance cameras, LiDAR, and stereo cameras, sometimes issuing real-time voice alerts.The cited systems demonstrate that multiple sensing modalities can support monitoring.
- Social distancing monitoring: The proposed system avoids recorded data and intrusive alerts by using softer omnidirectional audio-visual cues and modulating ROI inflow through critical social density.This combines breach warning with overcrowding control.
3. Preliminaries
Deep CNN-based object detectors dominate modern detection benchmarks, and their reported generalization supports using pre-trained pedestrian detectors for new social-distancing environments.
- Object detection with deep learning: Object detection identifies instances of semantic classes such as humans, cars, and buildings in images.Deep CNN models have driven major gains on benchmarks including MS COCO.
- Object detection with deep learning: Supervised training commonly uses data augmentation to increase the variety of training data.The passage describes augmentation as a standard training technique for these models.
- Model Generalization: Pre-trained detectors can perform well under differing camera models, viewing angles, and illumination conditions.This reported generalization is presented as sufficient for deployment in new environments.
- Model Generalization: The paper therefore proposes directly using a pre-trained state-of-the-art deep-learning pedestrian detector for social-distancing monitoring.The choice relies on the described generalization capability.
4. Method
The method detects pedestrians from monocular camera images, maps them into real-world coordinates, and computes inter-pedestrian distances. It uses these distances and social density to issue non-intrusive warnings and estimate a maximum safe critical density without recording data after deployment.
- A fixed monocular camera detects people in an ROI and measures interpersonal distances in real time without recording data.
- The control algorithm activates an audio-visual cue when d ≤ d_c and advises against entering the ROI when ρ > ρ_c.
- Deep CNN detections produce person bounding boxes, whose bottom-edge centers are mapped from image coordinates to 2D bird’s-eye-view coordinates.The mapping uses an inverse perspective transformation based on a transformation matrix M.
- Inter-pedestrian distance d_i,j is computed as the Euclidean distance between pedestrians’ real-world pose vectors, and distances below d_c count as violations.
- Critical density ρ_c is chosen as the maximum safe density whose violation probability remains below U_0, using linear regression of violations v on density ρ.The estimate uses the lower bound of the 95% prediction interval at v = 0; after fitting, deployment operates without recording data.
5. Experiments
The system was evaluated through three case studies using pedestrian datasets from an urban street, an indoor mall, and a train station. Experiments applied perspective transformation, pedestrian detection, real-world coordinate conversion, and ROI-based density and distance monitoring.
- Three case studies used the Oxford Town Center, Mall, and Train Station pedestrian crowd datasets.The datasets represent an urban street, an indoor mall, and New York City’s Grand Central Terminal.
- Each dataset required a perspective transformation matrix to map image measurements into real-world coordinates.The matrix came from an official source for Oxford, a measured floor plan for the train station, and a reference-object estimate for the mall.
- The experiments applied pedestrian detection and converted detected positions from image coordinates into real-world coordinates.
- Figure 3 illustrates pedestrian detection with Faster R-CNN and the corresponding social distancing.
- Monitoring considered only pedestrians inside the ROI and tracked social density, inter-pedestrian distances, and violation counts over time.
6. Results
The system evaluates real-time pedestrian detection, physical-distance metrics, and relationships between social density and distancing violations. It estimates critical density from regression analysis and finds consistent values across datasets.
- Pedestrian Detection: Faster R-CNN and YOLOv4 both achieved real-time pedestrian-detection performance.Detection accuracy was reported from the original MS COCO studies.
- Social Distancing Monitoring: For each pedestrian, the method computes the closest physical distance and derives minimum and average closest-distance metrics.These metrics track social-distancing conditions across detections.
- Social Distancing Monitoring: Social density ρ and average closest physical distance show a negative correlation over time across the evaluated datasets.Lower-density periods correspond to higher minimum and average distances, including examples at 85 s, 80 s, and 100 s.
- Critical Social Density: The number of social-distancing violations v increases with social density ρ, indicating a positive linear relationship.The relationship was investigated using two-dimensional histograms before regression.
- Critical Social Density: Critical density ρc is identified from the lower prediction-interval bound at v = 0, and the resulting values are similar across datasets.The regression analysis used the density distributions' normality assumption and compared critical densities with model intercepts.
7. Conclusion
The work proposes a real-time monocular-camera system for monitoring social distancing and using critical social density to modulate inflow and avoid overcrowding. It was verified on three pedestrian crowd datasets, but remains limited by missed detections in extreme-density occlusions and by not modeling pedestrian groups.
- Conclusion: The system monitors social distancing with AI and a monocular camera, using critical social density to modulate inflow into the ROI.The proposed system is intended to avoid overcrowding through inflow control.
- Conclusion: Table 1 lists information about each pedestrian dataset, while Tables 2 and 3 report detector performance and critical social density, respectively.Table 2 reports real-time detector performance; Table 3 defines critical density using the prediction interval at v = 0.
- Conclusion: The method was verified using 3 different pedestrian crowd datasets.
- Conclusion: The Train Station Dataset has missing detections in extremely dense areas because of occlusion.The authors state that most pedestrians were nevertheless captured and the critical-density approach remained valid.
- Conclusion: Pedestrians belonging to a group were not modeled as a group in the current work.The authors identify group modeling as a future direction.