SynTraC: A Synthetic Dataset for Traffic Signal Control from Traffic Monitoring Cameras

I. INTRODUCTION

• Existing traffic signal control datasets lack real images alongside traffic signal data, leaving a gap for methods that interpret visual cues, [9].Camera images can capture vehicle behavior, formation, pedestrians, and cyclists beyond vehicle counts.Source: §I · p. 1
• SynTraC is the first image-based traffic signal control dataset, generated with CARLA to provide real-style roadside-camera images and ground-truth annotations.It includes over 86,000 RGB images, signal states, three reward types, and varied weather and times of day.Source: §I · pp. 1–2, Figure 1 · p. 2
• Experiments reveal a performance gap between image-based approaches and traditional traffic signal control strategies, highlighting challenges for existing RL methods using visual data.The authors identify the need for new algorithms to fully utilize visual data in traffic management.Source: §I · p. 2

II. SYNTRAC DATASET GENERATION PIPELINE

• The CARLA-based generation pipeline has three stages: traffic scene configuration, simulation, and reward calculation.The pipeline extends CARLA because it lacks built-in reward calculation and lane detection.Source: §II · p. 2

A. Traffic Scene Configuration

• Scenario configuration randomizes weather and time of day and covers 12 distinct traffic flows across four intersection paths.Weather examples include sunny, rainy, foggy, and cloudy conditions, with different time slots.Source: A. Traffic Scene Configuration · p. 2
• Four RGB cameras capture 1920×1080 images with a 90° field of view, with frames stored every second.Traffic signals use green-light periods of either 10 or 15 seconds.Source: A. Traffic Scene Configuration · p. 2

B. Simulation and Data Generation

• Images and annotations are gathered from CARLA during simulation, with images captured every second.The pipeline overview is shown in Fig. 2.Source: B. Simulation and Data Generation · p. 2, Figure 2 · p. 3
• The pipeline projects CARLA’s 3D coordinates into 2D image coordinates using a camera projection matrix derived from image dimensions and field of view.The simulation runs at a fixed 0.08-second timestep and captures images every 13 timesteps to produce one image per second.Source: B. Simulation and Data Generation · p. 3
• Lane detection assigns vehicles to lanes by comparing projected lane geometry with each vehicle’s bounding-box center.A distance threshold defines the search region; the cross-product direction identifies right versus left lane orientation.Source: B. Simulation and Data Generation · p. 3, Figure 3 · p. 3

C. Reward Calculation

• SynTraC calculates waiting time, queue length, and throughput rewards to represent different traffic signal control objectives.Lower waiting time and queue length indicate better control, while higher throughput indicates better traffic flow.Source: C. Reward Calculation · p. 3
• Queue length and waiting time are calculated per camera, while throughput sums across cameras and accumulates during a simulation.Waiting time sums stopped vehicles moving below 0.1 m/s; throughput resets only for a new traffic scene.Source: C. Reward Calculation · p. 3

• Image-based traffic signal control methods are evaluated on both synthetic and real-world data.Source: §III · p. 4
• Online CARLA tests use a 240-second simulation across three traffic flows.Source: §III · p. 4
• Policies are first obtained from ground-truth lane counts, then supplied with detector-derived counts; end-to-end image-input training is omitted because image resolution makes training time unacceptable.Source: §III · p. 4

• The evaluation adopts commonly used detection models, RL models, and metrics.Source: A. Evaluation Setting · p. 4
• Four offline RL methods—DQN, DDQN, SAC, and CQL—are compared with the rule-based MaxPressure policy using ground-truth lane counts.Source: A. Evaluation Setting · p. 4
• The image-based methods use Masked R-CNN, Faster R-CNN, or RetinaNet, with results reported for pretrained and fine-tuned detectors.Source: A. Evaluation Setting · p. 4
• Detection is measured by vehicle-count MSE and MAE, while control uses total waiting time, queue length, throughput, and travel time.Source: A. Evaluation Setting · p. 4

Evaluation of Detection Models

• Faster R-CNN performs better across weather and times than the other evaluated detection models.Its inference time is also reported as an advantage for real-world use.Source: B. Evaluation Results · p. 4
• Detection-model performance drops outside sunny daytime, with nighttime causing the largest decline; Table I presents performance by weather and time.The authors report that SynTraC's image variety supports fine-tuning detectors for nighttime use.Source: B. Evaluation Results · p. 4, Table I · p. 4

Evaluation of Control Policies

• All RL models using ground-truth lane counts outperform rule-based policies on all metrics except DDQN, while CQL leads among RL models but loses the most performance when paired with detectors.The authors attribute CQL's detector-related drop to a better policy being more sensitive to incorrect information.Source: B. Evaluation Results · p. 4
• Pairing detectors with control policies substantially reduces performance even in sunny daytime, and nighttime losses are more significant than rainy-weather losses.Source: B. Evaluation Results · p. 4

C. Influence of Different Camera Angles

• The camera-angle evaluation includes raised and lowered camera angles and a raised camera location for bird-view images.Source: C. Influence of Different Camera Angles · pp. 4–5
• Changing camera angles significantly affects RL policies, and Table III reports that methods are not robust to camera angles.Source: C. Influence of Different Camera Angles · p. 5, Table III · p. 5
• Raising camera locations captures more comprehensive information for detection and gives CQL the best performance in that setting.SAC and DQN also approach performance using ground-truth counts.Source: C. Influence of Different Camera Angles · p. 5

D. Generalization on Real-world Data

• Fine-tuning RetinaNet with SynTraC increases detection performance on real-world data, based on 100 annotated images from a six-minute intersection video.The comparison is with the pre-trained detection model.Source: D. Generalization on Real-world Data · p. 5, Table IV · p. 5
• Because real-world signal states cannot be changed for testing, the authors do not report standard metrics such as waiting time for the real-world TSC evaluation.Source: D. Generalization on Real-world Data · p. 5
• In a real-world scenario, the SynTraC-trained end-to-end controller changes the signal after over 10 vehicles wait north and south for around 30 seconds, while only 3 cross-traffic cars pass.The authors describe this as the correct decision compared with the real-world controller's continued red light.Source: Figure 4 · p. 5, D. Generalization on Real-world Data · p. 6

IV. DISCUSSION

• Some traffic signal control datasets provide signal timings and traffic states, but none of these are suitable for RL because they contain only signal information.Source: §IV · p. 6
• Cityflow and SUMO datasets used by many RL approaches provide simulator-derived traffic information but do not contain traffic monitoring images.Source: §IV · p. 6
• SynTraC fills the gap identified by the authors for traffic signal control data that includes traffic monitoring images.Source: §IV · p. 6

V. CONCLUSION AND FUTURE WORK

• SynTraC is presented as the first image-based TSC dataset with reward data for offline reinforcement learning and a generation pipeline aligned with real-world conditions.It also includes information for downstream tasks such as vehicle detection.Source: §V · p. 6
• RL policies perform very well with ground truth information, while integrating detection models into the TSC system hurts policy performance.Source: §V · p. 6
• Future work includes control algorithms for uncertain information under varying weather and a thorough real-world evaluation of the end-to-end TSC system.Source: §V · p. 6