PaperlayerView exact revision on arXiv

Source-linked AI summary

A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle

Gustavo Claudio Karl Couto, Eric Aislan Antonelo, Gabriel George Zipperer

arXiv:2609.04147v1cs.LGcs.AIcs.RO

TL;DR

The paper addresses the need for controlled physical testbeds connecting simulation-based end-to-end driving methods with real-world sensing and control constraints. It presents an open miniature Ackermann platform and evaluates command-conditioned behavior cloning across physical and simulated settings. The learned policy follows physical routes near human demonstration accuracy, while wider camera views and mixed sim-to-real data improve closed-loop route completion.

  • Problem

    Simulation-based driving methods do not expose practical physical-system constraints such as limited field of view, actuator imperfections, calibration errors, and odometry drift.

  • Method

    The paper builds an open platform combining a miniature Ackermann vehicle, printed track, trajectory registration, Webots digital twin, command-conditioned behavior cloning, and sim-to-real-translated synthetic data.

  • Results

    The higher-capacity mixed-data policy is the only configuration completing all four routes; the compact policy reaches 6.1 cm mean cross-track error versus 4.7 cm for human demonstrations, while widening field of view reduces simulated error from 35.6 to 3.3 cm.

  • Takeaways & Limitations

    The released platform provides a practical testbed for sim-to-real studies and an initial command-conditioned imitation-learning baseline for reproducible research.

  • Takeaways & Limitations

    The study is limited by few physical runs, manual trajectory registration, and preliminary ablations that confound capacity with input resolution.

Abstract

from arXiv · show

This paper presents a low-cost, open experimental platform for research in end-to-end autonomous driving with miniature Ackermann vehicles. The platform combines a physical vehicle, a printed urban track, data collection tools, trajectory registration, and a Webots digital twin, enabling controlled experiments that connect simulation-based autonomous-driving methods to real-world execution. As a first baseline, we implement command-conditioned behavior cloning, in which a neural policy receives an on-board camera image and a high-level navigation command and outputs steering and speed. The system is evaluated both on the physical vehicle and in simulation. In real closed-loop experiments, the learned policy follows lanes and executes commanded turns, reaching a mean cross-track error of 6.1 cm with respect to the reference route, close to the 4.7 cm observed in human demonstrations. In the digital twin, camera field of view has a strong effect on performance, reducing the mean cross-track error from 35.6 to 3.3 cm when widened from 58 to 120 degrees. Using the digital twin to generate synthetic driving data and a learned sim-to-real image translator to reduce the appearance gap, we further show that a higher-capacity policy trained on this synthetic data combined with real demonstrations is the only configuration that completes all four track routes in closed loop, whereas the compact baseline and the same network trained on real data alone complete fewer. These results establish the open platform as a practical testbed for sim-to-real studies and provide an initial command-conditioned imitation-learning baseline; we release it to support reproducible research.

1. Introduction

The paper introduces an open miniature-vehicle platform for controlled end-to-end autonomous-driving and sim-to-real experiments. It establishes command-conditioned behavior cloning as a baseline and reports closed-loop route-following results in physical and simulated environments.

  • Platform and motivation: Simulation provides repeatable evaluation, but omits practical constraints including camera latency, limited field of view, actuator imperfections, calibration errors, and odometry drift.
  • Platform and motivation: The platform combines a physical Ackermann vehicle, printed urban track, data-collection tools, trajectory registration, and Webots digital twin.It is designed to transfer, adapt, and compare simulation-based driving methods on real hardware.
  • Baseline: Command-conditioned behavior cloning maps an onboard camera image and high-level navigation command to steering and speed.The command resolves ambiguity when the same intersection image permits multiple maneuvers.
  • Results: The higher-capacity policy trained with sim-to-real-translated synthetic data plus real demonstrations completes all four routes, while the compact and real-data-only policies complete fewer.
  • Results: In the digital twin, camera field of view is a critical factor in route-following performance.

2. Related Work

Related work spans simulation-based end-to-end driving, conditional imitation learning, privileged information, and miniature physical vehicles. Model-scale platforms provide a repeatable intermediate setting between simulation-only research and full-size autonomous cars.

  • Simulation and end-to-end driving: Large-scale simulators support controllable urban scenarios, repeatable evaluation, privileged information, and comparison of end-to-end driving architectures.
  • Simulation and end-to-end driving: Conditional imitation learning uses high-level navigation commands to disambiguate visuomotor policies at intersections.
  • Miniature-vehicle platforms: DeepPicar demonstrated real-time CNN-based end-to-end control on a small embedded vehicle.
  • Miniature-vehicle platforms: F1TENTH provides a 1/10-scale autonomous racing platform with hardware and virtual environments for safe, repeatable experimentation.

3. Command-Conditioned Behavior Cloning Baseline

The baseline conditions a behavior-cloning policy on both camera imagery and a discrete navigation command. It predicts bounded steering and speed using weighted regression, with steering emphasized because route-following accuracy is dominated by it.

  • Policy and training objective: The policy receives an RGB camera image and high-level command, then predicts continuous steering and signed speed.Expert actions are collected during teleoperation.
  • Policy and training objective: Weighted mean squared error trains the policy, with wδ = 1 and wv ≤ 0.25 assigning less weight to speed.The speed term is downweighted because route-following accuracy is dominated by steering and closed-loop speed is capped.
  • Command conditioning: Three commands—follow_lane, turn_left, and turn_right—condition the policy on the intended maneuver.Commands are active within labelled intersection intervals for turns and elsewhere for lane following.
  • Command conditioning: Command conditioning resolves the ambiguity of a purely reactive policy at intersections.

4. Policy Architectures

The architecture uses a CNN image encoder combined with a one-hot navigation command to predict two actions. The paper compares a compact baseline with a higher-capacity variant while preserving the same command-conditioned interface and task.

  • Input and output representation: The input is a 160 × 120 RGB image paired with a one-hot navigation command; the higher-capacity variant uses 320 × 240 frames with a top crop.Actions are bounded in [−1, 1] and denormalized to steering and speed limits.
  • Compact CNN baseline: The compact CNN uses four strided convolutional blocks, global average pooling, command concatenation, and a two-layer MLP, totaling 94,882 parameters.
  • Compact CNN baseline: The compact architecture combines a visual embedding with the command before decoding the two desired actions.
  • Architecture comparison: The larger command-conditioned variant has about 1.0M parameters and preserves the compact model’s input-output interface and task.It enables comparisons of capacity and real-only versus mixed synthetic-plus-real training.

5. Experiments

The experiments use a low-cost Ackermann mini-vehicle, host-side software, and a rectified printed track with four selectable routes. The platform records physical driving data and supports quantitative comparison with a Webots counterpart.

  • Vehicle platform: The off-the-shelf Ackermann vehicle combines rear drive motors, servo steering, an onboard camera, and ESP32-based control electronics.The host runs policy inference and state estimation, while the vehicle closes per-wheel speed loops at 100 Hz.
  • Vehicle platform: The firmware derives per-wheel setpoints from commanded speed and steering using an electronic differential.A battery cutoff and 500 ms command watchdog provide additional execution safeguards.
  • Open platform: The system is released under the MIT license with firmware, host software, calibration files, and the deployed checkpoint.
  • Track and routes: The printed 3.2 × 2.8 m mat defines four one-way loops—outer, center_left, center_right, and center_straight—as selectable routes and high-level commands.The mat is photographed and perspective-rectified to create a flat top-down reference texture.

5.2. Dataset generation

The dataset combines human demonstrations on the physical track with scripted, perturbed Webots driving, then translates synthetic images toward the real camera appearance before mixing the data.

  • Real-data collection: Human teleoperation produces sessions containing camera frames, steering, speed, pose, inertial, encoder, and command-gate labels.Frames are recorded at 10 Hz in 320 × 240 resolution while a deadman trigger gates recording.
  • Real-data collection: The expert dataset contains 41 sessions and 19,206 frames across four route families, including 18,467 active driving frames.Commands are assigned from labelled intersection spans, with straight actions remapped to follow_lane.
  • Synthetic-data generation: The Webots twin generates 96 synthetic laps and 43,581 active frames across the four routes using scripted expert driving, actuator noise, and off-route recovery perturbations.Perturbations enrich visited states without changing the expert labels.
  • Sim-to-real translation: A four-level U-Net translates synthetic images using paired simulator–real data while preserving expert labels, producing a mixed dataset of 137 sessions and 62,048 active frames.The translator is trained on 2,559 registered image pairs from 16 real sessions.
  • Sim-to-real translation: The selected translator reduces mean route error for a real-trained policy from 28.0 to 13.1 cm in the digital twin.It outperforms the compared domain-randomized U-Net, pix2pix, and SimGAN configurations on closed-loop route error.

5.4. Training protocol

Policies are trained by weighted behavior cloning on active frames, with session-level validation and separate real-only versus mixed-data training configurations. Map registration converts camera observations into globally referenced route-deviation measurements.

  • Training: Behavior cloning uses a weighted loss on active frames, optimized with AdamW at learning rate 10^-3, weight decay 10^-4, and batch size 32.Speed and steering loss weights are 0.25 and 1.0, respectively.
  • Training: Whole sessions are held out for validation to prevent leakage between adjacent frames.
  • Training: The compact baseline uses real data, while the higher-capacity policy is trained either on real data alone or on mixed synthetic-plus-real data.
  • Trajectory evaluation: A calibrated ground-plane projection and operator alignment register camera trajectories to the rectified map, yielding globally consistent positions in metres.Ideal route centre lines are constructed from the track loops and marked intersections.
  • Trajectory evaluation: Runs are scored with mean, median, and 95th-percentile errors only when at least ten manual anchors span at least 85% of frames.The reported fully localized set contains 10 human sessions and three neural-control runs.

5.6. Physical closed-loop evaluation

The compact policy is evaluated in fully autonomous physical runs while an operator supplies only high-level commands. It follows several routes without intervention, but the more demanding left-turn route is completed only by the larger mixed-data policy.

  • Physical evaluation: Three compact-baseline runs drive continuously at a speed cap of 0.15 m/s while the operator issues only high-level commands.Each run repeats the outer, center-straight, and center-right loops.
  • Physical evaluation: The compact policy reaches a 6.1 ± 1.2 cm mean cross-track error versus 4.7 ± 1.8 cm for fully localized human demonstrations.The human value is a reference for the available demonstrations, not a strict upper bound.
  • Route completion: The center-left left-turn loop is completed only by the larger mixed-data policy.The compact baseline completes the other evaluated loops without operator interventions.
  • Trajectory registration: The ground-plane homography maps metric track coordinates to camera pixels and relates onboard observations to the rectified map.The projection uses camera intrinsics and extrinsics because the track is planar.

5.7. Digital twin and camera field-of-view ablation

In the Webots digital twin, camera field of view strongly affects route-following accuracy: widening the lens substantially reduces mean cross-track error.

  • 35.6 ± 12.1 cm mean cross-track error with the calibrated 58° lens, including per-route errors of 25–53 cm and peaks above 1 m.
  • 3.3 ± 0.2 cm mean cross-track error with a 120° lens, with per-route errors of 3.1–3.5 cm.
  • Widening the simulated lens from 58° to 120° identifies field of view as a major limiting factor in the digital twin.

5.8. Architecture and command-conditioning ablations

The mixed synthetic-plus-real training condition is the only evaluated configuration that completes all four routes, while the compact and real-only larger policies complete fewer.

  • The compact baseline completes the outer, center-straight, and center-right loops but fails the center-left turn.
  • The larger policy trained on real data alone completes only the outer and center-straight loops.
  • The mixed-data policy is the only configuration completing all four routes in closed loop.
  • The mixed-data policy records per-route mean cross-track errors of 3.9–6.1 cm, 6.0–6.5 cm, 5.1 cm, and 4.2 cm, with an overall mean of about 5.1 cm.
  • The policy comparison figure groups runs by policy, shows route loops by column, and distinguishes complete, partial, and ideal laps by line style or color.

6. Conclusion

The paper presents an open miniature-vehicle platform and baseline for sim-to-real autonomous-driving research. Results include physical lane following, four-route completion with mixed data, and strong sensitivity to camera field of view.

  • 6.1 cm mean cross-track error was achieved by the compact policy on the physical track, compared with 4.7 cm for human demonstrations.
  • The higher-capacity network completed all four routes only when real data was augmented with sim-to-real-translated synthetic laps.
  • Widening the digital-twin camera from 58° to 120° reduced cross-track error from 35.6 to 3.3 cm.
  • The main limitations are few physical runs, manual trajectory registration, and preliminary ablations that confound capacity with input resolution.
Loading 2609.04147v1…