PaperlayerView exact revision on arXiv

Source-linked AI summary

Lightweight Machine Learning-Driven Monocular Sidewalk Path Extraction for Embedded Micromobility Navigation

Lkhanaajav Mijiddorj, Yang Yan, Tyler Beringer, Bilguunzaya Mijiddorj, Alex N. Ho, Bin Xu, Binbin Weng

arXiv:2608.25178v1cs.CVcs.AI

TL;DR

Embedded micromobility needs efficient, robust sidewalk path extraction despite cluttered scenes, variable geometry, scarce maps, and unreliable GNSS. This paper compares monocular BEV and image-space planning and finds that simple image-space geometry consistently outperforms BEV in accuracy, reliability, and latency.

  • Problem

    Embedded micromobility requires efficient, robust sidewalk perception and navigation despite clutter, changing geometry, scarce maps, and unreliable GNSS.

  • Method

    The paper develops a modular monocular pipeline combining lightweight sidewalk segmentation with five path-planning methods across BEV and image-space domains.

  • Results

    14.3 px lateral center error at 2.2 ms made image-space midpoint planning 421× faster than BEV distance-transform planning, while image-space geometry consistently outperformed BEV overall.

  • Takeaways & Limitations

    The recommended image-space midpoint architecture with distance-transform fallback is transparent, lightweight, and deployable on single-board computers.

  • Takeaways & Limitations

    Dynamic obstacle handling is limited to implicit mask avoidance, with explicit obstacle detection and prediction deferred to future work.

Abstract

from arXiv · show

Sidewalk-scale path extraction demands perception and planning that run reliably on compact, low-power hardware in cluttered, map-sparse environments. We present a monocular vision pipeline for sidewalk path extraction in micromobility systems that progresses through three design iterations, from a skeleton-graph baseline through distance-transform corridor planning to a lightweight image-space architecture, and provides a systematic comparison of five path-planning methods across both bird's-eye-view (BEV) and image-space domains. A compact SegFormer-B0 student model, trained with a semi-supervised teacher-student framework using OneFormer Swin-L pseudo-labels, achieves a hand-annotated IoU of 0.946 at 11.7 ms per frame, improving over the baseline checkpoint (IoU 0.758, 18.9 ms). In a controlled planner comparison on 32 hand-labeled frames, image-space midpoint planning achieves the lowest lateral center error (14.3 px) at 2.2 ms, a 421x speedup over BEV distance-transform planning (926.8 ms, 65.0 px center error), while maintaining comparable mask-path alignment (98.5% versus 98.6%). A full-video replay across six campus sequences (22,679 frames) confirms that the improved segmentation reduces temporal instability from 1.46% to 0.33% and increases template-path availability from 73.7% to 79.3%. We further show that BEV-only path extraction is fragile in monocular settings: in one profiled run, 99.3% of frames produced no valid BEV path. The final recommended architecture, image-space midpoint primary, image-space distance-transform fallback, and BEV reserved for visualization, runs the full perception-to-path stack in under 50 ms per frame on CPU, making it suitable for embedded pedestrian-speed micromobility systems.

1 Introduction

The paper presents a transparent, lightweight monocular pipeline for extracting local sidewalk paths on embedded hardware, motivated by cluttered, variable, and map-sparse sidewalk environments. It evaluates segmentation and path generation offline while comparing BEV and image-space geometric planning.

  • Motivation: Sidewalk micromobility robots must navigate cluttered, irregular environments under tight payload, power, and cost constraints.Sidewalk widths, surfaces, markings, lighting, maps, and GNSS reliability vary substantially.
  • Approach: A lightweight monocular segmentation model extracts traversable sidewalk regions, followed by geometric path extraction in BEV or image space.BEV is obtained through planar homography, while image-space planning uses direct boundary analysis.
  • Approach: Five path-planning methods are systematically compared across BEV and image-space domains.The study reports that simple image-space geometry consistently outperforms the compared BEV alternatives, according to the supplied introduction passage.
  • Evaluation: The offline evaluation isolates segmentation quality, planner accuracy, runtime efficiency, and the BEV-versus-image-space trade-off.Video from a forward-facing camera mounted on an electric scooter is processed frame by frame without closing the vehicle-control loop.
  • Contributions: 0.946 hand-annotated IoU is achieved by a compact SegFormer-B0 student trained with pseudo-labels from a OneFormer Swin-L teacher, at 11.7 ms on CPU.This is presented as a semi-supervised segmentation training recipe for embedded hardware.
  • Contributions: The work contributes a modular monocular sidewalk-path pipeline designed to be transparent and lightweight, with three design iterations and backward-compatible improvements.The contribution targets embedded hardware deployment.

2 Related Work

Prior work spans end-to-end navigation, monocular BEV reconstruction, efficient segmentation, generative free-space prediction, corridor following, skeleton geometry, and image-space planning. This paper favors lightweight segmentation paired with deterministic geometric planning, using image-space alternatives to avoid BEV distortion and latency issues.

  • Target-driven and end-to-end navigation: End-to-end navigation simplifies runtime with one model and policy but can be difficult to interpret under shifted conditions and may require GPU-class compute.
  • Dense monocular bird’s-eye reconstruction: Monocular BEV reconstruction performs strongly on modern GPUs, whereas calibrated homography offers predictable metric framing for near-planar sidewalk patches with lower overhead.
  • Segmentation for drivable/free space: Compact backbones reduce embedded latency and memory, while this approach carries segmentation masks into geometry and planning to form a complete pixel-to-motion path.
  • Generative free-space priors and diffusion: Diffusion-based free-space predictors can hallucinate plausible corridors under weak signals, but their footprint and tuning complexity mismatch embedded scooter requirements.
  • Corridor following and agricultural rows: Classical thinning, histogram-based corridor following, and agricultural row navigation provide fast geometric precedents, but sidewalks add junctions, driveways, and uneven edges.
  • Image-space planning and distance transforms: Image-space midpoint extraction and distance-transform maxima can produce smooth centerlines while avoiding homography distortion and coverage failures at substantially lower latency.

3 Methodology · 3.1 Segmentation Module and Supervision Strategy

The methodology maps monocular RGB camera frames to controller-ready waypoints through modular segmentation, optional BEV projection, path planning, and waypoint smoothing. SegFormer-B0 is trained with hand-labeled and cleaned pseudo-labeled data in a teacher–student framework, improving hand-annotated IoU from 0.758 to 0.946.

  • 3 Methodology: The pipeline converts monocular RGB frames into controller-ready waypoints through segmentation, optional BEV projection, five path-planning methods, and waypoint smoothing.The architecture supports both BEV-domain and image-space planning.
  • 3.1 Segmentation Module and Supervision Strategy: SegFormer-B0, with 3.7M parameters, is selected for deployment because of its low latency and memory footprint.SegFormer is adopted for its urban-benchmark performance and adaptability across model scales.
  • 3.1 Segmentation Module and Supervision Strategy: The semi-supervised teacher–student framework addresses noisy masks from small models trained on limited data.Two teacher architectures were explored across the design iterations.
  • 3 Methodology: Five path-planning methods are compared across BEV and image-space domains after semantic segmentation and geometric processing.The comparison is part of a modular sidewalk-navigation architecture.
  • 3.1 Segmentation Module and Supervision Strategy: Iteration 2 uses a OneFormer Swin-L teacher to generate dense binary pseudo-labels from up to 1,419 unlabeled frames across 10 campus video sequences.Iteration 1 instead uses a SegFormer-B2 teacher fine-tuned on 300 hand-labeled frames and applied to 2,300 unlabeled frames.
  • 3.1 Segmentation Module and Supervision Strategy: Pseudo-labels are cleaned with connected-component filtering and morphological smoothing before student training.These operations refine the generated training masks.
  • 3.1 Segmentation Module and Supervision Strategy: 1,419 image–mask pairs train the best-performing student, combining 699 pairs from earlier sequences with 720 newly extracted frames from four additional campus videos.Hand labels take precedence wherever hand-labeled and pseudo-labeled frames overlap.
  • 3.1 Segmentation Module and Supervision Strategy: 0.960 validation IoU is achieved by the OneFormer teacher with expanded data, while hand-annotated IoU improves from 0.758 to 0.946 over the baseline.The hand-annotated evaluation is external to the internal validation split.

3.2 Resolution Trade-Off Analysis · 3.3 Bird’s-Eye View Projection

The analysis selects 640×360 SegFormer-B0 inference as a practical accuracy–latency trade-off, then projects segmented masks into a metric-scale bird’s-eye representation using a planar homography. This BEV transformation aligns the mask with the robot frame but depends on a flat ground plane and fixed camera pose.

  • 3.2 Resolution Trade-Off Analysis: 640×360 was selected as the practical operating point, providing accurate and consistent segmentation with sub-50 ms inference on CPU.Lower resolutions produced overly coarse, fragmented masks, while 960×540 and higher incurred prohibitive latency.
  • 3.2 Resolution Trade-Off Analysis: 320×180 and below produced overly coarse masks that fragmented continuous sidewalk segments.The resolution sweep evaluated SegFormer-B0 on CPU to balance segmentation accuracy and real-time performance.
  • 3.2 Resolution Trade-Off Analysis: 960×540+ incurred prohibitive latency despite improving segmentation detail.The sweep therefore favored a lower practical resolution rather than the highest available image quality.
  • 3.3 Bird’s-Eye View Projection: A planar homography transforms the segmented image mask into a spatially consistent top-down BEV binary mask.In this representation, pixel distances approximate ground distances.
  • 3.3 Bird’s-Eye View Projection: The homography H is estimated from four point correspondences between the camera image and a flat ground reference.The image mask Mimg is warped into BEV coordinates through the homogeneous-pixel mapping.
  • 3.3 Bird’s-Eye View Projection: The resulting BEV mask aligns with the robot reference frame, placing the ego position at bottom-center and forward motion along the vertical axis.This arrangement supports spatial reasoning relative to the robot’s motion direction.
  • 3.3 Bird’s-Eye View Projection: BEV projection enables metric-scale reasoning but assumes a flat plane and fixed camera pose.The paper tests the validity of these assumptions empirically in Section 4.3.

3.4 BEV Mask Refinement · 3.5 Path Planning Methods

BEV mask refinement removes perspective-warp artifacts and disconnected regions before planning. Five planners span BEV and image-space domains, with image-space methods offering distinct speed and robustness tradeoffs.

  • 3.4 BEV Mask Refinement: Morphological cleanup suppresses gaps and spurs, while connected-component filtering retains the largest region touching the ego anchor row.Optional flood-fill, boundary smoothing, and distance-transform selection stages are independently configurable for ablation.
  • 3.5 Path Planning Methods: Five path-planning methods operate across two geometric domains, receiving a binary sidewalk mask and producing a centerline trajectory.
  • 3.5.1 BEV Skeleton-Graph Planner: The BEV skeleton-graph planner thins masks into 1-pixel skeletons, builds 8-neighbor graphs, prunes branches, and uses Dijkstra’s algorithm to enumerate candidate paths.It exposes junction topology but is computationally expensive because it constructs graphs per pixel.
  • 3.5.2 BEV Distance-Transform Ridge Planner: The BEV distance-transform ridge planner traces maximum-clearance paths by minimizing an EDT-based cost field with Dijkstra’s algorithm.Using ε = 0.5 and α = 1.5, it avoids skeletonization noise but retains BEV, EDT, and graph-search costs.
  • 3.5.3 BEV Template Arc Planner: The BEV template arc planner scores five fixed arc templates for corridor containment and smoothness, selecting the highest-confidence candidate or falling back to the DT ridge planner.Its small fixed arc bank trades generality for predictable runtime and interpretability.
  • 3.5.4 Image-Space Midpoint Planner: 2.2 ms is the image-space midpoint planner’s reported runtime, obtained by smoothing row-wise sidewalk-boundary midpoints while bypassing BEV.Rows with fewer than the minimum number of road pixels are skipped before centerline smoothing.
  • 3.5.5 Image-Space Distance-Transform Planner: 108.1 ms is the image-space distance-transform planner’s reported runtime, using dynamic programming with lateral drift constraints to follow the widest corridor regions.It is more robust than midpoint extraction for irregular boundaries or partial gaps, but slower.

3.6 Temporal Smoothing

The pipeline uses optional temporal filters to reduce trajectory jitter from segmentation noise. Path coefficients and heading angles are smoothed with adaptive reset mechanisms that preserve responsiveness to abrupt path or heading changes.

  • Segmentation noise can produce jittery frame-by-frame path trajectories, motivating two optional temporal filters.
  • Path smoothing applies an exponential moving average to cubic polynomial coefficients, with α→0.85 for high-confidence paths and α→0.35 for low-confidence paths.A topology-change detector resets the filter when coefficient jumps exceed a threshold, preventing averaging across distinct path segments.
  • Heading smoothing uses a circular EMA to handle the ±180◦ wraparound discontinuity.The filter resets when the heading delta exceeds 45◦, preserving rapid responses to sharp turns while suppressing straight-segment noise.

3.7 Evaluation Metrics

The pipeline is evaluated offline on held-out campus video using four quantitative metrics covering segmentation quality, path centering, mask–path alignment, and temporal stability.

  • Segmentation quality: Segmentation IoU averages per-frame overlap between predicted binary masks and hand-annotated ground truth.Intersection-over-union quantifies segmentation quality.
  • Path centering: Lateral path-centering error measures sample-point deviation from the sidewalk mask’s geometric centerline in pixels.Results include mean and standard deviation, convertible to meters using the known BEV scale.
  • Path alignment: Mask–path alignment reports the fraction of path pixels inside the ground-truth sidewalk mask.This metric is also called the inside-GT ratio.
  • Temporal stability: Temporal stability uses frame-to-frame IoU between consecutive cleaned masks to identify unstable frames and quantify segmentation flicker.Frames below a threshold are flagged as unstable, and the unstable rate captures temporal noise.

4 Results

The candidate segmentation model improves accuracy, speed, temporal stability, and path availability across hand-annotated and full-video evaluations. Image-space midpoint planning is the recommended primary method, with image-space distance-transform fallback, while BEV planning is slower and unreliable in monocular settings.

  • Segmentation results: 0.946 IoU and 11.7 ms inference improve on the baseline’s 0.758 IoU and 18.9 ms, a 38% speedup.The improved model uses a OneFormer Swin-L teacher with threshold 0.60 and was evaluated on 32 hand-annotated frames.
  • Full-video evaluation: 77% lower temporal instability, from 1.46% to 0.33%, accompanies a 5.6-percentage-point increase in template-path availability.These results come from full-video replay across six campus sequences totaling 22 679 frames.
  • Planner comparison: 14.3 px lateral center error and 2.2 ms runtime make image-space midpoint the best primary planner, versus 65.0 px and 926.8 ms for the best BEV method.Mask–path alignment remains comparable at 98.5% for image-space midpoint versus 98.6% for the best BEV method.
  • Planner comparison: 99.4% inside-GT ratio makes image-space distance transform the best fallback when midpoint extraction fails, while remaining 8.6× faster than BEV distance transform.Midpoint failures occur with discontinuous boundaries or irregular masks.
  • BEV reliability: 99.3% of 4,407 profiled frames produced no valid BEV path, because monocular projection leaves only a narrow occupied strip in the BEV grid.Distance-transform and skeleton algorithms frequently fail when operating on the resulting tiny sliver; oracle-mask experiments indicate better segmentation alone cannot resolve this geometry-related fragility.
  • Runtime analysis: 2.2 ms planner runtime instead of 380.3 ms enables the recommended image-space architecture to exceed 59 FPS on CPU.The architecture uses image-space midpoint primary planning with distance-transform fallback and eliminates the BEV warp.

5 Conclusion

The study concludes that simple image-space geometry provides a more accurate, reliable, and lower-latency approach to monocular sidewalk navigation on embedded platforms than complex BEV processing. It recommends a lightweight image-space architecture while identifying offline evaluation, fixed-camera assumptions, and limited annotated data as remaining constraints.

  • Conclusion: The modular pipeline evolves from skeleton-graph and distance-transform planning to a lightweight image-space architecture, with systematic comparison of five planning methods.The conclusion frames this progression as both an engineering improvement and a scientific contribution.
  • Quantitative findings: 0.946 hand-annotated IoU at 11.7 ms is achieved by the OneFormer-trained SegFormer-B0 student, versus 0.758 IoU and 18.9 ms for the baseline.The student model improves both segmentation quality and runtime relative to the baseline.
  • Quantitative findings: 14.3 px lateral center error at 2.2 ms makes image-space midpoint planning 421× faster and more accurate than BEV DT planning at 65.0 px and 926.8 ms.The conclusion presents image-space geometry as consistently outperforming the more complex BEV pipeline in accuracy, reliability, and latency.
  • Quantitative findings: 99.3% of frames produce no valid BEV-only path in one profiled sequence because of insufficient monocular mask coverage.This failure motivates reserving BEV processing rather than using it as the primary path-extraction approach.
  • Limitations and future work: Offline evaluation, unmodeled dynamic obstacles, a fixed homography, and comparison on only 32 hand-annotated frames remain limitations requiring broader validation.The paper notes that larger annotated benchmarks and physical closed-loop scooter evaluation would strengthen the findings.
  • Limitations and future work: Future work targets closed-loop scooter deployment with obstacle avoidance, temporal video segmentation, and integration of local planning with global routing and mapping.These directions address deployment, temporal stability, and longer autonomous traversals.

Funding

The research received no specific grant funding; the authors also disclosed using Claude for code development, data analysis, and language editing, with subsequent human review and editing.

  • The research received no specific grant from public, commercial, or not-for-profit funding agencies.
  • The authors used Claude to assist with code development, data analysis, and language editing, then reviewed and edited the content.The authors stated that they take full responsibility for the paper’s content.

CRediT authorship contribution statement

The contribution statement assigns L. Mijiddorj the broadest technical and writing responsibilities, while the other listed authors contribute primarily through conceptualization, investigation, data curation, supervision, or writing support.

  • CRediT authorship contribution statement: L. Mijiddorj is credited with conceptualization, methodology, software, validation, formal analysis, investigation, data curation, visualization, and both stages of writing.The listed writing roles are original-draft writing and review and editing.
  • CRediT authorship contribution statement: B. Mijiddorj and T. Beringer contributed to conceptualization, investigation, and writing review and editing.
  • CRediT authorship contribution statement: Y. Yan contributed to conceptualization, data curation, and writing review and editing, while A. Ho contributed to conceptualization and writing review and editing.
  • CRediT authorship contribution statement: B. Xu is listed with conceptualization and supervision before the supplied contribution statement is truncated.
Loading 2608.25178v1…