NavAble: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation

1 Introduction

• Accessibility-critical classes are absent, coarsely defined, or underrepresented in large-scale vision datasets, while some open-vocabulary models degrade substantially in zero-shot settings.The examples include APS, handrails, and door-activation buttons.Source: §1 · p. 2
• Existing BLV datasets have narrow scopes and do not provide dense detection or segmentation across broad BLV-relevant classes, limiting scale, semantic coverage, and intra-class diversity.Prior datasets include obstacle detection, surface-discontinuity hazards, tactile walking surface indicators, and image-level presence labels.Source: §1 · p. 2
• Most vision datasets use human-eye-level or vehicle-mounted views, unlike assistive platforms’ egocentric viewpoints; limited diversity and environmental coverage compound the mismatch.The passage links these distribution differences with poor zero-shot segmentation performance.Source: §1 · p. 2
• NavAble combines 41.5K densely annotated real-world images across 11 accessibility-critical classes with a generator that produces 452K densely annotated synthetic frames from 500 class-validated assets across 11 categories.The real-world set comprises 5.5K newly collected egocentric and 36K curated open-source images; synthetic frames pair RGB with semantic segmentation, depth, surface normals, and 2D/3D bounding boxes.Source: §1 · pp. 2–3
• Fine-tuning on NavAble improves accessibility-object segmentation, synthetic augmentation adds up to +3.5 mIoU, and gains transfer to egocentric data from two assistive platforms.The reported transfer evaluation uses data from a quadruped robot and a cane-shaped robot.Source: §1 · p. 3

2 Real-world data: curation and collection

• The 11 selected classes support mobility decisions such as crossing streets, entering buildings, locating transit infrastructure, and navigating vertical transitions.Selection criteria include navigational utility and frequent occurrence in assistive settings, spanning indoor and outdoor environments.Source: §2 · p. 3
• Open-source datasets have inconsistent masks, severe class imbalance, fragmented semantic coverage, and mislabeled or ambiguous samples, motivating curation and targeted collection.Some categories are absent, while crosswalk and elevator-button samples are heavily overrepresented.Source: §2 · pp. 3–4

2.1 Data curation from open-source datasets

• Curation standardizes class definitions across sources and filters low-quality or label-inconsistent samples, yielding over 36.5K annotated images while leaving severe class imbalance unresolved.Figure 2 depicts curated and normalized online data merged with newly collected dense annotations.Source: Figure 2 · p. 3, §2.1 · p. 4

2.2 Targeted real-world data collection

• Targeted collection addresses underrepresented classes and unresolved mask definitions with dense pixel-level labels across all 11 classes, diverse indoor and outdoor environments, egocentric viewpoints, and contextual co-occurrence.Images were captured at chest height across 50 locations, manually filtered, polygon-annotated, and reviewed through three full passes.Source: §2.2 · p. 4

3 Scalable synthetic data generation (SDG) pipeline

• Real-world data cannot provide the scale, diversity, and viewpoint variability needed for robust perception, making synthetic data a complement for rare-class coverage and generalization.The pipeline uses NVIDIA Isaac Sim with controllable scene composition and viewpoints.Source: §3 · p. 4

3.1 From web images to simulation-ready 3D assets using foundation models

• The asset pipeline crawls class-specific web images, filters unsuitable samples with a vision-language model, detects objects, segments them, and reconstructs textured 3D meshes for Isaac Sim.Grounding DINO and SAM2.1 support localization and masks; SAM 3D Objects reconstructs the image-mask pairs.Source: §3.1 · pp. 4–5
• Reconstruction quality is bounded by segmentation performance, which remains challenging for long-tail accessibility classes.The pipeline yields over 500 class-validated assets across 11 categories.Source: §3.1 · p. 5
• Recorded camera trajectories can be replayed across environments, lighting, and asset styles, while scene-graph annotations preserve consistency and make interactive collection a one-time cost.Five 100-frame trajectories replayed across four environments and 10 asset variants produce 20,000 annotated frames.Source: §3.1 · p. 5

3.2 NavAble synthetic dataset

• NAVABLEsyn contains 452K RGB frames and 781K instance annotations from over 25K trajectories across 37 environments, with pixel-aligned segmentation, boxes, normals, and depth.The split uses 488 locations populated with retained assets from the 11 NAVABLE classes.Source: §3.2 · p. 5

4 Experiments

• The experiments assess zero-shot recognition, real-data fine-tuning, synthetic augmentation, and transfer to real-time embodied deployment.The embodiment study uses SegFormer because it is the only benchmark architecture that runs in real time on both platforms.Source: §4 · pp. 5–6, Does the model improvement… · p. 8

4.1.1 Model training and setup

• The held-out test set contains 1,482 real-world images from locations excluded from training and validation; four configurations compare real, curated, and synthetic data, with turnstile excluded from mean metrics.Synthetic budgets are matched by foreground instances per class, and the validation and test splits are identical across configurations.Source: §4.1 · p. 6, §4.1.1 · pp. 6–7

4.2 Evaluation results

• Closed-vocabulary models omit most target classes, so SAN is the only evaluated open-source model with a relevant zero-shot segmentation baseline.The experiments benchmark five segmentation architectures, initialized from released ADE20K weights except SAN, which uses COCO-Stuff-164K weights.Source: §4.1.1 · p. 6, Table 1 · p. 6, §4.2 · p. 7
• Adding curated open-source data improves SegFormer, SAN, and Mask2Former mIoU, but degrades DeepLabv3+ and EncNet, consistent with challenges from fragmented, variably annotated data.Mask2Former reaches 71.29% mIoU with Real+Cur; DeepLabv3+ falls to 59.49%.Source: §4.2 · p. 7, Table 3 · p. 8
• 65.96% mIoU, with a +4.90 mIoU improvement over the Real baseline, is SegFormer’s best result using Real+Cur+Syn.Synthetic augmentation also raises Mask2Former from 68.84% mIoU (Real) to 72.67% (Real+Syn), while CNN-based models fail to exceed their Real baselines.Source: §4.2 · pp. 7–8, Table 3 · p. 8
• Across modern architectures, synthetic-augmented configurations achieved best performance on 10 of 11 foreground classes; for Segformer, Real+Syn improved pedestrian signals by +11.18 mIoU and elevator buttons by +8.23 mIoU.Source: §4.2 · p. 8

Does the model improvement meaningfully transfer to real-time embodied deployment?

• On two out-of-distribution robot platforms, SegFormer fine-tuned on NAVABLE-Real exceeds SAN zero-shot, while Real+Syn adds consistent mIoU and precision gains, especially on long-tail classes.Reported per-class gains reach +9.64 mIoU on crosswalk for the cane-shaped robot and +8.50 mIoU on APS button for the quadruped.Source: Table 4 · p. 8, Does the model improvement… · p. 8

• NAVABLE combines curated open-source data, new real-world annotations, and controllable synthetic generation for accessibility-critical perception in BLV navigation.The authors describe the approach as addressing data scarcity and annotation inconsistency.Source: §5 · p. 9
• Fine-tuning existing models with NAVABLE improves segmentation reliability, and synthetic augmentation consistently improves detection performance across assistive robotic platforms.The authors characterize accessibility-critical perception as a distinct long-tail vision problem requiring dedicated datasets, evaluation protocols, and embodiment-aware data generation.Source: §5 · p. 9
• The current taxonomy excludes important BLV-relevant objects and environmental cues, and the real-world data have limited geographic diversity.The authors say broader data collection is motivated by the geographic limitation.Source: §5 · p. 9
• Parts of the synthetic pipeline still require human intervention for object placement and trajectory recording.Source: §5 · p. 9
• Real-world deployment of the model with assistive mobile-robot navigation systems remains an open challenge.Source: §5 · p. 9
• The authors see opportunities in foundation models, language-guided scene generation, and text-to-3D synthesis to further automate and scale accessibility-oriented simulation pipelines.They hope the work motivates research at the intersection of embodied AI and accessibility.Source: §5 · p. 9

A Detailed training settings

• Within each architecture, training configurations share augmentation, optimizer, and schedule settings; only the training corpus and iteration budget differ.The final checkpoint is selected by best foreground mIoU on the held-out NAVABLE-Real validation split.Source: A Detailed training settings · p. 13
• Iteration budgets scale with corpus size to keep approximate epoch coverage comparable, while base architecture settings remain fixed across configurations.Table 5 reports architecture settings and iteration budgets; the appendix also details Mask2Former’s full-precision training and effective-batch adjustments.Source: A Detailed training settings · p. 13, Table 5 · p. 14
• The embodiment-viewpoint YOLOv26-seg variants are fine-tuned for 70 epochs using the corresponding official recipe, without additional augmentation changes.Training used a university GPU cluster with NVIDIA L40S nodes and a local workstation with RTX PRO 6000 Blackwell Max-Q GPUs.Source: A Detailed training settings · p. 13

B List of generated assets

• The synthetic asset inventory includes APS buttons, door buttons, pedestrian signals, elevator buttons, elevators, bus stops and signs, revolving doors, escalators, handrails, and crosswalks.All assets except crosswalks and escalators were generated through the synthetic data pipeline.Source: Figure 5 · p. 14

C Details of expanded NAVABLEsyn

• Additional lighting conditions are stored as toggleable USD layers, enabling paired observations of the same trajectory under different illumination without additional data collection.Outdoor variants include day, sunset, and night; indoor variants include ambient and directional lighting.Source: C Details of expanded NAVABLEsyn · pp. 14–15

• Per-class IoU (%) is reported for Mask2Former, SAN, SegFormer, and DeepLabv3+ across the four training-data configurations.The held-out NAVABLE-Real test split contains 1,482 images; the tables mark the best configuration per class within each architecture.Source: D Per-class evaluation results · p. 15, Table 6 · p. 15, Table 7 · p. 15
• Turnstile has no real-world test instances: predicting a turnstile mask receives IoU = 0, while never predicting one receives no score.Source: D Per-class evaluation results · p. 15
• The Stage-2D mixed-reference result uses REAL × 40 + CUR × 1 + SYNv2 (full) and a different training budget, so it is included for completeness only.Source: D Per-class evaluation results · p. 15

E Per-class embodiment evaluation results

• The rolling guide-cane split contains 157 images and five labelled classes, while the Unitree Go2 split contains 155 images and nine labelled classes.Classes with no ground-truth instances are excluded from each split’s mean.Source: E Per-class embodiment evaluation… · p. 15
• On Go2, elevator, elevator_button, and door_button each have only one ground-truth annotation, making their per-class IoU dependent on a single frame.The appendix reports these values for completeness only; Tables 8 and 9 give per-class IoU for SegFormer and DeepLabv3+.Source: E Per-class embodiment evaluation… · p. 15, Table 8 · p. 16, Table 9 · p. 16

1. Claims

• The claims checklist answers yes, stating that the abstract and introduction present two technical contributions and four empirical findings substantiated elsewhere in the paper.Source: §1 · p. 16
• The checklist says claims should match the results and reflect expected generalization, while distinguishing aspirational goals from outcomes attained by the paper.Source: §1 · p. 16

2. Limitations

• The limitations response indicates that the paper discusses its limitations.Source: §2 · p. 16
• Tables 10–12 report per-class IoU, with Glide covering 157 images and five labelled classes and Go2 covering 155 images and nine labelled classes.Table 10 concerns EncNet (ResNet-101); Table 12 marks classes with only one ground-truth instance.Source: Table 11 · p. 17, Table 10 · p. 17, Table 12 · p. 18
• The authors identify geographic scope and long-tail segmentation quality as limitations discussed elsewhere in the paper.The limitations response points to real-world data geography and Section 3.1, respectively.Source: §2 · p. 17

3. Theory assumptions and proofs

• The paper states that it presents no theoretical results because it is an empirical dataset and pipeline paper.Source: §3 · p. 18

4. Experimental result reproducibility

• The released materials include the full dataset, synthetic data generation pipeline, and Isaac Sim extension.Source: §4 · p. 18
• Training and evaluation details are documented in Section 4.1.1 and Appendix A.Source: §4 · p. 18

5. Open access to data and code

• The project repository provides a README with instructions for dataset use, model fine-tuning, and running the synthetic generation pipeline.Source: §5 · p. 19

6. Experimental setting/details

• Architectures, training-data configurations, augmentations, optimizers, learning-rate schedules, and per-configuration iteration budgets are specified.Source: §6 · p. 19
• Checkpoints are selected by validation mIoU on a held-out real split.Source: §6 · p. 19
• The authors state that training and test details are provided in Section 4.1.1 and Appendix A.Source: §6 · p. 19

7. Experiment statistical significance

• Error bars across multiple seeds are not reported because fine-tuning runs require expensive multi-GPU training over 16K–324K iterations.Source: §7 · p. 20
• The authors report that all configurations use identical validation/test splits and checkpoint-selection protocols for fair comparison.Source: §7 · p. 20

8. Experiments compute resources

• Appendix A specifies NVIDIA L40S 48 GB cluster nodes and RTX PRO 6000 Blackwell Max-Q workstations as training hardware.Source: §8 · p. 20

9. Code of ethics

• The authors state that data were collected in publicly accessible environments, involved no human subjects, and captured no personally identifiable information.Source: §9 · p. 21

10. Broader impacts

• The paper frames assistive perception as supporting safer and more independent navigation for BLV individuals.Source: §10 · p. 21

11. Safeguards

• The released artifacts are a segmentation dataset, a 3D asset library, and a data generation pipeline, not pretrained generative models or other high-misuse-risk artifacts.Source: §11 · p. 21

13. New assets

• The authors report that documentation, dataset cards, and usage instructions accompany the NAVABLE dataset, 3D asset library, and Isaac Sim extension.Source: §13 · p. 22

14. Crowdsourcing and research with human subjects

• The paper states that annotations were performed by the authors and that it involved no crowdsourcing or human-subject research.Source: §14 · p. 23