PaperlayerView exact revision on arXiv

Source-linked AI summary

CORNAV: Construction-Aware Reasoning for Robot Navigation on Active Worksites

Parastoo Ali Pour, Deepak Prakash Kumar, Tommy Zhou, Pramod Khargonekar, Mohammad Abdullah Al Faruque

arXiv:2610.03622v1cs.RO

TL;DR

Active construction sites change over time, while navigation systems lack a unified use of semantic, architectural, schedule, and safety information. CORNAV combines blueprints, scene graphs, schedules, and LLM-based safety validation; blueprint grounding improves task success, and schedule awareness improves compliance with active hard zones.

  • Problem

    Active construction changes traversable regions and safety zones over time, while existing approaches do not unify scene understanding, architectural information, schedules, and operational safety constraints.

    Source: §I · p. 1

  • Method

    CORNAV combines architectural CAD, open-vocabulary 3D scene graphs, project schedules, and LLM-based hazard validation in a navigation framework.

    Source: §I · p. 2

  • Results

    72.2% TSR with the complete CORNAV pipeline versus 13.0% TSR without blueprint-based grounding; the complete pipeline achieves 100% CCR.

    Source: F. Ablation Study · p. 7

  • Takeaways & Limitations

    In indoor office and active construction settings, blueprint grounding improves object localization, schedule-aware planning prevents navigation through active work zones, and safety validation guards against incorrectly labeled hazards.

    Source: §V · p. 8

Abstract

from arXiv · show

The construction industry faces persistent labor shortages, low productivity that costs the global economy over $1.6 trillion annually, and one of the highest injury rates among major industries. These factors motivate the use of autonomous robots to improve efficiency and worker safety. Existing language-grounded navigation systems, however, rely on semantic scene understanding alone and lack access to construction-specific context such as architectural plans, evolving work schedules, and safety constraints. As a result, they localize permanent building features unreliably and cannot safely navigate active jobsites. We present CORNAV, a blueprint-grounded, schedule-aware navigation framework that operates from 2D CAD drawings and project schedules without requiring a Building Information Model. CORNAV aligns architectural blueprints against hierarchical open-vocabulary 3D scene graphs to ground object queries, converts project schedules into time-varying navigation constraints, and validates requests through an LLM-based safety module that escalates hazardous zones before planning. An A* planner then enforces mandatory exclusion zones while preferentially avoiding higher-risk areas. Across an indoor office and a real construction site, blueprint grounding raises task success from 13.0% to 72.2% over semantic retrieval alone, schedule awareness eliminates all hard-zone violations, and the safety module correctly rejects hazardous requests arising from mislabeled project schedules.

I. INTRODUCTION

Construction combines labor shortages, serious safety risks, and declining productivity, motivating robotics for efficiency and worker safety. Yet active sites change over time, while existing approaches do not unify semantic scene understanding, architectural information, schedules, and operational safety constraints.

I. INTRODUCTION

  • Construction faces persistent workforce shortages, high injury rates, and long-term productivity decline, motivating robotics to improve worker safety and operational efficiency.The cited introduction reports more than 1,000 worker fatalities annually since 2016 and a global economic cost of roughly $1.6 trillion per year from the productivity gap.Source: §I · p. 1
  • Changing structures, equipment, and work zones make maps outdated and require robots to reason about operational context and evolving safety constraints, not just obstacles.Temporary exclusion areas and task-specific safety procedures may change throughout the day.Source: §I · p. 1
  • Scene graphs, blueprints, schedules, and safety planning have advanced largely in isolation, leaving robots without a unified account of what exists, where it is, and when hazards apply.Scene graphs can vary in room-level structure; blueprint-based approaches commonly treat workspaces as static, and safety constraints often come from robot designers rather than operational documents.Source: §I · p. 2
  • CORNAV integrates CAD drawings, open-vocabulary 3D scene graphs, project schedules, and safety rules, combining semantic retrieval with deterministic blueprint-based room resolution.It also uses an LLM to interpret hazard descriptions and safety rules, and is evaluated on robot-collected data in an indoor environment and an active construction site.Source: §I · p. 2

II. RELATED WORK

Architectural drawings provide spatial priors, while open-vocabulary scene graphs support language-based object and region retrieval. CORNAV combines these sources because scene graphs alone may not reliably establish room identity.

A. Architectural Drawings for Robot Localization

  • Architectural plans have primarily served as localization and mapping priors, without accounting for temporal and safety constraints from ongoing construction.CORNAV instead uses CAD as the authoritative representation for room identities, spatial hierarchy, and safety-constraint grounding.Source: A. Architectural Drawings for Robot… · p. 2

B. 3D Scene Graphs for Semantic Environment

  • Open-vocabulary scene graphs enable language-based retrieval, but room mislabeling can prevent reliable association of scheduled hazards with the correct region.CORNAV combines HOV-SG with blueprint-derived spatial information and construction-specific operational knowledge.Source: B. 3D Scene Graphs for Semantic… · p. 2

C. BIMs for Robot Navigation

BIM-based navigation work incorporates schedules and safety information, but often depends on BIM/IFC models and does not provide open-vocabulary grounding for objects absent from those models. CORNAV uses site CAD and schedules in a unified offline-and-online navigation pipeline.

C. BIMs for Robot Navigation

  • Existing BIM- and IFC-based systems incorporate schedules or model semantics into navigation, but require those models and commonly treat declared hazards as static zones.They also use structured model lookups rather than open-vocabulary grounding for objects observed on site but absent from the model.Source: C. BIMs for Robot Navigation · pp. 2–3
  • CORNAV draws on site CAD and schedules, distinguishes mandatory hard constraints from cautionary soft constraints, and uses LLM validation to escalate hazardous zones.Its open-vocabulary scene graph grounds queries to both blueprint-defined fixtures and movable objects observed on site.Source: C. BIMs for Robot Navigation · p. 3

III. SYSTEM OVERVIEW

  • Offline processing builds a semantic world model and construction-knowledge representation; online execution combines them with a task request to produce schedule-aware, safety-compliant plans.The per-query pipeline takes a natural-language task and requested date and time, returning robot-frame waypoints or an infeasibility verdict with a reason.Source: §III · p. 3, Figure 2 · p. 3

A. Offline Pre-Processing

Offline preparation aligns CAD-derived room structure with the robot occupancy map, converts project schedules into time-windowed hazard data, and builds an open-vocabulary scene graph. At runtime, the system uses the blueprint and scene graph together to resolve queries and rooms.

A. Offline Pre-Processing

  • CAD floor plans are parsed into room polygons and a hierarchical blueprint scene graph, then aligned to the robot occupancy map through coarse rotation search and ICP refinement.The alignment connects blueprint room structure to the robot map; Fig. 4 shows an aligned overlay for construction floor 1.Source: A. Offline Pre-Processing · p. 3, Figure 3 · p. 3, Figure 4 · p. 4
  • Project spreadsheets become structured schedules with activities, room assignments, active time windows, and hazard severity, while survey observations build an HOV-SG scene graph with labels and CLIP embeddings.The schedule supplies temporal information for activating and deactivating navigation constraints.Source: A. Offline Pre-Processing · p. 3

B. Object Grounding and Room Resolution

  • Runtime grounding retrieves observed transient objects by CLIP similarity, resolves permanent CAD fixtures from a blueprint registry, and assigns the object’s room by checking its coordinate against the aligned room boundary.This supports reasoning about both observed objects and planned features not yet present or visible during mapping.Source: B. Object Grounding and Room Resolution · p. 4

C. Temporal Constraint

Schedule entries become time-specific geometric constraints, while a rule-guided language model can escalate hazardous soft zones before planning.

C. Temporal Constraint

  • At query time, scheduled activities map room assignments and hazard severity to active blueprint-derived hard exclusion zones or preferentially avoided soft zones.Hard zones prohibit entry; soft zones remain traversable unless safety validation promotes them.Source: C. Temporal Constraint · p. 4

D. Safety Validation

  • Active soft zones are scored against embedded construction-safety rules, and scores of 0.7 or higher promote them to hard constraints before planning.The rule-guided GPT-4o module uses activity hazard notes and runs at zero temperature for reproducible escalation decisions.Source: D. Safety Validation · p. 4
  • The embedded rules address unapproved temporary structures, unprotected edges, active work hazards, and unstable or unfinished surfaces.Examples include scaffolding, open slabs, overhead work, fresh concrete, and loose gravel.Source: D. Safety Validation · p. 4

E. Constraint Aware Planning

The planner combines obstacle-clearance costs with schedule-dependent zone penalties, enforcing hard exclusions while encouraging avoidance of soft zones. Evaluation uses recorded sensor data from an office and an active construction site, with six navigation metrics.

E. Constraint Aware Planning

  • Hard zones become occupied grid cells that paths cannot intersect, while soft zones remain traversable with a fivefold cost multiplier in A* planning.Soft-zone traversal can remain feasible when detours are excessive or no alternative exists; the cost field also penalizes low-clearance cells.Source: E. Constraint Aware Planning · p. 4, Figure 5 · p. 5

A. Sites and Platforms

  • Quantitative evaluation uses recorded RGB, depth, and pose data from office and construction-site surveys, with live demonstrations on a quadruped and humanoid.The planning pipeline was evaluated offline across scenarios spanning days and times as scheduled zones became active.Source: A. Sites and Platforms · p. 5

B. Evaluation Metrics

  • Six metrics measure hard-zone compliance, soft-zone avoidance, feasible-task success, relative path length, hazardous-zone entry, and feasible planning.CCR measures trajectories outside active hard zones; SACR measures feasible trajectories avoiding active soft zones; TSR measures feasible-task completion; PER compares path lengths against unconstrained plans; ZER tracks entry into hazardous zones mislabeled soft; FPR measures requests receiving feasible paths.Source: B. Evaluation Metrics · p. 5

C. Blueprint-Based Room Resolution

The section compares blueprint-based room resolution with scene-graph labeling and reports registration quality using blueprint-to-map wall alignment. Blueprint room identities support schedule and safety constraints defined per room.

C. Blueprint-Based Room Resolution

  • Blueprint-to-map alignment is evaluated by RMSE between transformed blueprint wall points and corresponding occupancy-map walls; RMSE variation reflects map quality and available CAD geometry.Larger mapped areas may accumulate localization or odometry error, while observable wall geometry and map completeness can also affect ICP alignment.Source: Table I · p. 5, C. Blueprint-Based Room Resolution · p. 5
  • Blueprint lookup assigns a room label by testing which CAD room polygon contains the target, whereas HOV-SG uses geometric segments and CLIP text-image similarity.Because schedules and safety constraints are room-specific, unstable segment boundaries and labels can propagate into incorrect constraint activation.Source: C. Blueprint-Based Room Resolution · pp. 5–6
  • 3.3% of HOV-SG room-segment labels matched blueprint labels, with 0% accuracy on construction floors and 16.7% in the office.HOV-SG produced 30 segments while the blueprint defines 54 rooms; manual inspection found segment boundaries often diverged from architectural rooms.Source: C. Blueprint-Based Room Resolution · p. 5, Table II · p. 6

D. Schedule-Aware Navigation

Across schedule timestamps, the planner avoided all active hard zones and reported infeasibility when a start or goal lay inside one. Soft-zone avoidance depended on available alternate routes, while path efficiency remained near 1.0 on average.

D. Schedule-Aware Navigation

  • The evaluation comprised 89 trials across distinct schedule timestamps from four weekly look-ahead project schedules.The same navigation request was issued at different times as work zones became active, and trajectories were compared using SACR and PER.Source: D. Schedule-Aware Navigation · p. 6
  • Zero hard-zone violations occurred across 74 feasible trials; the other 15 were correctly reported infeasible because an endpoint lay inside an active hard zone.There were six blocked goals and nine blocked starts, with no failures from route severance between otherwise accessible endpoints.Source: D. Schedule-Aware Navigation · p. 6
  • SACR varied from 77.3% on construction floor 0 to 23.8% in the office, partly because hard-blocked requests and unavoidable soft-zone crossings do not count as compliant.In the office, the active soft region occupied the only corridor in 12 trials; only five no-active-zone trials contributed to its SACR.Source: D. Schedule-Aware Navigation · p. 6
  • Construction floor 1 routed around active soft regions in 78.9% of soft-active trials, while other sites' crossings reflected unavoidable geometry; mean path-efficiency ratio stayed near 1.0.On floor 1, longer routes often avoided soft zones; on floor 0, path-length variation came from clearance detours around narrow passages.Source: D. Schedule-Aware Navigation · p. 6

E. Safety Rule Validation

The safety validator escalated all four hazardous cases mislabeled as soft and retained all four non-hazardous cases as soft, reducing hazardous-case ZER from 100% to 0.0%.

E. Safety Rule Validation

  • All four hazardous cases were escalated to hard constraints and all four non-hazardous cases remained soft, matching every expected outcome.Validation reduced hazardous-case ZER from 100% to 0.0% without changing non-hazardous cases.Source: E. Safety Rule Validation · p. 6
  • Across 64 repeated calls, no escalation decision crossed the threshold in the wrong direction, and all escalation decisions were identical across repetitions.Seven cases had zero score variance; SR2 hazardous had a standard deviation of 0.024.Source: E. Safety Rule Validation · p. 6

F. Ablation Study

The ablations show that blueprint grounding most affects task success, while schedule awareness and safety validation protect compliance; the full pipeline achieves 100% CCR and 72.2% TSR.

F. Ablation Study

  • The ablations remove schedules, blueprint-based room resolution, or LLM safety validation; all methods were evaluated on the same 54 requests.HOV-SG was used as the semantic-navigation baseline because no directly comparable blueprint- and schedule-aware method without BIM was available.Source: F. Ablation Study · pp. 6–7
  • 100% CCR and 72.2% TSR were achieved by the complete CORNAV pipeline, with mean PER 0.988 and feasible paths for 46.3% of requests.Every returned path satisfied mandatory safety requirements; PER uses clearance-weighted cost, so values marginally below 1.0 do not indicate error.Source: F. Ablation Study · p. 7, Table V · p. 7
  • CORNAV and the variant without safety validation tie at 29.6% SACR, but CORNAV declines hazardous tasks while the ablation attempts and fails them.Their CCR differs: 100% for CORNAV versus 83.3% without validation.Source: F. Ablation Study · p. 7
  • Removing blueprint grounding drops TSR from 72.2% to 13.0%; permanent-fixture retrieval falls to 0.0% TSR and CLIP-based retrieval to 17.1%.Table VI reports task success by object source; deterministic blueprint room grounding supports reliable localization of permanent building features.Source: F. Ablation Study · p. 7, Table VI · p. 7
  • Removing schedule awareness reduces CCR from 100.0% to 53.7%, while feasible-path rate remains at the blueprint ablation’s 40.7% FPR.Without temporal information, the planner cannot identify active hard zones and routes through them, showing that finding paths and complying with constraints are distinct.Source: F. Ablation Study · p. 7
  • Without safety validation, CCR falls to 83.3% while FPR remains 46.3%, because hazardous paths are executed rather than refused.Both CORNAV and this ablation find feasible paths for the same share of requests; the difference lies in validation, not path planning.Source: F. Ablation Study · p. 7
  • HOV-SG achieves 7.4% TSR and 42.6% CCR, producing feasible paths for 31.5% of navigation requests.It lacks blueprint grounding, schedule awareness, and construction-specific safety reasoning.Source: F. Ablation Study · p. 7
  • In 12 experiments with the Go2 quadruped and G1 humanoid, the full pipeline accounted for schedules and reached destinations, or identified infeasibility and kept the robots still.The tests covered various missions; robots did not move when schedule constraints made a path infeasible.Source: F. Ablation Study · pp. 7–8

V. CONCLUSION

CORNAV combines architectural blueprints, open-vocabulary 3D scene graphs, project schedules, and LLM-based safety validation for navigation in dynamic construction environments. Experiments showed improved object localization, avoidance of active work zones, and protection against incorrectly labeled hazards.

V. CONCLUSION

  • CORNAV unifies blueprint grounding, schedule-aware planning, and LLM-based safety validation to reason about where objects are and when areas are safe to access.The framework combines architectural blueprints, hierarchical open-vocabulary 3D scene graphs, and project schedules.Source: §V · p. 8
  • Experiments in an indoor office and active construction settings showed that blueprint grounding improves object localization, schedule-aware planning prevents navigation through active work zones, and safety validation guards against mislabeled hazards.Together, these capabilities enable navigation under construction-specific constraints.Source: §V · p. 8
  • Future work includes multi-robot coordination on a shared construction site and tighter coupling between task planning and mobile manipulation for long-horizon tasks.Source: §V · p. 8
Loading 2610.03622v1…