PaperlayerView exact revision on arXiv

Source-linked AI summary

BIMScript: Material-Aware Structured Scene Programs for BIM Ingestion

Prakash Kondibhau Naikade, Thomas B. Moeslund, Andreas Møgelmose

arXiv:2608.21447v1cs.CV

TL;DR

Existing structured scene models leave material and condition semantics, decoding speed, and coordinate precision unresolved for automated BIM ingestion. BIMScript addresses these within one grammar using material-passport supervision, overhead-removing and schema-aware decoding, and geometric refinement; it validates direct Revit/IFC ingestion while motivating sustainability-aware AEC reasoning.

  • Problem

    Automated as-built BIM ingestion requires structured scene programs to represent what elements are made of, produce them quickly, and locate them precisely.

  • Method

    BIMScript extends a scene-program grammar with material and condition attributes, material-passport supervision, CUDA-graph and schema-aware decoding, and discrete–continuous coordinate refinement.

  • Results

    BIMScript preserves layout performance while adding material and condition semantics, achieves 3.4× output-exact decoding acceleration, and validates one-to-one ingestion into native Revit objects and IFC entities.

  • Takeaways & Limitations

    The resulting language-form BIM representation supports direct authoring-tool ingestion and provides a machine-readable basis for sustainability-aware AEC reasoning.

  • Takeaways & Limitations

    The study relies on synthetic supervision with VLM-distilled pseudo-labels, leaving real-scan generalization and label denoising open.

Abstract

from arXiv · show

Structured-language models such as SceneScript reconstruct a scene as a short program of parametric commands, an inherently editable and semantically explicit representation. We ask three questions that stand between such models and their most compelling application, automated ingestion of existing buildings into BIM tools, studied here on synthetic scans: \emph{what} is the scene made of, \emph{how fast} can it be produced, and \emph{exactly where} is each element. BIMScript answers all three within one grammar. First, we extend the layout language with per-element \emph{material} and \emph{condition} attributes, supervised by a vision-language-model material-passport corpus we build over 100k synthetic scenes (1.9M pseudo-labeled elements), and route image appearance to the material tokens through a lifted-feature point encoder. Second, we show that autoregressive decoding of these programs is dominated not by compute but by kernel-launch and host-synchronization overhead, and remove it with an output-exact CUDA-graph decoder (1.9 vs 6.4\,ms/step, $3.4\times$) plus a grammar-parallel, tolerance-verified draft-and-verify scheme that exploits the deterministic entity schema. Third, we address the model's 5cm token-grid granularity with training-free geometric snapping and a hybrid discrete--continuous decoder head that regresses a sub-bin offset, and measure how much of the residual error each recovers. Because each command maps one-to-one onto a native Revit object, we validate direct ingestion into a BIM authoring tool end to end with a working add-in and its IFC4 export, and the same program's language form is designed to support LLM-driven, sustainability-aware reasoning over the built asset.

1 Introduction

BIMScript extends structured scene programs toward automated as-built BIM ingestion by adding material and condition semantics, accelerating decoding, and improving coordinate precision. Its one-to-one mapping from commands to native BIM objects supports direct Revit/IFC ingestion and later sustainability-oriented reasoning.

  • BIM ingestion: Each command maps one-to-one onto a native Revit object, and a working adapter creates elements carrying material and condition parameters.The implementation supports direct BIM authoring-tool ingestion and IFC exchange.
  • What: BIMScript adds per-element material and condition attributes to a backward-compatible scene grammar for BIM ingestion.The language targets properties beyond geometry while preserving the existing structured-program formulation.
  • What: 1,900,908 pseudo-labeled elements over ≈100k synthetic scenes supervise the material-aware language through a lifted-feature point encoder.The encoder routes image appearance to material tokens.
  • Fast: 3.4× decoding acceleration comes from an output-exact CUDA-graph decoder that removes kernel-launch and host-synchronization overhead.A grammar-parallel, tolerance-verified draft-and-verify scheme further exploits the deterministic entity schema.
  • Exactly where: Training-free geometric snapping and a hybrid head with bounded within-bin offsets address the 5cm token-grid limitation and quantify residual coordinate error.The hybrid head regresses sub-bin offsets while retaining coarse tokens for structure.

2 Related Work

Prior work largely reconstructs geometry and topology or recognizes materials in 2D, while scan-to-BIM remains multi-stage and material and condition information is often added manually. BIMScript combines element-level semantics with program-based reconstruction and schema-aware acceleration.

  • Scan-to-BIM: Scan-to-BIM methods typically use multi-stage geometric-fitting pipelines, with material and condition added manually afterward.Related layout and CAD-alignment methods primarily target geometry and topology.
  • Structured reconstruction: BIMScript predicts geometry, material, and condition jointly in one autoregressive pass and emits a program shaped for BIM objects.This contrasts with prior methods focused on geometry and topology.
  • Efficient decoding: Fast SceneScript reduces iteration count with multi-token prediction and self-speculative decoding, but incurs a small quality cost.BIMScript instead uses schema-determined slot types for its draft-and-verify acceleration.
  • Efficient decoding: Because the decoder is tiny at ∼2 MFLOPs per step, BIMScript targets kernel-launch and synchronization overhead rather than compute-heavy inference.This reframes which inference optimizations matter for structured scene programs.
  • Material semantics: Material recognition is usually 2D image labeling, whereas BIMScript connects appearance evidence to 3D building elements at scale using VLM-distilled labels.Its material-passport pipeline avoids per-scene manual annotation in the described synthetic setting.
  • AEC applications: Element-level material inventories support material passports, circularity indicators, and BIM-based life-cycle assessment, which BIMScript produces automatically.The resulting program is a machine-readable inventory combining geometry and materials.

3 Background and a Reproducible Baseline

SceneScript represents layouts as constrained programs whose numeric geometry is discretized on a 5cm grid, but reproducing its published baseline is complicated by artifact, protocol, and tokenizer inconsistencies.

  • SceneScript representation: SceneScript predicts continuous geometry as categorical tokens over B bins, constrained by a finite-state machine.The public-checkpoint configuration uses a [0,32] m range and B=640, yielding a 5cm grid.
  • Compatibility issues: A tokenizer mismatch can bias every coordinate by up to one bin and push wall-corner error beyond the F1@5cm threshold regardless of model quality.The adopted mitigation rounds during discretization, reconstructs at bin edges, and uses a margin-anchored frame with bounded clamping.
  • Reproducibility: The published 0.848 mean F1@5cm cannot be audited because the dataset splits and compatible architecture are not public.The released checkpoint has 646 value-embedding rows and 2,000 positions, differing from the paper-described 2,048-token vocabulary.
  • Reproducibility: 0.581 mean F1@5cm is obtained by the released weights on the reproduction test split, versus 0.848 on the paper’s non-public split.The difference is not attributed because the splits differ.
  • Token-grid effects: The round-trip tokenizer oracle reaches 0.994 F1@5cm at 640 bins, while frame-boundary censoring removes 0.6% of entities during evaluation.These measurements indicate representation and anchoring effects are separately measurable rather than fully resolved.

4 BIMScript: What Is in the Scene

BIMScript extends SceneScript’s flat command grammar with inline material and condition attributes, while adding appearance features and preserving direct object-level BIM compatibility.

  • Material-aware grammar: BIMScript adds inline material and condition attributes to each building-element command without changing the decoder.The attributes reuse the existing categorical value-token space and avoid vocabulary growth.
  • Material-aware grammar: Material tokens attend to their element’s own geometry tokens, avoiding brittle cross-references from separate attribute commands.Separate set_material commands would roughly double sequence length because element ids are positional and regenerated during decoding.
  • End-to-end representation: Every emitted command maps one-to-one onto a native Revit/IFC object, while GraphStep and SchemaDraft accelerate decoding and SubBin refines coordinates below the token grid.The overview presents material, condition, speed, precision, and BIM-object mapping as components of one program representation.
  • Material-passport corpus: 99,990 scenes and 1,900,908 elements form a 14-material, 5-condition pseudo-labeled corpus with zero join failures.The distribution is heavily imbalanced, so the paper reports macro-F1 alongside accuracy.
  • Appearance encoding: A lifted-feature encoder projects frozen 2D image features onto 3D points, aggregates observations, and appends them to XYZ before sparse encoding.The decoder, tokenizer, and grammar remain unchanged; depth-based occlusion filtering removes wrong-surface correspondences.
  • Validation example: On an 18-element validation scene, coverage is 1.0, F1@5cm is 0.944, 16 of 18 materials are correct, and condition accuracy is 0.944.The visible material error is a tile wall predicted as wallpaper.

5 FastBIMScript: Efficient Decoding

FastBIMScript accelerates structured decoding by targeting overhead rather than compute, combining output-exact CUDA graphs with schema-aware draft-and-verify decoding.

  • Where the time actually goes: Per-step decode time is flat with sequence length and only about 2 MFLOPs, indicating kernel-launch and host-synchronization overhead rather than compute dominates.KV-caching alone delivers only 1.2× because it removes FLOPs from a loop that was not FLOP-bound.
  • GraphStep: 6.40 → 1.91 ms per step yields 3.4× acceleration with GraphStep while emitting identical tokens and requiring no retraining.The graph compiles the decode step and synchronizes with the host only every 16 steps to scan for termination.
  • SchemaDraft: SchemaDraft drafts all remaining parameter slots in one forward pass because the command token deterministically fixes their types, then verifies and accepts the longest matching prefix.Structural slots must match exactly, while numeric slots may differ by at most τ=2 bins; strict τ=0 restores exactness.
  • System boundary: Further decode optimization is bounded by the sparse encoder, which already takes 0.7 s while decoding takes 0.85–0.97 s per scene.The encoder is therefore co-dominant at the reported operating point.

6 Exactly Where: Sub-bin Geometric Refinement

BIMScript separates coarse structural quantization from within-bin coordinate refinement, measuring the 5cm grid’s contribution and testing geometric snapping alongside a learned offset head.

  • Quantization baseline: With 640 bins over [0, 32] m, the coordinate grid spacing is 5cm and nearest-bin quantization has a 2.5 cm worst-case error.Tight F1 thresholds from 0.1–5 cm are added as evaluation overrides without changing legacy default thresholds.
  • Evaluation protocol: Appearance-source ablation compares six arms on the same 2,250-scene subset, 20k-step budget, and 250 validation scenes, with condition macro-F1 measured over good, new, and worn.Because the study is single-seed, the reported ordering is evidence rather than a settled ranking.
  • SubBin: SubBin preserves the coarse token for structure and regresses a bounded within-bin offset for each numeric slot using the decoder’s hidden state.The offset is bounded to (−0.5, 0.5), trained with Huber loss on pre-rounding residuals, and added through an optional head compatible with legacy checkpoints.

7 Experiments

Experiments evaluate attributes, speed, and precision under fixed scene and hardware protocols, showing usable material prediction, weaker condition recognition, and residual precision errors beyond grid quantization.

  • Appearance ablation: DINOv2 leads condition F1 at 0.366, DINOv3 leads material accuracy at 0.370, and raw per-point RGB reaches 0.324 versus 0.322 for point-only features.The ablation indicates that learned, view-stable features rather than color itself provide the useful appearance signal.
  • Material and condition: Material prediction reaches 0.626 validation accuracy and 0.629 test accuracy on matched elements, with the full-corpus model matching 70% of ground-truth elements.Attributes are scored only over elements matched within 10 cm.
  • Material and condition: Condition macro-F1 is 0.353 on validation and 0.352 on test, with performance carried mainly by the good class at 0.914.New reaches only 0.144, while worn is never recovered despite 485 matched test instances.
  • Precision: Mean F1@2 cm doubles from 0.034 to 0.071 with the offset head, while door and window MAE fall by 10.3% and 3.2%.Wall MAE remains near 8.7 cm, and F1@0.5 cm is zero for every variant; these measurements condition on already-correct detection.
  • Speed: The two speed mechanisms are reported separately because their composition requires graph families keyed by varying draft lengths rather than a single captured graph.The authors do not assume that the individual speed factors multiply.

8 BIM Ingestion and LLM-Driven AEC Reasoning

BIMScript programs are ingested into native Revit objects with semantic parameters, while their language representation provides a compact interface for sustainability-aware AEC reasoning.

  • BIM ingestion: Each BIMScript command maps one-to-one to a native authoring-tool object, with material and condition attached to that object’s own parameters rather than a side channel.This preserves the program’s object-model structure during ingestion.
  • BIM ingestion: The pyRevit adapter imports decoded programs as native Revit walls, attaching predicted material, condition, and source id as shared parameters.The tested workflow includes an autocreated type named BIMScript - concrete.
  • LLM-driven AEC reasoning: The material- and condition-annotated program serves as a compact, machine-readable inventory for LLM reasoning over embodied carbon, life-cycle estimation, maintenance, adaptive reuse, and disassembly.The condition attribute supports maintenance triage, which prior structured-scene models lacked.

9 Limitations and Future Work

The paper identifies synthetic supervision, limited domain coverage, and incomplete validation as boundaries for BIMScript’s current evidence and deployment scope.

  • Limitations: Real-scan generalization and denoising of VLM-distilled material labels remain open because supervision is synthetic and pseudo-labeled.The material labels come from a VLM-distilled synthetic corpus.
  • Limitations: BIMScript’s rare-class tails make macro-F1 sensitive to which classes appear in a matched population, including damaged conditions at 0.05%.Glass represents 0.04% of materials, while damaged represents 0.05% of conditions.
  • Limitations: The Revit and IFC path is validated on programs decoded from synthetic scenes rather than professional as-built scans.The supplied limitation passage identifies this as an unresolved validation boundary.
  • Future Work: Future expansion targets facades, structural elements, and MEP runs, requiring real scans or sufficiently realistic synthetic data.The paper also leaves LLM-driven reasoning as future work.

10 Conclusion

BIMScript represents building elements with geometry, material, and condition in an extensible structured language, while preserving direct BIM mappings. The conclusion reports matching layout-only performance, overhead-bound decoding, and detection as the remaining metric bottleneck.

  • Conclusion: BIMScript assigns each element geometry, material, and condition, supervised by a VLM-distilled passport covering 1.9M elements.Attributes use the existing token space and condition material tokens on each element’s geometry.
  • Conclusion: The material-aware model matches the layout-only baseline on the same splits, while new attributes can append without renumbering or decoder changes.Each command also maps one-to-one onto a native Revit object and IFC entity.
  • Conclusion: Decoding is bound by overhead rather than compute, and detection now limits metric precision more than the token grid.The paper connects these findings to BIM ingestion and language-based sustainability-aware reasoning.

Supplementary Material

The supplementary material details material-passport extraction, CUDA-graph and draft decoding, discretization conventions, translation-frame effects, and grid-resolution diagnostics.

  • Extraction: Material-passport extraction projects observing keyframes onto elements, filters occlusions, and writes VLM labels back as material- and condition-annotated plans.Independent dense-segmentation and classical-CV cues are retained as checks rather than merged into labels.
  • Decoding: CUDA-graph decoding uses context-length buckets and device-side FSM tables to avoid host synchronization when advancing parameter types.Intra-element redrafting addresses ambiguity in coupled coordinates.
  • Discretization and Reconstruction: Quantization conventions bound per-axis errors to [−2.5, +2.5) cm, with height bounded to [−2, +2) cm under the described pairing.The reconstruction analysis gives a worst-case per-corner error of 4.06 cm.
  • Discretization and Reconstruction: 0.570 F1@5cm is obtained with edge reconstruction, whereas center reconstruction scores 0.154 on 100 validation scenes.This identifies a convention consistent with the released weights rather than directly reproducing the training recipe.
  • Translation Frame: The translation-frame issue caused 46,218 removal events across 14,395 scene-visits in one run, with at least 14.5% of scenes affected and 92% of losses being walls.The paper adopts a margin-anchored frame with bounded clamping.
  • Grid Resolution: The finer grid opens headroom at strict thresholds: the AvgF1 ceiling rises from 0.845 at 640 bins to 0.970 at 2,048 bins.The matched evaluation reports a strict-F1 advantage for the finer grid and a coarse-average disadvantage.

D BIMScript-to-BIM Mapping and Additional Results

The additional results show qualitative geometric agreement, correct opening placement, and material predictions that vary by scene. The figures also expose missed wall segments and residual corner offsets behind strict-F1 measurements.

  • Qualitative Layout Results: Qualitative layouts compare ground truth, BIMScript predictions, and zoomed corner insets, showing centimeter-level agreement alongside missed wall segments and corner offsets.Doors use swing arcs and windows use blue sill glyphs.
  • 3D Results: 3D renderings show geometry tracking ground truth closely and openings placed on their host walls.The same renderings visualize predicted materials over the input point cloud.
  • Material Results: Material behavior differs across scenes: one recovers all five ground-truth materials, while another collapses tile and wood paneling into frequent wallpaper.The latter scene has material accuracy 0.63 and macro-F1 0.29.
Loading 2608.21447v1…