PaperlayerView exact revision on arXiv

Source-linked AI summary

Roomer: Reflective Object-Grounded Model Editing and Repair for 3D Indoor Layout Synthesis

Lingwei Dang, Ziyan Qiu, Jiajia Cheng, Shishuo Shang, Zhenhao Zhang, Yufei Zhu, Qingxin Xiao, Pan Liu, Shenghui Huang, Yun Hao, Juntong Li, Qingyao Wu

arXiv:2608.01973v1cs.ROcs.CV

TL;DR

Indoor layout generators can produce plausible scenes with residual local violations, while existing approaches offer limited object-grounded local repair. Roomer performs verification-gated repairs and, across external generators, preserves valid regions while improving physical validity and practical usability.

  • Problem

    Existing indoor layout generators may leave local violations, motivating object attribution and local repair without disrupting already-valid configurations.

  • Method

    Roomer attributes measured violations to implicated objects, proposes structured local StatePatches, and commits only candidates that pass full-scene verification.

  • Results

    Across external generators, Roomer repairs residual violations while preserving valid regions and improving physical validity and practical usability.

  • Takeaways & Limitations

    Roomer supports localized, verification-gated correction of indoor layouts and evaluates usability alongside distributional quality and physical validity.

Abstract

from arXiv · show

Existing indoor layout generators produce globally plausible layouts yet may retain local violations such as collisions, out-of-bounds placements, obstructed openings, and blocked circulation. Most prior work focuses on full-scene synthesis or scene-level optimization, with limited support for identifying responsible objects and locally repairing affected regions. We present Roomer, a reflective repair framework that casts these violations as sparse, object-grounded repair problems. Roomer encodes layouts as ``RoState'' and uses ``RoReview'' to bind measured violations to implicated objects. A geometry-conditioned vision-language model planner proposes a structured local edit, while a deterministic solver validates it and generates a finite set of candidate edits when needed. Each candidate is committed only if full-scene verification confirms that it resolves the target violation without new hard violations or broken protected constraints. We train the planner on Roomer-CC, a controlled-corruption dataset that pairs faulty layouts with object-grounded violation evidence and known-feasible inverse StatePatches. Since existing benchmarks rarely assess whether physically valid layouts are usable, we introduce Roomer-Eval to assess distributional quality, physical validity, and practical usability. Experiments show that Roomer repairs residual violations while preserving valid regions, improves physical validity and usability, and transfers across external generators.

Introduction

Roomer treats sparse, localized layout violations as verification-gated, object-grounded repair problems rather than full-scene regeneration. It introduces a reflective repair framework, Roomer-CC training data, and Roomer-Eval for physical validity and practical usability.

  • Motivation: Generated layouts can remain globally plausible while containing local collisions, out-of-bounds placements, obstructed openings, and blocked circulation.These violations may be visually subtle despite semantic coherence.
  • Roomer Framework: Roomer uses measured evidence to attribute residual violations to implicated objects, propose structured local StatePatches, and commit edits only after full-scene verification.Its RoState representation is object-addressable, while RoReview links violations to geometric measurements and implicated objects.
  • Training Data: Roomer-CC provides 67,550 paired repair examples from valid 3D-FRONT layouts, combining six object-level corruptions, attributed violation evidence, and known-feasible inverse StatePatch targets.The dataset supplies supervision absent from existing indoor-scene datasets.
  • Evaluation: Roomer-Eval combines distributional-quality and 3D-physical-validity metrics with five reproducible rule families for practical spatial usability.The protocol addresses the limited usability assessment in conventional layout benchmarks.
  • Evaluation: Experiments on 3D-FRONT layouts and external-generator outputs show that Roomer repairs residual violations while preserving valid content.The supplied introduction describes this result without reporting quantitative values.

Related Work

Prior indoor-layout synthesis methods use explicit priors and optimization for feasible configurations, while data-driven approaches learn complete-layout distributions. Reflective closed-loop frameworks iteratively evaluate outputs and use feedback to guide subsequent updates.

  • Indoor Layout Synthesis: Early indoor-layout methods arrange furniture with design rules, exemplar statistics, or activity-based priors, using search or optimization to obtain feasible configurations.These methods offer explicit control but depend on handcrafted priors and scene-specific optimization.
  • Reflective Closed-Loop Scene Editing: Reflective closed-loop frameworks evaluate intermediate outputs and use self-generated critiques, environmental feedback, or external tools to guide subsequent updates.Related work also explores closed-loop co-refinement for multimodal human–object interaction synthesis.

A. Goal-conditioned Layout B. Object-Grounded Residual Diagnosis

Roomer converts generated layouts into object-addressable states, diagnoses violations through implicated-object evidence, and performs localized edits validated against the full scene. Its evaluation distinguishes distributional quality, physical validity, and practical spatial usability.

  • B. Object-Grounded Residual Diagnosis: Roomer converts each generated layout into an object-addressable RoState and constructs an instance-grounded RoReview.The RoReview binds measured violations to implicated objects.
  • A. Goal-conditioned Layout: The geometry-conditioned vision-language planner predicts an action-specific StatePatch seed, while a deterministic solver evaluates it before ordered fallback candidates.Fallback traversal begins only after the seed is rejected.
  • A. Goal-conditioned Layout: Roomer commits the first candidate passing full-scene verification and otherwise rolls back, while evaluation separately measures distributional quality, physical validity, and practical spatial usability.Collision and out-of-bounds rates assess geometric feasibility, whereas FID, KID, generative precision and recall, and category statistics assess distributional or compositional similarity.

Method

Roomer performs iterative, object-grounded local repair: it attributes detected violations, predicts StatePatches, and commits only edits that pass full-scene verification. Its geometry-conditioned planner and deterministic solver support generator-agnostic repair, while Roomer-CC provides supervision from controlled object-level corruptions and feasible inverse patches.

  • Iterative repair: Roomer iteratively attributes one violation to implicated entities, predicts a local StatePatch, and commits successful edits without regenerating the entire scene.Rejected attempts leave the committed layout unchanged, and the framework requires only conversion of upstream output into its canonical representation.
  • Object-grounded representation: RoState and RoReview make violations object-addressable by recording stable entity references, roles, violation types, and type-specific geometric measurements.Examples include bedside-clearance deficits and intrusion ratios, with protected regions and obstructing furniture explicitly identified.
  • Planner and action space: The planner combines semantic rendering, room requirements, serialized state, and geometry-conditioned repair queries to predict a schema-constrained StatePatch.It selects one of six actions—MOVE, ROTATE, SCALE, INSERT, DELETE, or the remaining listed operation—while the deterministic solver realizes precise candidates rather than committing the seed directly.
  • Deterministic validation: The solver tests the planner seed first, then an action-specific neighborhood, and accepts the first candidate that passes verification; otherwise it returns ∅.Verification recomputes functional regions and rule instances and enforces target effectiveness, physical safety, protected-relation preservation, structural validity, and global progress.
  • Repair supervision: Roomer-CC pairs valid 3D-FRONT layouts with controlled object-level corruptions and known inverse StatePatches that restore the reference layout.Pairs are retained only when the intended violation is triggered, attributed to the expected object or role, removed by the inverse patch, and free of new hard violations or broken protected relations.

Roomer-Eval: Unified Evaluation Protocol

Roomer-Eval unifies distributional and physical metrics with Practical, a rule-based measure of spatial usability that detects functional failures beyond collisions and boundary violations. Practical is recomputed from complete outputs as an instance-level compliance score across five rule families.

  • Metric suite: Roomer-Eval computes FID, KID, and SCA Gap on shared semantic renderings, and OOB and COL on assembled scenes.Practical complements these metrics by capturing functional failures that can persist in collision-free, in-bounds layouts.
  • Practical scoring: Practical is the micro-average over applicable rule instances, with each applicable instance weighted equally and N/A instances excluded.Object-level rule families may contribute multiple instances per scene, while room-level families contribute at most one.
  • Rule coverage: The five Practical rule families assess living-room organization, dining-table clearance, door-swing avoidance, walkable connectivity, and bedside clearance.Definitions for applicability, association, exemption, geometry, and contact are provided in Supplementary Sec. J.
  • Evaluation protocol: Final Practical scores are recomputed from each method’s complete output without reusing repair-time detections or decisions.The measure evaluates instance-level rule compliance, not scene-level all-pass performance, building-code compliance, exhaustive ergonomics, or aesthetic quality.

Experiments

Across common-1100 and external-generator evaluations, Roomer improves physical validity and practical usability while preserving distributional quality, and ablations identify object-grounded diagnosis and deterministic search as key components. Professional judgments also validate Practical as an indicator of spatial usability.

  • Main results: Roomer-Final achieves the best SCA Gap, OOB, COL, and Practical, while ReSpace remains strongest on FID and KID.Relative to Ours-Initial, Roomer reduces OOB and COL by 15.74 and 12.02 percentage points and raises Practical by 10.48 points.
  • Main results: Roomer satisfies required furniture content and avoids highlighted geometric and practical failures across bedrooms, living rooms, and dining rooms.The comparison uses representative final layouts from common-1100.
  • Transfer: OOB, COL, and Practical improve for every external generator, while distributional metrics improve in 10 of 12 pairs.ReSpace KID and DiffuScene-RS SCA Gap are the only distributional exceptions.
  • Component ablation: Without RoReview, Target Resolution falls to 2.44% and Practical drops below the initial layout, showing object identification is a prerequisite for repair.Removing geometry conditioning causes a smaller but substantial degradation, while deterministic search supplies most robustness and planner seeds provide complementary coverage.
  • Iterative benefit and adaptive computation: Roomer averages 3.756 planner calls per scene, 62.44% fewer than fixed ten-round execution, while later rounds continue improving safe repair.Most gains occur early; later rounds resolve a shrinking tail of difficult scenes through adaptive stopping.
  • Professional validation: Professional approval increases monotonically from Low to High Practical, with a significant ordered trend (Z = 4.67, p = 3.07 × 10−6).Ten evaluators made 900 binary judgments on an independent frozen set of 90 layouts, supporting Practical as a meaningful indicator of professionally judged usability.

Conclusions

Roomer formulates residual 3D indoor-layout failures as measurable, object-grounded repair tasks and combines diagnosis, StatePatch planning, candidate search, and verification-gated commitment. Roomer-CC and Roomer-Eval support repair supervision and joint assessment, while experiments show consistent gains across frozen layouts and four external generators.

  • Framework: Roomer treats residual 3D indoor-layout failures as measurable, object-grounded repair tasks.Its framework combines RoReview-based diagnosis, geometry-conditioned StatePatch planning, deterministic candidate search, and verification-gated commitment.
  • Datasets and evaluation: Roomer-CC provides object-grounded repair supervision, while Roomer-Eval jointly assesses distributional quality, physical validity, and practical spatial usability.These resources address both training for local repair and evaluation beyond physical validity alone.
  • Empirical findings: Experiments on frozen layouts and four external generators show consistent gains in physical validity and Practical.Practical’s agreement with professional judgments is independently supported.

Roomer: Reflective Object-Grounded Model Editing and Repair for … Qwen Parsing and 3D Assembly

Roomer represents indoor layouts with object-grounded state and violation evidence, then repairs local problems through planner-proposed StatePatches validated by deterministic verification. Its controlled-corruption supervision and Qwen parsing pipeline support reproducible repair and 3D scene assembly.

  • A Instance-Grounded Repair Context: RoState combines immutable architecture, committed furniture, geometry-derived functional regions, and stable object references that preserve grounding across planning and execution.Stable references connect semantic rendering, numerical state, geometric evidence, planner output, and StatePatch execution.
  • A Instance-Grounded Repair Context: RoReview binds each measured violation to implicated entities and relational roles without prescribing the repair action.Roles include functional anchor, protected region, obstructing object, and expected category; affected functional regions are rebuilt on temporary candidate layouts.
  • B Roomer-CC: Controlled-Corruption: Roomer-CC generates faulty layouts from valid 3D-FRONT scenes using MOVE, ROTATE, SCALE, INSERT, DELETE, or REPLACE corruptions.Each retained example pairs the corrupted RoState and violation evidence with a known-feasible inverse StatePatch.
  • B Roomer-CC: Controlled-Corruption: 61,010/68,789 Roomer-CC construction attempts were retained, corresponding to an 88.69% end-to-end rate.Internal placement samples for inserted objects count as one outer corruption attempt rather than separate attempts.
  • B Roomer-CC: Controlled-Corruption: Roomer-CC uses room-level partitions, with 89.98%/5.01%/5.01% source-room proportions for train/validation/test and a fully held-out test split.The dataset is built from the 3,952 Qwen-Image training rooms, while the final Practical metric excludes collision, out-of-bounds, window-blocking, and binary-content families.
  • C Qwen-Image Training and Optimization Details: Qwen-Image is fine-tuned on 3,952 rooms, comprising 2,822 bedrooms, 543 living rooms, and 587 dining rooms, with LoRA adapters used for optimization.The 571-room validation split is reserved for checkpoint selection, and no common-1100 test room contributes a fine-tuning example.
  • Qwen Parsing and 3D Assembly: Qwen outputs are decoded into structured furniture instances, matched to category-compatible 3D assets, and oriented with APM or a deterministic fallback.Assembly statistics distinguish scene-, component-, object-, and query-level units.
  • Qwen Parsing and 3D Assembly: Raw assembly succeeds for 1,090 scenes, deterministic object-level fallback recovers eight, and frozen scene-level candidate traversal recovers the remaining two.The recovery paths together account for the scenes described in the supplied assembly results.

D Geometry-Conditioning Implementation

Roomer separates semantic and geometric repair context while preserving shared object references across representations. Geometry evidence is bounded, normalized, and integrated through initialized adapters that preserve the pretrained backbone at startup.

  • Aligned Context: The semantic stream combines the current top-down rendering, room specification, human-readable RoState, and selected RoReview entry.Structured geometric evidence uses the same stable object references as the textual state and StatePatch schema, maintaining aligned object grounding.
  • Evidence Selection: Each RoReview retains at most Ne = 16 evidence entities, prioritizing anchors, obstructions, protected elements, and nearby relevant objects.Excess positions are padded and masked in every attention operation.
  • Geometry Normalization: Numerical geometry is normalized using room width W, depth D, and lower floor-plane bounds before aggregation.The passage specifies the normalization inputs as W, D, xmin, and zmin.
  • Parameter Initialization: Geometry-conditioning projections and attention matrices use Xavier-uniform initialization, with zero biases and LayerNorm parameters initialized to one and zero.The initialization applies to evidence-output projections, aggregator attention and FFN matrices, and geometry-conditioning adapter query and cross-attention matrices.
  • Gated Integration: With ℓ= 0 and γℓ= 1, the geometry branch initially contributes zero while the modified network matches the pretrained backbone and preserves gradient flow.The nonzero gate preserves gradient flow to the output projection.

E Training and Optimization Details … H StatePatch Parameter Serialization and Autoregressive Learning

Roomer combines frozen-backbone, parameter-grouped training with verification-gated repair execution. Its deterministic pipeline validates structured StatePatches, protects unaffected satisfied relations, searches fallbacks only after seed rejection, and trains autoregressively on serialized edits.

  • E Training and Optimization Details: Base language and vision weights remain frozen, while AdamW assigns distinct learning rates and weight-decay settings to LoRA, geometry, normalization, query, and residual-gate parameters.LoRA uses learning rate 10−5 with zero weight decay; most other listed groups use learning rate 10−4.
  • F Verification-Gated Commit: Formal Definitions: Verification uses stable issue keys and protected satisfied relations, preventing candidates from replacing one hard violation with another or breaking unaffected dependencies.Protected relations are verifier-derived, while relations depending on edited targets remain controlled by aggregate residual.
  • F Verification-Gated Commit: Formal Definitions: A candidate commits only when the target issue is fully resolved after recomputation, structural validity holds, and the aggregate residual satisfies the verification conditions.Partial severity reduction is insufficient, and structural validity is distinct from physical all-pass.
  • G Deterministic Candidate Instantiation and Repair Scheduling: The normalized planner-seed candidate is evaluated first, and deterministic action-specific fallback search begins only if that seed is rejected, stopping at the first verified fallback.Fallbacks are ordered primarily by parameter distance from the seed and then by edit magnitude.
  • G Deterministic Candidate Instantiation and Repair Scheduling: Roomer executes one target-grounded StatePatch per round using MOVE, ROTATE, SCALE, INSERT, DELETE, and REPLACE, while scheduling one unblocked issue at a time.Blocked issues can be reconsidered after a successful commit changes the scene.
  • H StatePatch Parameter Serialization and Autoregressive Learning: StatePatch floating-point values are rounded to at most three decimal places, use explicit units, and are learned through ordinary next-token prediction rather than a regression head.Positions and sizes use meters, orientations use degrees, and scale is dimensionless.
  • H StatePatch Parameter Serialization and Autoregressive Learning: The autoregressive objective supervises only assistant StatePatch tokens with field-dependent weights, prioritizing action and target selection over categorical, numeric, and schema fields.The weighting order is w_action = w_target > w_categorical > w_numeric > w_schema, leaving numerical refinement to deterministic search.

I Roomer-Eval Protocol and Complete Transfer Results · J Complete Roomer-Eval Practical Rule Definitions

Roomer-Eval uses a frozen common-1100 cohort and deterministic, standardized evaluation to measure distributional quality, physical validity, and practical usability, including transfer across four external generators. Its Practical metric micro-averages explicitly defined, applicability-aware rules for functional organization, clearance, door avoidance, connectivity, and bed access.

  • I Roomer-Eval Protocol and Complete Transfer Results: The frozen common-1100 cohort contains 777 bedrooms, 155 living rooms, and 168 dining rooms and is excluded from training, selection, and tuning.It is shared by final-layout comparison, cross-generator refinement, and controlled repair analysis.
  • I Roomer-Eval Protocol and Complete Transfer Results: Evaluation standardizes parsed scenes through unified 256 × 256 RGB top-down semantic renderings while preserving each baseline’s native conditioning and assembly procedure.Adapters perform coordinate conversion and field standardization without repairing layouts or replacing successfully assembled assets.
  • I Roomer-Eval Protocol and Complete Transfer Results: Deterministic traversal evaluates frozen candidate pools and same-category asset orders when primary outputs fail parsing or assembly, without using metrics for candidate selection.Final outputs are reported after this fixed traversal, enabling reproducible fallback behavior.
  • I Roomer-Eval Protocol and Complete Transfer Results: Ten of twelve paired FID, KID, and SCA Gap comparisons improve; the exceptions are ReSpace KID and DiffuScene-RS SCA Gap.These paired distributional-quality results correspond to the physical-validity and usability comparisons.
  • I Roomer-Eval Protocol and Complete Transfer Results: Roomer’s refined OOB, COL, and Practical results for all four external generators are micro-averages over evaluated objects or applicable rule instances, not scene all-pass rates.OOB counts out-of-bounds furniture objects, COL counts unique collision objects, and Practical counts applicable rule instances.
  • J Complete Roomer-Eval Practical Rule Definitions: Practical micro-averaging excludes N/A instances, while multiple coffee tables, dining tables, or beds create separate instances and door or connectivity rules create at most one room-level instance.This aggregation is defined over scene-rule pairs with binary applicability and satisfaction indicators.
  • J Complete Roomer-Eval Practical Rule Definitions: Functional rules require a sofa, coffee table, and TV for living organization, 0.60 m dining-table side clearance, and θ ≥135° for each coffee-table viewing arrangement.Dining chairs associated with a table are excluded from its obstacle set.
  • J Complete Roomer-Eval Practical Rule Definitions: Usability rules evaluate door swing-proxy avoidance, 0.60 m passage-width connectivity, and bed-side access using cusable/wbed ≥0.30.Connectivity uses 1 cm/pixel rasterization, 0.30 m obstacle dilation, 8-connectivity, and no required semantic anchor; bed clearance scales with bed width.

K Controlled Analysis of the Roomer Repair Loop · L Additional Component and Solver Ablations · M Professional Validation of Practical Usability

The controlled analyses define target-grounded repair and distinguish local safety from complete hard validity, while ablations isolate geometry conditioning, sparse edits, verification, and deterministic fallback. Professional validation further tests practical usability with blinded interior-design evaluators and finds a significant association between Practical level and approval.

  • K Controlled Analysis of the Roomer Repair Loop: Roomer evaluates repairs on 1,100 shared-initialization scenes with 4,800 admitted, object-grounded targets and fixed actions, scheduling, round limits, and stopping conditions.The controlled protocol uses byte-identical initial layouts and common RoReview entries across variants.
  • K Controlled Analysis of the Roomer Repair Loop: 96.27% Strict Safe Repair and 72.64% Hard Validity measure distinct properties because local target resolution need not eliminate pre-existing non-target hard violations.Strict Safe Repair additionally requires no new hard keys, preserved protected objects, and complete resolution of each initial target.
  • K Controlled Analysis of the Roomer Repair Loop: Round-prefix evaluation uses actual states from frozen K = 10 trajectories, preventing smaller budgets from benefiting from resampling or separate runs.Target Resolution is scene-macro averaged, whereas pooled Target Load counts unresolved issue keys before normalization.
  • L Additional Component and Solver Ablations: The component ablations isolate geometry-conditioning evidence, planner-seed execution, scene-wide JSON rewriting, and verification with rollback under shared inputs and repair constraints.The complete system retains deterministic fallback, full-scene verification, and rollback, while variants selectively remove these mechanisms or replace sparse StatePatches with full-scene rewrites.
  • L Additional Component and Solver Ablations: The geometry-branch ablation identifies eight Repair Queries, a two-layer aggregator, and interval injection into the final four even-numbered decoder layers as the best overall trade-off.The comparison includes text-only and serialized-evidence alternatives alongside the Repair Query plus geometry-adapter design.
  • M Professional Validation of Practical Usability: Professional validation samples 90 frozen layouts evenly across low, medium, and high Practical strata and room types, with 10 blinded interior-design evaluators producing 900 randomized binary judgments.Each stratum contains 10 bedrooms, 10 dining rooms, and 10 living rooms; a scene is approved when at least seven evaluators answer “Yes.”
  • M Professional Validation of Practical Usability: Z = 4.67 and p = 3.07×10^-6 support a monotonic association between Practical level and professional scene approval.Inter-rater reliability is α = 0.65 with bootstrap 95% confidence interval [0.54, 0.75], alongside 84.0% pooled agreement and 68.9% unanimous scenes.
Loading 2608.01973v1…