Source-linked AI summary
FuncRoom-Agent: Sequential Feed-Forward 3D Functional Indoor Scene Generation
Hao Feng, Zhi Zuo, MingJian Liang, Jingyu Hu, Xiaowei Hu, Liupengfei Wu, Dian Zhang, Guoxin Fang, Zhengzhe Liu
TL;DR
Function-Room Generation addresses the challenge of producing indoor 3D rooms that satisfy explicit functional goals without costly test-time generate–evaluate–revise loops. It combines a recursive executable DSL, sequential feed-forward expert construction, and ScenePRM-guided reinforcement learning. The method achieves state-of-the-art performance across general and function-oriented scene generation while reducing generation time to 25 minutes per scene.
Problem
Function-Room Generation addresses the need to generate executable indoor rooms that satisfy explicit functional, relational, and geometric requirements beyond visual plausibility.
Method
The framework uses a recursive staged DSL, visual-context trace distillation, deterministic sequential execution, and ScenePRM-guided reinforcement learning with execution-grounded feedback.
Results
The method achieves state-of-the-art performance on general indoor and function-oriented scene generation, with stronger functional completeness, relation correctness, support validity, and geometric executability; generation takes 25 minutes per scene.
Takeaways & Limitations
Distilling construction, evaluation, and refinement into training enables direct sequential feed-forward generation without iterative repair at inference time.
Abstract
from arXiv · showhide
We introduce Function-Room Generation, a new indoor 3D scene generation setting that creates rooms supporting explicit functional goals rather than merely visually plausible layouts. Existing agentic and executable methods improve controllability, but often depend on costly test-time generate--evaluate--revise loops, making functional room generation slow and computationally expensive. We address this challenge with three technical contributions. First, we design a recursive domain-specific language to effectively organize the hierarchical object compositions required by functional rooms, from room structure and major furniture to dense support-surface and nested small objects. It represents rooms as staged executable programs with explicit geometric and functional relations. Second, we propose a sequential feed-forward scene construction framework that distills recursive construction traces into a scene construction expert. At inference time, the expert writes executable DSL code stage by stage, and a deterministic executor directly instantiates each stage without teacher agents, online critics, or iterative repair. Third, we introduce ScenePRM, an execution-grounded process reward framework that improves the expert through reinforcement learning with functional, geometric, relational, and future-constructability feedback. We further establish a function-oriented benchmark and show state-of-the-art performance on both general indoor scene generation and function-room generation, achieving stronger functional completeness, relation correctness, geometric executability, and generation efficiency.
Introduction
Function-Room Generation targets executable indoor rooms that satisfy explicit functional goals, addressing limitations of generic layouts and costly iterative agentic refinement. The framework combines a recursive DSL, sequential feed-forward construction, ScenePRM, and a function-oriented benchmark.
- Function-Room Generation evaluates whether executable rooms contain required functional zones and objects in usable, geometrically valid layouts.
- The recursive DSL represents hierarchical functional rooms as staged executable programs with functional hierarchies, support relations, placements, and geometric constraints.
- Visual-context supervised fine-tuning distills reviewed multimodal construction traces into an expert that generates executable DSL segments stage by stage without test-time iterative repair.
- 25 minutes per scene reduces generation time by 24.2% versus the closest baseline while avoiding hours-long test-time refinement in several agentic systems.
- ScenePRM further optimizes the scene construction expert through execution-grounded reinforcement learning feedback.
Related Work
Prior indoor scene-generation methods emphasize plausible layouts, while newer agentic and executable approaches improve controllability through staged construction and code-like representations. FuncRoom-Agent instead uses execution-grounded process supervision to retain sequential feed-forward inference without online critic-and-repair.
- Data-driven generation: Data-driven indoor scene methods model object sets, bounding boxes, or relational graphs conditioned on room types, floor plans, text, or partial observations.These methods capture common spatial statistics and object co-occurrence patterns.
- Agentic systems: Agentic systems decompose scene construction into planning, generation, evaluation, and refinement using agents, critics, renderers, simulators, and retrieval tools.Their objectives include semantic alignment, visual realism, physical plausibility, and simulation readiness.
- Executable representations: Executable scene representations encode indoor scenes as code, structured programs, DSLs, or hierarchical scene trees to support execution, validation, editing, and interaction.Examples include Blender/Python programs, structured asset requests, indoor DSL specifications, BEV-grid layout programs, and hierarchical scene trees.
- Process supervision: Process reward models provide step-level supervision for intermediate decisions, complementing outcome rewards that evaluate only final success.Prior agentic reinforcement-learning work applies process supervision to tool-use, web navigation, question answering, retrieval, and code agents.
- FuncRoom-Agent: ScenePRM uses geometric, relational, visual, and functional checks on partial scenes while preserving sequential feed-forward DSL generation instead of online critic-and-repair.Each partial scene can be parsed, executed, rendered, and evaluated for progress toward functional-room goals.
Overview
The framework represents functional rooms as staged executable programs and trains a compact scene construction expert from agent-generated traces. At inference, the expert generates one DSL segment per partial scene state, and a deterministic executor executes each segment once without online search, critique, or repair.
- Overview: A recursive DSL organizes each room into five coarse-to-fine construction levels as a staged executable program.The representation supports the hierarchical structure of room construction.
- Overview: Visual-context SFT distills DSL construction traces produced by a multimodal code agent and execution harness into a compact scene construction expert.The traces provide the training basis for sequential feed-forward construction.
- Overview: ScenePRM-guided GRPO improves the expert by executing, rendering, and comparing candidate stage rollouts from the same partial scene.The process operates during training rather than as an inference-time repair loop.
- Overview: At inference, the expert generates one executable DSL segment from each partial scene state, which the deterministic executor executes once without teacher agents, search, critique, or repair.This yields sequential feed-forward scene construction.
Function-Room Generation Task Formulation
Function-Room Generation creates an executable indoor scene from a functional intent and optional reference image. The generated room must support specified activities while satisfying functional, relational, accessibility, circulation, and geometric requirements.
- Function-Room Generation Task Formulation: Given functional intent p and optional reference image I0, the task generates an executable scene S = G(p, I0) that supports specified activities.The prompt specifies intended functions, required objects, and appearance preferences; the image may guide geometry, composition, organization, or style.
- Function-Room Generation Task Formulation: Evaluation checks functional zones and objects, spatial, support, and mounting relations, accessibility and circulation, containment, and collision avoidance.These criteria distinguish functional-room generation from generic indoor scene generation focused primarily on visual plausibility.
Recursive DSL Representation
The recursive DSL represents functional rooms as executable program trees that organize hierarchical object composition, explicit relations, and dependency-aware construction from room structure to nested details.
- Motivation: A flat object list cannot represent the functional and geometric dependencies spanning furniture, mounted objects, support-surface arrangements, and nested details.The DSL addresses these dependencies by structuring the room as a recursive executable program.
- Program-tree structure: Each room is represented as a rooted, ordered program tree whose nodes encode function-oriented construction units and directed parent–child relations.The ordered tree determines how construction units are organized and executed.
- Node representation: Each node combines a local functional goal, an executable DSL segment, and a recursive expansion decision.The functional goal identifies the group, the DSL segment realizes it, and the expansion decision controls recursion.
- Node representation: A node’s goal and DSL segment specify function groups, objects, semantic roles, placements, constraints, asset requests, and spatial or functional relations.Examples include study areas, desk arrangements, and tabletop reading groups.
- Recursive expansion: Recursive expansion decomposes function groups into finer-grained child groups, while leaf nodes terminate recursion and child ordering fixes generation and execution order.A desk arrangement can expand into monitor, reading-tool, and desktop-detail groups.
- Dependency levels: The DSL uses five dependency levels: room structure; activity zones and major furniture; mounted objects; support-surface objects; and recursively nested objects.This order ensures walls, supporting furniture, and parent objects exist before dependent objects are generated.
Sequential Feed-Forward Scene Construction
Sequential feed-forward construction distills verified recursive traces into a compact expert that generates executable DSL segments from the current scene state and visual context, progressing from coarse structure to fine details.
- Trace collection: Verified recursive DSL traces are collected with a multimodal code agent and deterministic execution and rendering harness, then manually reviewed for functional and geometric validity.Invalid samples are corrected or discarded before supervision.
- Trace generation: At each step, the trace agent conditions on functional intent, visual context, the partial scene, construction level, and preceding history to generate the next goal, DSL segment, and expansion decision.These outputs are represented as (gt, at, ρt), where ρt determines recursive expansion.
- Trace generation: Each generated DSL segment is immediately parsed, executed, and rendered, returning an updated scene and visual observation for the next construction step.Repeating this process produces state-conditioned traces from coarse functional structure to fine local details.
- Expert distillation: The verified traces are decomposed into step-level samples whose inputs contain intent, reference image, rendered observation, partial-scene state, construction level, and history.Targets contain the next local goal, executable DSL segment, and recursive expansion decision.
- Expert distillation: Visual-context supervised fine-tuning distills the general-purpose multimodal code agent into a compact scene construction expert.The expert learns to generate the next executable DSL segment directly from the current executed scene state and visual context.
ScenePRM-Guided Reinforcement Learning
ScenePRM improves the supervised construction expert with execution-grounded, stage-level rewards that assess candidate rollouts for realism, functionality, layout, prompt following, and future constructability.
- Motivation: ScenePRM addresses the limitation that token-level likelihood training may produce executable segments that omit functional objects, incomplete zones, or relations hindering later construction.It optimizes the SFT expert with group-relative policy optimization.
- Candidate rollouts: At each construction level, K candidate stage rollouts are sampled from the current policy starting from the same accepted partial scene.Candidates independently generate the content required at that level.
- Execution-grounded evaluation: Each candidate is executed step by step to produce a terminal scene, and execution-failed candidates are directly penalized.Successful candidates are scored according to the quality of their resulting partial scenes.
- Process reward: A VLM-based evaluator scores successful candidates for visual realism, functionality, layout quality, and prompt following.Functionality and layout assessment includes completeness, relation correctness, support validity, wall consistency, geometric validity, and future constructability.
- Policy optimization: Candidate rewards are converted into group-relative advantages using the mean and standard deviation at each construction level.The highest-reward candidate initializes the next construction level during recursive training.
- Policy optimization: Candidate sampling, ScenePRM evaluation, and selection occur only during training, while inference uses the resulting expert without these online procedures.The training objective uses clipped GRPO with KL regularization against the frozen SFT reference policy.
Sequential Feed-Forward Inference
Sequential feed-forward inference constructs a room incrementally by conditioning each prediction on the previously executed partial scene. Each stage uses one model generation and one deterministic execution, without teacher agents, ScenePRM, candidate sampling, or iterative repair at inference.
- Sequential Feed-Forward Inference: At inference, the expert predicts a local functional goal, executable DSL segment, and recursive expansion decision from the current construction state.The DSL segment is executed and rendered once before the next prediction is made.
- Sequential Feed-Forward Inference: The expansion decision controls whether construction continues within the current functional group, recursively expands it, or proceeds to the next level.
- Sequential Feed-Forward Inference: Sequential feed-forward inference is sequential because predictions depend on the partial scene and feed-forward because each state receives one generation and deterministic execution.Teacher agents, ScenePRM, and candidate sampling are used only during training.
Experiments
Experiments evaluate the method on general indoor and function-oriented benchmarks using automatic metrics, ablations, qualitative comparisons, and a blinded user study. The method achieves strong functional, geometric, perceptual, and efficiency results, while ablations associate process supervision and recursive staged construction with improved coverage.
- Experimental Setup: The evaluation uses SceneEval and a 50-prompt Function-Room benchmark, with shared prompts and criteria across baselines.SceneEval uses 30 room-level prompts and nine metrics; Function-Room prompts specify multiple functional zones and required furniture and equipment.
- Experimental Setup: The study combines automatic benchmark evaluation, a 15-participant blinded user study, and ablations of training components and construction strategies.The user study includes five professional interior designers and ten non-experts.
- General Indoor Scene Generation: On SceneEval, the method achieves the best CNT, ATR, OOR, OAR, SUP, and ACC scores, ties for best NAV, and obtains the lowest COL and OOB rates.It generates an average of 64.3 objects per room while maintaining strong accessibility, collision avoidance, and boundary compliance.
- Function-Room Evaluation: 82.5, 86.3, and 65.7 are the method’s FZC, FOC, and FOP scores on Function-Room, respectively.It also achieves the best ACC and SUP, ties for best NAV, obtains the lowest OOB and COL rates, and requires 25 minutes per scene.
- Qualitative Results: Qualitative comparisons identify baseline errors in functional visibility, activity orientation, and object alignment, while the proposed method generally produces more complete functional regions and appropriate spatial relations.
- Function-Room Evaluation: 7.4, 8.3, and 7.9 are the highest realism, aesthetics, and functionality user-study scores, respectively.The table caption defines Real., Aesth., and Func. as these user-study dimensions and reports generation time in seconds converted to hours and minutes.
- Ablation Study: ScenePRM further raises FZC, FOC, and FOP by 6.1, 13.3, and 13.8 points over the standard RL baseline, respectively.The comparison supports process-level supervision over outcome-only training.
- Ablation Study: Staged construction improves functional coverage over one-shot generation, while recursive construction further improves FZC, FOC, and all reported spatial metrics.One-shot has higher FOP because it generates substantially fewer objects, trading coverage for precision; these pre-training results are not directly comparable with the trained expert.
Conclusion
The paper introduces Function-Room Generation to shift indoor 3D scene generation toward explicit functional usability. Its recursive DSL, sequential feed-forward construction, and ScenePRM-guided training support efficient generation with improved functional and spatial quality.
- Conclusion: Function-Room Generation targets explicit functional usability rather than generic visual plausibility in indoor 3D scenes.
- Conclusion: The framework combines a recursive DSL, sequential feed-forward scene construction, and ScenePRM-guided reinforcement learning with execution-grounded process feedback.
- Conclusion: Experiments on SceneEval and Function-Room demonstrate improved functional completeness, spatial and support relations, perceptual quality, and generation efficiency.