Source-linked AI summary
Contact-Rich Robotic Manipulation in Construction via Zero-Shot Learning: A Diffusion Policy-Guided Adaptive Control
Roman Ibrahimov, Salma Mozaffari, Arash Adel
TL;DR
Contact-rich construction assembly requires precise, robust control under contact and uncertainty, while real-world policy-training data are costly to collect. This paper combines simulation-trained diffusion policies with adaptive disturbance compensation, achieving perfect real-world success across evaluated tasks and sequential full-scale truss deployment.
Problem
Contact-rich construction assembly requires precise geometric alignment and robust control under contact forces because small pose errors can cause misalignment, jamming, or failure.
Method
The framework combines simulation-generated force-feedback trajectories, diffusion policy learning, and an L1-inspired adaptive controller for zero-shot real-world deployment.
Results
Perfect success rates were achieved across both evaluated real-world tasks, while independently trained policies were deployed sequentially in a full-scale timber truss assembly.
Takeaways & Limitations
The framework supports reliable zero-shot sim-to-real transfer and scalable multi-stage construction assembly with lower, more stable interaction forces.
Takeaways & Limitations
The framework lacks vision-based perception, assumes a fixed target pose, and has been evaluated on a limited set of representative construction assembly tasks.
Abstract
from arXiv · showhide
Construction robotics and automation offer promising means of improving productivity, alleviating workforce shortages, and reducing workers' exposure to physically demanding tasks. However, reliable contact-rich robotic assembly remains challenging under tight tolerances, fabrication inaccuracies, and uncertain contact dynamics. To address this challenge, we present a framework coupling diffusion policies trained on simulation-generated pose and force/torque data with an L1-inspired adaptive controller that corrects policy-predicted actions online to compensate for unmodeled contact dynamics. We benchmark the framework against baselines in timber joinery, pipe fitting, and sequential full-scale truss assembly. It achieves 100% success on single-task assemblies and 90-100% success across sequential truss assembly subtasks, with lower, more stable contact forces than the baselines. By enabling zero-shot sim-to-real transfer for force-aware contact-rich assembly, the framework reduces costly, labor-intensive real-world data collection for policy training and advances scalable, robust automation of multistage assembly, motivating extension to broader contact-rich manipulation tasks in construction.
1. Introduction
Construction robotics must address low productivity, workforce shortages, and physically demanding work by enabling reliable force-aware assembly under uncertain contact conditions. This paper proposes simulation-trained diffusion policies combined with online adaptive control for zero-shot, construction-scale contact-rich manipulation.
- Motivation: 456,000 additional workers are projected to be needed in the U.S. construction industry in 2027 to meet demand.
- Challenges: Robotic construction assembly must manipulate full-scale components while handling physical interactions with materials and evolving as-built structures.
- Challenges: Force feedback is crucial for closing the loop and correcting positional and contact uncertainty during contact-rich assembly.
- Approach: The framework trains diffusion policies entirely on simulated robot-position and force/torque data, reducing costly, time-consuming, and labor-intensive physical demonstrations.
- Approach: An L1-inspired adaptive controller corrects policy-predicted actions online to improve robustness against material imperfections and unmodeled contact dynamics.
- Contributions: The combined framework enables zero-shot sim-to-real transfer, lower and more stable contact forces, and sequential full-scale truss assembly through multiple contact-rich insertions.
2. Related work
Prior construction-robotics research has progressed from preplanned assembly toward learned and closed-loop methods, but tight-tolerance contact-rich tasks remain difficult under uncertainty. Classical controllers, learned policies, and simulation-based transfer each address parts of the problem while leaving gaps in online correction, contact modeling, or data efficiency.
- Construction robotic assembly: Preplanned construction assembly demonstrated customized full-scale robotic fabrication but generally assumed accurate positioning and provided limited online correction after contact-induced deviations emerged.These approaches suit loose-tolerance tasks or interactions where moderate deviations do not critically affect success.
- Construction robotic assembly: Tight-fitting timber and other contact-rich assemblies are sensitive to calibration errors, fabrication inaccuracies, and material imperfections that can cause misalignment, jamming, or incomplete mating.Precomputed motion alone is unreliable when small deviations interact with geometric constraints and contact sensitivity.
- Simulation-to-real transfer: Simulation-based learning reduces real-world data burdens, but inaccurate contact models and unreliable sim-to-real transfer remain obstacles under tight tolerances and unmodeled contact dynamics.Real-world data collection, repeated physical evaluation, and task-specific fine-tuning can also be costly, hazardous, and damaging to materials or equipment.
- Model-based control: Classical hybrid force–position and impedance controllers regulate contact but depend on predefined task frames, contact constraints, desired impedances, switching logic, and task-specific tuning.Their performance therefore depends on the accuracy of task representations and assumed contact structures.
- Learning-based control: Learning-based methods can capture difficult contact dynamics, while diffusion policies model multimodal action distributions for tasks with multiple valid contact instances.However, many demonstrated tasks allow moderate contact deviations, unlike tight-clearance insertion and construction-scale manipulation.
3. Methods
The framework combines simulation-based diffusion-policy training with real-time force- and position-feedback correction for zero-shot sim-to-real transfer in contact-rich construction manipulation. It uses randomized, perturbed demonstrations and an L1-inspired adaptive controller to compensate for contact disturbances during physical execution.
- Framework overview: The framework trains diffusion policies in simulation and corrects their predicted actions online using real-time force and position feedback.The adaptive controller is designed to improve execution performance and enable zero-shot sim-to-real transfer.
- Experimental setup: The physical platform uses a six-axis UR20 arm with flange-mounted six-axis force/torque sensing and a custom pneumatically actuated parallel gripper.The arm has 1750 mm reach and a 20 kg rated payload; experimental end-effector and component mass ranges from 6 to 15.2 kg.
- Simulation and sim-to-real setup: Simulation uses NVIDIA Isaac Sim 5.0 with GPU-accelerated parallel contact simulation, URDF robot models, and native six-axis force/torque sensing.The simulated platform reproduces nominal task geometry and physical observation/action interfaces without assuming exact fabrication deviations, compliance, friction, or contact dynamics.
- Demonstration generation: Demonstrations execute randomized two-phase approach and insertion motions, with 100 Hz trajectories perturbed by low-frequency lateral noise up to 2 mm.Parallel simulation environments independently randomize initial poses, while differential inverse kinematics tracks fixed-rotation Cartesian trajectories with a constant gripper-closed command.
- Adaptive control: The L1-inspired controller estimates disturbances, activates adaptation during significant contact forces, and scales policy actions to attenuate motion under high forces.Negligible forces suppress adaptation, whereas contact forces activate disturbance compensation; filtered disturbance estimates support corrective action injection.
4. Experiments
The experiments evaluate the proposed method in simulation and real-world environments through single-assembly and full-scale sequential assembly case studies. These studies cover contact-rich tasks spanning different geometries, clearances, material properties, and multistage conditions.
- Experimental Design: The evaluation combines experiments conducted in simulation and real-world environments.
- Experimental Design: The experiments comprise two categories: single assembly on two contact-rich tasks and full-scale assembly involving sequential contact-rich tasks.
- Single Assembly Experiments: The single-assembly studies use tasks with different geometries, clearances, and material properties.
- Full-Scale Assembly Experiment: The full-scale study evaluates multistage sequential contact-rich assembly tasks.
4.1. Single assembly experiments
Single-assembly experiments evaluated the proposed framework on timber tenon–mortise insertion and PVC pipe fitting across simulation and real-world environments. The study compared uncorrected diffusion-policy actions, adaptive force-and-position feedback, and a benchmark compliance controller under defined rollout and force limits.
- Tasks: The experiments tested vertically inserting a timber tenon into a mortise and inserting a PVC pipe into a three-way tee fitting.The tasks differed in geometry, clearance, and material properties to assess generalization across contact-rich assembly scenarios.
- Policy training: Diffusion policies were trained exclusively on simulation data using 1000 demonstrations per task and four independently trained models.The models used identical parameters and accounted for stochasticity during evaluation.
- Execution modes: The evaluation compared baseline policy execution, the proposed adaptive controller, and a compliance controller in simulation and zero-shot real-world transfer.The adaptive controller refined policy actions online using force and position feedback, while the compliance controller provided the benchmark.
- Evaluation protocol: Rollouts were successful when the tenon reached the goal within 3 mm, and were terminated after 600 inference steps, contact forces above 200 N, or position deviations beyond 100 mm.Success rates averaged 200 simulation rollouts and 40 real-world rollouts per execution mode and task.
- Force analysis: Peak contact-force magnitudes were compared across execution modes using Kruskal-Wallis tests followed by pairwise one-sided Mann-Whitney U tests with Bonferroni correction.Common Language Effect Size was the primary rank-based effect size, supplemented by Cohen’s d.
4.2. Full-scale assembly experiment
The full-scale experiment evaluates the proposed framework on sequential, contact-rich assembly of a four-member gable-shaped timber truss. Three task-specific diffusion policies, trained in simulation, are tested in independent real-world insertion rollouts with adaptive control.
- Assembly configuration: The experiment assembles a four-member, gable-shaped timber truss containing a bottom chord, central post, two rafters, mortise-and-tenon joints, and lap joints.The post and rafters are inserted into the bottom chord, while each rafter connects to the post near the apex.
- Assembly sequence: Three sequential contact-rich tasks insert the post, left rafter, and right rafter after the bottom chord is positioned and secured.The workcell includes pickup and assembly stations, with three grippers securing the bottom chord.
- Adaptive control: The adaptive controller is integrated into each insertion policy and tuned to be less reactive while retaining sufficient adaptation during contact.The controller parameters are tuned on the real setup and reported in Table 4.
- Policy training: Three separate diffusion policies are trained in simulation using 1000 demonstrations per task and task-specific insertion parameters.The policies use rafters’ z_app = 110 mm, the post’s z_app = 60 mm, r_samp = 5 mm, and z_max = 20 mm.
- Evaluation protocol: Each insertion task is evaluated through 10 independent real-world rollouts using its corresponding simulation-trained policy and adaptive control framework.The evaluation targets robustness of the individual insertion tasks and complements later assessment of the integrated truss workflow.
5. Results and discussion
The proposed adaptive framework improves robustness in contact-rich timber assembly, achieves comparable performance on compliant pipe fitting, and transfers to sequential full-scale truss assembly. It also produces lower peak forces, with the strongest benefits observed for the stiffer timber task.
- Single assembly success rates: 100% real-world success was achieved for the proposed mortise-and-tenon method, compared with 95% for the compliant policy, while eliminating reported jamming and force-threshold violations.In simulation, the proposed method outperformed both the baseline and compliant policy for the timber task.
- Single assembly success rates: 100% real-world success was achieved by all execution modes for pipe fitting, whose PVC compliance accommodates small misalignments through deformation and sliding.The methods therefore showed less pronounced differences on pipe fitting than on the stiffer timber joint.
- Peak contact forces: For timber joints, the proposed method consistently produced lower peak forces than both baselines, with real-world reduction versus the compliant policy significant at pcorr < 0.001 and d = 1.60.In simulation, both comparisons were highly significant, with CLES values of 0.74 versus the baseline and 0.72 versus the compliant policy.
- Peak contact forces: For pipe fitting, the proposed method produced significantly lower real-world forces than the compliant policy (pcorr = 0.004, CLES= 0.69, d = 0.67), but not than the baseline (pcorr = 1.000, CLES= 0.43, d = −0.30).These weaker differences were consistent with the similar pipe-fitting success rates and the PVC components’ compliance.
- Full-scale truss assembly: 100% post-insertion success, 100% right-rafter-insertion success, and 90% left-rafter-insertion success were achieved across 10 real-world rollouts per task in sequential full-scale truss assembly.The framework completed the gable truss assembly and remained stable after the manipulators released the structure.
6. Conclusion
The framework combines synthetic-data diffusion policy learning with L1-inspired adaptive disturbance compensation for zero-shot contact-rich construction assembly, achieving reliable transfer and sequential truss deployment. Remaining limitations concern perception, task diversity, manual tuning, and evaluation breadth.
- Contributions: The framework integrates synthetic trajectory generation, diffusion policy learning, and adaptive disturbance compensation to enable efficient zero-shot real-world construction assembly.Simulation generates motion-planned trajectories with force feedback, reducing reliance on costly real-world data.
- Contributions: The CNN-based diffusion policy predicts nominal actions, while an L1-inspired controller uses position and force feedback to compensate for unmodeled contact dynamics, friction, and misalignment.This architecture bridges the sim-to-real gap through real-time residual-disturbance estimation and compensation.
- Results: Perfect real-world success across timber-joint and pipe-fitting tasks, plus sequential full-scale timber-truss deployment, demonstrates reliable zero-shot transfer and scalability to multistage workflows.The truss experiment used independently trained policies deployed sequentially with larger components, higher contact forces, and sequential operations.
- Results: The adaptive method produces lower and more stable peak contact forces than baseline and compliant policies, especially during timber-joint assembly.The findings show that contact quality is not fully characterized by success rate alone.
- Results: Material stiffness and geometric tolerance influence performance: stiff timber benefits more from disturbance compensation, whereas deformable, lower-friction PVC accommodates misalignment through sliding.Timber is more sensitive to misalignment; PVC’s deformation and sliding reduce sensitivity to control inaccuracies despite smaller clearances.
- Limitations and future work: The framework assumes fixed target poses, excludes vision-based perception, covers limited assembly scenarios, requires manual controller tuning, and omits broader metrics such as energy and insertion time.Future work proposes richer multimodal perception, more complex construction tasks, automatic parameter tuning, and expanded performance evaluation.