PaperlayerView exact revision on arXiv

Source-linked AI summary

RoomShift: Room-scale Dynamic Haptics for VR with Furniture-moving Swarm Robots

Ryo Suzuki, Hooman Hedayati, Clement Zheng, James Bohn, Daniel Szafir, Ellen Yi-Luen Do, Mark D. Gross, Daniel Leithinger

arXiv:2008.08695v1cs.ROcs.HC

TL;DR

VR scenes can show objects that users cannot physically touch, sit on, or lean against. RoomShift reconfigures a room with robots that move furniture to match virtual content. Its five-participant evaluation supported realistic and enjoyable VR experiences, while the prototype remains limited by object clearance and load-bearing constraints.

  • Problem

    VR objects such as walls and furniture are only visual, limiting users’ ability to touch, sit on, or place objects on them.

    Source: INTRODUCTION · p. 1

  • Method

    RoomShift uses a swarm of shape-changing robots to move furniture and synchronize the physical environment with virtual scenes for tour and architectural-design interactions.

    Source: INTRODUCTION · pp. 1–2

  • Results

    In a five-participant evaluation, RoomShift was reported to provide a realistic and enjoyable VR experience.

    Source: CONCLUSION · p. 8

  • Takeaways & Limitations

    Participants gave positive feedback and saw potential for virtual tourism and architectural design.

    Source: Results and Feedback · p. 6

    The prototype requires at least 30 cm of clearance under objects, and its lift and wheeled robot are not robust enough to carry heavy objects or people.

    Source: LIMITATIONS AND FUTURE WORK · p. 8

Abstract

from arXiv · show

RoomShift is a room-scale dynamic haptic environment for virtual reality, using a small swarm of robots that can move furniture. RoomShift consists of nine shape-changing robots: Roombas with mechanical scissor lifts. These robots drive beneath a piece of furniture to lift, move and place it. By augmenting virtual scenes with physical objects, users can sit on, lean against, place and otherwise interact with furniture with their whole body; just as in the real world. When the virtual scene changes or users navigate within it, the swarm of robots dynamically reconfigures the physical environment to match the virtual content. We describe the hardware and software implementation, applications in virtual tours and architectural design and interaction techniques.

INTRODUCTION

RoomShift addresses the gap between visually immersive VR and environments users can physically touch, sit on, and lean against. It combines furniture-moving robots with interaction techniques for virtual tours and architectural design.

INTRODUCTION

  • Dynamic environments can support walking, sitting, and leaning, unlike haptic interfaces focused on fingertips or wearable sensations.Earlier actuated environments were limited by slow transformation or by supporting only walking.Source: INTRODUCTION · p. 1
  • RoomShift uses a swarm of shape-changing robots to move existing furniture, making physical objects available for whole-body interaction in VR.The system tracks and coordinates the robots to place objects near users while avoiding collisions.Source: INTRODUCTION · pp. 1–2
  • The paper contributes a furniture-moving robot concept, its hardware and software implementation, and application scenarios with interaction techniques.Source: INTRODUCTION · p. 2

Haptic Feedback using Passive Objects

Hand-held and Wearable Haptic Interfaces

RoomShift contrasts portable hand-held and wearable haptics with encountered-type systems that augment the environment. Its approach uses robots to reposition furniture as haptic proxies across a room-scale VR scene.

Hand-held and Wearable Haptic Interfaces

  • Hand-held and wearable haptics are portable, but remain limited in their ability to emulate a whole environment.Examples include actuated controllers, exoskeletons, haptic suits, electrical muscle stimulation, and actuated shoes.Source: Hand-held and Wearable Haptic Interfaces · p. 2

Encountered-type Haptic Feedback with Robotic Devices

Large-scale Haptic Feedback

  • Human-actuated systems require helpers, while inflatable room-scale systems are slow; RoomShift explores swarm robots for faster spatial transformation.The cited work identifies transformation speed and degrees of freedom as limitations for real-time dynamic haptics.Source: Large-scale Haptic Feedback · p. 2

Human-Robot Interaction with Swarm Robots

ROOMSHIFT: SYSTEM AND IMPLEMENTATION

RoomShift uses a swarm of mobile, shape-changing robots to reposition physical props as a VR scene changes. Its scissor-lift design is intended to move varied furniture while remaining compact beneath objects.

Overview

  • Each RoomShift robot combines a Roomba base with a scissor lift, letting it drive beneath and pick up furniture.The closed lift fits beneath tables or chairs with 30 cm clearance.Source: Overview · p. 3, Figure 2 · p. 3
  • Motion capture tracks the robots, props, and user; when scenes or layouts change, the system calculates prop positions and drives the swarm to reconstruct the room.The VR scene runs on an Oculus Go, with position information synchronized between the headset and the desktop controller.Source: Overview · p. 3

Mechanical Design

  • Nine robots use a scissor structure actuated by two linear actuators, extending vertically from 30 cm to 100 cm.The scissor lift moves at 1.3 cm/sec, and each robot moves at 20 cm/sec.Source: Mechanical Design · p. 3, Figure 4 · p. 3
  • The scissor structure was chosen as an inexpensive, lightweight option that can expand fourfold to move varied objects, unlike mechanisms with mobility or load-bearing trade-offs.The 2 kg structure costs $15 for the rack and $35 per linear actuator; extending it further by combining structures trades off stability.Source: Mechanical Design · pp. 3–4

Object Actuation

RoomShift can lift lightweight furniture, and its placed furniture carries user weight rather than the robot. Object fit and load capacity constrain which props the system can actuate.

Object Actuation

  • After placing furniture, the robot need not support a user's weight because the load goes to the furniture.Source: Object Actuation · p. 4
  • 22 kg is the maximum weight the robot can lift and carry; the scissor structure began to break above 23 kg.Tested furniture weighed 3.5 to 11.2 kg; multiple robots can coordinate on heavier objects when there is enough space underneath.Source: Object Actuation · p. 4
  • The height-adjustable lift actuates desks, long tables, chairs, side tables, and custom props such as a wall mounted to a side table.Source: Object Actuation · p. 4, Figure 5 · p. 4

Tracking System

RoomShift tracks robots, props, and users with optical motion capture, then continuously plans robot movement around obstacles. Its control system uses defined approach points and target furniture heights to guide pickup and placement.

Tracking System

  • Twenty IR cameras track six-degree-of-freedom object positions across a 10 m × 10 m area at 60 FPS.The system uses retro-reflective markers for tracking.Source: Tracking System · p. 4
  • Five markers on parallel scissor-lift bars keep each robot’s marker pattern fixed as lift height changes, while its orientation estimates lift height.Markers on props can also stand in for robot markers hidden beneath carried objects, though tracking can be lost when furniture is enclosed by walls on two sides.Source: Tracking System · p. 4

Control System

  • Path planning uses robot positions, obstacle positions, and target locations to set each robot’s next goal, with commands sent over WiFi at 30 Hz.The system uses RVO to determine movement vectors and PID control to set wheel speeds.Source: Control System · p. 5
  • For pickup and placement, robots follow a predefined sequence using user-defined entry and exit points to avoid furniture legs.The system uses registered target furniture heights to extend the lift; a sensor could provide closed-loop control.Source: Control System · p. 5

Electronics Implementation

Each robot uses a microcontroller and motor driver to control its lift actuators. Because the Roomba battery cannot supply the actuators’ current, they use a separate rechargeable battery.

Electronics Implementation

  • Each robot is controlled by an ESP8266 microcontroller chip (Wemos D1 mini) and is powered by the Roomba through a voltage regulator.Figure 9 illustrates the circuit schematic.Source: Electronics Implementation · p. 5, Figure 9 · p. 5
  • The linear actuators draw 600–800 mA on average and 1.5–2 A at peak, beyond the Roomba battery’s capacity.The actuators are powered by an external 12 V, 6000 mAh rechargeable battery.Source: Electronics Implementation · p. 5
  • Figure 10 depicts the communication software, while Figure 9 depicts the robot circuit.The supplied passages identify the figures’ subjects but do not describe their internal layouts.Source: Figure 10 · p. 5, Figure 9 · p. 5

Software Implementation

RoomShift synchronizes tracking data, a browser-rendered VR scene, and robot commands so physical props can move when the virtual scene changes. A preliminary chair task compared robot-moved chairs with static chairs.

Software Implementation

  • The interaction design space names walking and touching, physically moving objects, teleporting, and virtually moving objects, alongside controller-based interactions.Figure 11 is identified as the interaction design space of RoomShift.Source: Software Implementation · p. 6, Figure 11 · p. 6
  • When the virtual scene changes, the server determines prop types and target positions relative to the user, then sends robot commands over WiFi.The desktop streams motion-capture data to a Node.js server, synchronized with an A-Frame scene in an Oculus Go browser through WebSocket communication.Source: Software Implementation · pp. 5–6

Preliminary System Evaluation

  • A preliminary evaluation asked five VR-experienced participants to locate and sit on highlighted chairs in a within-subjects comparison of static chairs and robot-moved chairs.The task used four virtual chairs and repeated eight times across the two conditions; participants were later asked which condition felt more realistic or whether they felt the same.Source: Preliminary System Evaluation · p. 6

Results and Feedback

Participants reported the same realism for robot-moved and static chairs, while some noted robot noise and the absence of a rendered body affected their experience. Their feedback also highlighted virtual tourism and architectural design as promising applications.

Results and Feedback

  • All five participants perceived the realism of the robot-moved and static-chair conditions as the same.Three noticed robot noise through headphones but said it did not distract; four said the absent VR body diminished realism and confidence touching or sitting rapidly in both conditions.Source: Results and Feedback · p. 6
  • Participants gave positive feedback and were enthusiastic about virtual tourism and architectural design applications.They described using the system to familiarize themselves with unfamiliar places and interact with furniture in proposed spaces.Source: Results and Feedback · p. 6

Target Application Domain

  • Virtual tours reduce the time and cost of on-site viewings but lack bodily experiences such as touching surfaces and sitting; VR architectural design supports experiencing, discussing, and modifying proposals before construction.The paper motivates body-based experience and co-design for people with varied physical abilities.Source: Target Application Domain · pp. 6–7

Interaction Design Space

RoomShift spans embodied and controller-based interactions, from walking among nearby haptic props to physically moving furniture and editing or navigating the virtual scene. It also supports remote architectural co-design by synchronizing physical furniture with a designer’s changes.

Interaction Design Space

  • RoomShift proposes four interaction types across embodied and controller-based use: walking and touching, physically moving furniture, teleporting, and virtually moving furniture.Embodied use includes walking or physically manipulating furniture; controller-based use includes gestures and teleportation.Source: Interaction Design Space · p. 7

Experiencing Architectural Spaces: Walking and Touching

Architectural Co-Design: Physically Moving Furniture

  • At remote co-design sites, robots can move physical furniture in real time to reflect a designer’s changes, letting clients touch, walk around, and modify the design.RoomShift extends collaborative design scenarios in which a designer and client discuss a layout remotely.Source: Architectural Co-Design: Physically… · p. 7

Interaction Design Space

  • When users teleport, robots move furniture to match the new scene position.Source: Figure 13 · p. 7

Navigating Large Spaces: Teleporting in VR

RoomShift supports navigation and editing in virtual scenes by coordinating physical furniture with users’ changing locations and scene layouts. The discussion also describes flexible robot coordination and possible props beyond furniture.

Navigating Large Spaces: Teleporting in VR

Virtual Scene Editing: Virtually Moving Furniture

DISCUSSION

  • The nine-robot swarm can hand off tasks to address slow single-robot transformations, and robot roles can change with virtual content.Robots may also hand off tasks to recharge; nine robots were estimated to cover furniture and props within 3–4 m of one user.Source: DISCUSSION · p. 8
  • RoomShift could use furniture combinations, building blocks, cardboard props, or mannequins for varied scenes, and could simulate dynamic objects such as height-changing desks or transforming walls.Source: DISCUSSION · p. 8

LIMITATIONS AND FUTURE WORK

The prototype has mechanical and deployment constraints, and its preliminary evaluation leaves questions for larger-scale study. The paper concludes that RoomShift can provide realistic, enjoyable VR experiences and demonstrates applications in tours and architectural design.

LIMITATIONS AND FUTURE WORK

  • The scissor lift requires at least 30 cm of clearance beneath an object; pushing or dragging was possible only for lightweight objects.The robot could not drag a sofa.Source: LIMITATIONS AND FUTURE WORK · p. 8
  • The current structure and wheeled robot are not robust enough to carry heavy objects or people; stronger actuators and a more robust structure are proposed for that use.Source: LIMITATIONS AND FUTURE WORK · p. 8
  • The authors propose that future shape-changing robots could render the environment itself rather than carry an existing prop, potentially increasing haptic flexibility and dynamism.Source: LIMITATIONS AND FUTURE WORK · p. 8
  • The feedback came from a preliminary evaluation; the authors say a larger-scale study would provide further insight into swarm-robot use for VR haptics.Source: LIMITATIONS AND FUTURE WORK · p. 8
  • The current work focuses on wheel-based furniture rearrangement, while future work could explore construction swarms, drones, or cable-based robots.Source: LIMITATIONS AND FUTURE WORK · p. 8
  • Deployment beyond the lab is constrained by a dedicated tracking setup, alongside technical challenges in tracking, interaction, and multi-person collision avoidance.The authors suggest a depth camera and AR markers as one possible way to reduce hardware requirements.Source: LIMITATIONS AND FUTURE WORK · p. 8

CONCLUSION

  • The conclusion reports that five participants found RoomShift realistic and enjoyable, alongside demonstrations for virtual tours and architectural design.Source: CONCLUSION · p. 8
Loading 2008.08695v1…