An Integrated System for Perception-Driven Autonomy with Modular Robots

1 Introduction

• Existing systems had not demonstrated fully autonomous, reactive self-reconfiguration for high-level tasks in unknown environments.Prior work showed partial capabilities, including beacon-dependent mapping, human-directed high-level decisions, or swarm self-assembly for basic tasks.Source: §1 · p. 2
• The system selects behaviors and reconfigures when task and perceived-environment requirements call for a capability the current configuration lacks.Its architecture combines flexible, distributed robot modules with centralized sensing, perception, and high-level mission planning.Source: §1 · p. 2
• Three hardware demonstrations show autonomous exploration, reactive reconfiguration decisions, and object manipulation from high-level task specifications.The authors present the system as groundwork for modular robots addressing real-world tasks.Source: §1 · p. 2

2 Results

• Across three demonstrations, the robot explored unknown environments and autonomously completed complex tasks by selecting behaviors and reconfiguring when its current configuration could not perform them.The planner chose from a behavior library based on the task and perceived environment; different configurations provide different capabilities.Source: §2 · p. 3
• In Demonstration I, the robot switched from Car to Proboscis to retrieve a spool from a narrow gap, then returned to Car to deliver it.The planner classified the surroundings as a tunnel and selected the configuration used to reach between the trash cans.Source: §2 · p. 3
• In Demonstration II, the robot recognized stairs, reconfigured from Scorpion to Snake, climbed to deliver a circuit board to a mailbox, then returned to Scorpion.Source: §2 · pp. 3–4
• In Demonstration III, the robot recognized a high target about 25 cm above ground, reconfigured to Proboscis, and placed the stamp using highReach.Source: §2 · pp. 3–4
• The demonstrations used the same software architecture and SMORES-EP modules, but II and III used an improved 3D sensor and different driver than I.The behavior library was extended as abilities were added; motor speeds, SLAM parameters, and the low-level reconfiguration controller also received minor adjustments.Source: §2 · p. 4

3 Discussion

• The architecture distributes low-level hardware while centralizing high-level planning and perception, combining elements of both design paradigms.Source: §3 · p. 4
• The demonstrations span flat-ground movement, tight spaces, reaching, rough terrain, and object manipulation, with this range accessible only by reconfiguring between morphologies.Source: §3 · pp. 4–5
• The same high-level delivery task led to a Proboscis for reaching up high and a Snake for climbing stairs after environment characterization.For object retrieval, the system used a Car for an object in the open and reconfigured into the Proboscis for an object in a tunnel.Source: §3 · p. 5
• About one minute per reconfiguration action, versus 5–15 minutes in past distributed systems, made the system faster for the complex demonstrated tasks.The authors attribute the faster transformations to centralized sensing and control using AprilTags and a centralized path planner.Source: §3 · p. 5
• The sensor module’s distinct shape complicates planning, and centralized sensing restricts independent module movement to the sensor’s vicinity, precluding disparate clusters.Configurations and behaviors must also provide the sensor an adequate view and support its weight and elongated shape.Source: §3 · p. 5
• In 24 Demonstration II attempts, 42% of failures were due to hardware errors and 38% to low-level software failures, under a planner that assumes reliable components.The authors describe low-level failures cascading into unexpected planner behavior and task failure.Source: §3 · p. 5
• Open-loop stair climbing and reaching were vulnerable to hardware errors and environmental variation; the authors report that sensing made exploration and reconfiguration less vulnerable to error.For example, stair climbing is likely to fail when actual stairs exceed the library entry’s height property; proposed improvements include more feedback and failure-recovery frameworks.Source: §3 · p. 6
• The environment characterization function assumes a simplified set of environment types and does not generalize well to completely unstructured environments or scale well to many types.The authors call for a more general characterization function for more realistic environments.Source: §3 · p. 6

4.1 Hardware

• SMORES-EP modules connect through electro-permanent magnets, while a separate sensor module supplies RGB-D sensing, computation, and AprilTag tracking.Centralized sensing and computation facilitate control and rapid reconfiguration, but the sensor module’s different shape complicates configuration and behavior design.Source: §4.1 · pp. 6–7

4.2 Perception and Planning for Information

• The system uses SLAM, waypoint-based exploration, and object recognition to gather information, with implementation choices intended to fit the robot system and expected environments.The implementation uses RTAB-MAP, Octomap, Next Best View exploration, and color-based object detection.Source: §4.2 · pp. 7–8
• Environment characterization classifies an object’s surroundings from a 3D occupancy grid and returns a waypoint for acting or reconfiguring.The implementation distinguishes tunnel, stairs, high, and free environments using obstacle reachability, distance, and object height.Source: §4.2 · pp. 7–8

4.3 Library of Configurations and Behaviors

• Each library entry pairs a configuration and behavior with properties describing its capabilities and suitable environments, allowing the planner to match tasks to robot abilities.For example, the planner searches for an entry capable of climbing stairs when a mailbox is characterized as being in a stairs environment.Source: §4.3 · pp. 8–9

4.4 Reconfiguration

• For a requested topology change, the controller disconnects, moves, and reconnects modules; plans between configurations are created manually and stored in the library.AprilTags tracked by a downward-facing camera provide module localization within a 0.75m × 0.5m area.Source: §4.4 · p. 9

4.5 High-Level Planner

• The high-level planner synthesizes a controller from task specifications and library properties, then selects behaviors and configurations using environment information.If no controller or suitable library entry exists, users are advised to revise the task specification or add behaviors.Source: §4.5 · p. 10

• Figure 1 is titled as a depiction of the environments and tasks for the demonstrations.Source: Figure 1 · p. 14
• Figures 2–5 are titled as demonstrations, a system overview flowchart, SMORES-EP hardware, and environment characterization.Source: Figure 2 · p. 15, Figure 3 · p. 16, Figure 4 · p. 16
• Figures 6–7 and Tables 1–2 are titled as module reconfiguration, a task specification with synthesized controller, a behavior library, and demonstration failure reasons.Source: Figure 6 · p. 17, Figure 7 · p. 18, Table 1 · p. 18

• Millibot and Swarm-Bots exploration relied on beacons with known locations, while human operators retained high-level or manipulation decisions.Source: §5 · p. 20
• Other cited work used known environments or fixed obstacle-specific assembly, whereas this system self-reconfigures between specific morphologies based on RGB-D characterization.Source: §5 · pp. 20–21
• Hardware demonstrations show autonomous self-reconfiguration adapting to perceived environments to complete complex tasks.Source: §5 · p. 21

6 Library of Configurations and Behaviors

• Library entries combine a robot configuration, its behavior, behavior properties, and suitable environment types.Configurations specify module structure; behaviors specify movement controllers; properties describe capabilities and appropriate environments.Source: §6 · p. 21
• Behavior properties express desired actions without naming a configuration, while environment types let the planner select a suitable behavior from the sensed setting.For a drop action in a tunnel, the system can select Proboscis rather than Car because the library marks it suitable for tunnels.Source: §6 · p. 22
• Parametric behaviors allow joint commands to change at run time, expanding motion beyond the fixed commands of static behaviors.A collision-free path from perception and exploration can determine real-time velocity, which is converted into joint values.Source: §6 · p. 22
• The Car supports object handling and flat-terrain locomotion, whereas Proboscis reaches through narrow gaps or upward but has limited locomotion.Proboscis is limited to forward and backward motion, making it unsuitable for general navigation.Source: §6 · p. 22

7 High-Level Planner

• LTLMoP synthesizes a finite-state robot controller from user task specifications over abstract robot and environment propositions.States specify high-level actions, and transitions depend on environment propositions evaluated through sensing programs.Source: §7 · p. 23
• At each step, the planner evaluates sensed environment propositions and chooses a library behavior matching the action properties and current environment type.The system maps propositions such as mailBox to sensing functions and navigation actions such as explore to drive behaviors and path planning.Source: §7 · pp. 23–24
• The controller reuses the current configuration when possible and otherwise reconfigures to one that can execute the selected behavior.When multiple configurations are suitable, it selects one at random; added specification constraints ensure library behaviors can implement the synthesized controller.Source: §7 · p. 24

8 Reconfiguration

• A downward-facing camera tracks AprilTag fiducials to localize modules within a 0.75m × 0.5m area in front of the sensor module.The system supports any module in that area equipped with an AprilTag marker.Source: §8 · p. 24
• The controller changes topology by detaching modules, driving them to new positions, and reconnecting them in the goal configuration.Before movement, the robot checks that the reconfiguration zone is flat and obstacle-free; plans are currently created manually and stored in the library.Source: §8 · pp. 24–25
• Docking uses a pre-docking waypoint, slow heading alignment, and a final straight approach with the magnets firing to secure connection.For disconnection, the module turns off its electro-permanent magnets and bends its tilt joint to mechanically separate.Source: §8 · p. 25