Source-linked AI summary
The STRANDS Project: Long-Term Autonomy in Everyday Environments
Nick Hawes, Chris Burbridge, Ferdian Jovan, Lars Kunze, Bruno Lacerda, Lenka Mudrová, Jay Young, Jeremy Wyatt, Denise Hebesberger, Tobias Körtner, Rares Ambrus, Nils Bore, John Folkesson, Patric Jensfelt, Lucas Beyer, Alexander Hermans, Bastian Leibe, Aitor Aldoma, Thomas Fäulhammer, Michael Zillich, Markus Vincze, Eris Chinellato, Muhannad Al-Omari, Paul Duckworth, Yiannis Gatsoulis, David C. Hogg, Anthony G. Cohn, Christian Dondrup, Jaime Pulido Fentanes, Tomas Krajník, João M. Santos, Tom Duckett, Marc Hanheide
TL;DR
STRANDS addresses the challenge of operating multi-purpose service robots in variable everyday environments, where not all situations can be anticipated. It combines an application-neutral architecture with monitoring, recovery, scheduling, and long-term learning; deployments included a 28-day continuous run and improved application-specific behaviour from experience.
Problem
Multi-purpose indoor service robots must operate predictably and remain available in environments whose people, doors, and furniture change unpredictably.
Method
The STRANDS Core System combines component-robustness strategies with task-based scheduling, monitored recovery, and restartable behaviours for long-term autonomy.
Results
28 days was the longest single continuous autonomous run reported for a multi-task indoor mobile service robot, while long-term experience also increased successful human-robot interactions per day.
Takeaways & Limitations
Long autonomous runtimes give robots opportunities to gather data, make active data-collection choices, and improve application-specific behaviour.
Takeaways & Limitations
The system assumes an appropriate robot and computational platform, and future work includes understanding human activities and closing gaps in that understanding.
Abstract
from arXiv · showhide
Thanks to the efforts of the robotics and autonomous systems community, robots are becoming ever more capable. There is also an increasing demand from end-users for autonomous service robots that can operate in real environments for extended periods. In the STRANDS project we are tackling this demand head-on by integrating state-of-the-art artificial intelligence and robotics research into mobile service robots, and deploying these systems for long-term installations in security and care environments. Over four deployments, our robots have been operational for a combined duration of 104 days autonomously performing end-user defined tasks, covering 116km in the process. In this article we describe the approach we have used to enable long-term autonomous operation in everyday environments, and how our robots are able to use their long run times to improve their own performance.
2 Long-Term Autonomy in STRANDS
STRANDS defines long-term autonomy for indoor mobile service robots as sustained operation amid changing environments and varied tasks. Its Core System is designed to support this operation while using accumulated experience to improve performance.
- Indoor service robots must handle short- to medium-term variability while remaining predictable, retaskable, and highly available.People, doors, and furniture can change, while robots must perform scheduled and on-demand tasks in confined environments.
- The project defines long-term operation for mobile service robots as at least multiple weeks of continuous operation.Such operation requires software and hardware robustness, active consumable-resource management, and autonomy-supporting capabilities that remain effective over long runtimes.
- STRANDS treats long-term operation as an opportunity for robots to learn environmental structure and dynamics, improving robustness over time.The project aims to create a cycle in which longer operation provides experience that supports better performance and greater autonomy.
- The STRANDS Core System was evaluated across four end-user deployments and supported over 100 days of autonomous operation.The architecture combines common-sense and novel elements for long-term service-robot applications.
3 Application Scenarios
STRANDS evaluates long-term autonomy in security and care environments, using robots for monitoring, information provision, guidance, and therapy support. These scenarios require adaptation to changing routines and robust operation.
- The project evaluates its systems in two application scenarios: security and care.The scenarios are intended to meet end-user demands through deployments in real environments.
- Security scenario: Security deployments use a robot to monitor an indoor office, model environmental changes, and detect prohibited, unusual, or anomalous events.The robot creates models of 3D structure, objects, and people, then uses them to identify unusual situations and patterns.
- Care scenario: Care deployments use a robot to guide visitors, provide information to residents, and assist walking-based therapies in an elderly care facility.Direct interaction with staff and patients makes long-term robustness and adaptation to facility routines especially important.
- Deployment environments: Figure 1 shows Bob at G4S’s Challenge House in Tewkesbury and Henry in the Haus der Barmherzigkeit reception in Vienna.The two robots are shown in their respective application environments.
4 Robot Technology
The deployed STRANDS systems run on MetraLabs SCITOS A5 robots with ROS-based software and onboard sensing, computation, long runtimes, and autonomous charging.
- All deployed systems use the MetraLabs SCITOS A5, an industry-standard mobile robot capable of 12 hours per charge and autonomous charging.The platform includes SICK S300 lasers, two Asus Xtion PRO RGB-D cameras, and an embedded Intel Core i7 computer with two additional networked PCs.
- The systems are developed in ROS, released under open source licenses, and binary packaged for Ubuntu LTS.Although most work is platform neutral, the deployed hardware is based on the SCITOS A5.
5 The Core STRANDS System
The Core STRANDS System supports long-term autonomy through modular robustness, monitoring and recovery, structured navigation, task scheduling, and persistent data-driven adaptation.
- The application-neutral architecture improves software robustness by minimizing component assumptions, containing errors, and automatically bringing components and subsystems up or down.These design-time practices reduce propagation of local failures across the system.
- Run-time monitoring relaunches crashed components, detects problematic topic rates, and triggers restarts while subsystems run only when required.Continuous integration tests components and the whole system in isolation, on recorded data, and in simulation.
- Navigation: Navigation combines a fixed laser-built map, adaptive Monte Carlo localisation, and dynamic-window navigation over 3D obstacle information.A manually built topological map adds task and navigation locations, with support for door passing, charging-station docking, and adaptive navigation near humans.
- Navigation: Topological movement constraints restrict navigation to known good areas, while long-term experience can adapt route planning and execution.Navigation failures can prevent a robot from reaching its charging station, limiting autonomous runtime.
- Task execution: Tasks encode actions, locations, durations, and execution windows, while the executive schedules them, manages navigation, handles failures, and controls maintenance actions.Charging, batch learning, and database backups are represented as tasks so the executive can manage limited resources.
- Data and learning: The system stores ROS messages in MongoDB-based tools and uses FreMEn to integrate sparse, irregular observations into spatio-temporal models.These data support learning, monitoring, and debugging.
6 Metrics
STRANDS evaluated long-term autonomy across four security and care deployments using system lifetime and task-activity metrics. The deployments combined extended operation with substantial task execution and travel, while long runs also enabled service improvements.
- 6 Metrics: TSL measures autonomous system availability, while A% measures the proportion of permitted time spent actively performing tasks.TSL resets after unrecoverable failure or unrequested expert intervention.
- 6 Metrics: 2014’s longest run was seven days against a 15-day target, whereas the 2015 security deployment reached 28 days against a 30-day target.Most 2014 failures involved insufficient software robustness; 2015 failures mostly involved external computer or hardware issues.
- 6 Metrics: Task counts and travel distances varied mainly with task type and deployment environment.Information serving could take tens of minutes with little travel, whereas door checking was brief and required travel before and during the task.
- 6 Metrics: 28 days was the longest 2015 continuous run, while the four deployments produced 2,545 hours and 116 km across 43 continuous runs.The 28-day run ended when the robot’s motors stopped responding, later fixed by a firmware update.
- 6 Metrics: The system’s long run times were designed to support tasks that use experience to improve robot service performance.This links deployment duration to the project’s broader objective of learning from everyday operation.
7 Monitored Navigation
STRANDS monitored navigation failures and applied recovery behaviours during deployment. Successful recovery prevented a failure from immediately recurring and helped preserve continuous autonomous operation.
- 7 Monitored Navigation: Successful recoveries were not followed by another failure within one minute or one metre of travel.Unsuccessful recovery sequences could arise from repeated monitored-navigation attempts or task-directed replanning after a failure.
- 7 Monitored Navigation: Successful recoveries clustered near difficult-to-navigate areas, including doors and desks.Figure 6 locates these recoveries in the 2015 security deployment.
- 7 Monitored Navigation: Each recovered failure could potentially have otherwise ended a continuous autonomous run.The authors identify monitored recovery as a significant contributor to deployment-level long-term autonomy.
8 Adaptive Topological Navigation
STRANDS used an adaptive topological navigation model to learn how reliably and how quickly the robot traversed map edges. This experience supported behaviour adaptation to the deployed environment.
- 8 Adaptive Topological Navigation: The robot aggregated navigation experience into a Markov Decision Process built automatically from the topological map.The MDP represented uncertainty in edge-traversal success and expected duration.
- 8 Adaptive Topological Navigation: Figure 6 maps successful recovery locations over the deployment area and topological map.Colors distinguish bumper-triggered or other human-help requests, reversing along the previous path, and simple retries.
- 8 Adaptive Topological Navigation: Traversal logs recorded edge duration and success in the robot’s database for online model learning.The learned success probabilities and durations enabled continual adaptation of navigation behaviour.
9 Predicting Human-Robot Interaction
In the care deployment, the robot adapted information-serving schedules to learn when and where people were likely to interact. Over deployment, this approach increased successful interactions per day, although there was no control group.
- 9 Predicting Human-Robot Interaction: The care robot scheduled information-terminal tasks at different topological nodes for staff and residents.Displayed information included the date, daily menu, and news, including for residents with potentially severe dementia.
- 9 Predicting Human-Robot Interaction: Scheduling balanced exploration of uncertain interaction opportunities against exploitation of times and locations with higher interaction likelihood.The two objectives were maintaining spatio-temporal models and maximizing interaction chances.
- 9 Predicting Human-Robot Interaction: Successful interactions increased on average per day over the course of the care deployment.The authors had no control group, but on-site observations indicated a positive effect from the robot’s choices.
- 9 Predicting Human-Robot Interaction: Figure 7 reports results from selecting interaction times and locations using FreMEn models learned during the 2015 care deployment.The figure concerns successful interactions, defined as information being offered and someone interacting with the screen.
10 Activity Learning
STRANDS learns models of normal human activity from walking trajectories and uses long-term operation to improve trajectory-cluster prediction. Unusual trajectories can trigger identity-confirmation tasks, while learned clusters represent desk-approach and desk-leaving activity.
- 42,850 individual trajectories were detected during the 2015 security deployment and generalized into spatio-temporal activity models using QSTAGs.QSTAGs generalize individual trajectories into spatial and temporal relations between trajectories and landmarks.
- Trajectories sufficiently far from cluster centres triggered the robot to approach the tracked human and request identity confirmation with a card reader.Fast response requires accurately matching the trajectory’s start to a cluster.
- Accuracy of predicting a trajectory’s cluster from its initial 20% improves as the robot gathers more data over its lifetime.This illustrates application-specific performance improvement enabled by long-term operation.
- Three learned trajectory clusters represented two desk-approach activities and one desk-leaving activity.The trajectories came from the bottom-left region of Figure 8, with direction indicated from red to green.
11 Conclusions and Future Work
STRANDS combines monitoring, component restarts, and learning from failures to support long-term autonomy, while acknowledging that catastrophic and difficult-to-predict failures remain unmanaged. Its broader conclusion is that extended operation enables robots to gather and actively select data that improves performance, with future work targeting deeper understanding of human activity.
- Monitoring and on-demand restarts help the system cope with unexpected internal and environmental situations during long-term operation.The approach spans individual components, navigation, and task behaviours.
- Catastrophic or hard-to-predict failures, including PC component and networking failures, remain beyond the robots’ current failure-management capabilities.Future work includes redundancy, online reconfiguration, and more general failure detection.
- Online learning from unusually long robot experiences allows performance to improve as additional data is gathered and actively selected.The paper identifies this coupling between runtime and data collection as its novel element.
- Future work will focus on understanding human activities and actively closing gaps in the robot’s understanding accumulated over weeks of autonomous runtime.