PaperlayerView exact revision on arXiv

Source-linked AI summary

Artificial Intelligence for Long-Term Robot Autonomy: A Survey

Lars Kunze, Nick Hawes, Tom Duckett, Marc Hanheide, Tomáš Krajník

arXiv:1807.05196v1cs.ROcs.AI

TL;DR

Robots must operate autonomously for weeks, months, or years in complex, changing, and partly unknown real-world scenarios. The paper surveys AI techniques and their integration in long-running systems, finding that navigation and perception are universal while interaction, learning, and system-level integration remain open challenges.

  • Problem

    Long-term autonomy requires robots to handle complex, changing, and partly unknown real-world environments, tasks, and tools over extended periods.

  • Method

    The paper surveys AI techniques for long-term autonomy, deployed real-world systems, promising laboratory or dataset methods, and future challenges.

  • Results

    Navigation and mapping and perception were present in all surveyed systems, while interaction and learning were only partially or not supported in most domains.

  • Takeaways & Limitations

    AI techniques must be reintegrated within autonomous systems to address long-term operation across diverse real-world domains.

Abstract

from arXiv · show

Autonomous systems will play an essential role in many applications across diverse domains including space, marine, air, field, road, and service robotics. They will assist us in our daily routines and perform dangerous, dirty and dull tasks. However, enabling robotic systems to perform autonomously in complex, real-world scenarios over extended time periods (i.e. weeks, months, or years) poses many challenges. Some of these have been investigated by sub-disciplines of Artificial Intelligence (AI) including navigation & mapping, perception, knowledge representation & reasoning, planning, interaction, and learning. The different sub-disciplines have developed techniques that, when re-integrated within an autonomous system, can enable robots to operate effectively in complex, long-term scenarios. In this paper, we survey and discuss AI techniques as 'enablers' for long-term robot autonomy, current progress in integrating these techniques within long-running robotic systems, and the future challenges and opportunities for AI in long-term autonomy.

I. INTRODUCTION

Long-term autonomy requires robots to remain operational in complex, changing, and partly unknown real-world settings over weeks, months, or years. This survey examines AI techniques, their integration in deployed systems, and future challenges for long-term autonomy.

  • I. INTRODUCTION: Long-term autonomy extends beyond robustness in known, static environments to changing platforms, environments, tasks, and observability conditions.The challenges vary with application requirements and with how these elements change over time, whether they are structured, and how observable they are.
  • I. INTRODUCTION: Open-world operation includes unknown environments, newly appearing objects, changing user tasks, and adaptation to new tools, techniques, or knowledge.
  • I. INTRODUCTION: The survey covers AI techniques used in real-world long-term deployments and promising techniques demonstrated only in datasets or laboratories.
  • I. INTRODUCTION: The survey excludes long-duration applications in relatively static, known settings, including manufacturing and intra-logistics.Those settings typically achieve longevity through known environments and controlled dynamics, limiting LTA-specific challenges.
  • I. INTRODUCTION: The paper presents itself as the first survey focused specifically on AI techniques for enabling long-term robot autonomy.

II. DOMAINS

Long-term autonomy spans diverse domains and requires integrating multiple AI abilities under domain-specific environmental, task, and operational conditions. Navigation and perception appear across all surveyed systems, while system-level integration, interaction, and learning remain important challenges.

  • II. DOMAINS: The survey characterizes domains by environment variability, task diversity, semantics, dynamics, partial observability, cost and criticality, interaction and cooperation, and autonomy level.Deployment duration and integration of AI areas are also assessed using days, months, or years and none, partial, or full integration.
  • II. DOMAINS: Long-term systems operate across space, marine, air, field, road, and service domains with differing autonomy requirements.Examples include Mars rovers, autonomous wave gliders, aerial systems constrained by energy, field robots, autonomous vehicles, and robots working alongside people.
  • II. DOMAINS: Multiple experience-based representations enabled long-term autonomous field navigation despite drastic changes in environmental appearance.
  • II. DOMAINS: Navigation and mapping and perception were the only AI areas present in all surveyed systems.Knowledge representation and reasoning and planning were supported by most systems, whereas interaction and learning were usually partial or absent except in service robotics.
  • II. DOMAINS: System-level integration has no standard solution and remains insufficiently researched despite being essential to combining localization, navigation, perception, planning, and scheduling.ROS and specialized frameworks ease component integration, but understanding and evaluating integrated AI methods remain major challenges.

III. AI AREAS

The paper discusses six AI areas as enablers for autonomous robots operating in real-world environments over extended periods. These areas are navigation and mapping, perception, knowledge representation and reasoning, planning, interaction, and learning.

  • III. AI AREAS: Navigation and mapping support autonomous robot operation in real-world environments over extended periods.
  • III. AI AREAS: Perception, knowledge representation and reasoning, and planning are included among the AI areas discussed for long-term autonomy.
  • III. AI AREAS: Interaction and learning are also treated as AI areas relevant to extended autonomous operation.

A. NAVIGATION & MAPPING

Long-term navigation and mapping must handle changing environments rather than assume a static world. The survey reviews complementary strategies using multiple representations, stable features, dynamic models, and long-term applications.

  • Long-term navigation: Over 140 km of autonomous driving achieved a 99.6% autonomy rate, including night-time operation, using visual teach-and-repeat navigation.The robot learns a map while being driven along a training route and then repeats the route autonomously.
  • Changing environments: Long-term mapping treats environment modelling as an ongoing process requiring decisions about what information to remember or forget.Deleting observations is risky because apparent changes may be temporary and environments may revert to earlier states.
  • Changing environments: Multiple maps at different timescales support localisation by selecting the model that best explains current sensor data.Short-term maps are updated online, while longer-term maps are adapted offline from long-term experience.
  • Robust representations: Seasonally robust features and image-sequence matching improve place recognition across appearance changes.SeqSLAM matches sequences rather than individual images, while other work learns features stable across mapping sessions.
  • Dynamic modelling: Dynamic and periodic environment models use long-term experience to predict future states, improving localisation and navigation in human-populated environments.Dynamic occupancy grids model cell occupancy as a two-state Markov process, while Fourier-based models represent rhythmic processes.
  • Applications: Long-term mapping has supported a 14-month lake-shore survey and 4D crop-monitoring reconstruction over time.The crop-monitoring system combines 3D SLAM, cross-session data association, and full 4D reconstruction optimisation.

B. PERCEPTION

Long-term perception must distinguish persistent structure from movable or changing objects and support open-ended adaptation. The survey highlights progress in object reasoning while identifying ongoing adaptation of learned models as an open challenge.

  • Object perception: Objects that are dynamic but temporarily stationary require long-term observations to distinguish movable objects from static environment structure.Several 3D point clouds recorded over weeks can identify movable objects while refining the static structure.
  • Object perception: Explicit object categories support long-term localisation by distinguishing mapped objects, unmapped static objects, and unmapped dynamic objects.
  • Lifelong learning: Open-ended perception approaches learn new object categories through web-based semantic context, human-robot interaction, or user-directed data collection.These approaches support classifier training and manipulation of newly encountered categories during long-term operation.
  • People perception: Long-term people perception can integrate experience for tracking, activity-pattern learning, and person re-identification across multiple temporal scales.Person-specific appearance-model adaptation over long periods remains an open challenge.
  • Open challenges: Most prior perception work focuses on pre-deployment training rather than ongoing adaptation of learned models during long-term operation.

C. KNOWLEDGE REPRESENTATION & REASONING

Knowledge representation and reasoning enable long-term robots to represent changing worlds, reason across space and time, and use accumulated experience. The survey also links novelty detection to adaptation and learning.

  • Foundations: Long-term robots require knowledge representations and reasoning capabilities that capture world changes over time.The survey identifies spatio-temporal reasoning, non-monotonic reasoning, and belief revision as important AI areas.
  • Spatio-temporal reasoning: Spatio-temporal models infer where entities are located across space and time and can inform AI planners.Examples model object persistence, semantic-world changes, and frequency-based patterns in people’s locations.
  • Experience memory: OPEN-EASE makes robot and human manipulation experiences semantically accessible for querying perceptions, reasoning, and actions.Other frameworks store and retrieve perceptual memories for long-term manipulation tasks.
  • Open-world reasoning: Novelty and anomaly detection supports recognition and interpretation of unfamiliar or familiar objects in unexpected contexts.Anomaly reasoning is linked to adaptation and learning because detected anomalies can trigger learning in long-term autonomy systems.

D. PLANNING

Planning and executive control adapt robot behaviour to changing tasks, environments, resources, opportunities, and failures. Long-term systems increasingly combine planning with models learned during execution and broader optimisation processes.

  • Planning: Planning and scheduling determine action sequences and adapt robot behaviour online to environment or task dynamics.Planning systems have been deployed across long-term systems including the Opportunity rover and service robots.
  • Executive control: Executive control systems manage tasks, respond to opportunities and failures, prevent behavioural loops, and provide recovery mechanisms.Long-term autonomy architectures use executives to handle aspects omitted from planning models, such as replanning after failed actions or charging on low battery.
  • Planning models: Planning systems differ in how they represent action effects, time, resources, uncertainty, and choices among multiple goals.These differences affect how systems incorporate long-term experience and handle oversubscription, including exploration choices.
  • Learning and planning: STRANDS and CoBot adapted task and navigation plans using models learned during execution.STRANDS learned MDP models of environmental dynamics, while CoBot predicted when humans would be available to help complete tasks.
  • Future directions: Planning is expected to become more prominent in logistics and autonomous transport as systems manage greater variation in environments, demand, and execution resources.

E. INTERACTION

Long-term interaction with diverse users introduces dynamics and unpredictability, while also enabling robots to learn user-specific representations and adapt their autonomous behaviour.

  • E. INTERACTION: Humans and other actors make long-running interaction scenarios dynamic and non-predictable.This creates challenges for autonomous systems interacting with diverse users over extended periods.
  • E. INTERACTION: A survey of 45 studies found both a lack of and need for more learning and adaptation in long-term interactive robots.The studies covered health care and therapy, education, work environments and public spaces, and at-home settings.
  • E. INTERACTION: Personalised user models and individualised profiles help robots maintain interaction context in long-running autonomous systems.Several systems develop personalised models, and individualised user profiles are identified as a key required ability.
  • E. INTERACTION: In-situ learning from long-term interactions can improve autonomous behaviour by adapting representations to users and usage patterns.Examples include predicting when humans are available to help and learning spatio-temporal usage patterns to maximise robot utility.

F. LEARNING

Learning during deployment is presented as crucial for long-term autonomy because open and dynamic worlds exceed what robots can know at design time. Exploration and continual-learning methods provide ways to gather experience and improve capabilities online.

  • F. LEARNING: Learning during operation is crucial because initial knowledge is unlikely to cover behaviour in open, dynamic worlds.The survey distinguishes this from learning performed only during the design phase.
  • F. LEARNING: Fixed viewpoints limit the experiences and training data robots can generate, motivating exploration to gather new observations.CoBot and STRANDS are cited as examples of systems balancing exploration with experience gathering or exploitation.
  • F. LEARNING: Continual learning from experience, including reinforcement learning and learning from demonstration, can support online capability improvement.These methods respectively learn continually or focus on experiences such as failures and novelty.
  • F. LEARNING: Limited training in a particular domain or open world can be addressed through learning approaches suited to the robot’s available experience.The passage frames diverse machine-learning techniques as potential contributors to future on-the-job learning.

IV. FUTURE CHALLENGES

Future long-term autonomy requires integrating AI techniques with human input, information sharing, dependable architectures, domain-specific knowledge, and evaluation methods for changing systems and environments. The paper also identifies broader ethical, social, and legal issues and argues that long-term experience may help address difficult robotics problems.

  • Human-in-the-Loop Systems: Human-in-the-loop systems must obtain and integrate information from users, maintainers, and domain experts when unforeseen situations arise.Inputs may include teleoperation, natural interaction, labelled examples, or datasets, including potentially conflicting or untrustworthy information.
  • Knowledge Transfer between LTA Systems: Knowledge transfer between deployed LTA systems requires deciding what information to exchange and when, while addressing privacy and security concerns.Exchanging all logged information is considered unrealistic.
  • Systems Integration: Systems integration is a software-engineering challenge because long-term operation requires close integration of different AI techniques and reliable architectures.The paper notes that ROS offers few instruments for ensuring reliable and robust system architectures, while model-based approaches may help.
  • More Domain Specialisation: Domain specialisation can create synergies between application-specific capabilities, such as crop recognition improving navigation robustness and accuracy in precision agriculture.Relative positioning and navigation may matter more than absolute positioning for crop-row tasks.
  • Verification and Evaluation of LTA Systems: Verification and evaluation must handle robot systems, models, task specifications, and environments that change at different timescales.The paper calls for records of internal models and formal guarantees under probabilistically changing environments.
  • Conclusion: AI methods may address difficult open robotics problems through long-term experience, although many challenges remain and ethical, social, and legal issues are beyond the paper’s scope.The conclusion gives perception-based mobile manipulation in real-world settings as an example of an open problem.
Loading 1807.05196v1…