Source-linked AI summary
Lio -- A Personal Robot Assistant for Human-Robot Interaction and Care Applications
Justinas Miseikis, Pietro Caroni, Patricia Duchamp, Alina Gasser, Rastislav Marko, Nelija Miseikiene, Frederik Zwilling, Charles de Castelbajac, Lucas Eicher, Michael Fruh, Hansruedi Fruh
TL;DR
Care facilities face ageing-related demand and staff shortages, while existing robots often provide narrower interaction, logistics, or manipulation capabilities. Lio addresses this gap with a mobile, arm-equipped platform integrating autonomous operation, local processing, safety features, and care-oriented interfaces. Deployments in seven healthcare institutions and positive feedback support its practical use, although speech understanding, navigation in constrained environments, and some manipulation tasks remain limited.
Problem
Ageing populations and healthcare staff shortages motivate care robotics, while existing platforms often have limited task or manipulation capabilities.
Method
Lio combines a mobile platform and robotic arm with perception, memory, planning, autonomous navigation, local computing, and safety-oriented hardware and software.
Results
Lio has been deployed in seven healthcare institutions, where it performs daily routines, patient engagement, reminders, and object-manipulation tasks with positive feedback from patients, residents, and staff.
Takeaways & Limitations
Lio is presented as an all-in-one personal-care platform that combines autonomous operation, human interaction, and manipulation for healthcare and home-care applications.
Takeaways & Limitations
Language understanding, navigation in cluttered or narrow environments, tabletop picking, and some door-opening actions remain constrained by current sensing, markers, and online language services.
Abstract
from arXiv · showhide
Lio is a mobile robot platform with a multi-functional arm explicitly designed for human-robot interaction and personal care assistant tasks. The robot has already been deployed in several health care facilities, where it is functioning autonomously, assisting staff and patients on an everyday basis. Lio is intrinsically safe by having full coverage in soft artificial-leather material as well as having collision detection, limited speed and forces. Furthermore, the robot has a compliant motion controller. A combination of visual, audio, laser, ultrasound and mechanical sensors are used for safe navigation and environment understanding. The ROS-enabled setup allows researchers to access raw sensor data as well as have direct control of the robot. The friendly appearance of Lio has resulted in the robot being well accepted by health care staff and patients. Fully autonomous operation is made possible by a flexible decision engine, autonomous navigation and automatic recharging. Combined with time-scheduled task triggers, this allows Lio to operate throughout the day, with a battery life of up to 8 hours and recharging during idle times. A combination of powerful on-board computing units provides enough processing power to deploy artificial intelligence and deep learning-based solutions on-board the robot without the need to send any sensitive data to cloud services, guaranteeing compliance with privacy requirements. During the COVID-19 pandemic, Lio was rapidly adjusted to perform additional functionality like disinfection and remote elevated body temperature detection. It complies with ISO13482 - Safety requirements for personal care robots, meaning it can be directly tested and deployed in care facilities.
I. INTRODUCTION
Lio is presented as a personal robot platform designed for autonomous operation, human-robot interaction, and personal care in healthcare and home-care settings. It addresses workforce pressures and limitations of existing care robots by combining interaction, manipulation, and safety-oriented design.
- Motivation: Ageing populations, projected nursing-cost increases, and healthcare staff shortages motivate robotics for care applications.In Switzerland, the population over 80 is expected to double from 2015 to 2040, while nursing costs are expected to nearly triple.
- Related work: Existing care robots cover interaction, patient transport, monitoring, logistics, or manipulation, but many lack broad task and manipulation capabilities.Social robots may support positive emotions or therapy without assisting with tasks, while other platforms have narrower healthcare or logistics roles.
- Lio: Lio is a personal robot platform specifically designed for autonomous operation in healthcare facilities and home care.The platform builds on experience with the P-Care robot and is designed to be intrinsically safe for human-robot interaction.
- Paper scope: The paper describes Lio’s hardware, software, interfaces, algorithms, hospital-operation challenges, use cases, deployments, evaluation, and limitations.It concludes by positioning Lio within personal care and research applications and discussing future work.
II. SYSTEM DESCRIPTION
Lio combines a mobile platform, robotic arm, heterogeneous sensing, local computing, and ROS-based software to support safe interaction and manipulation in care environments. Its design emphasizes adaptable operation and direct access to robot modules.
- System architecture: Lio combines a robotic arm with a mobile platform to perform complex care tasks in existing environments without significant deployment adaptations.The platform can manipulate objects and was iteratively developed using observations and interactions with staff and patients.
- System architecture: Heterogeneous sensors support navigation, environmental understanding, and interaction, while ROS and myP provide modular communication and software control.The node-based setup supports communication, overview, control, interchangeability, and custom access to sensor data and robot control.
- Mechanical design: The mobile platform and arm are designed to reach floor objects and tabletops while maintaining a comfortable interaction height for wheelchair users.The robot’s height was constrained to avoid intimidating seated users.
- Safety and interaction: Soft artificial-leather covers provide collision protection and a friendlier appearance intended to support acceptance.Most of the body has a soft feel compared with hard-shell robots.
- Privacy and computing: Four embedded computing units allow complex algorithms, including deep-learning methods, to run locally without transferring data outside the robot.The computing units communicate over an internal secure Ethernet network, with additional modules integrated through ROS topics.
B. Mobile Platform and Navigation
Lio’s mobile platform integrates differential-drive mobility, redundant sensing, mapping, localisation, obstacle handling, and autonomous charging. Its interfaces support interaction and operational control during deployment.
- Platform: The 790x580 mm differential-drive platform is sized for wheelchair-prepared environments and can turn in small spaces.Two caster wheels support manoeuvrability in constrained areas.
- Sensing and safety: LiDARs, distance sensors, mechanical bumpers, and floor sensors provide overlapping obstacle and collision detection around the platform.The front and rear LiDAR measurements are merged into a 360° scan, while bumpers stop the robot after contact detection.
- Mapping and autonomy: Lio creates facility maps from two LiDARs, uses adapted gmapping and adaptive Monte Carlo localisation, and can navigate autonomously to its charging station.Virtual objects can be added to maps to prevent access to selected areas.
- Navigation feedback: Custom waypoints and areas can be stored, while sensor information supports navigation and an LED strip communicates driving direction and robot state.
- Interaction: A non-touch display, loudspeakers, and a multidirectional microphone support status display, voice interaction, and sound-based communication.Voice and touch interaction with the robot arm were preferred over a touch-screen interface in cited use-case studies.
- Emergency handling: An emergency button releases the robot and mobile platform for manual movement and returns Lio to normal operation when released.
C. Robot Arm
Lio’s six-degree-of-freedom arm combines trajectory planning, compliant control, interchangeable end-effectors, sensing, and gripper-based interaction. These components support manipulation, collision-aware operation, and task-specific tooling.
- Robot arm: The P-Rob 3 arm has six degrees of freedom, a 3-5 kg payload, and startup calibration that takes under 3 seconds.Each joint moves by a maximum of 5° during calibration, which can be executed from any position.
- Motion planning: myP provides joint-space and tool-space trajectory planning with kinematic handling of singularities, configurations, and positional or directional constraints.A standalone C++ kinematics library enables high-frequency calculations and external-project use.
- Compliant control: A compliant position-control mode uses an adjustable-gain and adjustable-stiffness feed-forward PD controller for soft motion during human interaction and contact with objects.Users can push the robot’s head to initiate actions through compliant control.
- End-effectors: Interchangeable end-effectors connect through a one-screw mechanical interface with electrical power, signals, and gigabit Ethernet available for gripper devices.
- Gripper: The P-Grip uses soft-covered fingers to grasp and manipulate objects up to 130 mm, with proximity sensing for detection, collision avoidance, and interactive actions.Finger position and force controllers support grasping, while sensor readings and finger positions can be used to learn object classes.
- Gripper: The gripper supports interchangeable fingers, an under-finger camera providing 30 FPS video, programmable-button input, and force-sensitive detection of physical presses.
E. Software and Interfaces
Lio provides layered interfaces for users, developers, and technical operators, supporting runtime monitoring, task scheduling, remote control, and ROS-based system management.
- Software architecture: The MCM and myP architectures provide low-level control, path planning, machine learning, databases, APIs, and browser-based application development.MCM operates at a 100 Hz update cycle, while myP includes a Python IDE and user-friendly browser interface.
- Software architecture: Different myP access levels allow users to work according to granted permissions, while scripts can be copied, version-controlled, and updated remotely.
- User interfaces: The Home Interface launches scripts, switches autonomy, and enables remote control; the Nursing Interface schedules tasks and monitors robot status.
- User interfaces: Nursing Interface information includes robot location, current actions and calendar, action history and logs, camera feeds, patient data, face-recognition teaching, and manual steering.
F. Perception and Behaviour Algorithms
Lio combines reusable perception and behaviour algorithms with sensor fusion for manipulation and autonomous interaction, while some unstructured-environment tasks still require physical markers.
- Algorithm framework: Lio’s sensor-based behaviours can be enabled, disabled, or overridden so projects reuse core functionality while targeting specific improvements.
- Integrated algorithms: Integrated AI capabilities include object detection and recognition, face detection and recognition, door opening and closing, and voice recognition and synthesis.
- Sensor fusion: Grasping fuses object detection, RealSense 3D point clouds, and mobile-platform position to drive to targets and move the arm for grasping.
- Sensor fusion: Gripper-finger proximity sensors fine-tune the grasping point and confirm successful grasping.
- Current limitation: Special markers are currently required beside objects such as door handles for localisation in unstructured environments.Visual and LiDAR fusion otherwise observes door state, calculates trajectories, and confirms opening success.
G. Adaptations for COVID-19
Lio was adapted during COVID-19 to add healthcare-support functions alongside its regular routines, including contactless delivery, UV-C disinfection, and remote temperature measurement.
- Pandemic adaptations: During COVID-19, Lio added functionality for healthcare professionals while continuing its regular healthcare routines.Autonomous item delivery can be performed contactlessly for staff and patients.
- Disinfection: Lio performs room disinfection with an approved UV-C light targeting frequently touched surfaces such as door handles, switches, elevator buttons, and handrails.
- Temperature measurement: Remote elevated-body-temperature measurement uses a thermal camera on the gripper coupled with a colour camera to map detected faces between images.
- Operational reliability: Reliable operation in unpredictable elderly-care environments requires sophisticated error handling across hardware and software components.Hardware handling covers arm collisions, emergency stops, and sensor failures, with collision recovery options including pausing, resuming, or stopping movement.
III. ROBOT OPERATION
Lio’s operation combines proactive decision-making with perception, memory, planning, and facility-specific routines, while addressing privacy, connectivity, navigation restrictions, and system integration.
- Autonomous operation: The decision engine selects actions from robot-status and environmental information to control Lio’s proactive autonomy.
- Autonomous operation: Perception, memory, and planning supply the decision engine with real-time sensory, AI, introspective, and remembered information for reactive behaviour selection.
- Decision logic: SWI Prolog rules evaluate available information, then prioritize proposals against manual commands and scheduled calendar actions.
- Deployment routines: Facility routines can include distributing mail or collecting blood samples, with custom holders requested from staff before execution.
- Privacy and deployment: Visual and navigation data are processed on-board, while stored face-recognition data are anonymised and encoded so original images cannot be recovered.
- Privacy and deployment: Full-feature operation requires secure WiFi, and hospital-system integration can be challenging because standards vary, though APIs enable institution-specific integration.
B. Lio Deployment
Lio has been deployed through site-specific preparation, staff training, and adaptation across health care institutions and home care. Its deployments support autonomous institutional routines and assistive manipulation tasks for a person with paraplegia.
- Institutional deployment: Lio deployment begins with facility analysis, task definition, developer adaptation, and preparation of the client’s WiFi network.The process evaluates operating areas, navigation, traversability, and relevant environmental features such as floors, doors, rooms, and handles.
- Institutional deployment: After delivery, developers install Lio, create a facility map, designate a charging spot, and train staff in operation, restarting, charging, and interface use.
- Institutional deployment: Seven health care institutions in Germany and Switzerland use Lio for deliveries, entertainment, reminders, menu ordering, and other scheduled assistance.Examples include transporting lab samples and mail, encouraging exercises, opening doors for reminders, playing music, offering snacks, and sending menu orders to catering.
- Home care deployment: In a home deployment lasting several weeks, Lio assisted a woman with paraplegia and increased her independence during daily activities.The deployment involved a person’s home rather than an institutional setting.
- Home care deployment: The robot opened and handed over bottles using an automatic bottle opener, while smartphone control supported more complex tasks such as removing a jacket.
A. Usability Evaluation
Usability evaluation combined contextual inquiries, interviews, formative evaluations, and natural-environment testing with operational data and user feedback. The studies revealed positive engagement alongside accessibility, perception, language, navigation, and manipulation challenges.
- Evaluation approach: Usability work followed laboratory simulation with tests involving actual end-users in their natural environments.Development was guided by human factors and usability engineering, using contextual inquiries, interviews, and formative evaluations.
- Accessibility and safety: Lio’s head was not always reachable for wheelchair users because safety programming prevented it from extending beyond the mobile base.
- Operational evaluation: 85.5% was the autonomous daily delivery success rate across 186 planned deliveries from February 1 to May 8, 2020.Lio also averaged 16.8 km monthly for delivery tasks and 25.2 km during entertainment tasks, with entertainment triggered 96 times per month.
- User acceptance: Elderly people and health care staff were generally curious and open, while some users expected Lio to have a personality.An ethical evaluation described an elderly person engaging with Lio as a play companion, and passersby often patted its head.
- Speech interaction: A deeper male voice was easier for elderly people to understand, although Lio uses a male voice despite female voices being perceived as friendlier.Realistic human voices also made users more likely to reply by voice rather than through other inputs.
- Current limitations: Combined voice and tactile inputs remained unreliable for some people with disabilities and elderly users, while cluttered or narrow environments challenged navigation.
- Current limitations: Without a gripper depth camera, precise picking from higher surfaces is more complicated; door opening still requires markers, and language understanding relies on keyword matching.The language-understanding service still relies on an online service, so constant listening is disabled because of privacy concerns.
- Current limitations: Because Lio does not proactively approach people, some users’ interest declined over time, motivating gradual additions of functionality.
V. CONCLUSIONS AND FUTURE WORK
Lio combines safe manipulation, autonomous operation, configurable software, and healthcare deployments in an all-in-one platform for human-robot interaction and personal care. Future work focuses on expanding autonomy, interaction, sensing, communication, and simulation support.
- Contributions: Lio combines a mobile platform, robotic arm, intrinsic safety measures, and ISO13482 compliance for personal care and human-robot interaction.Its safety design includes padded covers, limited forces and speed, a soft mode, and advanced navigation and behaviour algorithms.
- Deployments and evaluation: Lio has been deployed multiple times in healthcare institutions, receiving positive feedback from patients, residents, and medical staff.Autonomous operation also allows deployment after only brief staff training.
- Manipulation capabilities: Its arm and gripper distinguish Lio through door opening, grasping, handing over water bottles, and handling UV-C disinfection tools.Customisable inventory space allows the robot to carry and operate a broad range of tools.
- Software and research use: Researchers can enable or disable onboard basic, navigation, perception, and AI algorithms according to project needs.This lets developers reuse existing algorithms while focusing improvements on their own areas of expertise.
- Interfaces and integration: Multiple user interfaces support both inexperienced and tech-savvy users, while TCP communication enables integration with external systems.Users can observe robot status and schedule or adjust its behaviour.
- Future autonomy: Planned development includes multi-floor mapping, elevator use, marker-less object localisation, proactive interaction, and scene understanding.One example is checking whether a person drank water after receiving it.
- Future development: Planned hardware and language improvements include a 3D gripper sensor, emotional user-facing appearance, wakeword recognition, and chatbot capabilities.A full-featured ROS and ROS2 simulation is also being developed to improve testing and pre-acquisition evaluation.