Source-linked AI summary
FollowUpBot: An LLM-Based Conversational Robot for Automatic Postoperative Follow-up
Chen Chen, Jianing Yin, Jiannong Cao, Zhiyuan Wen, Mingjin Zhang, Weixun Gao, Xiang Wang, Haihua Shu
TL;DR
Postoperative follow-up is labor-intensive and error-prone, while existing digital approaches can be inflexible or risk privacy. FollowUpBot combines robotic navigation, privacy-preserving multimodal LLM conversations, and structured report generation, achieving 100% symptom coverage versus 53.8% for WiNGPT2 and improving report accuracy across field types.
Problem
Bedside postoperative follow-up is labor-intensive and error-prone, motivating high-quality, labor-efficient support for monitoring recovery and complications.
Method
FollowUpBot integrates robotic navigation, adaptive privacy-preserving multimodal LLM conversations, and automatic structured report generation.
Results
100% symptom coverage versus 53.8% for WiNGPT2, with higher satisfaction and improved report-generation accuracy across multiple field types.
Takeaways & Limitations
FollowUpBot demonstrates high-coverage, high-satisfaction postoperative interactions and accurate structured report generation in simulated patient evaluations.
Abstract
from arXiv · showhide
Postoperative follow-up plays a crucial role in monitoring recovery and identifying complications. However, traditional approaches, typically involving bedside interviews and manual documentation, are time-consuming and labor-intensive. Although existing digital solutions, such as web questionnaires and intelligent automated calls, can alleviate the workload of nurses to a certain extent, they either deliver an inflexible scripted interaction or face private information leakage issues. To address these limitations, this paper introduces FollowUpBot, an LLM-powered edge-deployed robot for postoperative care and monitoring. It allows dynamic planning of optimal routes and uses edge-deployed LLMs to conduct adaptive and face-to-face conversations with patients through multiple interaction modes, ensuring data privacy. Moreover, FollowUpBot is capable of automatically generating structured postoperative follow-up reports for healthcare institutions by analyzing patient interactions during follow-up. Experimental results demonstrate that our robot achieves high coverage and satisfaction in follow-up interactions, as well as high report generation accuracy across diverse field types. The demonstration video is available at https://www.youtube.com/watch?v=_uFgDO7NoK0.
I. INTRODUCTION
FollowUpBot addresses labor-intensive, error-prone postoperative follow-up and limitations of existing digital approaches by integrating autonomous navigation, adaptive privacy-preserving interaction, and structured report generation. Experiments on 100 synthetic postoperative cases show complete symptom coverage, higher satisfaction than WiNGPT2, and improved report accuracy with targeted ablations.
- Motivation: Traditional bedside follow-up is labor-intensive and error-prone because hospitals rely on nurses to visit patients and manually record recovery conditions.The need is intensified by increasing surgery volumes and limited clinical staff.
- Limitations of Existing Approaches: Existing digital platforms ease nurses’ workload but lack physical embodiment, depend on cloud services, and often use inflexible scripted interactions.These limitations reduce patient engagement, raise privacy concerns, and prevent adaptation to patients’ specific conditions and emotional responses.
- System Contributions: FollowUpBot integrates dynamic obstacle-avoiding navigation, adaptive privacy-preserving follow-up, and automatic structured report generation.Its follow-up module personalizes medical responses using patients’ clinical profiles and real-time interaction data.
- Evaluation: 100 synthetic postoperative cases were used to evaluate follow-up interaction quality and report generation accuracy.The evaluation used GPT-4o simulated patients.
- Results: 100% symptom coverage and higher satisfaction scores than the WiNGPT2 baseline were achieved in interactions with GPT-4o simulated patients.Ablation studies found that NLI-based Answer verification and explicit field tracking improved report accuracy and formatting consistency.
II. RELATED WORKS
Digital health technologies support postoperative follow-up through symptom tracking, medication reminders, telemedicine, wearables, and emerging LLM-based systems. However, existing approaches face uneven engagement, measurement-accuracy concerns, and adherence limitations.
- Digital health technologies: Mobile health applications support postoperative care by tracking symptoms and reminding patients to take medication, helping streamline nurses’ follow-up work.These capabilities have been reported in recent digital health research.
- Digital health technologies: Telemedicine platforms became important for postoperative care during the COVID-19 pandemic but showed uneven engagement across demographic groups, especially among elderly patients.The passage identifies demographic variation as a limitation of telemedicine-based follow-up.
- Digital health technologies: Wearable devices enable continuous vital-sign tracking and early complication detection through passive, low-effort monitoring.They may help address engagement gaps across patient populations.
- Digital health technologies: In real-world clinical settings, wearables are limited by concerns about measurement accuracy and long-term patient adherence.These concerns constrain their effectiveness for postoperative monitoring.
- Digital health technologies: AI-based follow-up systems powered by large language models show strong potential for postoperative care.The supplied passage introduces this potential without specifying particular system capabilities or evaluation results.
III. FOLLOWUPBOT · A. Overview and Workflow
FollowUpBot is an end-to-end robotic system for automating in-hospital postoperative follow-up through navigation, adaptive privacy-preserving interaction, and local report generation. Its closed-loop workflow maps the ward, visits assigned patients, collects structured follow-up fields, and stores completed reports locally.
- A. Overview and Workflow: FollowUpBot integrates automatic navigation, adaptive privacy-preserving follow-up, and automatic report generation in one end-to-end robotic system.It uses an RGB camera, touchscreen, audio I/O, LiDAR, an RGB-D camera, and dual edge devices for local real-time inference and control.
- A. Overview and Workflow: At startup, the robot builds a SLAM-based 3D ward map and waits for new follow-up tasks.Tasks are issued by the Operation Room Information System (ORIS).
- A. Overview and Workflow: After receiving a task, the robot retrieves the patient’s ID, bed number, and surgery details, then plans a global bedside path while avoiding obstacles dynamically.The task information comes from ORIS.
- A. Overview and Workflow: At the bedside, a local medical LLM sequentially queries prioritized follow-up fields and adapts the dialogue to patient profiles and real-time responses.Example fields include headache, dizziness, and nausea.
- A. Overview and Workflow: Patients can interact with the robot through speech, touch, or text inputs during the adaptive follow-up dialogue.The robot maintains a prioritized list of follow-up fields and queries each field sequentially.
- A. Overview and Workflow: Field descriptions guide content extraction, while an NLI module maps free-form outputs to valid options for strict single-choice and numerical formats.Confirmed fields are marked complete before the workflow advances to the next field.
- A. Overview and Workflow: After all required fields are completed, the robot generates a hospital-template report, saves it locally, checks for remaining tasks, or returns to standby.This closed-loop workflow supports safe navigation, adaptive interaction, and report generation.
B. Automatic Navigation Module
FollowUpBot autonomously navigates hospital wards using multimodal sensing, online SLAM, and hierarchical path planning. It identifies target beds and plans routes while accounting for environmental constraints.
- Sensing and Mapping: FollowUpBot uses 2D LiDAR, an RGB-D camera, wheel encoders, and IMU-assisted localization for mobile-robot navigation.These sensors support localization and environmental perception in the hospital ward.
- Sensing and Mapping: It performs online SLAM by fusing LiDAR, RGB-D, and IMU data to construct a semantic 3D map identifying beds, walls, and restricted zones.The semantic map represents clinically relevant landmarks and navigational constraints.
- Path Planning: After receiving a task, the robot locates the target bed and initiates automatic navigation using a hierarchical planner.The planner computes a global path through A*-based search on a topological graph, with costs adjusted by distance.
C. Follow-up Interaction and Report Generation Modules
FollowUpBot combines a locally deployed medical LLM with multimodal patient interaction, dynamic field tracking, and template-aware verification to automate privacy-preserving postoperative follow-up. After required fields are completed, a report LLM generates and stores a structured hospital-format report locally.
- Multimodal, Privacy-Preserving Follow-up: At the bedside, FollowUpBot conducts personalized conversations using a locally deployed medical LLM and accepts speech, touch, and text responses.Patient information from ORIS, such as age and surgery type, personalizes interactions; replies are rendered on-screen and through speech synthesis.
- Field Tracking for Structured Completion: The robot dynamically selects unfilled template fields, guides response elicitation, and uses a report LLM to extract field values from dialogue.Each field includes a label, type, and description, enabling structured completion across the hospital’s follow-up template.
- Answer Verification via NLI: The nli-deberta-v3-base cross-encoder maps free-form answers to predefined options by selecting the option with the highest entailment score.This NLI-based verification supports semantic correctness and format consistency for strict-format fields.
- Report Generation and Completion: After all required fields are verified and completed, the report LLM generates a structured report in the hospital’s format and stores it locally.The workflow processes fields sequentially and combines on-device LLMs, dynamic tracking, and template-aware postprocessing.
IV. DEMONSTRATIONS
Figure 2 presents two main robot interfaces: a Patient Interaction Interface for multimodal postoperative follow-up and a Report Generation Interface for concise postoperative data records.
- Robot interfaces: The Patient Interaction Interface supports speech, touch, and text interactions through a patient panel, real-time dialogue module, and input area.The patient panel displays basic information, while the dialogue module presents follow-up conversations in real time.
- Robot interfaces: The Report Generation Interface provides a concise record of postoperative data.
V. EXPERIMENT AND EVALUATION
FollowUpBot was evaluated through real-world hospital deployment and a synthetic dataset of 100 postoperative follow-up cases. The evaluation also examined patient satisfaction across six aspects and report-generation performance by field type.
- Real-world deployment: FollowUpBot successfully navigated inpatient wards and completed automatic follow-up with real patients at Guangdong Provincial People’s Hospital.This demonstrated clinical feasibility in a real-world hospital environment.
- Quantitative evaluation: A synthetic dataset of 100 postoperative follow-up cases was constructed using GPT-4o to quantitatively evaluate robot components.Each case contained a patient profile, a multi-turn dialogue with field-level annotations, and a structured report.
- Evaluation metrics: The evaluation included simulated patient satisfaction across six aspects.This assessment is summarized in Fig. 4.
- Evaluation metrics: Report-generation performance was assessed by field type.The results are presented in Table I.
A. Follow-up Interaction Quality
FollowUpBot was evaluated in simulated follow-up dialogues against WiNGPT2 using 100 patient profiles. It achieved complete clinical-symptom coverage, substantially exceeding the prompting-only baseline.
- Evaluation setup: The evaluation simulated follow-up interactions with GPT-4o using 100 patient profiles.FollowUpBot was compared with the prompting-only baseline WiNGPT2.
- Evaluation metrics: Coverage measures the proportion of clinically required symptoms addressed during each follow-up dialogue.The evaluation also assessed simulated patient satisfaction after each dialogue.
- Coverage results: 100% coverage was achieved by FollowUpBot, compared with 53.8% for WiNGPT2.This result indicates that FollowUpBot addressed all clinically required symptoms in the simulated dialogues, whereas the baseline addressed just over half.
B. Report Generation Accuracy · VI. CONCLUSION
FollowUpBot combines multimodal postoperative follow-up with structured clinical report generation. Ablation results show that NLI alignment and field tracking improve report-generation performance, including strong accuracy on single-choice and numerical fields.
- B. Report Generation Accuracy: Ablation studies evaluate field-specific descriptions, NLI-based option matching, and explicit field tracking.Each experiment is repeated five times on a 100sample dataset, with results averaged.
- B. Report Generation Accuracy: NLI alignment significantly improves accuracy, especially for structured fields.
- B. Report Generation Accuracy: 91.44% accuracy and 0.9912 BERTScore F1 are achieved on single-choice fields with field tracking.
- B. Report Generation Accuracy: 99.20% accuracy and 0.0300 MAE are achieved on numerical fields with field tracking.
- VI. CONCLUSION: FollowUpBot automatically conducts postoperative follow-up through multimodal interaction and generates structured clinical reports with high accuracy.
- VI. CONCLUSION: The robot integrates real-time navigation, a medical LLM, field-aware dialogue tracking, NLI-based output normalization, and report generation into one pipeline.
- VI. CONCLUSION: Experimental results show high coverage and satisfaction in follow-up interaction and improved report generation.