Source-linked AI summary
Multi-Robot Scanner for Automated Full-Body Dermoscopic Imaging
Valerio Franchi, Rafael Garcia, Nuno Gracias, Ricard Campos, Josep Quintana, Sandra González-Villà, Mark Ventura, Nuria Ferrera, Clément Lenoir, Josep Malvehy
TL;DR
The paper addresses the need to combine whole-body coverage with dermatoscopic detail without requiring a separate manual examination. It presents a four-cobot, non-contact scanner with automated reconstruction, lesion detection, view planning, and dermoscopic imaging, and reports performance exceeding Vectra across most clinically relevant features while remaining competitive with contact dermoscopy for most features. The authors conclude that the system is technically feasible but requires further clinical validation and workflow optimization.
Problem
Commercial total-body photography lacks dermoscopic resolution, so lesions identified during screening still require separate manual handheld examination.
Method
The paper develops a four-cobot scanner with dermoscopic cameras, 3D reconstruction, automatic mole detection, view planning, and comparisons with contact dermoscopy and Vectra.
Results
The scanner consistently outperformed Vectra across most clinically relevant lesion features and achieved competitive quality with contact dermoscopy for most features assessed.
Takeaways & Limitations
Automated full-body dermoscopic acquisition can integrate total-body photography and dermoscopic imaging into one non-contact workflow, reducing reliance on separate manual dermoscopic examinations.
Takeaways & Limitations
Quantitative acquisition-time and clinical-workflow efficiency comparisons remain future work, and further validation is needed before clinical deployment.
Abstract
from arXiv · showhide
This paper outlines the specifications and design approach used to construct a full body imaging scanner capable of capturing skin lesions at a dermatoscopic level using cameras mounted on the end-effectors of four UR10 manipulators. The system possesses a view-planning algorithm capable of appropriately selecting the best camera position to acquire images of moles, a high-level controller to allow the manipulators to work simultaneously and a collision-detector that halts the manipulators when they make contact with an object or a person. We evaluate the system through real-patient full-body scans, comparing acquired images against contact dermoscopy and an existing total-body photography system (Vectra) across clinically relevant lesion features, and quantify true optical resolving power using a USAF 1951 resolution target, yielding a smallest resolvable feature size of 22.1 microns for our scanner compared to 8.8 microns for contact dermoscopy. Results show the scanner consistently outperforms Vectra across most clinically relevant features and achieves comparable performance to contact dermoscopy for the majority of features assessed. By acquiring dermatoscopic-quality images automatically and without contact, and without requiring a separate manual dermoscopic examination, the scanner closes part of the gap between total-body photography and handheld dermoscopy, suggesting potential for future integration into screening workflows.
1. Introduction
The paper addresses the gap between broad total-body photography and time-intensive handheld dermoscopy by presenting and validating an automated, non-contact full-body dermoscopic imaging scanner.
- Motivation: Full-body examinations with handheld dermoscopy are time-consuming, especially for patients with many or atypical naevi.Dermatologists inspect pigmented lesions individually for melanoma signs.
- Motivation: Commercial total-body photography provides broad standardized coverage but not dermoscopic-resolution images, so suspicious lesions still require separate manual examination.
- System contribution: The proposed platform uses four collaborative robots to acquire simultaneous, non-contact dermatoscopic-level images across the patient’s skin.High-resolution cameras with liquid lenses are mounted on the cobots’ end-effectors.
- Evaluation: The system is evaluated through automated patient scans against contact dermoscopy and a commercial total-body photography system, with objective optical-resolution measurements.
2. System Overview
The scanner combines four collaborative robots, a medical bed, imaging and control hardware, and software for reconstruction, lesion detection, and dermoscopic image acquisition within defined patient dimensions.
- Subsystems: The scanner’s mechanical design integrates patient support, four cobots, end-effector imaging hardware, an electrical cabinet, a central computer, and a hospital touchscreen interface.
- Imaging workflow: The vision system first reconstructs the patient in 3D and detects pigmented lesions larger than 3 mm, then positions dermoscopic cameras 30 cm from each lesion.
- Cobot assembly: The four cobots are arranged around the patient, with each covering one body quadrant and carrying a protected vision system on its end-effector.
- Patient specifications: The scanner is dimensioned for patients 140–190 cm tall, with thorax heights of 20–45 cm and elbow-to-elbow widths of 40–50 cm.
- Patient positioning: Patients lie on the bed with their arms beside the body, opened approximately 30° and positioned near hip level.
- Cobot assembly: The UR10e CSI cobots provide a 12.5 kg payload and 130 cm reach, with safety control designed for human-robot collaboration.
2.2. Liquid Lens System
The scanner uses electrically tunable liquid lenses for rapid depth refocusing, while compensating for gravity-induced coma that can occur when the optical axis is horizontal.
- Liquid-lens operation: Liquid lenses change shape electrically, allowing rapid refocusing across different object distances without moving parts.
- Liquid-lens operation: The scanner captures focus-stacked images by varying lens power across several diopters so each skin detail is sharply focused in at least one image.The lenses change focal power within a few tens of milliseconds.
- Gravity-induced aberration: Horizontal optical-axis orientations reduce liquid-lens performance because hydrostatic pressure asymmetrically deforms the membrane and produces coma aberration.Vertical orientation distributes pressure more evenly across the lens surface.
- Gravity compensation: A second liquid reservoir and membrane passively compensate gravity-induced coma by producing an inverted deformation matched through liquid density, refractive index, and membrane stiffness.
- Optical design: The compensated tunable lens accommodates a large 1-inch-format sensor selected to reduce the number of images required for full-body capture.
2.3. Vision System
The vision system combines dermoscopic, 2D, and 3D cameras with polarized lighting to reconstruct the patient, detect lesions, and capture detailed close-up images.
- System architecture: The multicamera system combines wide-area body mapping with specialized dermoscopic imaging of individual lesions.
- Hardware components: The vision system includes an Oryx 10GigE RGB dermoscopic camera, liquid lens, polarizer, ring light, 2D RGB camera, and 3D ToF camera.
- Dermoscopic imaging: The dermoscopic camera integrates an Optotune liquid lens to enable rapid focal-length changes.
- Body reconstruction: The 2D and 3D cameras jointly generate a patient 3D reconstruction for mapping the body surface.
- Illumination: Cross-polarized lighting for the 2D and 3D cameras reduces skin reflections that could interfere with lesion visualization.
2.4. Computer Vision System
The computer vision system automates lesion-image acquisition through calibration, view planning, mole detection, and image enhancement. It combines patient reconstruction with camera-specific imaging to support dermoscopic assessment.
- The computer vision system integrates calibration, view planning, mole detection, and image enhancement for end-to-end automated lesion imaging.These modules work together to acquire skin-lesion images, detect moles, and enhance dermoscopic images for expert assessment.
- View planning uses the patient’s 3D reconstruction to acquire wide-area images for mole detection and higher-resolution models for body-surface coverage.The planner uses these reconstructions and detections to compute appropriate camera positions.
- YOLOv8 performs real-time mole detection on Triton 2D images, supporting fast lesion localization in clinical environments.The architecture was selected for its real-time object-detection capabilities.
- The Oryx liquid lens rapidly changes focal power, enabling multiple depth-focused images of the same skin area during non-contact dermoscopy.This addresses the challenge of capturing dermoscopic images across varying lesion depths.
2.5. Calibration
Calibration establishes the cameras’ intrinsic properties and the spatial relationships among cameras, robot end-effectors, and the global reference frame. Camera-specific ChAruco patterns and sequential calibration steps define the scanner’s geometry and liquid-lens focusing.
- Calibration estimates camera projection and distortion parameters together with camera-to-end-effector and robot-base poses for coordinated movement.These parameters enable a unified coordinate system for the four cobots.
- Camera-specific ChAruco patterns use 1 cm Triton squares, 3 cm Helios squares, and 0.5 cm Oryx squares to match field of view and resolution.The pattern is affixed beneath the scanner table’s removable mattress.
- The sequential process includes Triton intrinsic and eye-in-hand calibration, stereo calibration, and later calibration of the remaining camera relationships.Observing the pattern from multiple camera poses supplies the measurements for these steps.
- After camera-frame relationships are established, liquid-lens calibration maps dioptre values to physical focus distances for patient-surface imaging.The calibrated relationship supports focus selection from reconstructed patient depth.
2.6. Scanning Procedure
The scanning procedure begins with a low-resolution 3D reconstruction, detects moles, and then directs individual robots to capture focused dermoscopic images. The sequence is repeated for both patient-facing sides.
- The four robots first acquire a low-resolution textured 3D reconstruction using Triton 2D and Helios 3D cameras from predefined exploration poses.The reconstruction provides an initial approximation of the patient’s skin surface.
- Detected moles are localized in 3D, assigned surface normals, and used to compute closer camera poses and robot-specific acquisition sequences.Each robot is commanded to visit different mole targets before Oryx imaging.
- The system estimates skin depth relative to the Oryx camera from the reconstructed model or an additional local 3D capture when the model is noisy.This depth estimate determines the focus range for image acquisition.
- Liquid-lens settings generate a stack of images at varying focal depths, which are combined by focus stacking into one sharp image for each detected mole.The process repeats until all detected moles have been captured.
- After one patient-facing side is scanned, the operator asks the patient to turn and the system repeats the same sequence.
2.7. Anonymized 3D Avatar
The anonymized 3D-avatar system fits a deformable human model to the reconstructed body so lesion locations can be interpreted semantically and compared across scans. It also supports patient-data anonymization through generic avatar representations.
- A deformable human model adds semantic body-topology understanding to the raw 3D skin mesh.This makes interactions with scan data more meaningful than using geometry alone.
- The DHM supports anatomical queries within an examination and standardized lesion-location comparisons across longitudinal scans.Examples include retrieving moles on a named body region and tracking existing or newly detected lesions.
- The fitted patient-specific body shape can be transformed into a generic avatar, preserving patient-data anonymity during on-screen display.A generic pose and shape can be shown when patients are uncomfortable viewing their naked body.
- SMPL represents the body with a template mesh and associated skeleton whose shape and pose parameters deform the model.The model was selected as the basis for the deformable human representation.
- SMPL fitting is divided into sequential pose and shape-related optimization steps because jointly optimizing all model parameters is impractical.OpenPose detections from multiple rendered views estimate pose, while optimization minimizes joint and mesh discrepancies under anatomical constraints.
2.8. Interface for the Operator
The scanner provides touchscreen and physical controls for operating scans, switching modes, and activating safety functions. Remote mode guides the operator through patient setup and scan initiation, while emergency stops halt actuators and engage brakes.
- Operator interface: The 24-inch touchscreen HMI connects to the vision system and lets the operator select the scanner’s operational mode.Local mode supports manual functions and maintenance; Remote mode lets the computer vision system control the scanner.
- Operator interface: In Remote mode, the interface requests the patient ID, prompts patient positioning, and starts the full-body scan after operator confirmation.The operator presses OK once the patient is settled and ready.
- Physical interfaces: The Bosch Easy Panel provides one-click access to controls for powering actuators, selecting automatic or homing modes, acknowledging errors, and starting operations.It is positioned next to the mattress for convenient operator access.
- Safety functions: Emergency-stop controls immediately halt all actuators and engage the brakes on every axis until the system is rearmed.Multiple emergency-stop buttons are distributed around the machine.
- Physical interfaces: Pause and Stop controls respectively suspend the current process or terminate the scanning process.A paused process resumes when the operator presses Play again.
3. Experiments and Results
The experiments compare scanner images with contact dermoscopy and Vectra using optical-resolution testing and dermatologist assessment of lesion features. The scanner matched dermoscopy for many features and generally outperformed Vectra, despite lower true optical resolving power than contact dermoscopy.
- Data collection: Native images use a GPEN-based pipeline at ≈68 px/mm, whereas SR images use Real-ESRGAN processing at ≈137 px/mm.Both pipelines include sharpening, super-resolution, and contrast enhancement.
- Data collection: The dataset contains 156 lesions captured with Vectra, contact dermoscopy, and the scanner’s Native and SR image modalities.Lesion sizes range from 0.7 mm to 16 mm, with an average of 3.2(19) mm.
- Optical resolution: 22.1 µm was the scanner’s true optical resolution versus 8.8 µm for contact dermoscopy, with identical values for Native and SR.The USAF 1951 target measures true optical resolution rather than sensor pixel sampling.
- Feature evaluation: Dermatologists assessed 17 clinically relevant lesion features across structural characteristics, pigmentation colours, and vascular structures.Features were rated as excellent, good, mild, or not visible.
- Scanner versus contact dermoscopy: Compared with contact dermoscopy, the scanner outperformed for shiny white streaks and inverted network, while both modalities were comparable for several other features.Dermoscopy remained superior for regression areas, peppering, vascular structures, and light-brown pigmentation.
- Scanner versus Vectra: Across most assessed features, scanner images outperformed Vectra, especially for dots, globules, pigment network, streaks and pseudopods, inverted network, and vascular structures.Vectra performed slightly better for light-brown coloration, while peppering was comparable.
4. Conclusions and Future Work
The scanner combines four robotic manipulators, multimodal imaging, and software for automated, near-dermoscopic full-body acquisition. Results show stronger lesion-feature imaging than Vectra and competitive quality with contact dermoscopy, while validation, workflow efficiency, and clinical deployment remain future work.
- Across clinically relevant features, scanner images consistently outperformed Vectra, while Vectra performed better only for light-brown pigmentation and was broadly comparable for peppering.
- Native and SR scanner pipelines achieved competitive quality against contact dermoscopy for most features, although dermoscopy remained superior for regression areas, peppering, vascular structures, and light-brown pigmentation.
- Automated acquisition integrates total-body photography and dermoscopic imaging into one workflow, reducing the need for separate manual dermoscopic image acquisition.
- The contact-free system may benefit infection control and patient comfort, while avatar-based anatomical references support anonymized longitudinal lesion monitoring.
- Quantitative acquisition-time and workflow-efficiency comparisons remain future work, alongside larger clinical validation, workflow optimization, and improvements in image quality and speed.
CRediT authorship contribution statement
The supplied authorship statement assigns contributions across methodology, software, validation, investigation, analysis, data curation, writing, supervision, project administration, and funding acquisition.
- Contributions span methodology, software, validation, formal analysis, investigation, data curation, writing, visualization, supervision, project administration, and funding acquisition.