Source-linked AI summary
Reinforcement Learning of Artificial Microswimmers
Santiago Muiños-Landin, Keyan Ghazi-Zahedi, Frank Cichos
TL;DR
Artificial microswimmers lacked implemented adaptive learning despite living systems adapting through environmental interactions. The paper applies reinforcement learning to real microswimmers controlled by real-time microscopy, demonstrating navigation, obstacle avoidance, and information sharing while examining Brownian-motion effects. The work provides a platform for studying adaptive and collective behavior in microsystems, but its sensing and control remain externalized to computer and microscopy hardware.
Problem
Adaptive behavior and learning had not yet been implemented in artificial microswimmers, limiting the integration of learning strategies into microscopic systems.
Method
The study uses self-thermophoretic microswimmers in a real-world gridworld, applying Q-learning through externally provided sensing, control, and real-time microscopy.
Results
The swimmers learn to navigate to a target, avoid virtual obstacles, share information, and adapt learned behavior to Brownian-motion-dependent noise levels.
Takeaways & Limitations
The demonstration provides an experimental platform for studying adaptive and collective behavior and for integrating learning strategies into microsystems.
Takeaways & Limitations
The hybrid system externalizes the microswimmers’ brain and sensing capabilities to a computer and microscopy setup, while physical or chemical signaling within a swimmer remains a distant goal.
Abstract
from arXiv · showhide
The behavior of living systems is based on the experience they gained through their interactions with the environment [1]. This experience is stored in the complex biochemical networks of cells and organisms to provide a relationship between a sensed situation and what to do in this situation [2-4]. An implementation of such processes in artificial systems has been achieved through different machine learning algorithms [5, 6]. However, for microscopic systems such as artificial microswimmers which mimic propulsion as one of the basic functionalities of living systems [7, 8] such adaptive behavior and learning processes have not been implemented so far. Here we introduce machine learning algorithms to the motion of artificial microswimmers with a hybrid approach. We employ self-thermophoretic artificial microswimmers in a real world environment [9, 10] which are controlled by a real-time microscopy system to introduce reinforcement learning [11-13]. We demonstrate the solution of a standard problem of reinforcement learning - the navigation in a grid world. Due to the size of the microswimmer, noise introduced by Brownian motion if found to contribute considerably to both the learning process and the actions within a learned behavior. We extend the learning process to multiple swimmers and sharing of information. Our work represents a first step towards the integration of learning strategies into microsystems and provides a platform for the study of the emergence of adaptive and collective behavior.
Methods
The experiments use gold nanoparticle-coated melamine resin microswimmers in a custom inverted dark-field microscopy setup with laser heating.
- Microswimmers are prepared from 2.19 µm melamine resin particles coated with gold nanoparticles covering about 30% of the surface.The gold nanoparticles are 8–30 nm in diameter.
- A custom-built inverted dark-field microscopy setup images the sample with an emCCD camera and uses a 532 nm laser focused into the sample plane for heating.The setup uses oil-immersion condenser and objective optics.