Source-linked AI summary
Deep Learning in Robotics: A Review of Recent Research
Harry A. Pierson, Michael S. Gashler
TL;DR
Robotics research has rapidly adopted deep learning, but evidence about its use with physical robotic systems remains distributed across applications, benefits, and limitations. This review synthesizes contemporary research, identifies major robotics challenges, and discusses DNN structures and practical considerations. It concludes that deep learning shows promise across sensing, cognition, and action while significant obstacles remain.
Problem
Robotics needs methods that can learn complex, high-dimensional, and novel dynamics without time-consuming expert derivation, which remains difficult for physical systems.
Method
The review synthesizes contemporary robotics research, discusses common DNN structures and practical considerations, and identifies major challenges and mitigation strategies.
Results
Deep learning has shown promise in sensing, cognition, and action, with DNNs able to process raw sensor data and fuse high-dimensional, multimodal data.
Takeaways & Limitations
The review communicates recent advances and highlights deep learning’s potential for broader application in robotics.
Takeaways & Limitations
Deep learning still has many obstacles to overcome, including barriers to adoption in robotics and dangers in some applications.
Abstract
from arXiv · showhide
Advances in deep learning over the last decade have led to a flurry of research in the application of deep artificial neural networks to robotic systems, with at least thirty papers published on the subject between 2014 and the present. This review discusses the applications, benefits, and limitations of deep learning vis-à-vis physical robotic systems, using contemporary research as exemplars. It is intended to communicate recent advances to the wider robotics community and inspire additional interest in and application of deep learning in robotics.
1 University of Arkansas
This review examines recent applications of deep artificial neural networks to physical robotic systems, discussing their applications, benefits, and limitations. It aims to communicate advances and encourage further robotics research.
- At least thirty papers on deep learning applications in robotic systems were published between 2014 and the present.
- The review uses contemporary research as exemplars to discuss deep learning’s applications, benefits, and limitations for physical robotic systems.
- The paper aims to communicate recent advances to the wider robotics community.
- It also aims to inspire additional interest in and application of deep learning in robotics.
1. Introduction
The introduction presents deep neural networks as flexible models for robotics and outlines how the review organizes their structures, applications, practical considerations, and limitations.
- Deep neural networks can model complex functions such as nonlinear dynamics and form compact representations.
- The review explains common DNN structures used in robotics and illustrates them with examples from recent literature.
- It provides practical considerations for roboticists wishing to use DNNs.
- The paper discusses limitations, mitigation strategies, and future directions for deep learning in robotics.
2. Deep learning
The section traces neural networks from earlier regression and robotics-control applications to modern deep learning, then introduces common DNN structures and training approaches.
- History: Neural networks supported robotics control as early as the 1980s, including nonlinear regression for dynamical systems and ALVINN’s lane-keeping demonstration.
- History: GPUs enabled parallel neural-network computation, while Hinton’s 2006 training method supported many-layered networks and helped trigger renewed research interest.
- History: Earlier work showed neural networks could self-organize control schemas, model locomotion, and learn robot state and dynamics from raw images.
- Training: Neural networks are trained on example pairs by minimizing prediction loss, with stochastic gradient descent, RMSProp, and Adam among the optimization methods discussed.
- Common DNN structures: Structure A uses a DNN to regress arbitrary functions, while Structure B uses an encoder-decoder pair for unsupervised learning from x.
- Common DNN structures: Autoencoders compress observations into latent state representations and can autonomously identify useful values when designers do not know them in advance.
- Common DNN structures: Structure C is a recurrent neural network that models dynamic systems by representing controls, future state, anticipated observations, and state transitions over time.
3. Deep learning in robotics
The paper identifies seven robotics challenges where deep learning may have high impact and reviews how DNN structures address them. It emphasizes both current progress and important gaps, especially in dynamics, manipulation, and planning.
- The paper identifies seven general robotics challenges critical for reaching future goals and areas where DNN technology has high potential for impact.
- Challenge 1: Learning complex, high-dimensional, and novel dynamics remains difficult because analytic derivations require human experts, are time consuming, and pose danger in some applications.
- Challenge 2: Robotic systems must learn control policies for dynamic, uncertain environments while accommodating high degrees of freedom and limited state information.
- Challenge 3: Robust, general solutions remain elusive for advanced manipulation involving deformable objects, complex geometries, tools, and environmental actuation in novel situations.
- Challenge 4: Advanced recognition requires estimating deformable-object pose and state, specifying semantic tasks and paths, and recognizing hazardous object or surface properties.
- Challenge 5: Interpreting human actions supports collaborative robotics, teaching by demonstration, and task specification by people without robotics or automation expertise.
- Challenge 6: Sensor fusion and dimensionality reduction seek meaningful state representations from rich, high-dimensional, multimodal sensing data.
- Challenge 7: High-level task planning remains a limited DNN application, and standard benchmarks are needed to quantify decision quality.
3.1 Classifiers and discriminative models (Structure A) in robotics
Structure A uses deep learning to approximate robotics functions from sample observations and to perform classification, including perception, grasping, and scene understanding. Across reviewed applications, these models achieved strong recognition, grasping, navigation, and prediction results while reducing reliance on hand-engineered features.
- The role of Structure A in robotics: Structure A learns mappings such as actions to state changes, state changes to actions, and forces to motions from sample input-output pairs.It is useful when physical equations are unavailable or too complex to achieve acceptable accuracy.
- Detection and perception: Deep neural networks can operate directly on high-dimensional inputs, reducing dependence on human experts who design application-specific feature vectors.This direct processing is a major attraction in detection and perception applications.
- Detection and perception: Visual door detection was passed to a navigation algorithm, and the authors suggested this approach can be superior to map-building in dynamic environments.The result links learned perception directly to mobile-robot navigation.
- Grasping and object manipulation: 84% and 89% grasp success rates on Baxter and PR2 exceeded 31% for a state-of-the-art reference algorithm.The network selected successful grasps from RGB-D images without prior object-geometry knowledge.
3.2 Generative and Unsupervised models (Structure B) in robotics
Structure B uses generative and unsupervised models to learn compact representations of high-dimensional observations, predict observations or physical outcomes, and support sensorimotor control. Autoencoders, recurrent integrations, and latent-state methods address complex dynamics and sensor-fusion problems without requiring fully analytic models.
- Autoencoders and latent representations: Autoencoders encode observations into compact internal representations and decode them back to the original observations.They are primarily used when high-dimensional observations are available but a low-dimensional state representation is desired.
- Generative models: Generative models use decoding to map succinct representations into full high-dimensional observations and can predict observations or physical outcomes.They were applied to physics simulations and nonlinear dynamics of simple physical systems.
- Sensorimotor integration: Unsupervised and recurrent methods integrated RGB images, sound data, and joint angles for visuomotor control.The reported system outperformed the state of the art in real-time learning evaluations and converged quickly with noisy sensor data.
- Dynamics modeling: A fading-memory feature enabled inverse-dynamics adaptation as payload and mechanical wear changed.Analytic dynamic models have difficulty coping with such changes and with highly compliant serial-elastic manipulators.
- Practical recommendations: Convolutional autoencoders tend to be especially effective for estimating state from robot images.Convolutional layers are advantageous in the encoding portion, but offer little advantage for decoding.
- Practical recommendations: Predicting a single pixel experimentally seems to lead to much faster training times.The approach parameterizes the decoder so the user can specify which pixel it should predict.
3.3 Recurrent models (Structure C) in robotics
Structure C uses recurrent connections as memory for modeling time-dependent state and complex dynamics. Applications include traffic prediction, food cutting, gesture-guided manipulation, and multimodal control, but recurrent networks remain difficult to train because temporal unfolding produces vanishing gradients.
- The role of recurrent models: Recurrent connections provide memory that helps models remember current state and represent effects of time in changing environments.This makes recurrent models suitable for anticipating complex dynamics.
- Applications: Recurrent models addressed robotic food cutting involving friction, deformation, hysteresis, and changing food-knife contact.These effects are difficult to model directly because surface contact changes throughout the cut.
- Applications: Model-predictive control driven by learned dynamics increased mean cutting rate from 1.5 cm/s to 5.1 cm/s over fixed-trajectory stiffness control.Training data came from operation under fixed-trajectory stiffness control.
- Applications: A recurrent model coordinated gesture recognition with attention switching, object perception, and grasping, achieving an 85% successful-grasp rate in simulation.The simulated robot focused on a human collaborator’s gesture, selected the indicated object, and found an acceptable grasp.
- Limitations: Recurrent models are difficult to train because temporal unfolding makes them effectively deeper, causing gradients to become vanishingly small.The error-carousel idea in LSTM networks largely solved this problem.
- Practical recommendations: Inferring intrinsic state from images can remove the need for recurrent processing when state is accurately inferred.Even imperfect state inference can provide pre-training that significantly reduces recurrent-model training time.
3.4 Policy learning models (Structure D) in robotics
Structure D learns control policies through reinforcement learning, maximizing long-term reward rather than minimizing prediction error. These models can control dynamic systems efficiently at runtime and handle complex tasks, but require extensive computation and training before reliable policies converge.
- Capabilities: Reinforcement-learning controllers operate from partial state information, infer priorities from rewards, and can be computationally efficient at runtime.The models are intended to learn near-optimal or reasonably acceptable control policies.
- The role of Structure D: Deep Q-networks model action values and seek policies that maximize long-term reward through exploration and exploitation.This training objective differs from minimizing prediction error against a fixed sample set.
- Practical benefits: Using fewer onboard sensors can reduce power consumption, cost, and weight while increasing mobile-robot range or performance.The stated basis is that these systems need not access full state information.
- Applications: A learned-cost-function pipeline completed tasks involving complex second-order dynamics and hard-to-model manipulator-object interactions.Demonstrated tasks included navigation, reaching, peg insertion, placing a dish, and pouring.
- Visuomotor control: End-to-end visuomotor reinforcement learning performed tasks reliably even with moderate visual distractors.Applications included shape sorting, screwing a cap onto a bottle, fitting a hammer claw to a nail, and placing a coat hanger on a rack.
- Limitations and recommendations: Reinforcement-learning models require significantly more training pattern presentations before converging to reliable control policies.Despite slow training, they are highly efficient after training; simulation can reduce wear on physical equipment and training time.
4. Current Shortcomings of DNNs for Robotics
Deep neural networks pose practical challenges for robotics, including large data and training-time demands, computational constraints, and safety concerns. Researchers mitigate these issues through simulation, data manipulation, staged networks, and off-board training.
- Data requirements: Simulation can generate extensive training data and physics-based grasp-quality labels, which researchers validate against human classifications.One simulated database contained over 300,000 grasps on more than 700 objects, and computer- and human-generated labels showed good correlation.
- Data requirements: Digital manipulation expands datasets without acquiring additional samples by simulating variation such as different conversational-gesture speeds.Neverova et al. varied video playback speed to model temporal variance among people.
- Training and computation: DNNs may involve millions of parameters and take days to train, making them practical mainly for frequently repeated tasks with adequate payback.Training-time costs arise from model size and the expense of generating data on physical systems.
- Training and computation: Multiple smaller DNNs can reduce training and runtime costs by dividing functions, while staged grasp detection trades some first-stage accuracy for computational efficiency.Two-stage designs first eliminate unlikely grasps, then apply a larger network to the remaining candidates; reported combinations were robust and computationally efficient.
- Safety and hardware constraints: Deep learning is difficult to deploy on weight-, power-, and cost-sensitive robots, and unsupervised learning is unsafe where a single failure is catastrophic.Aerial systems addressed resource constraints by training through a ground-based wireless control system with off-board computing, making training safe and automatic.
5. Conclusion
Deep learning offers robotics systems capabilities in sensing, cognition, action, multimodal data fusion, and adaptation, while significant data, training-time, computational, and generalization barriers remain.
- Deep learning has shown promise across sensing, cognition, and action in physical robotic systems.
- DNNs can process raw sensor data, infer features without human assistance, and fuse high-dimensional multimodal data.These properties may reduce up-front engineering time and support integrated robotic systems.
- Performance can improve with experience, facilitating adaptation in dynamic, unstructured robotic environments.
- Training remains constrained by large data requirements, long training times, and the expense of generating physical-system data.Cloud robotics, non-robot cooking videos, local parallel processing, and distributed computing are discussed as possible responses, although communication speeds can limit distributed approaches.
- DNNs remain vulnerable to adversarial samples and struggle with 3D spatial layouts involving object invariance.
- DNNs are powerful robotics tools, but their ability to deliver dexterous adaptation in general environments remains unresolved.The review notes that cognitive training datasets currently do not exist and that human-level abilities remain an ambitious future objective.