Computer Science · Literature

Research papers on Reinforcement learning in robotics

Recent and highly-cited academic work on reinforcement learning in robotics, gathered from Semantic Scholar, CrossRef and OpenAlex.

1. Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper)

   Esper, Khalil, Spieck, Jan, Sixdenier, Pierre-Louis, et al. · 2017 · arXiv (Cornell University) · 11,341 citations

   Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety

2. Reinforcement Learning: A Survey

   Leslie Pack Kaelbling, Michael L. Littman, Andrew Moore · 1996 · Journal of Artificial Intelligence Research · 8,926 citations

   This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations o

3. Playing Atari with Deep Reinforcement Learning

   Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. · 2013 · arXiv (Cornell University) · 5,106 citations

   We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.

4. Deep Reinforcement Learning: A Brief Survey

   Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, et al. · 2017 · IEEE Signal Processing Magazine · 4,407 citations

   Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey wil

5. A Comprehensive Survey of Multiagent Reinforcement Learning

   Lucian Buşoniu, Robert Babuška, Bart De Schutter · 2008 · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · 2,232 citations

   Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multiagent reinforcement learning (MARL). A central issue in the field is the formal statement of the multiagent learning goal. Different viewpoints on this issue have led to the proposal of many differe

6. Continuous control for robot based on deep reinforcement learning

   Shansi Zhang · 2019 · 949 citations

   One of the main targets of artificial intelligence is to solve the complex control problems which have high-dimensional observation spaces. Recently, the combination of deep learning and reinforcement learning has made remarkable progress, including the high-level performance in the video and board games, 3D navigations and robotic control. In this thesis, deep reinforcement learning algorithms are studied to perform some robotic tasks with continuous action spaces.

7. Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation

   Lei Tai, Giuseppe Paolo, Ming Liu · 2017 · 820 citations

   We present a learning-based mapless motion planner by taking the sparse 10-dimensional range findings and the target position with respect to the mobile robot coordinate frame as input and the continuous steering commands as output. Traditional motion planners for mobile ground robots with a laser range sensor mostly depend on the obstacle map of the navigation environment where both the highly precise laser sensor and the obstacle map building work of the environment are indispensable. We show that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be trained end-to-end without any manually designed features and prior demonstrations. The trained planne

8. Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

   Lukas Brunke, Melissa Greeff, Adam W. Hall, et al. · 2022 · Annual Review of Control Robotics and Autonomous Systems · 720 citations

   The last half decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. It includes learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safet

9. Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators

   Thomas George Thuruthel, Egidio Falotico, Federico Renda, et al. · 2018 · IEEE Transactions on Robotics · 403 citations

   Dynamic control of soft robotic manipulators is an open problem yet to be well explored and analyzed. Most of the current applications of soft robotic manipulators utilize static or quasi-dynamic controllers based on kinematic models or linearity in the joint space. However, such approaches are not truly exploiting the rich dynamics of a soft-bodied system. In this paper, we present a model-based policy learning algorithm for closed-loop predictive control of a soft robotic manipulator. The forward dynamic model is represented using a recurrent neural network. The closed-loop policy is derived using trajectory optimization and supervised learning. The approach is verified first on a simulate

10. Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

    Zhongyu Li, Xuxin Cheng, Xue Bin Peng, et al. · 2021 · 2021 IEEE International Conference on Robotics and Automation (ICRA) · 301 citations

    Developing robust walking controllers for bipedal robots is a challenging endeavor. Traditional model-based locomotion controllers require simplifying assumptions and careful modelling; any small errors can result in unstable control. To address these challenges for bipedal locomotion, we present a model-free reinforcement learning framework for training robust locomotion policies in simulation, which can then be transferred to a real bipedal Cassie robot. To facilitate sim-to-real transfer, domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. The learned policies enable Cassie to perform a set of diverse and dynamic

11. Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control

    Zhongyu Li, Xue Bin Peng, Pieter Abbeel, et al. · 2024 · The International Journal of Robotics Research · 241 citations

    This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the r

12. Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control

    Fangyi Zhang, Jürgen Leitner, Michael Milford, et al. · 2015 · arXiv (Cornell University) · 216 citations

    This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works

13. Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

    Rongrong Liu, F. Nageotte, P. Zanne, et al. · 2021 · Robotics · 202 citations

    Deep learning has provided new ways of manipulating, processing and analyzing data. It sometimes may achieve results comparable to, or surpassing human expert performance, and has become a source of inspiration in the era of artificial intelligence. Another subfield of machine learning named reinforcement learning, tries to find an optimal behavior strategy through interactions with the environment. Combining deep learning and reinforcement learning permits resolving critical issues relative to the dimensionality and scalability of data in tasks with sparse reward signals, such as robotic manipulation and control tasks, that neither method permits resolving when applied on its own. In this p

14. Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis

    Yue Wen, Jennie Si, Andrea Brandt, et al. · 2019 · IEEE Transactions on Cybernetics · 189 citations

    Robotic prostheses deliver greater function than passive prostheses, but we face the challenge of tuning a large number of control parameters in order to personalize the device for individual amputee users. This problem is not easily solved by traditional control designs or the latest robotic technology. Reinforcement learning (RL) is naturally appealing. The recent, unprecedented success of AlphaZero demonstrated RL as a feasible, large-scale problem solver. However, the prosthesis-tuning problem is associated with several unaddressed issues such as that it does not have a known and stable model, the continuous states and controls of the problem may result in a curse of dimensionality, and

15. Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks

    Xiang Zhang, Liting Sun, Zhian Kuang, et al. · 2021 · IEEE Robotics and Automation Letters · 122 citations

    Many manipulation tasks require robots to interact with unknown environments. In such applications, the ability to adapt the impedance according to different task phases and environment constraints is crucial for safety and performance. Although many approaches based on deep reinforcement learning (RL) and learning from demonstration (LfD) have been proposed to obtain variable impedance skills on contact-rich manipulation tasks, these skills are typically task-specific and could be sensitive to changes in task settings. This letter proposes an inverse reinforcement learning (IRL) based approach to recover both the variable impedance policy and reward function from expert demonstrations. We e

16. Improving Vision-Language-Action Model with Online Reinforcement Learning

    Yanjiang Guo, Jianke Zhang, Xiaoyu Chen, et al. · 2025 · 2025 IEEE International Conference on Robotics and Automation (ICRA) · 114 citations

    Recent studies have successfully integrated large vision-language models (VLMs) into low-level robotic control by supervised fine-tuning (SFT) with expert robotic datasets, resulting in what we term vision-language-action (VLA) models. Although the VLA models are powerful, how to improve these large models during interaction with environments remains an open question. In this paper, we explore how to further improve these VLA models via Reinforcement Learning (RL), a commonly used fine-tuning technique for large models. However, we find that directly applying online RL to large VLA models presents significant challenges, including training instability that severely impacts the performance of

17. Robust and Versatile Bipedal Jumping Control through Reinforcement Learning

    Zhongyu Li, Xue Bin Peng, P. Abbeel, et al. · 2023 · Robotics: Science and Systems XIX · 83 citations

    This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world. We present a reinforcement learning framework for training a robot to accomplish a large variety of jumping tasks, such as jumping to different locations and directions. To improve performance on these challenging tasks, we develop a new policy structure that encodes the robot's long-term input/output (I/O) history while also providing direct access to a short-term I/O history. In order to train a versatile jumping policy, we utilize a multi-stage training scheme that includes different training stages for different obj

18. Closed-loop Dynamic Control of a Soft Manipulator using Deep Reinforcement Learning

    Andrea Centurelli, Luca Arleo, A. Rizzo, et al. · 2022 · IEEE Robotics and Automation Letters · 81 citations

    The focus of the research community in the soft robotic field has been on developing innovative materials, but the design of control strategies applicable to these robotic platforms is still an open challenge. This is due to their highly nonlinear dynamics which are difficult to model and the degree of stochasticity they often incorporate. Data-driven controllers based on neural networks have recently been explored as a viable solution to be employed for these manipulators. This paper presents a neural network-based closed-loop controller, trained by a deep reinforcement learning algorithm called Trust Region Policy Optimization (TRPO). The training takes place in simulation, using an approx

19. Adaptive Control and Intersections with Reinforcement Learning

    A. Annaswamy · 2023 · Annu. Rev. Control. Robotics Auton. Syst. · 59 citations

    This article provides an exposition of the field of adaptive control and its intersections with reinforcement learning. Adaptive control and reinforcement learning are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by signific

20. RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning

    Kun Lei, Huanyu Li, Dongjie Yu, et al. · 2025 · Science robotics · 56 citations

    Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass skilled human operators. We present a real-world reinforcement learning (RL) framework, RL-100, for achieving complete task success under a predefined evaluation protocol built on diffusion visuomotor policies. RL-100 unifies imitation and RL under a single clipped proximal policy optimization surrogate objective applied in the denoising process, yielding conservative, stable improvements across offline and online stages. To meet deployment latency, a lightweight consistency distillation compresses multistep diffusion into a one-step controller for high-frequency co

21. Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance

    S. Luo, G. Androwis, S. Adamovich, et al. · 2021 · Frontiers in Robotics and AI · 50 citations

    A significant challenge for the control of a robotic lower extremity rehabilitation exoskeleton is to ensure stability and robustness during programmed tasks or motions, which is crucial for the safety of the mobility-impaired user. Due to various levels of the user’s disability, the human-exoskeleton interaction forces and external perturbations are unpredictable and could vary substantially and cause conventional motion controllers to behave unreliably or the robot to fall down. In this work, we propose a new, reinforcement learning-based, motion controller for a lower extremity rehabilitation exoskeleton, aiming to perform collaborative squatting exercises with efficiency, stability, and

22. Control Policy with Autocorrelated Noise in Reinforcement Learning for Robotics

    Paweł Wawrzyński · 2015 · International Journal of Machine Learning and Computing · 37 citations

23. A learning architecture based on reinforcement learning for adaptive control of the walking machine LAURON

    Winfried Ilg, Karsten Berns · 1995 · Robotics and Autonomous Systems · 37 citations

24. Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments From Temporal Logic Specifications

    Mingyu Cai, Erfan Aasi, C. Belta, et al. · 2022 · IEEE Robotics and Automation Letters · 35 citations

    Model-free continuous control for robot navigation tasks using Deep Reinforcement Learning (DRL) that relies on noisy policies for exploration is sensitive to the density of rewards. In practice, robots are usually deployed in cluttered environments, containing many obstacles and narrow passageways. Designing dense effective rewards is challenging, resulting in exploration issues during training. Such a problem becomes even more serious when tasks are described using temporal logic specifications. This work presents a deep policy gradient algorithm for controlling a robot with unknown dynamics operating in a cluttered environment when the task is specified as a Linear Temporal Logic (LTL) fo