Computer Science · Literature

Research papers on Reinforcement learning in robotics

Recent and highly-cited academic work on reinforcement learning in robotics, gathered from Semantic Scholar, CrossRef and OpenAlex.

1. Diagnosing Non-Intermittent Anomalies in Reinforcement Learning Policy Executions (Short Paper)

   Esper, Khalil, Spieck, Jan, Sixdenier, Pierre-Louis, et al. · 2017 · arXiv (Cornell University) · 11,345 citations

   Due to the safety risks and training sample inefficiency, it is often preferred to develop controllers in simulation. However, minor differences between the simulation and the real world can cause a significant sim-to-real gap. This gap can reduce the effectiveness of the developed controller. In this paper, we examine a case study of transferring an octorotor reinforcement learning controller from simulation to the real world. First, we quantify the effectiveness of the real-world transfer by examining safety metrics. We find that although there is a noticeable (around 100%) increase in deviation in real flights, this deviation may not be considered unsafe, as it will be within > 2m safety

2. Q-learning

   Christopher J. Watkins, Peter Dayan · 1992 · Machine Learning · 9,001 citations

   This paper presents the design and implementation of an integrated robotic system capable of detecting objects through computer vision and making decisions based on logic strategies to perform physical tasks. For that, the system uses a robotic arm to play the Tic-Tac-Toe game utilizing a Q-learning algorithm to determine optimal moves. The system can be controlled using a graphical interface that enables real-time monitoring, facilitating seamless interaction between the user and the robotic arm. Three algorithms with different decision strategies were developed: a random decision algorithm, the MiniMax algorithm, and Q-learning, a reinforcement-learning algorithm. The results obtained high

3. Reinforcement Learning: A Survey

   Leslie Pack Kaelbling, Michael L. Littman, Andrew Moore · 1996 · Journal of Artificial Intelligence Research · 8,866 citations

   This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations o

4. Deep Reinforcement Learning: A Brief Survey

   Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, et al. · 2017 · IEEE Signal Processing Magazine · 4,334 citations

   Deep reinforcement learning (DRL) is poised to revolutionize the field of artificial intelligence (AI) and represents a step toward building autonomous systems with a higher-level understanding of the visual world. Currently, deep learning is enabling reinforcement learning (RL) to scale to problems that were previously intractable, such as learning to play video games directly from pixels. DRL algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of RL, then progress to the main streams of value-based and policy-based methods. Our survey wil

5. A Comprehensive Survey of Multiagent Reinforcement Learning

   Lucian Buşoniu, Robert Babuška, Bart De Schutter · 2008 · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · 2,207 citations

   <para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multiagent reinforcement learning (MARL). A central issue in the field is the formal statement of the mult

6. Continuous control for robot based on deep reinforcement learning

   Shansi Zhang · 2019 · 940 citations

   One of the main targets of artificial intelligence is to solve the complex control problems which have high-dimensional observation spaces. Recently, the combination of deep learning and reinforcement learning has made remarkable progress, including the high-level performance in the video and board games, 3D navigations and robotic control. In this thesis, deep reinforcement learning algorithms are studied to perform some robotic tasks with continuous action spaces.

7. Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation

   Lei Tai, Giuseppe Paolo, Ming Liu · 2017 · 809 citations

   We present a learning-based mapless motion planner by taking the sparse 10-dimensional range findings and the target position with respect to the mobile robot coordinate frame as input and the continuous steering commands as output. Traditional motion planners for mobile ground robots with a laser range sensor mostly depend on the obstacle map of the navigation environment where both the highly precise laser sensor and the obstacle map building work of the environment are indispensable. We show that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be trained end-to-end without any manually designed features and prior demonstrations. The trained planne

8. Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

   Lukas Brunke, Melissa Greeff, Adam W. Hall, et al. · 2022 · Annual Review of Control Robotics and Autonomous Systems · 675 citations

   The last half decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. It includes learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safet

9. Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators

   Thomas George Thuruthel, Egidio Falotico, Federico Renda, et al. · 2018 · IEEE Transactions on Robotics · 395 citations

   Dynamic control of soft robotic manipulators is an open problem yet to be well explored and analyzed. Most of the current applications of soft robotic manipulators utilize static or quasi-dynamic controllers based on kinematic models or linearity in the joint space. However, such approaches are not truly exploiting the rich dynamics of a soft-bodied system. In this paper, we present a model-based policy learning algorithm for closed-loop predictive control of a soft robotic manipulator. The forward dynamic model is represented using a recurrent neural network. The closed-loop policy is derived using trajectory optimization and supervised learning. The approach is verified first on a simulate

10. Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control

    Fangyi Zhang, Jürgen Leitner, Michael Milford, et al. · 2015 · arXiv (Cornell University) · 215 citations

    This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works

11. Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

    Zhongyu Li, Xuxin Cheng, Xue Bin Peng, et al. · 2021 · 201 citations

    Developing robust walking controllers for bipedal robots is a challenging endeavor. Traditional model-based locomotion controllers require simplifying assumptions and careful modelling; any small errors can result in unstable control. To address these challenges for bipedal locomotion, we present a model-free reinforcement learning framework for training robust locomotion policies in simulation, which can then be transferred to a real bipedal Cassie robot. To facilitate sim-to-real transfer, domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. The learned policies enable Cassie to perform a set of diverse and dynamic

12. Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis

    Yue Wen, Jennie Si, Andrea Brandt, et al. · 2019 · IEEE Transactions on Cybernetics · 188 citations

    Robotic prostheses deliver greater function than passive prostheses, but we face the challenge of tuning a large number of control parameters in order to personalize the device for individual amputee users. This problem is not easily solved by traditional control designs or the latest robotic technology. Reinforcement learning (RL) is naturally appealing. The recent, unprecedented success of AlphaZero demonstrated RL as a feasible, large-scale problem solver. However, the prosthesis-tuning problem is associated with several unaddressed issues such as that it does not have a known and stable model, the continuous states and controls of the problem may result in a curse of dimensionality, and

13. Control Policy with Autocorrelated Noise in Reinforcement Learning for Robotics

    Paweł Wawrzyński · 2015 · International Journal of Machine Learning and Computing · 37 citations

14. A learning architecture based on reinforcement learning for adaptive control of the walking machine LAURON

    Winfried Ilg, Karsten Berns · 1995 · Robotics and Autonomous Systems · 37 citations

15. Adaptive optimal control of stencil printing process using reinforcement learning

    Nourma Khader, Sang Won Yoon · 2021 · Robotics and Computer-Integrated Manufacturing · 24 citations

16. Skillful control under uncertainty via direct reinforcement learning

    Vijaykumar Gullapalli · 1995 · Robotics and Autonomous Systems · 16 citations

17. Addressing Challenges in Dynamic Modeling of Stewart Platform using Reinforcement Learning-Based Control Approach

    Hadi Yadavari, Vahid Tavakol Aghaei, Serhat İkizoğlu · 2024 · Journal of Robotics and Control (JRC) · 6 citations

    In this paper, we focus on enhancing the performance of the controller utilized in the Stewart platform by investigating the dynamics of the platform. Dynamic modeling is crucial for control and simulation, yet challenging for parallel robots like the Stewart platform due to closed-loop kinematics. We explore classical methods to solve its inverse dynamical model, but conventional approaches face difficulties, often resulting in simplified and inaccurate models. To overcome this limitation, we propose a novel approach by replacing the classical feedforward inverse dynamic block with a reinforcement learning (RL) agent, which, to our knowledge, has not been tried yet in the context of the Ste

18. Consensus of Multi-agent Reinforcement Learning Systems: The Effect of Immediate Rewards

    Neshat Elhami Fard, Rastko Selmic · 2022 · Journal of Robotics and Control (JRC) · 6 citations

    This paper studies the consensus problem of a leaderless, homogeneous, multi-agent reinforcement learning (MARL) system using actor-critic algorithms with and without malicious agents. The goal of each agent is to reach the consensus position with the maximum cumulative reward. Although the reward function converges in both scenarios, in the absence of the malicious agent, the cumulative reward is higher than with the malicious agent present. We consider here various immediate reward functions. First, we study the immediate reward function based on Manhattan distance. In addition to proposing three different immediate reward functions based on Euclidean, $n$-norm, and Chebyshev distances, we

19. Data-Driven Reinforcement Learning Control for Quadrotor Systems

    Ngoc Trung Dang, Phuong Nam Dao · 2024 · International Journal of Mechanical Engineering and Robotics Research · 1 citations

    This paper aims to solve the tracking problem and optimality effectiveness of an Unmanned Aerial Vehicle (UAV) by model-free data Reinforcement Learning (RL) algorithms in both sub-systems of attitude and position. First, a cascade UAV model structure is given to establish the control system diagram with two corresponding attitude and position control loops. Second, based on the computation of the time derivative of the Bellman function by two different methods, the combination of the Bellman function and the optimal control is adopted to maintain the control signal as time converges to infinity with the addition of a discount factor. Third, according to off policy technique, the two propose

20. Control of Dynamic Systems in Robotics Using Reinforcement Learning

    Dr. Emma J. Clarke · 2022 · American Journal of Instrumentation and Control Engineering

    The field of robotics has made significant strides in utilizing Reinforcement Learning (RL) for the control of dynamic systems, which are systems that evolve over time due to internal or external factors. Unlike traditional control techniques, RL allows robotic systems to learn from their environment and adapt to dynamic conditions without requiring an explicit model of the system. This article explores the application of RL in controlling dynamic robotic systems, focusing on tasks such as motion control, path planning, and robotic manipulation. We discuss key RL methods including Q-learning, Deep Q Networks (DQN), and Proximal Policy Optimization (PPO), and how these techniques are applied