Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

Training and Transferring Safe Policies in Reinforcement Learning

Qisong Yang, Thiago D. Simão, Nils Jansen, Simon H. Tindemans, and Matthijs T. J. Spaan. Training and Transferring Safe Policies in Reinforcement Learning. In Adaptive and Learning Agents, 2022. Workshop at AAMAS22

Download

pdf 

Abstract

Safety is critical to broadening the application of reinforcement learning (RL). Often, RL agents are trained in a controlled environment, such as a laboratory, before being deployed in the real world. However, the target reward might be unknown prior to deployment. Reward-free RL addresses this problem by training an agent without the reward to adapt quickly once the reward is revealed. We consider the constrained reward-free setting, where an agent (the guide) learns to explore safely without the reward signal. This agent is trained in a controlled environment, which allows unsafe interactions and still provides the safety signal. After the target task is revealed, safety violations are not allowed anymore. Thus, the guide is leveraged to compose a safe sampling policy. Drawing from transfer learning, we also regularize a target policy (the student) towards the guide while the student is unreliable and gradually eliminate the influence from the guide as training progresses. The empirical analysis shows that this method can achieve safe transfer learning and helps the student solve the target task faster.

BibTeX Entry

@InProceedings{Yang22ala,
  author =       {Qisong Yang and Thiago D. Sim{\~a}o and Nils Jansen
                  and Simon H. Tindemans and Matthijs T. J. Spaan},
  title =        {Training and Transferring Safe Policies in
                  Reinforcement Learning},
  booktitle =    {Adaptive and Learning Agents},
  year =         2022,
  note =         {Workshop at AAMAS22}
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Thu Feb 29, 2024 16:15:45 UTC