Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

CEM: Constrained Entropy Maximization for Task-Agnostic Safe Exploration

Qisong Yang and Matthijs T. J. Spaan. CEM: Constrained Entropy Maximization for Task-Agnostic Safe Exploration. In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence, pp. 10798–10806, 2023.

Download

pdf 

Abstract

In the absence of assigned tasks, a learning agent typically seeks to explore its environment efficiently. However, the pursuit of exploration will bring more safety risks. An under-explored aspect of reinforcement learning is how to achieve safe efficient exploration when the task is unknown. In this paper, we propose a practical Constrained Entropy Maximization (CEM) algorithm to solve task-agnostic safe exploration problems, which naturally require a finite horizon and undiscounted constraints on safety costs. The CEM algorithm aims to learn a policy that maximizes state entropy under the premise of safety. To avoid approximating the state density in complex domains, CEM leverages a k-nearest neighbor entropy estimator to evaluate the efficiency of exploration. In terms of safety, CEM minimizes the safety costs, and adaptively trades off safety and exploration based on the current constraint satisfaction. The empirical analysis shows that CEM enables the acquisition of a safe exploration policy in complex environments, resulting in improved performance in both safety and sample efficiency for target tasks.

BibTeX Entry

@InProceedings{Yang23aaai,
  author =       {Qisong Yang and  Matthijs T. J. Spaan},
  title =        {{CEM}: Constrained Entropy Maximization for Task-Agnostic Safe Exploration},
  booktitle =    {Proceedings of the Thirty-Seventh AAAI Conference on
                  Artificial Intelligence},
  pages =        {10798--10806},
  year =         2023
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Thu Feb 29, 2024 16:15:45 UTC