Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

Planning with Uncertainty: Deep Exploration in Model-Based Reinforcement Learning

Yaniv Oren, Matthijs T. J. Spaan, and Wendelin Böhmer. Planning with Uncertainty: Deep Exploration in Model-Based Reinforcement Learning. arXiv:2210.13455, 2022.

Download

pdf 

Abstract

Deep model-based Reinforcement Learning (RL) has shown super-human performance in many challenging domains. Low sample efficiency and limited exploration remain as leading obstacles in the field, however. In this paper, we demonstrate deep exploration in model-based RL by incorporating epistemic uncertainty into planning trees, circumventing the standard approach of propagating uncertainty through value learning. We evaluate this approach with the state of the art model-based RL algorithm MuZero, and extend its training process to stabilize learning from explicitly-exploratory trajectories. In our experiments planning with uncertainty is able to demonstrate effective deep exploration with standard uncertainty estimation mechanisms, and with it significant gains in sample efficiency.

BibTeX Entry

@Misc{Oren22arxiv,
  author =       {Yaniv Oren and Matthijs T. J. Spaan and Wendelin
                  B{\"o}hmer},
  title =        {Planning with Uncertainty: Deep Exploration in
                  Model-Based Reinforcement Learning},
  howpublished = {arXiv:2210.13455},
  year =         2022
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Fri Mar 24, 2023 12:14:28 UTC