Publications

| Sorted by Date | Classified by Publication Type | Classified by Research Category |

Safe Policies for Factored Partially Observable Stochastic Games

Steven Carr, Nils Jansen, Suda Bharadwaj, Matthijs T. J. Spaan, and Ufuk Topcu. Safe Policies for Factored Partially Observable Stochastic Games. In Robotics: Science and System XVII, 2021.

Download

pdf 

Abstract

We study planning problems where a controllable agent operates under partial observability and interacts with an uncontrollable opponent, also referred to as the adversary. The agent has two distinct objectives: To maximize an expectedvalue and to adhere to a safety specification. Multi-objective partially observable stochastic games (POSGs) formally model such problems. Yet, even for a single objective, the task of computing suitable policies for POSGs is theoretically hard and computationally intractable in practice. Using a factored state-space representation, we define a decoupling scheme for the POSG state space that---under certain assumptions on the observability and the reward structure---separates the state components relevant for the reward from those relevant for safety. This decoupling affects the possibility to compute provably safe and reward-optimal policies in a tractable two-stage approach. In particular, on the fully observable components related to safety, we exactly compute the set of policies that captures all possible safe choices against the opponent. We restrict the agent\textquoterights behavior to these safe policies and project the POSG to a partially observable Markov decision process (POMDP). Any reward-maximal policy for the POMDP is then guaranteed to be safe and reward-maximal for the POSG. We showcase our approach\textquoterights feasibility using high-fidelity simulations of two case studies that concern UAV path planning and autonomous driving. Moreover, to demonstrate the practical applicability, we design a physical experiment involving a robot decision making problem under energy constraints that is motivated by a paired helicopter with NASA\textquoterights Perseverance Mars rover.

BibTeX Entry

@inproceedings{Carr21rss,
  title =        {Safe Policies for Factored Partially Observable
                  Stochastic Games},
  author =       {Steven Carr and Nils Jansen and Suda Bharadwaj and
                  Matthijs T. J. Spaan and Ufuk Topcu},
  year =         2021,
  booktitle =    {Robotics: Science and System XVII}
}

Note: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Generated by bib2html.pl (written by Patrick Riley) on Thu Feb 29, 2024 16:15:45 UTC