Imagine learning to play a new video game. A natural approach would be to learn to make predictions about how the video game will respond to your actions (model learning) and to use those predictions to make decisions (planning). Model-based reinforcement learning (MBRL) is an analogous approach for artificial learning agents, where agents use their experience to create a predictive model of their environment and then use that model for planning purposes. Unfortunately, model-based learning has not been as successful in artificial agents as it is in natural ones! One major reason for this is that even tiny errors in the model can lead to catastrophic planning errors. In this project we will study this problem and potential remedies that may make MBRL more robust. We will work toward applying these ideas in the challenging domain of Atari 2600 games. Specific projects will be informed by the mutual interests and experience of student and mentor, but will likely center on questions about how to measure and represent model error/uncertainty, how to robustly make decisions using a flawed, uncertain model, and/or how to scale up MBRL techniques to complex, high-dimensional problems.
Learning Agents in Complex Environments (L.A.C.E.) Lab seeks to understand how artificial agents can be designed to flexibly learn to behave competently in a wide variety of complex, high-dimensional environments.
One reason to study this is that programs with this capability would be super useful! Flexibility and robust learning are key to many of the short- and long-term ambitious of artificial intelligence. If we ever want artificial agents that clean our houses, serve as personal assistants, and otherwise make sound decisions in our complicated and ever-changing world, it seems like they will have to be able to learn and reason about the world.
Another reason is that it is just so fundamentally weird that humans and other animals can just walk around in a world as complicated as this one and somehow get things done! By studying the problem of creating computational artifacts that can learn and behave flexibly, we may learn about the fundamental computational challenges that need to be overcome in order to make life in a complicated world possible.
Our work will of course not reach these grand ambitions in the short term, but every journey starts with one step...