Selectively Planning with a Flawed Predictive Model
One reasonable way for an artificial agent to learn to make good decisions is for it to learn a predictive model of its environment so it can “imagine” possible futures and plan its behavior to obtain positive outcomes. However, in the inevitable event that the model has flaws, and therefore makes incorrect predictions, the agent may make catastrophically bad decisions. In recent work, Prof. Talvitie and collaborators have been exploring how one might detect errors in the model and selectively use the model to take advantage of what it can predict accurately, while avoiding catastrophic planning failure from what it cannot. The plan for this summer is to continue building on some promising results, potentially including extending these ideas into more complicated model architectures or planning methods, using uncertainty measures to mediate between multiple models with different biases and limitations, or exploring ideas that we haven’t had yet!
(Note: if you are a US Citizen, you are encouraged to apply through https://etap.nsf.gov/award/1783/opportunity/10312; otherwise keep going through URO. Applications from both sites will be considered).
Learning Agents in Complex Environments (L.A.C.E.) Lab seeks to understand how artificial agents can be designed to flexibly learn to behave competently in a wide variety of complex, high-dimensional environments.
One reason to study this is that programs with this capability would be super useful! Flexibility and robust learning are key to many of the short- and long-term ambitious of artificial intelligence. If we ever want artificial agents that clean our houses, serve as personal assistants, and otherwise make sound decisions in our complicated and ever-changing world, it seems like they will have to be able to learn and reason about the world.
Another reason is that it is just so fundamentally weird that humans and other animals can just walk around in a world as complicated as this one and somehow get things done! By studying the problem of creating computational artifacts that can learn and behave flexibly, we may learn about the fundamental computational challenges that need to be overcome in order to make life in a complicated world possible.
Our work will of course not reach these grand ambitions in the short term, but every journey starts with one step....