UMBC ACM Student Chapter

Planning and Learning in Complex Stochastic Domains: AMDPs, Option Discovery, Learning Transfer, Language Learning, and More

Dr. Marie desJardins, University of Maryland, Baltimore County
1-2pm Friday, March 9th, 2018, ITE 456, UMBC

Robots acting in human-scale environments must plan under uncertainty in large state–action spaces and face constantly changing reward functions as requirements and goals change. We introduce a new hierarchical planning framework called Abstract Markov Decision Processes (AMDPs) that can plan in a fraction of the time needed for complex decision making in ordinary MDPs. AMDPs provide abstract states, actions, and transition dynamics in multiple layers above a base-level “flat” MDP. AMDPs decompose problems into a series of subtasks with both local reward and local transition functions used to create policies for subtasks. The resulting hierarchical planning method is independently optimal at each level of abstraction, and is recursively optimal when the local reward and transition functions are correct.

I will present empirical results in several domains showing significantly improved planning speed, while maintaining solution quality. I will also discuss related work within the same project on automated option discovery, abstraction construction, language learning, and initial steps towards automated methods for learning AMDPs from base MDPs, from teacher demonstrations, and from direct observations in the domain.

This work is collaborative research with Dr. Michael Littman and Dr. Stefanie Tellex of Brown University. Dr. James MacGlashan of SIFT and Dr. Smaranda Muresan of Columbia University collaborated on earlier stages of the project. The following UMBC students have also contributed to the project: Khalil Anderson, Tadewos Bellete, Michael Bishoff, Rose Carignan, Nick Haltemeyer, Nathaniel Lam, Matthew Landen, Keith McNamara, Stephanie Milani, Shane Parr (UMass), Shawn Squire, Tenji Tembo, Nicholay Topin, Puja Trivedi, and John Winder.

Dr. Marie desJardins is a Professor of Computer Science and the Associate Dean for Academic Affairs in the College of Engineering and Information Technology at the University of Maryland, Baltimore County. Prior to joining the faculty at UMBC in 2001, she was a Senior Computer Scientist in the AI Center at SRI International. Her research is in artificial intelligence, focusing on the areas of machine learning, multi-agent systems, planning, interactive AI techniques, information management, reasoning with uncertainty, and decision theory. She is active in the computer science education community, founded the Maryland Center for Computing Education, and leads the CS Matters in Maryland project to develop curriculum and train high school teachers to teach AP CS Principles.

Dr. desJardins has published over 125 scientific papers in journals, conferences, and workshops. She will be the IJCAI-20 Conference Chair, and has been an Associate Editor of the Journal of Artificial Intelligence Research and the Journal of Autonomous Agents and Multi-Agent Systems, a member of the editorial board of AI Magazine, and Program Co-chair for AAAI-13. She has previously served as AAAI Liaison to the Board of Directors of the Computing Research Association, Vice-Chair of ACM’s SIGART, and AAAI Councillor. She is a AAAI Fellow, an ACM Distinguished Member, a Member-at-Large for Section T (Information, Computing, and Communication) of the American Association for the Advancement of Science, the 2014-17 UMBC Presidential Teaching Professor, a member and former chair of UMBC’s Honors College Advisory Board, former chair of UMBC’s Faculty Affairs Committee, and a member of the advisory board of UMBC’s Center for Women in Technology.