Designing Artificial Intelligence for Open Worlds


March 21-23, 2022 (VIRTUAL)

Sponsored by the Association for the Advancement of Artificial Intelligence

Open-world learning has taken on new importance in recent years as AI systems continue to be applied and transitioned to real-world settings where unexpected events (‘novelties’) can, and do, occur. An open-learning framework has been defined as one that can ‘deal with both normal in-distribution inputs and undesired out-of-distribution (OOD) inputs.’ Designing AI that can operate in open worlds, including detecting, characterizing and adapting to novelty, is a critical goal on the path to building intelligent systems that can work alongside humans to solve complex problems while being reliable enough to handle the unexpected. We invite contributions that describe novel technical approaches for open-world learning and novelty, theoretical frameworks for understanding open-world learning and novelty (including theories of novelty), empirical studies, implementations (including simulators and experimental testbeds), and lessons learned from current implementations.

Key Dates

Plenary Speakers and Panelists

Plenary Speakers

Panelists

Format

All three days will comprise a mix of invited talks, paper presentations, panels, and a townhall (on the final day). The symposium will be held virtually. Details on how to join will be emailed to registered participants.

Call for Papers

All submissions should be made via the AAAI SSS-22 EasyChair website. Please be sure to select 'Designing Artificial Intelligence for Open Worlds' as the relevant track when making your submission. We accept the following types of submissions in AAAI format:


•Full papers (up to 8 pages + unlimited pages for references)
•Short and experience papers (up to 4 pages + unlimited pages for references)

Topics (Non-Exhaustive)


• Open-world learning
• Unexpected situations and novelty
• Open-world simulations and gameplaying
• Online reinforcement learning
• Out-of-distribution inputs
• Robustness and anti-fragility in AI
• Architectures for novelty detection, characterization and adaptation
• Experimental methods and frameworks for evaluating OOD
• Theories of open-world learning and novelty
• Datasets and methodologies for open-world learning and novelty
• Experimental results using existing methods, including negative results of interest
• Systems-level issues, including engineering best pracitces
• Applications

Vision, Opinion and Position Papers

We will also accept a small number of vision, opinion and position papers that provide discussions on challenges and roadmaps for designing AI for open worlds.

Papers that do not meet the formatting requirements will be rejected without review. The accepted papers will be published online and will not be considered archival. Select papers may be invited to a special journal issue on open world learning.

Please email any enquiries to kejriwal@isi.edu

Contributed Papers

Representation Edit Distance as a Measure of Novelty. Joshua Alspector. [Paper]

Information-Theoretic Approach to Detect Collusion in Multi-Agent Games. Trevor Bonjour, Vaneet Aggarwal and Bharat Bhargava. [Paper]

Adversarial Creation of a Smart Home Testbed for Novelty Detection. Jarren Briscoe, Assefaw Gebremedhin, Lawrence Holder and Diane Cook. [Paper]

Metacognitive Mechanisms for Novelty Processing: Lessons for AI. Giedrius Burachas, Scott Grigsby, Bill Ferguson, Jeffrey Krichmar and Rajesh Rao. [Paper]

An Environment Transformation-based Framework for Comparison of Open-World Learning Agents. Matthew Molineaux and Dustin Dannenhauer. [Paper]

An Architecture for Novelty Handling in a Multi-Agent Stochastic Environment: Case Study in Open-World Monopoly. Tung Thai, Ming Shen, Neeraj Varshney, Sriram Gopalakrishnan, Utkarsh Soni, Chitta Baral, Jivko Sinapov and Matthias Scheutz. [Paper]

Anticipatory Thinking Challenges in Open Worlds: Risk Management. Adam Amos-Binks, Dustin Dannenhauer and Leilani Gilpin. [Paper]

Toward Defining Domain Complexity Measure Across Domains. Katarina Doctor, Christine Task, Eric Kildebeck, Mayank Kejriwal, Lawrence Holder and Russell Leong. [Paper]

Open-Learning Framework for Multi-modal Information Retrieval with Weakly Supervised Joint Embedding. Kma Solaiman and Bharat Bhargava. [Paper]

An Integrated Architecture for Online Adaptation to Novelty in Open Worlds using Probabilistic Programming and Novelty-Aware Planning. James Niehaus, Bryan Loyall and Avi Pfeffer. [Paper]

Measuring the Complexity of Domains Used to Evaluate AI Systems. Christopher Pereyda and Lawrence Holder. [Paper]

Decision making without prior knowledge in dynamic environments using Bootstrapped DQN. Bhargav Ganguly, Marina Haliem, Mridul Agarwal, Vaneet Aggarwal and Bharat Bhargava. [Paper]

Runtime Monitoring of Deep Neural Networks Using Top-Down Context Models Inspired by Predictive Processing and Dual Process Theory. Anirban Roy, Adam Cobb, Nathaniel D. Bastian, Brian Jalaian and Susmit Jha. [Paper]

NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty. Jonathan Balloch, Zhiyu Lin, Mustafa Hussain, Aarun Srinivas, Xiangyu Peng, Julia Kim and Mark Riedl. [Paper]

Self-Initiated Open World Learning for Autonomous AI Agents. Bing Liu, Eric Robertson, Scott Grigsby and Sahisnu Mazumder. [Paper]

Measurement of Novelty Difficulty in Monopoly. Kma Solaiman and Bharat Bhargava. [Paper]

Measuring the Performance of Open-World AI Systems. Vimukthini Pinto, Jochen Renz, Cheng Xue, Peng Zhang, Katarina Doctor and David Aha. [Paper]

Science Birds Novelty: an Open-world Learning Test-bed for Physics Domains. Cheng Xue, Vimukthini Pinto, Peng Zhang, Chathura Gamage, Ekaterina Nikonova and Jochen Renz. [Paper]

Continuous Learning Based Novelty Aware Emotion Recognition System. Mijanur Palash and Bharat Bhargava. [Paper]

Measuring Difficulty of Novelty Reaction. Ekaterina Nikonova, Cheng Xue, Vimukthini Pinto, Chathura Gamage, Peng Zhang and Jochen Renz. [Paper]

Invited Papers

Constraints on Theories of Open-World Learning. Pat Langley. [Paper]

L2Explorer: A Lifelong Reinforcement Learning Assessment Environment. Erik C. Johnson, Eric Q. Nguyen, Blake Schreurs, Chigozie S. Ewulum, Chace Ashcraft, Neil M. Fendley, Megan M. Baker, Alexander New, Gautam K. Vallabha. [Paper]

Program and Schedule

Day 1 (Mar. 21, 2022)

Session 1 (Plenary Talk I) | 12-1 pm EST / 9-10 am PST

• "Detecting, Coping with, and Adapting to Changing Worlds." Risto Miikkulainen, University of Texas at Austin.


Session 2 (Oral Presentations I) | 1-2 pm EST / 10-11 am PST

Matthew Molineaux and Dustin Dannenhauer. An Environment Transformation-based Framework for Comparison of Open-World Learning Agents | 20 minutes
Kma Solaiman and Bharat Bhargava. Open-Learning Framework for Multi-modal Information Retrieval with Weakly Supervised Joint Embedding | 10 minutes
Cheng Xue, Vimukthini Pinto, Peng Zhang, Chathura Gamage, Ekaterina Nikonova and Jochen Renz. Science Birds Novelty: an Open-world Learning Test-bed for Physics Domains | 20 minutes
Christopher Pereyda and Lawrence Holder. Measuring the Complexity of Domains Used to Evaluate AI Systems | 10 minutes


Session 3 (Panel I) | 2-3 pm EST / 11 am-noon PST

OWL: OWL: Open world learning challenges for AI
Panelists: Ser Nam Lim (Meta), Rob Hyland (Charles River Analytics), Jeff Krichmar (University of California, Irvine)


30-Minute Break | 3-3:30 pm EST / 12-12:30 pm PST


Session 4 (Oral Presentations II) | 3:30-5 pm EST / 12:30-2 pm PST

Anirban Roy, Adam Cobb, Nathaniel D. Bastian, Brian Jalaian and Susmit Jha. Runtime Monitoring of Deep Neural Networks Using Top-Down Context Models Inspired by Predictive Processing and Dual Process Theory | 20 minutes
Trevor Bonjour, Vaneet Aggarwal and Bharat Bhargava. Information-Theoretic Approach to Detect Collusion in Multi-Agent Games | 10 minutes
Adam Amos-Binks, Dustin Dannenhauer and Leilani Gilpin. Anticipatory Thinking Challenges in Open Worlds: Risk Management | 20 minutes
Ekaterina Nikonova, Cheng Xue, Vimukthini Pinto, Chathura Gamage, Peng Zhang and Jochen Renz. Measuring Difficulty of Novelty Reaction | 10 minutes
Jonathan Balloch, Zhiyu Lin, Mustafa Hussain, Aarun Srinivas, Xiangyu Peng, Julia Kim and Mark Riedl. NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty | 20 minutes
Mijanur Palash and Bharat Bhargava. Continuous Learning Based Novelty Aware Emotion Recognition System | 10 minutes


Day 2 (Mar. 22, 2022)

Session 1 (Plenary Talk II) | 12-1 pm EST / 9-10 am PST

Deva Kannan Ramanan, Carnegie Mellon University.


Session 2 (Oral Presentations III) | 1-2 pm EST / 10-11 am PST

Tung Thai, Ming Shen, Neeraj Varshney, Sriram Gopalakrishnan, Utkarsh Soni, Chitta Baral, Jivko Sinapov and Matthias Scheutz. An Architecture for Novelty Handling in a Multi-Agent Stochastic Environment: Case Study in Open-World Monopoly | 20 minutes
Bhargav Ganguly, Marina Haliem, Mridul Agarwal, Vaneet Aggarwal and Bharat Bhargava. Decision making without prior knowledge in dynamic environments using Bootstrapped DQN | 10 minutes
Giedrius Burachas, Scott Grigsby, Bill Ferguson, Jeffrey Krichmar and Rajesh Rao. Metacognitive Mechanisms for Novelty Processing: Lessons for AI | 20 minutes


Session 3 (Panel II) | 2-3 pm EST / 11 am-noon PST

Sim2Real: Challenges in transitioning AI advances from simulators and testbeds to the real-world
Panelists: Aniruddha (Ani) Kembhavi (Allen Institute), Luis A. Garcia (University of Southern California), Kevin Green (Oregon State University)


20-Minute Break | 3-3:20 pm EST / 12-12:20 pm PST


Session 4 (Invited Talk I) | 3:20-3:50 pm EST / 12:20-12:50 pm PST

Pat Langley. Constraints on Theories of Open-World Learning | 30 minutes


Session 5 (Oral Presentations IV) | 3:50-5 pm EST / 12:50-2 pm PST

Jarren Briscoe, Assefaw Gebremedhin, Lawrence Holder and Diane Cook. Adversarial Creation of a Smart Home Testbed for Novelty Detection | 20 minutes
Kma Solaiman and Bharat Bhargava. Measurement of Novelty Difficulty in Monopoly | 10 minutes
Katarina Doctor, Christine Task, Eric Kildebeck, Mayank Kejriwal, Lawrence Holder and Russell Leong. Toward Defining Domain Complexity Measure Across Domains | 20 minutes
Joshua Alspector. Representation Edit Distance as a Measure of Novelty | 10 minutes


Day 3 (Mar. 23, 2022)

Session 1 (Plenary Talk III) | 12-1 pm EST / 9-10 am PST

• "Human-centered Multimodal Machine Intelligence." Shri S. Narayanan, University of Southern California. [Slides]


Session 2 (Invited Talk II) | 1-1:30 pm EST / 10-10:30 am PST

Erik Johnson, Eric Nguyen, Blake Schreurs, Chigozie Ewulum, Chace Ashcraft, Neil Fendley, Megan Baker, Alexander New, Gautam Vallabha. L2Explorer: A Lifelong Reinforcement Learning Assessment Environment | 30 minutes


Session 3 (Oral Presentations V) | 1:30-2:30 pm EST / 10:30-11:30 am PST

Vimukthini Pinto, Jochen Renz, Cheng Xue, Peng Zhang, Katarina Doctor and David Aha. Measuring the Performance of Open-World AI Systems | 20 minutes
James Niehaus, Bryan Loyall and Avi Pfeffer. An Integrated Architecture for Online Adaptation to Novelty in Open Worlds using Probabilistic Programming and Novelty-Aware Planning | 20 minutes
Bing Liu, Eric Robertson, Scott Grigsby and Sahisnu Mazumder. Self-Initiated Open World Learning for Autonomous AI Agents | 20 minutes


30-Minute Break | 2:30-3:00 pm EST / 11:30 am-noon PST


Townhall: Open Discussion | 3-4:30 pm EST / 12-1:30 pm PST


Wrapup | 4:30-4:45 pm EST / 1:30-1:45 pm PST

Organizing Committee

  • Mayank Kejriwal, University of Southern California (Co-Chair)
  • Abhinav Shrivastava, University of Maryland (Co-Chair)
  • Eric Kildebeck, University of Texas at Dallas (Co-Chair)
  • Bharat Bhargava, Purdue University
  • Carl Vondrick, Columbia University
© 2021 The University of Southern California