Artificial Intelligence has made dramatic progress over just the last few years due primarily to deep machine learning using artificial neural networks. While traditional AI approaches require programmers to design hand-crafted, problem-specific rules for extracting features, deep learning uses models and algorithms that directly learn from low-level data, delivering unprecedented accuracy on a wide variety of problems. Despite these successes, however, state-of-the-art deep learning fails for important classes of problems. For example, deep learning-based image recognition struggles to detect objects that are small, that require fine-grained reasoning about appearance, that lack distinctive visual features, or for which few training examples exist to train a network’s vast number of parameters. Even with large training datasets, deep networks can be confused by minor noise caused randomly or by an adversary, and cannot explain their decisions to help debug these failures. These shortcomings will prevent deep learning from being safely applied in many domains that could most benefit from AI, including military, intelligence, security, medical, and legal applications where decisions must be justified, data is scarce, and the consequences of failure are great.
This project is investigating a path towards reliable, explainable, accurate machine learning by uniting deep learning with case-based reasoning (CBR), which are models that explicitly encode domain knowledge including common sense and human expertise. CBR makes decisions by consulting key training examples seen in the past through domain-specific similarity rules, adapting them to the target problem through adaptation rules, testing this solution, and using feedback to decide to add the example to the knowledge base. This approach has many advantages over deep learning, because it: (1) allows human expertise to be directly encoded by asking them to write cases and adaptation/similarity rules, (2) can reason from limited training data given appropriate adaptation and similarity rules, (3) improves over time in an inertia-free way without requiring retraining, (4) can explain its answers by citing the cases and adaptation rules that it used, and (5) mimics the prototype theory that humans are thought to use. Despite these advantages and numerous practical successes of CBR, deep learning has become much more popular because it delivers better accuracy on many practical problems when large datasets are available, can better discover statistical patterns that may not be apparent to a human, and high-quality, easy-to-use software frameworks are available.
We are investigating tight couplings of deep learning with case-based reasoning in order to directly inject domain knowledge into classification algorithms, taking advantage of the complementary strengths of these two approaches, in order to produce highly accurate classifiers that incorporate human expertise, are more robust to noisy exemplars, better tolerate small and biased training sets, and explain their decisions. The challenge is how to combine two techniques that have largely been studied and developed separately, with minimal communication or collaboration between the two communities, in part because they have largely focused on different application domain. Our team is ideal to address this gap, consisting of two established PIs with complementary expertise who nonetheless have been close colleagues for nearly a decade. The team will also include postdocs and Ph.D. students with developing expertise in deep learning and case-based reasoning.
This project is funded by the Office of Naval Research, “Combining Deep Learning and Case-based Reasoning for Robust, Accurate, Explainable Classification,” Oct 2019 – Sept 2022.
- Yue Chen, Ph.D. student in Computational Linguistics
- Larry Gates, Ph.D. student in Computer Science
- Chang Liu, M.S. student in Computer Science
- Vibhas Vats, M.S. student in Computer Science
- Xizi Wang, Ph.D. student in Computer Science
- Kaithleen Wilkerson, Ph.D. student in Computer Science
- Zachary Wilkerson, Ph.D. student in Computer Science
- Xiaomeng Ye, Ph.D. student in Computer Science
- Ziwei Zhao, Ph.D. student in Computer Science
The following publications are directly related to this project:
- David Leake, David Crandall, "Bringing Case Based Reasoning to Deep Learning," International Conference on Case-Based Reasoning Special Track on Challenges and Promises, 2020. [PDF]
- Xiaomeng Ye, David Leake, William Huibregtse, Mehmet Dalkilic, "Applying Class-to-Class Siamese Networks to Explain Classifications with Supportive and Contrastive Cases," International Conference on Case-Based Reasoning, 2020. [PDF]
Courses and educational materials
The PIs teach several courses that are related to the project. We make lecture notes and other materials available here with the hope that they may be useful to others.
- Probabilistic Approaches to Artificial Intelligence.
Uncertainty is a fact of everyday life, caused in part by incomplete and noisy observations, imperfect models, and (apparent) nondeterminism of the social and physical world. Much (and in some cases most) recent work across a range of computing disciplines (including artificial intelligence, robotics, computer vision, natural language processing, data mining, information retrieval, bioinformatics, etc.) has used probabilistic frameworks to explicitly address this uncertainty. This course will introduce the statistical, mathematical, and computational foundations of these frameworks, with a particular focus on a popular and very general framework called Probabilistic Graphical Models. We will also cover related topics in optimization and probability theory. We will study applications of these techniques across a range of AI disciplines, with perhaps a bias towards computer vision, and students will be encouraged to choose a final project that aligns with their own research interests.
- Computer Vision. This is an introductory-level graduate course that broadly introduces the field of computer vision. The course covers vision broadly, from fundamental signal processing through the latest approaches to key problems like recognition, stereo, and tracking. The course has also evolved to focus more on deep learning and machine learning in general, since these are the core interests of most students and the community at large. The course centers around 4 challenging, open-ended programming projects, and a final project in which students complete a research project of their choice and presente it at a public poster session. A gallery of the projects from this class is available.
- Elements of Artificial Intelligence. This is a graduate-level course that covers both AI fundamentals and cutting-edge techniques in machine learning, probabilistic inference, deep learning, and applications including computer vision and natural language processing. In addition to the residential section, the PI developed a new version of B551 for online data science students, including redesigned lectures, projects, and activities for the online format. The new version features video lectures organized in 5-10 minute segments, separated by short activities, in an attempt to replicate an in-person active learning classroom environment.
Examples of outreach and dissemination activities organized or conducted by the PIs include:
- The AI Talk Series, a new weekly seminar for topics related to computer vision, artificial intelligence, machine learning, data mining, and other broadly-related areas. Videos of talks are on made publicly available on the web to enhance the reach of the seminar.
- Talks and workshops at IU Mini University, a summer returning education program for alumni and community members.
Point of contact
This material is based upon work supported by the Office of Naval Research grant N00014-19-1-2655. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. Government.Last updated: August 14, 2020 at 15:44 pm