AI Talk Series

Interested in artificial intelligence, machine learning, robotics, computer vision, natural language processing, and/or broadly related areas? All are welcome! Most talks for Fall 2020 will be held Tuesdays 11am – 12pm via Zoom. Subscribe to our mailing list by sending a blank email to with the subject line: “subscribe ai-seminar-l”. Contact David Crandall,, with questions or suggested speakers.

Fall 2020

Personalized Speech Enhancement: Test-Time Adaptation Using No or Few Private Data

Prof. Minje Kim, IU Intelligent Systems Engineering
Tuesday September 15, 11am

One of the keys to success in machine learning applications is to improve each user’s personal experience via personalized models. A personalized model can be a more resource-efficient solution than a general-purpose model, too, because it focuses on a particular sub-problem, for which a smaller model architecture can be good enough. However, training a personalized model requires data from the particular test-time user, which is not always available due to their private nature. Furthermore, such data tend to be unlabeled as it can be collected only during the test time, once after the system is deployed to the user devices. One could rely on the generalization power of a generic model, but such a model can be too computationally/spatially complex for real-time processing in a resource-constrained device. In this talk, I will present some techniques to circumvent the lack of labeled personal data in the context of speech enhancement. Our machine learning models will require no or few data samples from the test-time users, while they can still achieve the personalization goal. To this end, we will investigate modularized speech enhancement models as well as the potential of adversarial optimization and self-supervised learning for no- or few-shot fine-tuning for personalized speech enhancement. Because our research achieves the personalization goal in a privacy-preserving and resource-efficient way, it is a step towards a more available and affordable AI for society, while modern AI tends to be in the form of a large-scale generalist, which sometimes makes the model underperform for the socially under-represented groups.

eHealth AI and Large Scale Lifestyle Data Inference

Prof. Xiaozhong Liu, IU SICE
Tuesday September 22, 11am

Using large scale lifestyle data, e.g., diet information, online shopping records, electric pharmacy prescription logs, and social networks, for various health-AI tasks is exciting but challenging. By leveraging sophisticated machine learning, data analysis and joint learning techniques, we could infer very rich user lifestyle knowledge, which can help us to explore various kinds of individual health risks and locate the most vulnerable communities during epidemic outbreaks. In this talk, I will share three most recent works, Health Diet Recommendation, Computational User Health Profiling, and COVID19 Public Awareness Modeling, while all of those employ very large scale user lifestyle data, e.g., 94 million users plus 150 billion purchase/query logs, for machine learning. Unlike prior efforts on this track, large scale lifestyle data enables innovative and comprehensive investigations of different populations. It provides important potential to enhance our understanding of individual/community health risks.

Learning to Connect Images and Text for Natural Communication

Prof. Malihe Alikhani, Pitt Cyber
Monday October 5, 11am

From the gestures that accompany speech to images in social media posts, humans effortlessly combine words with visual presentations. However, machines are not equipped to understand and generate such presentations due to people’s pervasive reliance on common-sense and world knowledge in relating words and images. I present a novel framework for modeling and learning a deeper combined understanding of text and images by classifying inferential relations to predict temporal, causal, and logical entailments in context. This enables systems to make inferences with high accuracy while revealing author expectations and social-context preferences. I proceed to design methods for generating text based on visual input that use these inferences to provide users with key requested information. The results show a dramatic improvement in the consistency and quality of the generated text by decreasing spurious information by half. Finally, I sketch my other projects on human-robot collaboration and conversational systems and describe my research vision: to build human-level communicative systems and grounded artificial intelligence by leveraging the cognitive science of language use.

Using Virtual Reality and Artificial Intelligence to Reverse Engineer the Origins of Intelligence

Prof. Justin Wood, IU SICE
Tuesday October 20, 11am

What are the origins and computational foundations of intelligence? How can we replicate biological intelligence in machines? To address these questions, my lab uses a two-pronged approach. First, we perform controlled-rearing experiments, using newborn chicks as a model system. We raise chicks in strictly controlled virtual worlds and record their behavior 24/7 as they learn to perceive and understand their environment. Using virtual reality, we explore how core cognitive abilities (e.g., object perception) emerge in newborn brains. Second, we perform parallel experiments on autonomous artificial agents, using virtual controlled-rearing chambers. We raise artificial agents in the same environments as newborn chicks, and test whether they develop the same abilities when given the same experiences. The agents can be equipped with different learning algorithms, so by comparing animals and agents, we can isolate the core learning mechanisms needed to mimic biological intelligence. To facilitate progress, we are creating an “Origins of Intelligence” testbed, which will include benchmarks for evaluating whether artificial brains learn like newborn brains across a range of tasks.

Rehabilitating Isomap: Euclidean Representation of Geodesic Structure

Prof. Michael Trosset, IU Statistics
Tuesday October 27, 11am

Manifold learning techniques for nonlinear dimension reduction assume that high-dimensional feature vectors lie on a low-dimensional manifold, then attempt to exploit manifold structure to obtain useful low-dimensional Euclidean representations of the data. Isomap, a seminal manifold learning technique, is an elegant synthesis of two simple ideas: the approximation of Riemannian distances with shortest path distances on a graph that localizes manifold structure, and the approximation of shortest path distances with Euclidean distances by multidimensional scaling. We revisit the rationale for Isomap, clarifying what Isomap does and what it does not. In particular, we explore the widespread perception that Isomap should only be used when the manifold is parametrized by a convex region of Euclideanspace. We argue that this perception is based on an extremely narrow interpretation of manifold learning as parametrization recovery, and we submit that Isomap is better understood as constructing Euclidean representations of geodesic structure. We reconsider a well-known example that was previously interpreted as evidence of Isomap’s limitations, and we re-examine the original analysis ofIsomap’s convergence properties, concluding that convexity is not required for shortest path distances to converge to Riemannian distances.

A Tale of Two Parsers – The Final Chapter?

Prof. Joakim Nivre, Uppsala University
Monday November 2, 11am

Data-driven dependency parsing has for almost two decades been dominated by two main approaches: graph-based parsing and transition-based parsing. In its simplest forms, the two approaches are radically different and have complementary strengths and weaknesses, which are reflected in distinct error profiles. Over the years, however, research aiming to mitigate the weaknesses of each approach (without sacrificing the strengths) has led to a gradual convergence in methodology, which has been further accelerated by the adoption of deep learning techniques. In this talk, I will survey the development of graph-based and transition-based dependency parsing from a historical perspective but with an emphasis on recent work analyzing the impact of deep learning on the architecture and behavior of these parsers.

NLP for Scoring Content in Education

Dr. Brian Riordan, Educational Testing Service
Monday November 30, 11am

Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks and applications. This includes education, where recent neural methods have demonstrated improved human-machine agreement across core tasks including speech proficiency evaluation, essay scoring, short-answer (content) scoring, and argument mining. This talk will provide a short overview of NLP research for scoring and feedback in education at ETS, focusing on scoring content in student responses. We will cover recent work demonstrating challenges for interpretability and robustness of state-of-the-art neural methods and discuss remaining hurdles to reliably capturing student ideas in educational applications.

Summer 2020

Scalable and Privacy-Preserving Hardware/Algorithm Co-design for Accelerating Genome Sequencing

Prof. Lei Jiang, IU Intelligent Systems Engineering
Tuesday June 9, 4pm

Genome sequencing is the cornerstone of personalized medicine. The output of the genome sequencing pipeline for molecular tumor testing can be used to prioritize anti-cancer therapy and direct patient management. Therefore, its latency is a matter of life and death. Despite the huge advancement of sequencing machines, Moore’s Law cannot catch up with the explosion of genomic data. It is challenging for state-of-the-art hardware platforms to process the genome sequencing pipeline efficiently, due to the lack of scalable and large-capacity main memories. Moreover, frequently fetching small DNA reads with little spatial locality significantly increases the memory I/O data transfer energy. At last, blindly uploading private genomic data to untrusted servers in the cloud is dangerous. In this talk, I will present a scalable ReRAM-based main memory hardware system, an algorithm/hardware co-designed read alignment processing-in-memory accelerator and a privacy-preserving neural network algorithm design for accelerating genome sequencing.

Deep Learning for Time Series and Science Data  in MLPerf

Prof. Geoffrey Fox, IU Intelligent Systems Engineering
Tuesday June 16, 4pm

Deep Learning has been applied to many time series and sequences to sequence mappings but in many areas, the best way forward is not clear. Probably Industry is in the leads with audio applications and ride-hailing. We discuss a few research examples including COVID daily data, solutions of ordinary differential equations, and earthquakes. Deep learning surrogates are very promising. We show how working with the industry consortium MLPerf, we may be able to establish best practices and help the community discover and apply ideas to new fields.

Autism-inspired AI for Visuospatial and Social Reasoning

Prof. Maithilee Kunda, Department of Computer Science, Vanderbilt University
Tuesday June 23, 4pm

Individuals on the autism spectrum think differently from neurotypical individuals, including many having strengths in visuospatial reasoning and challenges in social and theory-of-mind reasoning. Our research aims to use computational methods in AI to better model and understand cognitive differences in autism and other neurodiverse conditions, and likewise to use what we learn from studying neurodiversity to inform the development of new AI techniques. In this talk, I will discuss two current projects: 1) studying strategy differences in the context of standardized human tests of visuospatial reasoning, including how intelligent agents discover and exploit new strategies based on their individual cognitive resources; 2) studying cognitive models of social cognition in the context of developing a new educational game, Film Detective, for teaching theory of mind and social reasoning skills to middle school students on the autism spectrum in a very visually oriented way.

Conversation with Beth Plale on the National Science Foundation

Prof. Beth Plale, IU SICE
Tuesday June 30, 4pm

Dr. Plale, who is completing a 3-year term at the National Science Foundation (NSF), will speak about agency priorities, will call attention to AI programmatic activity, and will speak to CISE priorities including broadening participation and open science. Intended for the early career researcher, this conversation will also call attention to some of the mechanics of interacting with the agency that can be confusing to the newcomer, and will leave plenty of time for discussion.

Actionable Models of Control in Complex Systems for the Biomedical Domain

Prof. Luis Rocha, IU Informatics & Cognitive Science
*** Thursday July  9, noon ***

Modern artificial intelligence (AI) and machine learning (ML) techniques predict system behavior from previous observations. However, in biology, and in true complex systems in general, profound transformation in system behavior can occur from never before seen observations—think zoonosis and COVID-19 pandemic. Moreover, AI/ML techniques are often unable to provide an explanation for their predictions. I will argue that complex systems modeling, while using AI/ML techniques to estimate parameters, needs to produce actionable, dynamical models that can predict and explain system behavior for rare or unseen control events. This is particularly true for understanding biochemical regulation under dynamical perturbations from environmental and evolutionary events. Towards that goal, I will present our recent work uncovering effective control pathways in the canalizing dynamics of biochemical regulation. Our methodology centers on removing redundancy from systems biology models of development, cell cycle, cancer response, and others. Removing the large amounts of redundancy in these models, reveals preferred pathways for the spread of perturbations and the building blocks of dynamical response, leading to the prediction of actionable control interventions.

Towards Psychoacoustically and Scientifically Valid Machine Learning for Speech Processing

Prof. Donald Williamson, IU Computer Science
Tuesday July 14, 4pm

Machine learning has helped make speech processing applications, such as speech recognition, more prevalent and ubiquitous, where it has become a major component of everyday life for millions of people. For other fields within speech processing, however, the usage has been less pronounced. This partially occurs because machine (deep) learning has not resulted in realistic outputs that sound natural and authentic to human observers. In this talk, we will discuss our recent efforts to ensure that the outputs from machine-learning speech algorithms are psychoacoustically and scientifically valid. In particular, we will show how incorporating perceptual and scientific principles into the learning algorithms may lead to further improvements.

Using Cognitive Science and Neuroscience to Help Advance AI

Prof. Zoran Tiganj, IU Computer Science & Cognitive Science
Tuesday July 21, 4pm

Building artificial agents that can mimic human learning and reasoning has been a longstanding objective in artificial intelligence. I will discuss some of the empirical data and computational models from neuroscience and cognitive science that could help us advance towards this goal. Specifically, I will talk about the importance of structured representations of knowledge, particularly about mental or cognitive maps for space, time, and concepts. I will present data from recent behavioral and neural studies, which suggest that the brain maintains a compressed mental timeline of the past and uses it to construct a compressed mental timeline of the future. From the computational perspective, these findings illustrate how associative learning can play a role in building structured representations of knowledge. Finally, I will discuss possible strategies to incorporate these findings into building artificial agents, especially in memory-augmented and attention-based neural networks.

Optimization with L0 Norm and Its Applications

Prof. Yijie Wang, IU Computer Science
Tuesday July 28, 4pm

The ongoing surge in sparse learning with L0 norm has drawn a lot of attention across several scientific communities. In this talk, I will first go over the recent advances in sparse regression and show the advantages of L0 norm regularization over other types of sparsity-inducing regularization (such as Lasso, MCP, SCAD). In the second part of the talk, I will focus on the proximal alternative linearized minimization (PALM) algorithm, which is able to solve non-convex and non-smooth optimizations. I will talk about how to cast the deep neural network (DNN) models into the PALM framework to train DNNs with non-convex and non-smooth regularizations. Specifically, I will talk DNN pruning using L0 norm sparse group lasso regularization. Besides using the L0 norm to induce model sparsity, in the end, I will talk about how to use the binary nature of the L0 norm to generate label information to integrate the collaborative filtering model and dictionary learning model. And how to apply the integrated model to predict gene regulatory networks.

Robust Multi-view Visual Learning: A Knowledge Flow Perspective

Prof. Zhengming Ding, IUPUI Computer and Information Technology
Tuesday August 4, 4pm

Multi-view data are extensively accessible nowadays thanks to various types of features, view-points and different sensors. For example, the most popular commercial depth sensor Kinect uses both visible light and near infrared sensors for depth estimation; automatic driving uses both visual and radar/lidar sensors to produce real-time 3D information on the road; and face analysis algorithms prefer face images from different views for high-fidelity reconstruction and recognition. All of them tend to facilitate better data representation in different application scenarios. Essentially, multiple features attempt to uncover various knowledge within each view to alleviate the final tasks, since each view would preserve both shared and private information. Recently, there are a bunch of approaches proposed to deal with multi-view visual data. Our tutorial covers most multi-view visual data representation approaches from two knowledge flows perspectives, i.e., knowledge fusion and knowledge transfer, centered from conventional multi-view learning to zero-shot learning, and from transfer learning to few-shot learning. We will discuss the current and upcoming challenges, which would benefit the artificial intelligence community in both industry and academia, from literature review to future directions.

Linking Exploits from the Dark Web to Known Vulnerabilities for Proactive Cyber Threat Intelligence: An Attention-based Deep Structured Semantic Model Approach

Prof. Sagar Samtani, Kelley School of Business
Tuesday August 18, 4pm

The Dark Web has emerged as a valuable source to proactively develop cyber threat intelligence (CTI) capabilities. Despite its value, Dark Web data contains tens of thousands of unstructured, un-sanitized text records containing significant non-natural language. This prevents the direct application of standard CTI analytics (e.g., malware analysis, IP reputation services) and text mining methodologies to perform critical tasks. One such challenge pertains to systematically linking Dark Web exploits to known vulnerabilities present within modern organizations. In this talk, I will present my recent work in extending a deep learning technique known as the Deep Structured Semantic Model (DSSM) (drawn the neural information retrieval) to incorporate emerging attention mechanisms from interpretability for deep learning literature. The resultant Exploit Vulnerability Attention DSSM (EVA-DSSM) automatically links hacker forum exploits and vulnerabilities provided by enterprise vulnerability assessment tools based on their names, outputs interpretable and explainable text features that are critical for creating links, and provides prioritized links for subsequent remediation and mitigation efforts. Rigorous evaluation indicates that EVA-DSSM outperforms baseline methods drawn from distributional semantics, probabilistic matching, and deep learning-based short text matching algorithms in matching relevant vulnerabilities from major vulnerability assessment tools to 0-day to web applications exploits, remote exploits, local exploits, and denial of service exploits. The framework’s utility in two contexts: the systems of selected major US hospitals and Supervisory Control and Data Acquisition (SCADA) systems worldwide.

For previous talks, please visit: IIS Seminar